From dholmes at openjdk.org  Wed May  1 08:33:02 2024
From: dholmes at openjdk.org (David Holmes)
Date: Wed, 1 May 2024 08:33:02 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
Message-ID: <HAqZ1UXmEwazRLw9EGZlHP18rcyEL0lD-Jn3-xfsSUI=.5691bf5a-5dfe-442f-b359-b817c0c962c2@github.com>

On Tue, 23 Apr 2024 06:31:30 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removed extra blank line.
>
> This is a big change, but the pattern of the changes is quite easy to follow.
> 
> I do have a couple of queries below.
> 
> Thanks

> @dholmes-ora, I am not sure if you got all your comments addressed. Would you please, have a look at here? Thanks.

My comments were addressed - thanks - but I will leave it to the experts in this area to grant the approvals. I did spot one change in testHelpers.cpp that had not actually been made yet.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2088150756

From dholmes at openjdk.org  Wed May  1 08:33:03 2024
From: dholmes at openjdk.org (David Holmes)
Date: Wed, 1 May 2024 08:33:03 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <nwx8_k7fUWqYX7WR2uEhtGohrA4_-UPWDw0bJU2hnHs=.929ca2e7-217c-4c86-947d-3ba4564da2e0@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
 <4eN_yJUIi_0MTBROX0yxeIZIYo4W3KNlBGGOSA3glI4=.8e6ec837-1cb3-414f-959c-86fb3e3c9907@github.com>
 <nwx8_k7fUWqYX7WR2uEhtGohrA4_-UPWDw0bJU2hnHs=.929ca2e7-217c-4c86-947d-3ba4564da2e0@github.com>
Message-ID: <C_w0aa6Da9BBSTNuHgLH9yk2EgQ4ODME7KW7MEjBbnY=.b34b2c72-709b-434d-868c-cb283137d2b8@github.com>

On Mon, 15 Apr 2024 16:03:51 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/metaspace/testHelpers.cpp line 81:
>> 
>>> 79:   if (reserve_limit > 0) {
>>> 80:     // have reserve limit -> non-expandable context
>>> 81:     _rs = ReservedSpace(reserve_limit * BytesPerWord, Metaspace::reserve_alignment(), os::vm_page_size(), mtTest);
>> 
>> mtMetaspace
>
> Done

Not done yet

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1586016936

From azafari at openjdk.org  Wed May  1 09:33:44 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 1 May 2024 09:33:44 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v15]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <J3juKECbVB-fq2a1pkvn2R8OYzR1ape1f91zZou3-pI=.65b73189-4c60-47cc-9719-d2b0dfacc256@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  One missed mtTest is changed to mtMetaspace in testHelpers.cpp

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/72467f68..512144e0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=14
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=13-14

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Wed May  1 09:33:44 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 1 May 2024 09:33:44 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <C_w0aa6Da9BBSTNuHgLH9yk2EgQ4ODME7KW7MEjBbnY=.b34b2c72-709b-434d-868c-cb283137d2b8@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
 <4eN_yJUIi_0MTBROX0yxeIZIYo4W3KNlBGGOSA3glI4=.8e6ec837-1cb3-414f-959c-86fb3e3c9907@github.com>
 <nwx8_k7fUWqYX7WR2uEhtGohrA4_-UPWDw0bJU2hnHs=.929ca2e7-217c-4c86-947d-3ba4564da2e0@github.com>
 <C_w0aa6Da9BBSTNuHgLH9yk2EgQ4ODME7KW7MEjBbnY=.b34b2c72-709b-434d-868c-cb283137d2b8@github.com>
Message-ID: <DoKTK2ED3jegNDMhEZ-K_QTt48v9BPNMsAvUqkxdYT0=.34741c8a-862e-4e8d-8e79-64e335d8234c@github.com>

On Wed, 1 May 2024 08:28:09 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Done
>
> Not done yet

Should be done now.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1586061028

From dholmes at openjdk.org  Wed May  1 10:13:02 2024
From: dholmes at openjdk.org (David Holmes)
Date: Wed, 1 May 2024 10:13:02 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <kQ-4TZZW7fQOB7fMPXrlpblrlg5pjuNveFlwrh6VSUI=.6b2ad00e-5be5-489c-a8d4-e2b6e7957ac2@github.com>
 <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>
Message-ID: <kbIZ2ZVfs2T3XY-6gwXDSIXQ-gN92U-9uKU5LikRZ4I=.4e4a849f-b311-4f74-a089-56a116fe0764@github.com>

On Tue, 23 Apr 2024 08:54:42 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/metaspace/testHelpers.cpp line 81:
>> 
>>> 79:   if (reserve_limit > 0) {
>>> 80:     // have reserve limit -> non-expandable context
>>> 81:     _rs = ReservedSpace(reserve_limit * BytesPerWord, Metaspace::reserve_alignment(), os::vm_page_size(), mtMetaspace);
>> 
>> I would make this mtTest. This should not increase the metaspace counters in NMT
>
> Done.

Huh! Now these comments are appearing saying it should be `mtTest` ??? I think github is messing things up here with hidden comments that make the actual time flow invisible.

@tstuefe  can you please clarify what this should be. Thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1586128894

From kdnilsen at openjdk.org  Wed May  1 19:16:25 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 1 May 2024 19:16:25 GMT
Subject: Integrated: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC
In-Reply-To: <DKYYYui8zmVtGZCscDQfc5Gxio5NFVvMbxMbGeYSVLg=.e99b513e-354a-47c3-bb6a-73ad6d4e96fe@github.com>
References: <DKYYYui8zmVtGZCscDQfc5Gxio5NFVvMbxMbGeYSVLg=.e99b513e-354a-47c3-bb6a-73ad6d4e96fe@github.com>
Message-ID: <kchK3SnWwg55EBr0X4j-zprI3slDIaq6Ard5Wi1NPLI=.76b536d9-327c-4289-ad9a-b0a695743e90@github.com>

On Tue, 30 Apr 2024 01:00:25 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Reviewed-by: wkemper, ysr
> 
> This pull request contains a backport of commit bfb798a6 from the openjdk/shenandoah repository.
> 
> The commit being backported was authored by Kelvin Nilsen on 29 Apr 2024 and was reviewed by William Kemper and
> Y. Srinivas Ramakrishna.

This pull request has now been integrated.

Changeset: df0e30dc
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/df0e30dcca39ecc8359bcdadc59f894d1839d2db
Stats:     268 lines in 6 files changed: 199 ins; 57 del; 12 mod

8328307: GenShen: Re-enable old-has-grown trigger for old-generation GC

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/41

From azafari at openjdk.org  Thu May  2 07:23:08 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 2 May 2024 07:23:08 GMT
Subject: Integrated: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <0Lvalxj55jlSR1a0qw2-X0Xp_j-RIx0x3KZzOYhmlr0=.7457d51d-8eda-4b4e-b6d6-9c9a860612c0@github.com>

On Thu, 11 Apr 2024 15:54:38 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

This pull request has now been integrated.

Changeset: 4036d7d8
Author:    Afshin Zafari <azafari at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/4036d7d8246da0550adf8543848606c777da20a1
Stats:     449 lines in 62 files changed: 29 ins; 51 del; 369 mod

8330076: NMT: add/make a mandatory MEMFLAGS argument to family of os::reserve/commit/uncommit memory API

Reviewed-by: stefank, jsjolen, stuefe

-------------

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Thu May  2 07:23:07 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 2 May 2024 07:23:07 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <HAqZ1UXmEwazRLw9EGZlHP18rcyEL0lD-Jn3-xfsSUI=.5691bf5a-5dfe-442f-b359-b817c0c962c2@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
 <HAqZ1UXmEwazRLw9EGZlHP18rcyEL0lD-Jn3-xfsSUI=.5691bf5a-5dfe-442f-b359-b817c0c962c2@github.com>
Message-ID: <AV1pSDp4vnB5AbXdy1A4HHxZtSqbpo0V6xIz9Y7Mj3g=.b6545c65-2484-452e-a949-c525c0312492@github.com>

On Wed, 1 May 2024 08:30:34 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> This is a big change, but the pattern of the changes is quite easy to follow.
>> 
>> I do have a couple of queries below.
>> 
>> Thanks
>
>> @dholmes-ora, I am not sure if you got all your comments addressed. Would you please, have a look at here? Thanks.
> 
> My comments were addressed - thanks - but I will leave it to the experts in this area to grant the approvals. I did spot one change in testHelpers.cpp that had not actually been made yet.

Thank you @dholmes-ora for your review with a sharp eye that found that missed change.
Thank you @tstuefe, @stefank, @shipilev  and @jdksjolen for your comments and reviews.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2089780895

From stuefe at openjdk.org  Thu May  2 07:43:08 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Thu, 2 May 2024 07:43:08 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <HAqZ1UXmEwazRLw9EGZlHP18rcyEL0lD-Jn3-xfsSUI=.5691bf5a-5dfe-442f-b359-b817c0c962c2@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
 <HAqZ1UXmEwazRLw9EGZlHP18rcyEL0lD-Jn3-xfsSUI=.5691bf5a-5dfe-442f-b359-b817c0c962c2@github.com>
Message-ID: <2ZUb0sqaS1WXReRk0o8Ko2nywN0bYc3xgv2cPJzjWD0=.63938279-f252-427d-a944-1551ebda2acf@github.com>

On Wed, 1 May 2024 08:30:34 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> This is a big change, but the pattern of the changes is quite easy to follow.
>> 
>> I do have a couple of queries below.
>> 
>> Thanks
>
>> @dholmes-ora, I am not sure if you got all your comments addressed. Would you please, have a look at here? Thanks.
> 
> My comments were addressed - thanks - but I will leave it to the experts in this area to grant the approvals. I did spot one change in testHelpers.cpp that had not actually been made yet.

Thanks @dholmes-ora for the eagle eye. Background: I prefer mtTest to mtWhatever in gtests, since it is what I expect to see in NMT (storage for test data = mtTest, "live" metaspace memory = mtMetaspate).

And god, is the GH interface terrible when patches get large. Its almost impossiible to keep an overview.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2089813793

From shade at openjdk.org  Thu May  2 08:28:57 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 2 May 2024 08:28:57 GMT
Subject: RFR: 8331405: Shenandoah: Optimize ShenandoahLock with TTAS
In-Reply-To: <amhKGnHWJ3yuMV0UztstDW0nZE4XqJDHlBFbml8013k=.93124e7e-bf28-42e7-b144-95d1d336dbba@github.com>
References: <UDTvGn-vj89MGIceH4coaImzWYarTRENxxCXaDc3xGI=.2771ae59-73a8-481a-a49f-b6321eebd5e5@github.com>
 <amhKGnHWJ3yuMV0UztstDW0nZE4XqJDHlBFbml8013k=.93124e7e-bf28-42e7-b144-95d1d336dbba@github.com>
Message-ID: <zxACxQEgFvxi8N0lMZ4XebpoRLsRlBV2eG4sn-huCro=.73625b74-ef04-41ef-b70e-5ce218db0be7@github.com>

On Tue, 30 Apr 2024 19:55:51 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Can you share any comparative performance data, may be?

This was found by code inspection. I ran some tests, and the difference on very contended locks is small. It is certainly dominated by [JDK-8331411](https://bugs.openjdk.org/browse/JDK-8331411). So I think we better do some performance work with [JDK-8331411](https://bugs.openjdk.org/browse/JDK-8331411), not here.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19018#issuecomment-2089885166

From shade at openjdk.org  Thu May  2 08:28:57 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 2 May 2024 08:28:57 GMT
Subject: Integrated: 8331405: Shenandoah: Optimize ShenandoahLock with TTAS
In-Reply-To: <UDTvGn-vj89MGIceH4coaImzWYarTRENxxCXaDc3xGI=.2771ae59-73a8-481a-a49f-b6321eebd5e5@github.com>
References: <UDTvGn-vj89MGIceH4coaImzWYarTRENxxCXaDc3xGI=.2771ae59-73a8-481a-a49f-b6321eebd5e5@github.com>
Message-ID: <i52JcEaGLMiP2lnbxgrtoGNykzych14U06Q9A6PaeoM=.903d3284-89a4-446d-b6b9-e1f2ee7027d2@github.com>

On Tue, 30 Apr 2024 10:29:53 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> JDK-8325587 made the contended lock acquisition as the CAS loop with backoffs. Unfortunately, for `SpinPause` path, we always do the CAS and then spin. This is not efficient, as CAS likely requests memory for ownership, which exacerbates the contention and makes the spin loop run longer before blocking.
> 
> We should really use TTAS pattern there, like `Thread::SpinAcquire` does it.
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah`
>  - [ ] Linux AArch64 server fastdebug, `all` with `-XX:+UseShenandoahGC`

This pull request has now been integrated.

Changeset: c4fe5bf9
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/c4fe5bf90c2d368c29714de63a90eca444fb3ece
Stats:     2 lines in 1 file changed: 1 ins; 0 del; 1 mod

8331405: Shenandoah: Optimize ShenandoahLock with TTAS

Reviewed-by: zgu, ysr

-------------

PR: https://git.openjdk.org/jdk/pull/19018

From wkemper at openjdk.org  Thu May  2 14:21:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 2 May 2024 14:21:33 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <5-le93HW4HQagzNRKym1V2a70dI7OVKrZh5NQdXKp0M=.952cdb7c-5d4b-4709-89bd-3aeb7d7b7b0d@github.com>

Merges tag jdk-21.0.4+1

-------------

Commit messages:
 - 8326974: ODR violation in macroAssembler_aarch64.cpp
 - 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC
 - 8312014: [s390x] TestSigInfoInHsErrFile.java Failure
 - 8322962: Upcall stub might go undetected when freezing frames
 - 8325095: C2: bailout message broken: ResourceArea allocated string used after free
 - 8328168: Epsilon: Premature OOM when allocating object larger than uncommitted heap size
 - 8330156: RISC-V: Range check auipc + signed 12 imm instruction
 - 8321713: Harmonize executeTestJvm with create[Limited]TestJavaProcessBuilder
 - 8331113: createJMHBundle.sh support configurable maven repo mirror
 - 8309142: Refactor test/langtools/tools/javac/versions/Versions.java
 - ... and 232 more: https://git.openjdk.org/shenandoah-jdk21u/compare/05f18b10...3ab3f52b

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/42/files
  Stats: 33216 lines in 931 files changed: 18443 ins; 8016 del; 6757 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/42.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/42/head:pull/42

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/42

From shade at openjdk.org  Thu May  2 14:47:18 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 2 May 2024 14:47:18 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
Message-ID: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>

`CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).

Doing this rename separately guarantees we have caught and renamed all current uses.

Additional testing:
 - [ ] Linux AArch64 server fastdebug, `all`

-------------

Commit messages:
 - Fix

Changes: https://git.openjdk.org/jdk/pull/19064/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19064&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331573
  Stats: 64 lines in 27 files changed: 0 ins; 2 del; 62 mod
  Patch: https://git.openjdk.org/jdk/pull/19064.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19064/head:pull/19064

PR: https://git.openjdk.org/jdk/pull/19064

From stefank at openjdk.org  Thu May  2 17:08:53 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 2 May 2024 17:08:53 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
Message-ID: <fr25GpyrOEBZlCs0JyygHSuvMEQ9rSoMmGa-M7vxUxc=.2f9a3033-801d-48ac-ae44-d1115ae51f47@github.com>

On Thu, 2 May 2024 14:40:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> Doing this rename separately guarantees we have caught and renamed all current uses.
> 
> Additional testing:
>  - [ ] Linux AArch64 server fastdebug, `all`

Seems like a good change.

src/hotspot/share/gc/parallel/psParallelCompact.cpp line 1270:

> 1268: 
> 1269:   ParallelScavengeHeap* heap = ParallelScavengeHeap::heap();
> 1270:   assert(!heap->is_stw_gc_active(), "not reentrant");

While reading this I see that all these "not reentrant" asserts seems redundant given that we already do these checks in `IsSTWGCActiveMark`. Brownies points if you get rid of them. ;)

src/hotspot/share/gc/parallel/psParallelCompact.cpp line 1493:

> 1491:   PCAddThreadRootsMarkingTaskClosure(uint worker_id) : _worker_id(worker_id) { }
> 1492:   void do_thread(Thread* thread) {
> 1493:     assert(ParallelScavengeHeap::heap()->is_stw_gc_active(), "called outside gc");

Should this be updated to "called outside gc pause" as you did in `G1CollectedHeap::pin_object`? The same comment goes for the other occurrences below.

-------------

Changes requested by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19064#pullrequestreview-2036315901
PR Review Comment: https://git.openjdk.org/jdk/pull/19064#discussion_r1587988542
PR Review Comment: https://git.openjdk.org/jdk/pull/19064#discussion_r1587974562

From stefank at openjdk.org  Thu May  2 17:14:55 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 2 May 2024 17:14:55 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
Message-ID: <168Z8R7UX0uYymkZ_NCW17QbrwLu_82rg0j4nJ895_E=.5e1c4624-770f-4e94-9588-797982c2cca9@github.com>

On Thu, 2 May 2024 14:40:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> Doing this rename separately guarantees we have caught and renamed all current uses.
> 
> Additional testing:
>  - [ ] Linux AArch64 server fastdebug, `all`

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19064#pullrequestreview-2036349526

From shade at openjdk.org  Thu May  2 17:14:57 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 2 May 2024 17:14:57 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <fr25GpyrOEBZlCs0JyygHSuvMEQ9rSoMmGa-M7vxUxc=.2f9a3033-801d-48ac-ae44-d1115ae51f47@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
 <fr25GpyrOEBZlCs0JyygHSuvMEQ9rSoMmGa-M7vxUxc=.2f9a3033-801d-48ac-ae44-d1115ae51f47@github.com>
Message-ID: <sAM_-0wLAjqg0oLKFM8WEJ-Pm6nQeCWciNMoXiiKP5k=.26db98db-d5b0-4281-b980-c85036b98ff7@github.com>

On Thu, 2 May 2024 16:56:11 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
>> 
>> Doing this rename separately guarantees we have caught and renamed all current uses.
>> 
>> Additional testing:
>>  - [ ] Linux AArch64 server fastdebug, `all`
>
> src/hotspot/share/gc/parallel/psParallelCompact.cpp line 1493:
> 
>> 1491:   PCAddThreadRootsMarkingTaskClosure(uint worker_id) : _worker_id(worker_id) { }
>> 1492:   void do_thread(Thread* thread) {
>> 1493:     assert(ParallelScavengeHeap::heap()->is_stw_gc_active(), "called outside gc");
> 
> Should this be updated to "called outside gc pause" as you did in `G1CollectedHeap::pin_object`? The same comment goes for the other occurrences below.

I deliberately stopped myself from doing this for Parallel GC code, where every GC is STW GC :) I can change to "GC pause" if you want.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19064#discussion_r1587999105

From shade at openjdk.org  Thu May  2 17:29:52 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 2 May 2024 17:29:52 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <fr25GpyrOEBZlCs0JyygHSuvMEQ9rSoMmGa-M7vxUxc=.2f9a3033-801d-48ac-ae44-d1115ae51f47@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
 <fr25GpyrOEBZlCs0JyygHSuvMEQ9rSoMmGa-M7vxUxc=.2f9a3033-801d-48ac-ae44-d1115ae51f47@github.com>
Message-ID: <uhO3PJxmP13Eowq9xjkg_rg42uQhEet8jvvRDyGYnHo=.6c013ce1-aaeb-4f8d-b480-a4eb5fe39af9@github.com>

On Thu, 2 May 2024 17:04:44 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
>> 
>> Doing this rename separately guarantees we have caught and renamed all current uses.
>> 
>> Additional testing:
>>  - [ ] Linux AArch64 server fastdebug, `all`
>
> src/hotspot/share/gc/parallel/psParallelCompact.cpp line 1270:
> 
>> 1268: 
>> 1269:   ParallelScavengeHeap* heap = ParallelScavengeHeap::heap();
>> 1270:   assert(!heap->is_stw_gc_active(), "not reentrant");
> 
> While reading this I see that all these "not reentrant" asserts seems redundant given that we already do these checks in `IsSTWGCActiveMark`. Brownies points if you get rid of them. ;)

Ah, hm. Indeed! Separate PR? There is some light cleanup in G1 that can be associated with it. This PR would keep with just a mechanical rename.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19064#discussion_r1588015870

From stefank at openjdk.org  Thu May  2 17:29:53 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 2 May 2024 17:29:53 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <uhO3PJxmP13Eowq9xjkg_rg42uQhEet8jvvRDyGYnHo=.6c013ce1-aaeb-4f8d-b480-a4eb5fe39af9@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
 <fr25GpyrOEBZlCs0JyygHSuvMEQ9rSoMmGa-M7vxUxc=.2f9a3033-801d-48ac-ae44-d1115ae51f47@github.com>
 <uhO3PJxmP13Eowq9xjkg_rg42uQhEet8jvvRDyGYnHo=.6c013ce1-aaeb-4f8d-b480-a4eb5fe39af9@github.com>
Message-ID: <8rTp30vldMrfGYMh6uP-tirE9bjNGTBePOSztx95MD4=.8f9cdead-0301-42c4-acad-a2bfc26b4702@github.com>

On Thu, 2 May 2024 17:23:21 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> src/hotspot/share/gc/parallel/psParallelCompact.cpp line 1270:
>> 
>>> 1268: 
>>> 1269:   ParallelScavengeHeap* heap = ParallelScavengeHeap::heap();
>>> 1270:   assert(!heap->is_stw_gc_active(), "not reentrant");
>> 
>> While reading this I see that all these "not reentrant" asserts seems redundant given that we already do these checks in `IsSTWGCActiveMark`. Brownies points if you get rid of them. ;)
>
> Ah, hm. Indeed! Separate PR? There is some light cleanup in G1 that can be associated with it. This PR would keep with just a mechanical rename.

Sounds like a good idea.

>> src/hotspot/share/gc/parallel/psParallelCompact.cpp line 1493:
>> 
>>> 1491:   PCAddThreadRootsMarkingTaskClosure(uint worker_id) : _worker_id(worker_id) { }
>>> 1492:   void do_thread(Thread* thread) {
>>> 1493:     assert(ParallelScavengeHeap::heap()->is_stw_gc_active(), "called outside gc");
>> 
>> Should this be updated to "called outside gc pause" as you did in `G1CollectedHeap::pin_object`? The same comment goes for the other occurrences below.
>
> I deliberately stopped myself from doing this for Parallel GC code, where every GC is STW GC :) I can change to "GC pause" if you want.

Ah, I see. I wouldn't mind if it were changed to include "pause", but I'm also OK with you leaving it as is.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19064#discussion_r1588019866
PR Review Comment: https://git.openjdk.org/jdk/pull/19064#discussion_r1588018382

From zgu at openjdk.org  Thu May  2 17:34:52 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Thu, 2 May 2024 17:34:52 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
Message-ID: <tWHvsp_5iUx05gPof-L85Lk39aHhDECdwzT2kl2-Mfg=.c0bceceb-08d8-42f0-ab5f-bd11569b4223@github.com>

On Thu, 2 May 2024 14:40:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> Doing this rename separately guarantees we have caught and renamed all current uses.
> 
> Additional testing:
>  - [ ] Linux AArch64 server fastdebug, `all`

LGTM

-------------

Marked as reviewed by zgu (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19064#pullrequestreview-2036411170

From wkemper at openjdk.org  Thu May  2 23:26:17 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 2 May 2024 23:26:17 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations
Message-ID: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>

* The code for resizing and retrying young allocation requests according to what's available in the young generation has been removed. This code became redundant after the free set was rewritten and began providing the same functionality.
* An unused field and its attendant methods were removed from `ShenandoahThreadLocalData`.
* The code for managing `plab` expenditures and configuring other plab behavior has been moved from `ShenandoahHeap` into `ShenandoahOldGeneration`.

-------------

Commit messages:
 - Whitespace fixes
 - Refactor for clarity, add a unit-ish test
 - Move code out of shHeap into shOldGeneration
 - Factor out code to test if plab allocation is possible
 - Return early from plab allocation if old generation is exhausted
 - Simplify and document promotion reserve handling
 - Begin to clean up and isolate generational allocation path

Changes: https://git.openjdk.org/shenandoah/pull/427/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=427&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331609
  Stats: 495 lines in 7 files changed: 266 ins; 175 del; 54 mod
  Patch: https://git.openjdk.org/shenandoah/pull/427.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/427/head:pull/427

PR: https://git.openjdk.org/shenandoah/pull/427

From tschatzl at openjdk.org  Fri May  3 06:52:51 2024
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Fri, 3 May 2024 06:52:51 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
Message-ID: <bvCWv2N2-pnOGovIGFFPv2hyh1_ecNCi7IZ3dxe8R8Q=.709d2162-a8f7-4c42-bcee-caf828685f04@github.com>

On Thu, 2 May 2024 14:40:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> Doing this rename separately guarantees we have caught and renamed all current uses.
> 
> Additional testing:
>  - [ ] Linux AArch64 server fastdebug, `all`

Marked as reviewed by tschatzl (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19064#pullrequestreview-2037494618

From gli at openjdk.org  Fri May  3 08:20:53 2024
From: gli at openjdk.org (Guoxiong Li)
Date: Fri, 3 May 2024 08:20:53 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
Message-ID: <gEuar5Ns3svYSr0PsBbs5cVO_xJefh91YeuZl7APpBU=.be7f4e1c-05cb-4c4b-8a20-f3988e0f17af@github.com>

On Thu, 2 May 2024 14:40:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> Doing this rename separately guarantees we have caught and renamed all current uses.
> 
> Additional testing:
>  - [ ] Linux AArch64 server fastdebug, `all`

Looks good.

-------------

Marked as reviewed by gli (Committer).

PR Review: https://git.openjdk.org/jdk/pull/19064#pullrequestreview-2037637061

From azafari at openjdk.org  Fri May  3 10:14:04 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 3 May 2024 10:14:04 GMT
Subject: RFR: 8331540: [BACKOUT] NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API
Message-ID: <Iubos6Z11iQTSPTjMD0lRlNdihIaUDF46pkj5JVcqC8=.79c46a62-6b29-4826-967b-c9e9cce8b916@github.com>

reverted the changes.

-------------

Commit messages:
 - 8330076: [BACKOUT] NMT: add/make a mandatory MEMFLAGS argument to family of os::reserve/commit/uncommit memory API

Changes: https://git.openjdk.org/jdk/pull/19080/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19080&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331540
  Stats: 449 lines in 62 files changed: 51 ins; 29 del; 369 mod
  Patch: https://git.openjdk.org/jdk/pull/19080.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19080/head:pull/19080

PR: https://git.openjdk.org/jdk/pull/19080

From jwilhelm at openjdk.org  Fri May  3 10:19:56 2024
From: jwilhelm at openjdk.org (Jesper Wilhelmsson)
Date: Fri, 3 May 2024 10:19:56 GMT
Subject: RFR: 8331540: [BACKOUT] NMT: add/make a mandatory MEMFLAGS
 argument to family of os::reserve/commit/uncommit memory API
In-Reply-To: <Iubos6Z11iQTSPTjMD0lRlNdihIaUDF46pkj5JVcqC8=.79c46a62-6b29-4826-967b-c9e9cce8b916@github.com>
References: <Iubos6Z11iQTSPTjMD0lRlNdihIaUDF46pkj5JVcqC8=.79c46a62-6b29-4826-967b-c9e9cce8b916@github.com>
Message-ID: <dDtEa72WQeuH1D7hKo0Dh6cCt156b2PH64zEYWMDG-8=.c6c9c5d8-b1ad-416e-a92a-8c3074a6b456@github.com>

On Fri, 3 May 2024 10:09:06 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

> reverted the changes.

Looks good.

-------------

Marked as reviewed by jwilhelm (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19080#pullrequestreview-2037833419

From azafari at openjdk.org  Fri May  3 10:19:56 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 3 May 2024 10:19:56 GMT
Subject: RFR: 8331540: [BACKOUT] NMT: add/make a mandatory MEMFLAGS
 argument to family of os::reserve/commit/uncommit memory API
In-Reply-To: <dDtEa72WQeuH1D7hKo0Dh6cCt156b2PH64zEYWMDG-8=.c6c9c5d8-b1ad-416e-a92a-8c3074a6b456@github.com>
References: <Iubos6Z11iQTSPTjMD0lRlNdihIaUDF46pkj5JVcqC8=.79c46a62-6b29-4826-967b-c9e9cce8b916@github.com>
 <dDtEa72WQeuH1D7hKo0Dh6cCt156b2PH64zEYWMDG-8=.c6c9c5d8-b1ad-416e-a92a-8c3074a6b456@github.com>
Message-ID: <mF56_97zK7xshSDCtvN83YDgqLxPzWZbu6FvbtluT94=.55d7503b-d3bb-4034-85c2-8c6feabb0d90@github.com>

On Fri, 3 May 2024 10:14:59 GMT, Jesper Wilhelmsson <jwilhelm at openjdk.org> wrote:

>> reverted the changes.
>
> Looks good.

Thanks @JesperIRL.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19080#issuecomment-2092712777

From azafari at openjdk.org  Fri May  3 10:19:56 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 3 May 2024 10:19:56 GMT
Subject: Integrated: 8331540: [BACKOUT] NMT: add/make a mandatory MEMFLAGS
 argument to family of os::reserve/commit/uncommit memory API
In-Reply-To: <Iubos6Z11iQTSPTjMD0lRlNdihIaUDF46pkj5JVcqC8=.79c46a62-6b29-4826-967b-c9e9cce8b916@github.com>
References: <Iubos6Z11iQTSPTjMD0lRlNdihIaUDF46pkj5JVcqC8=.79c46a62-6b29-4826-967b-c9e9cce8b916@github.com>
Message-ID: <hmYwn8iTzrSzRZKq7K3cLzL9A-hHkWTwFDECX1fxUUs=.e38faf8f-70e7-46df-a3b3-07f483822a22@github.com>

On Fri, 3 May 2024 10:09:06 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

> reverted the changes.

This pull request has now been integrated.

Changeset: f665e07a
Author:    Afshin Zafari <azafari at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/f665e07ab223bdabb6cf3f653f799913d874bc55
Stats:     449 lines in 62 files changed: 51 ins; 29 del; 369 mod

8331540: [BACKOUT] NMT: add/make a mandatory MEMFLAGS argument to family of os::reserve/commit/uncommit memory API

Reviewed-by: jwilhelm

-------------

PR: https://git.openjdk.org/jdk/pull/19080

From wkemper at openjdk.org  Fri May  3 14:15:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 3 May 2024 14:15:21 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <H1kUL5WTyt7CUj75YNJq-UjWabUI9q1axiTsqBBCmII=.139432f1-1bf6-4384-8d84-31852979f9ae@github.com>

Merges tag jdk-23+21

-------------

Commit messages:
 - 8331222: Malformed text in the jpackage doc page
 - 8331427: Rename confusingly named ArraysSupport.signedHashCode
 - 8331212: Error recovery for broken switch expressions could be improved
 - 8331142: Add test for number of loader threads in BasicDirectoryModel
 - 8330969: scalability issue with loaded JVMTI agent
 - 8331393: AArch64: u32 _partial_subtype_ctr loaded/stored as 64
 - 8318682: SA decoding of scalar replaced objects is broken
 - 8331207: Misleading example in DateFormat#parse docs
 - 8329223: Parallel: Parallel GC resizes heap even if -Xms = -Xmx
 - 8331252: C2: MergeStores: handle negative shift values
 - ... and 58 more: https://git.openjdk.org/shenandoah/compare/87e864bf...e833bfc8

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/428/files
  Stats: 15145 lines in 1355 files changed: 4410 ins; 7219 del; 3516 mod
  Patch: https://git.openjdk.org/shenandoah/pull/428.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/428/head:pull/428

PR: https://git.openjdk.org/shenandoah/pull/428

From rkennke at openjdk.org  Fri May  3 16:29:13 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 3 May 2024 16:29:13 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
Message-ID: <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>

On Tue, 30 Apr 2024 17:40:47 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:
> 
>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>  - Merge branch 'openjdk:master' into master
>  - Merge branch 'openjdk:master' into master
>  - Remove unnecessary call to update_watermark
>  - Assert progress on find_next and find_prev
>  - Simplify partition_membership_name by code reuse
>  - Simplify by combining implemnetations of shrink_interval functions
>  - Fix NumPartition type
>    
>    Beautify the code by changing type of NumPartitions and adding Int and
>    UInt forms of NumPartitions.
>  - Refinements to support zero-build compiles
>  - Fix whitespace
>  - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c

This is much better.
I only looked at shenandoahSimpleBitMap.cpp, and only skimmed over the find-consecutive* methods. I'm not sure, but it looks like some of the search loops might result in quadratic behaviour?

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 31:

> 29: ShenandoahSimpleBitMap::ShenandoahSimpleBitMap(size_t num_bits) :
> 30:     _num_bits(num_bits),
> 31:     _num_words((num_bits + (BitsPerWord - 1)) / BitsPerWord),

I think it would be easier to read as `_num_words(align_up(num_bits, BitsPerWord) / BitsPerWord)`

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 49:

> 47:   uintx bit_number = start_idx & right_n_bits(LogBitsPerWord);
> 48:   uintx omit_mask = right_n_bits(bit_number);
> 49:   uintx mask = ((uintx) 0 - 1) & ~omit_mask;

Isn't that simply `mask = ~omit_mask`? If so, you could omit the line and fold it into the above line like `mask = ~right_n_bits(bit_number)`.

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 51:

> 49:   uintx mask = ((uintx) 0 - 1) & ~omit_mask;
> 50:   if ((element_bits & mask) == mask) {
> 51:     size_t counted_ones = BitsPerWord - bit_number;

So this is the case where all relevant bits are set, right? You might want to add a comment that says so, it took me quite a while to figure this out (including the mask-calculation that leads to it).

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 52:

> 50:   if ((element_bits & mask) == mask) {
> 51:     size_t counted_ones = BitsPerWord - bit_number;
> 52:     return counted_ones + count_leading_ones(start_idx - counted_ones);

If you have a long string of 1s, then recursively calling itself might blow up the stack. I guess writing it in an iterative way might be better (and perhaps perform better).

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 56:

> 54:     // Return number of consecutive ones starting with the_bit and including more significant bits.
> 55:     uintx aligned = element_bits >> bit_number;
> 56:     uintx complement = ~aligned;;

There is one too many ; at the end of the line.

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 79:

> 77: }
> 78: 
> 79: bool ShenandoahSimpleBitMap::is_forward_consecutive_ones(idx_t start_idx, idx_t count) const {

Is this a more general version of count_trailing_ones(), i.e. which takes a count (e.g. end-index) instead of implicitely counting until the end of the bitmap? If so, maybe write one to call the other? Also, it looks like here you are using an iterative approach - good!

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 109:

> 107: }
> 108: 
> 109: bool ShenandoahSimpleBitMap::is_backward_consecutive_ones(idx_t last_idx, idx_t count) const {

Same here.

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 167:

> 165:       return beg;
> 166:     } else {
> 167:       // There is at least one non-zero bit within the masked element_bits.  Find it.

I think you meant to say 'There is at least one zero bit...' ?

-------------

Changes requested by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-2038435936
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1589376266
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1589397374
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1589404576
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1589406131
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1589399565
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1589415216
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1589415470
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1589420905

From shade at openjdk.org  Mon May  6 11:21:56 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 6 May 2024 11:21:56 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
Message-ID: <6hpNRaO10dRNfshEfX3RHSxY5lcxrLeU1wH7zUDqTlA=.40086db0-2a55-4735-8a78-13f154e9155d@github.com>

On Thu, 2 May 2024 14:40:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> Doing this rename separately guarantees we have caught and renamed all current uses.
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `all`

Thanks all!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19064#issuecomment-2095775823

From shade at openjdk.org  Mon May  6 11:21:57 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 6 May 2024 11:21:57 GMT
Subject: RFR: 8331573: Rename CollectedHeap::is_gc_active to be explicitly
 about STW GCs
In-Reply-To: <8rTp30vldMrfGYMh6uP-tirE9bjNGTBePOSztx95MD4=.8f9cdead-0301-42c4-acad-a2bfc26b4702@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
 <fr25GpyrOEBZlCs0JyygHSuvMEQ9rSoMmGa-M7vxUxc=.2f9a3033-801d-48ac-ae44-d1115ae51f47@github.com>
 <uhO3PJxmP13Eowq9xjkg_rg42uQhEet8jvvRDyGYnHo=.6c013ce1-aaeb-4f8d-b480-a4eb5fe39af9@github.com>
 <8rTp30vldMrfGYMh6uP-tirE9bjNGTBePOSztx95MD4=.8f9cdead-0301-42c4-acad-a2bfc26b4702@github.com>
Message-ID: <a4-eSmGfS8T4pXNbZ2aeJOXQ_b3PmlRw9uRcMh_d1vo=.874668eb-e7a0-4847-86ec-5e2b03a4a591@github.com>

On Thu, 2 May 2024 17:26:58 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Ah, hm. Indeed! Separate PR? There is some light cleanup in G1 that can be associated with it. This PR would keep with just a mechanical rename.
>
> Sounds like a good idea.

Filed: https://bugs.openjdk.org/browse/JDK-8331719 -- I'll give it out to some of our folks as a starter task.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19064#discussion_r1590876454

From shade at openjdk.org  Mon May  6 11:21:57 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 6 May 2024 11:21:57 GMT
Subject: Integrated: 8331573: Rename CollectedHeap::is_gc_active to be
 explicitly about STW GCs
In-Reply-To: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
References: <ZXkiqLdTHyB_wVe2UFk9MyP8cZwgHl1XXFfGNv6X9Qo=.793fd4cd-f34d-4620-8b89-b43423e00b27@github.com>
Message-ID: <itqD6KuTJwl40Jq1Jo-M0I1bizP9HPDMb9e3eVj1I4g=.800cbaa3-f20e-45aa-9b8a-1d1968e1ed83@github.com>

On Thu, 2 May 2024 14:40:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `CollectedHeap::is_gc_active()` is confusing, since its name implies _any_ GC phase is running, while it actually only covers the STW GCs. It would be good to rename it for clarity. The freed-up name, `is_gc_active` could then be repurposed to track any (concurrent or STW) GC phase running. That would be useful to resolve [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> Doing this rename separately guarantees we have caught and renamed all current uses.
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `all`

This pull request has now been integrated.

Changeset: 1eec30a6
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/1eec30a6c03b7f4028405dc9bdb4d2a663b3987d
Stats:     64 lines in 27 files changed: 0 ins; 2 del; 62 mod

8331573: Rename CollectedHeap::is_gc_active to be explicitly about STW GCs

Reviewed-by: stefank, zgu, tschatzl, gli

-------------

PR: https://git.openjdk.org/jdk/pull/19064

From kdnilsen at openjdk.org  Mon May  6 20:46:20 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 6 May 2024 20:46:20 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations
In-Reply-To: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
Message-ID: <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>

On Thu, 2 May 2024 23:21:46 GMT, William Kemper <wkemper at openjdk.org> wrote:

> * The code for resizing and retrying young allocation requests according to what's available in the young generation has been removed. This code became redundant after the free set was rewritten and began providing the same functionality.
> * An unused field and its attendant methods were removed from `ShenandoahThreadLocalData`.
> * The code for managing `plab` expenditures and configuring other plab behavior has been moved from `ShenandoahHeap` into `ShenandoahOldGeneration`.

Changes requested by kdnilsen (Committer).

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1129:

> 1127:   if (result != nullptr && req.is_old()) {
> 1128:     old_generation()->configure_plab_for_current_thread(req);
> 1129:     // TODO: Is this only necessary for 'shared' allocations?

I believe this is only necessary for shared allocs.  I'm supportive of making this change as part of PR, though it might be safer to do in separate PR.

(If this is a PLAB allocation, the objects allocated within the PLAB will be independently registered when they are allocated.)

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 229:

> 227: 
> 228:   const size_t requested_bytes = req.size() * HeapWordSize;
> 229:   if (can_promote(requested_bytes)) {

Even though we don't check whether this req.is_promotion(), we know:
  if there's room to promote requested_bytes, there is also room to evacuate requested_bytes

A comment could be helpful here.

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 237:

> 235: 
> 236:   if (req.type() == ShenandoahAllocRequest::_alloc_plab) {
> 237:     // The promotion reserve is exhausted. Check if we still have room for evacuations.

We don't KNOW promotion reserve is entirely exhausted, just that what remains in promotion reserve is less than requested bytes.

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 243:

> 241:   }
> 242: 
> 243:   // This is a shared allocation promotion request. However, we do not have room for any

How do we know this is NOT a shared allocation evacuation request?  Is there a precondition on the method call?

Implementation below looks we're not sure this is a promotion.  We still need to check.

I think what we're saying here is:
  if req.is_promotion();   return false because we already checked above, and found that can_promote(requested_bytes) to be false.
  
  So what if req is not promotion?  wouldn't we want to confirm that get_evacuation_reserve() > requested_bytes?

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.hpp line 125:

> 123:   }
> 124: 
> 125:   // Test if there is enough memory available in the old generation to allocate a new PLAB.

Comment says this tests for memory to allocate a PLAB, in which case maybe we should rename as can_allocate_PLAB().  However implementation of can_allocate() checks whether req.type() == _alloc_plab.  Seems something's not right here.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/427#pullrequestreview-2041548822
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1591505837
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1591477363
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1591524664
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1591535274
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1591481761

From kdnilsen at openjdk.org  Mon May  6 21:09:08 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 6 May 2024 21:09:08 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
 <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
Message-ID: <ms14kxSPSBG2VcnNfFxP83ZjsYKvBTXjg5GEPj3MoRU=.0eed0405-dfc0-4acf-825d-9cd4c4b7f8f5@github.com>

On Fri, 3 May 2024 15:40:57 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Merge branch 'openjdk:master' into master
>>  - Remove unnecessary call to update_watermark
>>  - Assert progress on find_next and find_prev
>>  - Simplify partition_membership_name by code reuse
>>  - Simplify by combining implemnetations of shrink_interval functions
>>  - Fix NumPartition type
>>    
>>    Beautify the code by changing type of NumPartitions and adding Int and
>>    UInt forms of NumPartitions.
>>  - Refinements to support zero-build compiles
>>  - Fix whitespace
>>  - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c
>
> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 31:
> 
>> 29: ShenandoahSimpleBitMap::ShenandoahSimpleBitMap(size_t num_bits) :
>> 30:     _num_bits(num_bits),
>> 31:     _num_words((num_bits + (BitsPerWord - 1)) / BitsPerWord),
> 
> I think it would be easier to read as `_num_words(align_up(num_bits, BitsPerWord) / BitsPerWord)`

Agree that's cleaner.  Thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1591556062

From kdnilsen at openjdk.org  Mon May  6 21:16:10 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 6 May 2024 21:16:10 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
 <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
Message-ID: <bP8GfFfTmZ4oCPJJKMla-cgkS2A9KFA3T93spvOoWiM=.d6481151-e6a5-4a4b-bc38-0b0e8ce13912@github.com>

On Fri, 3 May 2024 15:59:14 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Merge branch 'openjdk:master' into master
>>  - Remove unnecessary call to update_watermark
>>  - Assert progress on find_next and find_prev
>>  - Simplify partition_membership_name by code reuse
>>  - Simplify by combining implemnetations of shrink_interval functions
>>  - Fix NumPartition type
>>    
>>    Beautify the code by changing type of NumPartitions and adding Int and
>>    UInt forms of NumPartitions.
>>  - Refinements to support zero-build compiles
>>  - Fix whitespace
>>  - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c
>
> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 49:
> 
>> 47:   uintx bit_number = start_idx & right_n_bits(LogBitsPerWord);
>> 48:   uintx omit_mask = right_n_bits(bit_number);
>> 49:   uintx mask = ((uintx) 0 - 1) & ~omit_mask;
> 
> Isn't that simply `mask = ~omit_mask`? If so, you could omit the line and fold it into the above line like `mask = ~right_n_bits(bit_number)`.

That's much better.  Thanks.

> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 56:
> 
>> 54:     // Return number of consecutive ones starting with the_bit and including more significant bits.
>> 55:     uintx aligned = element_bits >> bit_number;
>> 56:     uintx complement = ~aligned;;
> 
> There is one too many ; at the end of the line.

Fixed

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1591562180
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1591562827

From wkemper at openjdk.org  Mon May  6 21:36:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 6 May 2024 21:36:36 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v2]
In-Reply-To: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
Message-ID: <MfDso6KFlx2pWtdyizk0ty7JZoECGPModECN-1RS6GE=.cd301c81-e540-405d-8de0-d9abc32a4e09@github.com>

> * The code for resizing and retrying young allocation requests according to what's available in the young generation has been removed. This code became redundant after the free set was rewritten and began providing the same functionality.
> * An unused field and its attendant methods were removed from `ShenandoahThreadLocalData`.
> * The code for managing `plab` expenditures and configuring other plab behavior has been moved from `ShenandoahHeap` into `ShenandoahOldGeneration`.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix zero build, warnings in test

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/427/files
  - new: https://git.openjdk.org/shenandoah/pull/427/files/b6860298..b2163411

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=427&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=427&range=00-01

  Stats: 17 lines in 1 file changed: 2 ins; 0 del; 15 mod
  Patch: https://git.openjdk.org/shenandoah/pull/427.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/427/head:pull/427

PR: https://git.openjdk.org/shenandoah/pull/427

From kdnilsen at openjdk.org  Mon May  6 22:04:06 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 6 May 2024 22:04:06 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
 <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
Message-ID: <L4aILh2AEylsSOamjM1LcwbNeXCn_CBHH7lufo03g8A=.84207be3-2662-49b5-8560-9bb2c002ea54@github.com>

On Fri, 3 May 2024 16:07:43 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Merge branch 'openjdk:master' into master
>>  - Remove unnecessary call to update_watermark
>>  - Assert progress on find_next and find_prev
>>  - Simplify partition_membership_name by code reuse
>>  - Simplify by combining implemnetations of shrink_interval functions
>>  - Fix NumPartition type
>>    
>>    Beautify the code by changing type of NumPartitions and adding Int and
>>    UInt forms of NumPartitions.
>>  - Refinements to support zero-build compiles
>>  - Fix whitespace
>>  - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c
>
> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 52:
> 
>> 50:   if ((element_bits & mask) == mask) {
>> 51:     size_t counted_ones = BitsPerWord - bit_number;
>> 52:     return counted_ones + count_leading_ones(start_idx - counted_ones);
> 
> If you have a long string of 1s, then recursively calling itself might blow up the stack. I guess writing it in an iterative way might be better (and perhaps perform better).

Thanks.  Replacing recursion with iteration.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1591600333

From kdnilsen at openjdk.org  Tue May  7 05:09:11 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 7 May 2024 05:09:11 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
 <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
Message-ID: <esJAHOuP2ZnLWXEIV5w7lnS_7zca1hCr_lutmhUvxdM=.0a5062f4-4b37-403d-94ba-1ee8f7cea909@github.com>

On Fri, 3 May 2024 16:16:44 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Merge branch 'openjdk:master' into master
>>  - Remove unnecessary call to update_watermark
>>  - Assert progress on find_next and find_prev
>>  - Simplify partition_membership_name by code reuse
>>  - Simplify by combining implemnetations of shrink_interval functions
>>  - Fix NumPartition type
>>    
>>    Beautify the code by changing type of NumPartitions and adding Int and
>>    UInt forms of NumPartitions.
>>  - Refinements to support zero-build compiles
>>  - Fix whitespace
>>  - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c
>
> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 79:
> 
>> 77: }
>> 78: 
>> 79: bool ShenandoahSimpleBitMap::is_forward_consecutive_ones(idx_t start_idx, idx_t count) const {
> 
> Is this a more general version of count_trailing_ones(), i.e. which takes a count (e.g. end-index) instead of implicitely counting until the end of the bitmap? If so, maybe write one to call the other? Also, it looks like here you are using an iterative approach - good!

It would be possible to implement is_forward_consecutive_ones(idx start_idx, idx_t count) as count_leading_ones(start_idx) >= count.  However, that would be less efficient than having the independent implementations.  For example, suppose I want to check whether there are 3 consecutive ones, and I call count_leading_ones() and it happens to count 817 consecutive ones.  I didn't really NEED to count all 817 consecutive ones to find out that I have 3 consecutive ones.

> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 109:
> 
>> 107: }
>> 108: 
>> 109: bool ShenandoahSimpleBitMap::is_backward_consecutive_ones(idx_t last_idx, idx_t count) const {
> 
> Same here.

same issue here.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1591816910
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1591817009

From kdnilsen at openjdk.org  Tue May  7 05:56:09 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 7 May 2024 05:56:09 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
 <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
Message-ID: <l5tW7fWtAtEyn7e8BLX8A34CW_Fsz1UN499FQ_pha2A=.727eec80-f8e6-4891-b214-174e437c7e69@github.com>

On Fri, 3 May 2024 16:22:55 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Merge branch 'openjdk:master' into master
>>  - Remove unnecessary call to update_watermark
>>  - Assert progress on find_next and find_prev
>>  - Simplify partition_membership_name by code reuse
>>  - Simplify by combining implemnetations of shrink_interval functions
>>  - Fix NumPartition type
>>    
>>    Beautify the code by changing type of NumPartitions and adding Int and
>>    UInt forms of NumPartitions.
>>  - Refinements to support zero-build compiles
>>  - Fix whitespace
>>  - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c
>
> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 167:
> 
>> 165:       return beg;
>> 166:     } else {
>> 167:       // There is at least one non-zero bit within the masked element_bits.  Find it.
> 
> I think you meant to say 'There is at least one zero bit...' ?

The if clause handles the case when element_bits == 0.  The fact that we didn't take the if branch tells us there is at least one non-zero bit.  The else-if clause handles the case that the desired number of consecutive one bits are found at offset beg.  At this point, we know there is at least one non-zero bit and we know that we don't have num_bits consecutive one bits starting at beg.

I'm adding a comment to the code to explain how this works.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1591847964

From kdnilsen at openjdk.org  Tue May  7 06:00:32 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 7 May 2024 06:00:32 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v54]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <sDaGRecEP1ZOEcrdsEoENuMTZbtzwZo8mrNsUtbpL0w=.660ec5b8-6bdc-41e0-bacb-e7ddafa2cdfe@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 117 commits:

 - Respond to reviewer comments
 - Merge remote-tracking branch 'origin/master' into restructure-free-set
 - Merge branch 'openjdk:master' into master
 - Merge remote-tracking branch 'origin/master' into restructure-free-set
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Remove unnecessary call to update_watermark
 - Assert progress on find_next and find_prev
 - Simplify partition_membership_name by code reuse
 - Simplify by combining implemnetations of shrink_interval functions
 - ... and 107 more: https://git.openjdk.org/jdk/compare/b20fa7b4...be3a76de

-------------

Changes: https://git.openjdk.org/jdk/pull/17561/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=53
  Stats: 2578 lines in 8 files changed: 2194 ins; 189 del; 195 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue May  7 14:14:36 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 7 May 2024 14:14:36 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v55]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <mRCUVJVQvX3E1pG50XcFl1FHyq3MnrsVlz-4SvjEfoM=.e5a8e812-f6ef-4c5c-95b8-c82b934050aa@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Improve comment to clarify loop of find_first_consecutive_set_bits()

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/be3a76de..0fe4392d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=54
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=53-54

  Stats: 20 lines in 1 file changed: 0 ins; 11 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue May  7 14:19:23 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 7 May 2024 14:19:23 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v56]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <YNBowh0mA1hNu_gJkSYIB-f0XxUimZRL6hDJpLXHXUg=.cb820c5e-71fd-42a9-b2cd-cb1b0e7697da@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix typo in comment

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/0fe4392d..0bc2f9dc

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=55
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=54-55

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue May  7 14:35:33 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 7 May 2024 14:35:33 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Another attempt at improving clarity of comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/0bc2f9dc..bb2a5ac1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=56
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=55-56

  Stats: 8 lines in 1 file changed: 4 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue May  7 14:39:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 7 May 2024 14:39:12 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
 <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
Message-ID: <8u0Ga82-Nn-N6yJ3EW0avt0l5UmpTfBFmmylWGM07ec=.d0f7b430-8ab8-4233-b509-270e250c3c65@github.com>

On Fri, 3 May 2024 16:26:41 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Merge branch 'openjdk:master' into master
>>  - Remove unnecessary call to update_watermark
>>  - Assert progress on find_next and find_prev
>>  - Simplify partition_membership_name by code reuse
>>  - Simplify by combining implemnetations of shrink_interval functions
>>  - Fix NumPartition type
>>    
>>    Beautify the code by changing type of NumPartitions and adding Int and
>>    UInt forms of NumPartitions.
>>  - Refinements to support zero-build compiles
>>  - Fix whitespace
>>  - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c
>
> This is much better.
> I only looked at shenandoahSimpleBitMap.cpp, and only skimmed over the find-consecutive* methods. I'm not sure, but it looks like some of the search loops might result in quadratic behaviour?

@rkennke : Thanks for all your careful feedback.  If you could be more specific about loops that might result in quadratic behavior, I'll look more closely.  There are some loops within loops, but I'm thinking these are not quadratic because the inner loop bounds are constants.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17561#issuecomment-2098559363

From shade at openjdk.org  Tue May  7 17:20:01 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 7 May 2024 17:20:01 GMT
Subject: RFR: 8331721: Introduce CollectedHeap helper methods for any GC in
 progress
Message-ID: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>

Continuation of [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573). So far we only have a way to test for STW GCs. We would need these more generic sensing methods for [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).

This patch considers that there can be several concurrent GCs in progress. A notable example of that is Generational ZGC, where minor and major concurrent collections can run at the same time.
 
Initially, I thought to use `is_gc_active` name freed up by [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573), but then I thought better of it. My concern would be that some backports might be confused about what `is_gc_active` name really means, depending on whether JDK-8331573 and this PR is backported or not. To avoid this confusion, I chose `is_any_gc_active` name instead.

This PR is somewhat open for bikeshedding about the names :)

Additional testing:
 - [x] Linux AArch64 server fastdebug, `hotspot_gc`
 - [ ] Linux AArch64 server fastdebug, `all`

-------------

Commit messages:
 - Work

Changes: https://git.openjdk.org/jdk/pull/19122/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19122&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331721
  Stats: 48 lines in 9 files changed: 44 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/19122.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19122/head:pull/19122

PR: https://git.openjdk.org/jdk/pull/19122

From zgu at openjdk.org  Tue May  7 17:57:52 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Tue, 7 May 2024 17:57:52 GMT
Subject: RFR: 8331721: Introduce CollectedHeap helper methods for any GC in
 progress
In-Reply-To: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
References: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
Message-ID: <LduSLH5Ivv84B965GN1tzWFFdTElhZzivPVVMdHwOkI=.5c8d695a-8595-4e75-bfa3-f3105f826952@github.com>

On Tue, 7 May 2024 14:33:26 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Continuation of [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573). So far we only have a way to test for STW GCs. We would need these more generic sensing methods for [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> This patch considers that there can be several concurrent GCs in progress. A notable example of that is Generational ZGC, where minor and major concurrent collections can run at the same time.
>  
> Initially, I thought to use `is_gc_active` name freed up by [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573), but then I thought better of it. My concern would be that some backports might be confused about what `is_gc_active` name really means, depending on whether JDK-8331573 and this PR is backported or not. To avoid this confusion, I chose `is_any_gc_active` name instead.
> 
> This PR is somewhat open for bikeshedding about the names :)
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc`
>  - [ ] Linux AArch64 server fastdebug, `all`

Can you explain the purpose of this change? I assume you want to enable `oopMapCache` only when any gc is active, am I correct?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19122#issuecomment-2098998387

From shade at openjdk.org  Tue May  7 18:31:07 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 7 May 2024 18:31:07 GMT
Subject: RFR: 8331721: Introduce CollectedHeap helper methods for any GC in
 progress [v2]
In-Reply-To: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
References: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
Message-ID: <X_NQd5z8qzJ0SVGGTEYLfL7sAewGAILb6t0ryoDCo1E=.ca9ba290-2128-4e7a-92c1-fa32da73c0bd@github.com>

> Continuation of [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573). So far we only have a way to test for STW GCs. We would need these more generic sensing methods for [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> This patch considers that there can be several concurrent GCs in progress. A notable example of that is Generational ZGC, where minor and major concurrent collections can run at the same time. For G1, concurrent mark can interleave with collection pauses. This is why we track the number of GCs in progress as atomic.
>  
> Initially, I thought to use `is_gc_active` name freed up by [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573), but then I thought better of it. My concern would be that some backports might be confused about what `is_gc_active` name really means, depending on whether JDK-8331573 and this PR is backported or not. To avoid this confusion, I chose `is_any_gc_active` name instead.
> 
> This PR is somewhat open for bikeshedding about the names :)
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc`
>  - [ ] Linux AArch64 server fastdebug, `all`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Switch to atomic to accurately track concurrent GC count

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19122/files
  - new: https://git.openjdk.org/jdk/pull/19122/files/93d4abf9..943325bc

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19122&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19122&range=00-01

  Stats: 24 lines in 4 files changed: 6 ins; 9 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/19122.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19122/head:pull/19122

PR: https://git.openjdk.org/jdk/pull/19122

From shade at openjdk.org  Tue May  7 18:31:08 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 7 May 2024 18:31:08 GMT
Subject: RFR: 8331721: Introduce CollectedHeap helper methods for any GC in
 progress
In-Reply-To: <LduSLH5Ivv84B965GN1tzWFFdTElhZzivPVVMdHwOkI=.5c8d695a-8595-4e75-bfa3-f3105f826952@github.com>
References: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
 <LduSLH5Ivv84B965GN1tzWFFdTElhZzivPVVMdHwOkI=.5c8d695a-8595-4e75-bfa3-f3105f826952@github.com>
Message-ID: <bVlVG1FZ7MhXEH-xpKb4M63iMoZ8BBjiNKxOV5Oq_t4=.8a1e9777-eb8d-4ae1-a36d-96abf1a8570d@github.com>

On Tue, 7 May 2024 17:55:10 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Can you explain the purpose of this change? I assume you want to enable `oopMapCache` only when any gc is active, am I correct?

Yes, I want to replace the OopMapCache gating check with `is_any_gc_active()`. But I think tracking concurrent GCs in progress might be useful for other cases, _and_ `OopMapCache` change would require more touchups, so I don't want to cobble these together.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19122#issuecomment-2099050723

From shade at openjdk.org  Tue May  7 18:38:05 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 7 May 2024 18:38:05 GMT
Subject: RFR: 8331721: Introduce CollectedHeap helper methods for any GC in
 progress [v3]
In-Reply-To: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
References: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
Message-ID: <_yFkn6p3JH9DPML_qw04vIUwu-yOcOLap3zjoKOrM3c=.2d01cca2-2ee5-4c16-9aae-ffd417cc76b3@github.com>

> Continuation of [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573). So far we only have a way to test for STW GCs. We would need these more generic sensing methods for [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> This patch considers that there can be several concurrent GCs in progress. A notable example of that is Generational ZGC, where minor and major concurrent collections can run at the same time. For G1, concurrent mark can interleave with collection pauses. This is why we track the number of GCs in progress as atomic.
>  
> Initially, I thought to use `is_gc_active` name freed up by [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573), but then I thought better of it. My concern would be that some backports might be confused about what `is_gc_active` name really means, depending on whether JDK-8331573 and this PR is backported or not. To avoid this confusion, I chose `is_any_gc_active` name instead.
> 
> This PR is somewhat open for bikeshedding about the names :)
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc`
>  - [ ] Linux AArch64 server fastdebug, `all`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Cleanup

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19122/files
  - new: https://git.openjdk.org/jdk/pull/19122/files/943325bc..e622d256

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19122&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19122&range=01-02

  Stats: 3 lines in 1 file changed: 0 ins; 2 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/19122.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19122/head:pull/19122

PR: https://git.openjdk.org/jdk/pull/19122

From zgu at openjdk.org  Tue May  7 20:32:52 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Tue, 7 May 2024 20:32:52 GMT
Subject: RFR: 8331721: Introduce CollectedHeap helper methods for any GC in
 progress
In-Reply-To: <bVlVG1FZ7MhXEH-xpKb4M63iMoZ8BBjiNKxOV5Oq_t4=.8a1e9777-eb8d-4ae1-a36d-96abf1a8570d@github.com>
References: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
 <LduSLH5Ivv84B965GN1tzWFFdTElhZzivPVVMdHwOkI=.5c8d695a-8595-4e75-bfa3-f3105f826952@github.com>
 <bVlVG1FZ7MhXEH-xpKb4M63iMoZ8BBjiNKxOV5Oq_t4=.8a1e9777-eb8d-4ae1-a36d-96abf1a8570d@github.com>
Message-ID: <t9DJ7x7V5GF4ePkaK4zTqlxEd99aHmbKOZbZzp2za-Y=.3c2f62dc-5de5-458b-8767-001702825a34@github.com>

On Tue, 7 May 2024 18:26:07 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> > Can you explain the purpose of this change? I assume you want to enable `oopMapCache` only when any gc is active, am I correct?
> 
> Yes, I want to replace the OopMapCache gating check with `is_any_gc_active()`. But I think tracking concurrent GCs in progress might be useful for other cases, _and_ `OopMapCache` change would require more touchups, so I don't want to cobble these together.

But is it really necessary? IMO, some stack walk code can also benefit from `oopMapCache`, e.g. https://code.devsnc.com/dev/glide-jdk17/blob/995d513f19fadf233b64f1caee386daa5700148d/src/hotspot/share/prims/stackwalk.cpp#L298

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19122#issuecomment-2099255420

From wkemper at openjdk.org  Tue May  7 21:16:16 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 7 May 2024 21:16:16 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v2]
In-Reply-To: <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
Message-ID: <6aJTFhR071whY5rfEAzfl0MMGY6AAss_dUWBnNKm56A=.4f14bac3-6316-42e3-b7f0-993a8c337de4@github.com>

On Mon, 6 May 2024 20:13:00 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix zero build, warnings in test
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1129:
> 
>> 1127:   if (result != nullptr && req.is_old()) {
>> 1128:     old_generation()->configure_plab_for_current_thread(req);
>> 1129:     // TODO: Is this only necessary for 'shared' allocations?
> 
> I believe this is only necessary for shared allocs.  I'm supportive of making this change as part of PR, though it might be safer to do in separate PR.
> 
> (If this is a PLAB allocation, the objects allocated within the PLAB will be independently registered when they are allocated.)

I'm going to test making this change as part of this PR.

> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 229:
> 
>> 227: 
>> 228:   const size_t requested_bytes = req.size() * HeapWordSize;
>> 229:   if (can_promote(requested_bytes)) {
> 
> Even though we don't check whether this req.is_promotion(), we know:
>   if there's room to promote requested_bytes, there is also room to evacuate requested_bytes
> 
> A comment could be helpful here.

Done

> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.hpp line 125:
> 
>> 123:   }
>> 124: 
>> 125:   // Test if there is enough memory available in the old generation to allocate a new PLAB.
> 
> Comment says this tests for memory to allocate a PLAB, in which case maybe we should rename as can_allocate_PLAB().  However implementation of can_allocate() checks whether req.type() == _alloc_plab.  Seems something's not right here.

Fixed the comment.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1593101415
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1593098077
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1593100240

From wkemper at openjdk.org  Tue May  7 21:21:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 7 May 2024 21:21:25 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v2]
In-Reply-To: <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
Message-ID: <70rQgtKA4FhMxqFtX-9vSkFhtWEMs1Sxxc6Ovttk0YQ=.218ec561-139e-4981-9512-26e5d46e89b6@github.com>

On Mon, 6 May 2024 20:32:28 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix zero build, warnings in test
>
> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 237:
> 
>> 235: 
>> 236:   if (req.type() == ShenandoahAllocRequest::_alloc_plab) {
>> 237:     // The promotion reserve is exhausted. Check if we still have room for evacuations.
> 
> We don't KNOW promotion reserve is entirely exhausted, just that what remains in promotion reserve is less than requested bytes.

Fixed comment.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1593106661

From wkemper at openjdk.org  Tue May  7 21:40:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 7 May 2024 21:40:06 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v2]
In-Reply-To: <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
Message-ID: <SX_DjfpSJ8oQS7f3Lc0cIhlBrlIhS2O9xY_EsLl26YE=.743cbc7f-c062-4f9a-a8f2-c0c0f9820df9@github.com>

On Mon, 6 May 2024 20:43:33 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> How do we know this is NOT a shared allocation evacuation request?

We check if the request type is  `ShenandoahAllocRequest::_alloc_plab` on line 239.

> So what if req is not promotion? wouldn't we want to confirm that get_evacuation_reserve() > requested_bytes?

[The previous version of this code](https://github.com/openjdk/shenandoah/pull/427/files#diff-5ca0a05384b7b2604dd3c9b55d91a7010a42cd0ba246600c844e19821bd6b60bL1178) had a comment for this case (i.e., shared evacuation) that read:

// This is a shared allocation for evacuation. Memory has already been reserved for this purpose.

https://github.com/openjdk/shenandoah/pull/427/files#diff-5ca0a05384b7b2604dd3c9b55d91a7010a42cd0ba246600c844e19821bd6b60bL1178

I believe this change preserves the original behavior, but it makes sense to me to also enforce evacuation reserve constraints for `shared` evacuations.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1593131282

From wkemper at openjdk.org  Tue May  7 21:48:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 7 May 2024 21:48:45 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v3]
In-Reply-To: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
Message-ID: <tjIKWQJ1xkDnoD0cSCVRpxO19zk2ZbbzaT3MzHxWLXY=.17c2f2e1-cd9a-4f5f-ba6d-59c3ad4ff807@github.com>

> * The code for resizing and retrying young allocation requests according to what's available in the young generation has been removed. This code became redundant after the free set was rewritten and began providing the same functionality.
> * An unused field and its attendant methods were removed from `ShenandoahThreadLocalData`.
> * The code for managing `plab` expenditures and configuring other plab behavior has been moved from `ShenandoahHeap` into `ShenandoahOldGeneration`.

William Kemper has updated the pull request incrementally with three additional commits since the last revision:

 - Fix comments, add assert for clarity
 - Allocation only needs to register old, shared objects with card table
 - Fix missing includes

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/427/files
  - new: https://git.openjdk.org/shenandoah/pull/427/files/b2163411..a2c52db1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=427&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=427&range=01-02

  Stats: 33 lines in 4 files changed: 5 ins; 1 del; 27 mod
  Patch: https://git.openjdk.org/shenandoah/pull/427.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/427/head:pull/427

PR: https://git.openjdk.org/shenandoah/pull/427

From shade at openjdk.org  Wed May  8 07:58:51 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 8 May 2024 07:58:51 GMT
Subject: RFR: 8331721: Introduce CollectedHeap helper methods for any GC in
 progress
In-Reply-To: <t9DJ7x7V5GF4ePkaK4zTqlxEd99aHmbKOZbZzp2za-Y=.3c2f62dc-5de5-458b-8767-001702825a34@github.com>
References: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
 <LduSLH5Ivv84B965GN1tzWFFdTElhZzivPVVMdHwOkI=.5c8d695a-8595-4e75-bfa3-f3105f826952@github.com>
 <bVlVG1FZ7MhXEH-xpKb4M63iMoZ8BBjiNKxOV5Oq_t4=.8a1e9777-eb8d-4ae1-a36d-96abf1a8570d@github.com>
 <t9DJ7x7V5GF4ePkaK4zTqlxEd99aHmbKOZbZzp2za-Y=.3c2f62dc-5de5-458b-8767-001702825a34@github.com>
Message-ID: <HX7_IfyJJFJ8ecIn_D2-ChwROvJnNWufFLEZAXnjVyg=.d667d7b3-e98a-4d67-958c-5458796107be@github.com>

On Tue, 7 May 2024 20:30:44 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> But is it really necessary? IMO, some stack walk code can also benefit from `oopMapCache`, e.g. https://github.com/openjdk/jdk/blob/master/src/hotspot/share/prims/stackwalk.cpp#L345

Ah, hm. Looking closer, I don't quite see why we even want to gate `OopMapCache` with GC checks. I asked Coleen in the bug here: [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19122#issuecomment-2099976047

From wkemper at openjdk.org  Wed May  8 19:45:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 8 May 2024 19:45:37 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <5-le93HW4HQagzNRKym1V2a70dI7OVKrZh5NQdXKp0M=.952cdb7c-5d4b-4709-89bd-3aeb7d7b7b0d@github.com>
References: <5-le93HW4HQagzNRKym1V2a70dI7OVKrZh5NQdXKp0M=.952cdb7c-5d4b-4709-89bd-3aeb7d7b7b0d@github.com>
Message-ID: <1I0WREkSq3Ru_v9BRaPmDxlp3nMLqHwk50lQT3S8ZeU=.9200390e-78ac-45e3-83bb-a39f48efbdba@github.com>

> Merges tag jdk-21.0.4+1

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/42/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/42/files/3ab3f52b..3ab3f52b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=42&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=42&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/42.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/42/head:pull/42

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/42

From wkemper at openjdk.org  Wed May  8 19:45:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 8 May 2024 19:45:37 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <5-le93HW4HQagzNRKym1V2a70dI7OVKrZh5NQdXKp0M=.952cdb7c-5d4b-4709-89bd-3aeb7d7b7b0d@github.com>
References: <5-le93HW4HQagzNRKym1V2a70dI7OVKrZh5NQdXKp0M=.952cdb7c-5d4b-4709-89bd-3aeb7d7b7b0d@github.com>
Message-ID: <w1yOAvbEAuEKVBS_2Ooc1CvP3L_16tOiX49aBvIBcZc=.24423cc8-21e3-419d-b6d5-99c4e9c7d996@github.com>

On Thu, 2 May 2024 14:15:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.4+1

This pull request has now been integrated.

Changeset: 5bfceed0
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/5bfceed03a3e9f5dcb3176f3111d8278450c50c0
Stats:     11670 lines in 476 files changed: 6764 ins; 3059 del; 1847 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/42

From duke at openjdk.org  Wed May  8 22:28:05 2024
From: duke at openjdk.org (duke)
Date: Wed, 8 May 2024 22:28:05 GMT
Subject: Withdrawn: 8323634: Shenandoah: Document behavior of EvacOOM protocol
In-Reply-To: <6ciSyKdz9hA6RBOZeDicFetK_G4AUBpx40YX7yT1O1M=.870e1ba1-6f4b-48e9-8360-dab141a3041d@github.com>
References: <6ciSyKdz9hA6RBOZeDicFetK_G4AUBpx40YX7yT1O1M=.870e1ba1-6f4b-48e9-8360-dab141a3041d@github.com>
Message-ID: <1yzh8mTTC8e14_btIwc4AD2rlgUYdqwu3EkBjUqN8Q4=.b641ccf7-539d-49dd-8e1b-6a601eebdfef@github.com>

On Fri, 12 Jan 2024 01:14:24 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> The protocol for handling OOM during evacuation is subtle and critical for correct operation.  This PR does NOT change behavior.  It provides improved documentation of existing behavior.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/17385

From wkemper at openjdk.org  Wed May  8 23:23:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 8 May 2024 23:23:28 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
Message-ID: <BgkhvBEZJ7HqSboU62mvUE07NfLNdK138pi1Declmzc=.3c686d63-6306-40c6-9bd2-f8fb42689aac@github.com>

> * The code for resizing and retrying young allocation requests according to what's available in the young generation has been removed. This code became redundant after the free set was rewritten and began providing the same functionality.
> * An unused field and its attendant methods were removed from `ShenandoahThreadLocalData`.
> * The code for managing `plab` expenditures and configuring other plab behavior has been moved from `ShenandoahHeap` into `ShenandoahOldGeneration`.

William Kemper has updated the pull request incrementally with three additional commits since the last revision:

 - Improve comments, remove too strong assert
 - Initialize _evacuation_reserve member
 - Modify test fixture to avoid accessing ShenandoahHeap before we know if UseShenandoahGC is set

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/427/files
  - new: https://git.openjdk.org/shenandoah/pull/427/files/a2c52db1..4320206b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=427&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=427&range=02-03

  Stats: 81 lines in 3 files changed: 23 ins; 23 del; 35 mod
  Patch: https://git.openjdk.org/shenandoah/pull/427.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/427/head:pull/427

PR: https://git.openjdk.org/shenandoah/pull/427

From wkemper at openjdk.org  Wed May  8 23:39:22 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 8 May 2024 23:39:22 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <SX_DjfpSJ8oQS7f3Lc0cIhlBrlIhS2O9xY_EsLl26YE=.743cbc7f-c062-4f9a-a8f2-c0c0f9820df9@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
 <SX_DjfpSJ8oQS7f3Lc0cIhlBrlIhS2O9xY_EsLl26YE=.743cbc7f-c062-4f9a-a8f2-c0c0f9820df9@github.com>
Message-ID: <Z6XB8WJh-Aoigr3WolEiWd_DcsyFz--lFANc9s01Ux0=.a48e4051-d776-4497-a1e9-1be7a0d34339@github.com>

On Tue, 7 May 2024 21:36:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 243:
>> 
>>> 241:   }
>>> 242: 
>>> 243:   // This is a shared allocation promotion request. However, we do not have room for any
>> 
>> How do we know this is NOT a shared allocation evacuation request?  Is there a precondition on the method call?
>> 
>> Implementation below looks we're not sure this is a promotion.  We still need to check.
>> 
>> I think what we're saying here is:
>>   if req.is_promotion();   return false because we already checked above, and found that can_promote(requested_bytes) to be false.
>>   
>>   So what if req is not promotion?  wouldn't we want to confirm that get_evacuation_reserve() > requested_bytes?
>
>> How do we know this is NOT a shared allocation evacuation request?
> 
> We check if the request type is  `ShenandoahAllocRequest::_alloc_plab` on line 239.
> 
>> So what if req is not promotion? wouldn't we want to confirm that get_evacuation_reserve() > requested_bytes?
> 
> [The previous version of this code](https://github.com/openjdk/shenandoah/pull/427/files#diff-5ca0a05384b7b2604dd3c9b55d91a7010a42cd0ba246600c844e19821bd6b60bL1178) had a comment for this case (i.e., shared evacuation) that read:
> 
> // This is a shared allocation for evacuation. Memory has already been reserved for this purpose.
> 
> https://github.com/openjdk/shenandoah/pull/427/files#diff-5ca0a05384b7b2604dd3c9b55d91a7010a42cd0ba246600c844e19821bd6b60bL1178
> 
> I believe this change preserves the original behavior, but it makes sense to me to also enforce evacuation reserve constraints for `shared` evacuations.

I've fixed these comments.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1594808624

From ysr at openjdk.org  Thu May  9 01:53:11 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 9 May 2024 01:53:11 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
Message-ID: <dXsIR74awoU102ZQegywyQ68n1Uk_7QIdJbIQRBKrCI=.c7521f7e-2be3-42fb-9a89-86e0e802b90c@github.com>

On Tue, 7 May 2024 14:35:33 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Another attempt at improving clarity of comments

I'll re-review the changes since my last review which predates the new bitmap based set implementation. Will try and get that done ASAP in the next day or two. Sorry for the delay.

-------------

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-2047016684

From duke at openjdk.org  Thu May  9 03:29:58 2024
From: duke at openjdk.org (duke)
Date: Thu, 9 May 2024 03:29:58 GMT
Subject: Withdrawn: 8327522: Shenandoah: Remove unused references to
 satb_mark_queue_active_offset
In-Reply-To: <FHAGyiw9p9dB_4zEe4Wnd9LlJMK3xwqPhvEnWPiJyhU=.ebf25823-d198-4d87-8230-b52ab6919748@github.com>
References: <FHAGyiw9p9dB_4zEe4Wnd9LlJMK3xwqPhvEnWPiJyhU=.ebf25823-d198-4d87-8230-b52ab6919748@github.com>
Message-ID: <-gNBJE3fcagLhqWF86kgPgMyvbcoPNpAfRYLYvTd9WQ=.4f1c59de-add5-47f0-b5e9-517fd59e6da0@github.com>

On Thu, 7 Mar 2024 08:00:46 GMT, Yude Lin <duke at openjdk.org> wrote:

> Removed an unused variable (trivial)
> 
> Also, there is another place that uses satb_mark_queue_active_offset which is ShenandoahBarrierSetC2::verify_gc_barriers
> The current barrier pattern is different from what this code is expecting:
> If->Bool->CmpI->AndI->LoadUB->AddP->ConL(gc_state_offset)
> rather than
> If->Bool->CmpI->LoadB->AddP->ConL(marking_offset)
> However, this code isn't doing as much checking as its counterpart in G1 anyway (so I'm thinking removing the incorrect matching code altogether?) Looking forward to your suggestions.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/18148

From wkemper at openjdk.org  Thu May  9 14:21:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 9 May 2024 14:21:40 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <Royn_2WG7ByIGOjOVabugyyeG3T8MKtYo96aIuTy8qk=.b3f17e9c-60c8-464d-9ab3-de990eaab87d@github.com>

Merges tag jdk-21.0.4+2

-------------

Commit messages:
 - 8314573: G1: Heap resizing at Remark does not take existing eden regions into account
 - 8319376: ParallelGC: Forwarded objects found during heap inspection
 - 8310913: Move ReferencedKeyMap to jdk.internal so it may be shared
 - 8317007: Add bulk removal of dead nmethods during class unloading
 - 8310228: Improve error reporting for uncaught native exceptions on Windows
 - 8309890: TestStringDeduplicationInterned.java waits for the wrong condition
 - 8331639: [21u]: Bump GHA bootstrap JDK to 21.0.3
 - 8328938: C2 SuperWord: disable vectorization for large stride and scale
 - 8310513: [s390x] Intrinsify recursive ObjectMonitor locking
 - 8330011: [s390x] update block-comments to make code consistent
 - ... and 9 more: https://git.openjdk.org/shenandoah-jdk21u/compare/3ab3f52b...93d091ad

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/43/files
  Stats: 2980 lines in 80 files changed: 1993 ins; 795 del; 192 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/43.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/43/head:pull/43

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/43

From wkemper at openjdk.org  Thu May  9 16:30:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 9 May 2024 16:30:13 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <6aJTFhR071whY5rfEAzfl0MMGY6AAss_dUWBnNKm56A=.4f14bac3-6316-42e3-b7f0-993a8c337de4@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
 <6aJTFhR071whY5rfEAzfl0MMGY6AAss_dUWBnNKm56A=.4f14bac3-6316-42e3-b7f0-993a8c337de4@github.com>
Message-ID: <fXJEkwO9-SveZXaNlwzPSqC2rs85NGi2nc554Nsc6H4=.6a3f91b5-f896-453f-a2bb-cae10c9806f1@github.com>

On Tue, 7 May 2024 21:13:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1129:
>> 
>>> 1127:   if (result != nullptr && req.is_old()) {
>>> 1128:     old_generation()->configure_plab_for_current_thread(req);
>>> 1129:     // TODO: Is this only necessary for 'shared' allocations?
>> 
>> I believe this is only necessary for shared allocs.  I'm supportive of making this change as part of PR, though it might be safer to do in separate PR.
>> 
>> (If this is a PLAB allocation, the objects allocated within the PLAB will be independently registered when they are allocated.)
>
> I'm going to test making this change as part of this PR.

Pipelines look okay with only registering shared objects here.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1595688231

From kdnilsen at openjdk.org  Thu May  9 16:38:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 9 May 2024 16:38:12 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <BgkhvBEZJ7HqSboU62mvUE07NfLNdK138pi1Declmzc=.3c686d63-6306-40c6-9bd2-f8fb42689aac@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <BgkhvBEZJ7HqSboU62mvUE07NfLNdK138pi1Declmzc=.3c686d63-6306-40c6-9bd2-f8fb42689aac@github.com>
Message-ID: <Thh8lzfg6W7J0cjzAOU8LKuRuldbNfPL6_DzCdffG6w=.24eb94bf-93cb-4112-b7ed-fa37638de8c4@github.com>

On Wed, 8 May 2024 23:23:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> * The code for resizing and retrying young allocation requests according to what's available in the young generation has been removed. This code became redundant after the free set was rewritten and began providing the same functionality.
>> * An unused field and its attendant methods were removed from `ShenandoahThreadLocalData`.
>> * The code for managing `plab` expenditures and configuring other plab behavior has been moved from `ShenandoahHeap` into `ShenandoahOldGeneration`.
>
> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Improve comments, remove too strong assert
>  - Initialize _evacuation_reserve member
>  - Modify test fixture to avoid accessing ShenandoahHeap before we know if UseShenandoahGC is set

Looks good.  Thanks.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/427#pullrequestreview-2048367748

From wkemper at openjdk.org  Thu May  9 23:52:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 9 May 2024 23:52:33 GMT
Subject: Integrated: 8331609: GenShen: Refactor generational mode allocations
In-Reply-To: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
Message-ID: <2bqMgmQY8c4s5nGvwZiLC2FCw2ar7foVMZ1OinNP_8Q=.6502a02c-c686-4464-8eeb-25f8e07d8158@github.com>

On Thu, 2 May 2024 23:21:46 GMT, William Kemper <wkemper at openjdk.org> wrote:

> * The code for resizing and retrying young allocation requests according to what's available in the young generation has been removed. This code became redundant after the free set was rewritten and began providing the same functionality.
> * An unused field and its attendant methods were removed from `ShenandoahThreadLocalData`.
> * The code for managing `plab` expenditures and configuring other plab behavior has been moved from `ShenandoahHeap` into `ShenandoahOldGeneration`.

This pull request has now been integrated.

Changeset: 80e69877
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/80e69877f807499e811ff0353d7477057fa2f63d
Stats:     503 lines in 8 files changed: 272 ins; 175 del; 56 mod

8331609: GenShen: Refactor generational mode allocations

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/427

From ysr at openjdk.org  Fri May 10 00:31:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 10 May 2024 00:31:17 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <BgkhvBEZJ7HqSboU62mvUE07NfLNdK138pi1Declmzc=.3c686d63-6306-40c6-9bd2-f8fb42689aac@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <BgkhvBEZJ7HqSboU62mvUE07NfLNdK138pi1Declmzc=.3c686d63-6306-40c6-9bd2-f8fb42689aac@github.com>
Message-ID: <w_ej9YQzyXNYXJV3QE9pf6ZxEQRNYMWRBNmCOJw1XOA=.4dde962f-1337-418c-ada9-0f594d578c62@github.com>

On Wed, 8 May 2024 23:23:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> * The code for resizing and retrying young allocation requests according to what's available in the young generation has been removed. This code became redundant after the free set was rewritten and began providing the same functionality.
>> * An unused field and its attendant methods were removed from `ShenandoahThreadLocalData`.
>> * The code for managing `plab` expenditures and configuring other plab behavior has been moved from `ShenandoahHeap` into `ShenandoahOldGeneration`.
>
> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Improve comments, remove too strong assert
>  - Initialize _evacuation_reserve member
>  - Modify test fixture to avoid accessing ShenandoahHeap before we know if UseShenandoahGC is set

A few minor comments; overall looks good, but is still a bit difficult to reason about because of having to carry several case analyses simultaneously in these allocation paths.

Was there any performance difference measured in any of the code pipeline performance tests on account of avoiding some of the redundant tests that you removed?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1117:

> 1115:   ShenandoahHeapLocker locker(lock(), req.is_mutator_alloc());
> 1116: 
> 1117:   // Make sure the old generation has room for either evacuations or promotions before trying to allocate.

I'd rephrase this more directly:


// Don't try allocation if there's no room, or if promotion budget exhausted.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1123:

> 1121: 
> 1122:   // If TLAB request size is greater than available, allocate() will attempt to downsize request to fit within available
> 1123:   // memory.

I am confused. Why do we need this comment here? Could this not be a non-TLAB allocation request? Why are we specifically talking about TLABs here?

If we believe this can only be a TLAB, we should assert that `req.type() == _alloc_tlab` ?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1147:

> 1145:       // The thread allocating b and the thread allocating c can "race" in various ways, resulting in confusion, such as
> 1146:       // last-start representing object b while first-start represents object c.  This is why we need to require all
> 1147:       // register_object() invocations to be "mutually exclusive" with respect to each card's memory range.

One can simply and more succinctly state:


  // Register with object start array while holding heap lock to protect against
  // update races between threads allocating to the same card.

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 247:

> 245:   // This is a shared allocation request. We've already checked that it can't be promoted, so if
> 246:   // it is a promotion, we return false. Otherwise, it is a shared evacuation request, and we allow
> 247:   // the allocation to proceed.

I'd rephrase to:
// We already dealt with PLAB allocations above. If it isn't a promotion, we'll
// allow the allocation attempt, because it must be a shared allocation.

Can we assert here that this is `_is_promotion` or else `_alloc_shared` or `_alloc_shared_gc` just to be sure we catch any changes to this or calling code in the future?

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 271:

> 269:         ShenandoahThreadLocalData::set_plab_actual_size(thread, actual_size);
> 270:       } else {
> 271:         // Disable promotions in this thread because entirety of this PLAB must be available to hold old-gen evacuations.

I would rephrase, removing the "because" :


  // Disable promotions in this thread's PLAB. The entirety of the PLAB
  // will be used for old gen evacuations.


I must say I find the flow of allocation logic in this and earlier code difficult to clearly understand and reason about, because a lot of allocation paths are aggregated into common code, so one needs to continually carry a lot of case analyses through the control flow.

Addition of assertions (as stated control point invariants) might help readability and comprehension, if refactoring for clarity isn't a good idea. (But any of this for the future.)

src/hotspot/share/gc/shenandoah/shenandoahThreadLocalData.hpp line 64:

> 62:   PLAB* _plab;
> 63: 
> 64:   // Heuristics will grow the desired size of plabs.

~"grow" or "adjust" (i.e. are downwards adjustments also done?)~ Scratch that, I see that it only grows starting at initial size.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/427#pullrequestreview-2048797700
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596072178
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596074282
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596121597
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596068015
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596109882
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596023955

From ysr at openjdk.org  Fri May 10 00:31:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 10 May 2024 00:31:17 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <w_ej9YQzyXNYXJV3QE9pf6ZxEQRNYMWRBNmCOJw1XOA=.4dde962f-1337-418c-ada9-0f594d578c62@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <BgkhvBEZJ7HqSboU62mvUE07NfLNdK138pi1Declmzc=.3c686d63-6306-40c6-9bd2-f8fb42689aac@github.com>
 <w_ej9YQzyXNYXJV3QE9pf6ZxEQRNYMWRBNmCOJw1XOA=.4dde962f-1337-418c-ada9-0f594d578c62@github.com>
Message-ID: <XM2esjXuz34-Fzlv4AqiZBQU_1SnSt3C7idJtj7mHog=.3f9a4bad-1d72-4a08-96cb-c5a2b7e1ce8e@github.com>

On Thu, 9 May 2024 23:01:14 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Improve comments, remove too strong assert
>>  - Initialize _evacuation_reserve member
>>  - Modify test fixture to avoid accessing ShenandoahHeap before we know if UseShenandoahGC is set
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1123:
> 
>> 1121: 
>> 1122:   // If TLAB request size is greater than available, allocate() will attempt to downsize request to fit within available
>> 1123:   // memory.
> 
> I am confused. Why do we need this comment here? Could this not be a non-TLAB allocation request? Why are we specifically talking about TLABs here?
> 
> If we believe this can only be a TLAB, we should assert that `req.type() == _alloc_tlab` ?

Indeed, we see below at line 1129 that this could be a "shared" allocation of a single object(?) in which case, the comment above singling out TLAB allocations appears confusing and misplaced here.

> src/hotspot/share/gc/shenandoah/shenandoahThreadLocalData.hpp line 64:
> 
>> 62:   PLAB* _plab;
>> 63: 
>> 64:   // Heuristics will grow the desired size of plabs.
> 
> ~"grow" or "adjust" (i.e. are downwards adjustments also done?)~ Scratch that, I see that it only grows starting at initial size.

Thank you for the good documentation below!

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596091811
PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596024871

From ysr at openjdk.org  Fri May 10 00:31:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 10 May 2024 00:31:17 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <6aJTFhR071whY5rfEAzfl0MMGY6AAss_dUWBnNKm56A=.4f14bac3-6316-42e3-b7f0-993a8c337de4@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
 <6aJTFhR071whY5rfEAzfl0MMGY6AAss_dUWBnNKm56A=.4f14bac3-6316-42e3-b7f0-993a8c337de4@github.com>
Message-ID: <RO4KSCyaqwm4LSVpwmVMVJULlcL8BXtJyxXfw08Fuuw=.c66ffbd5-7c4f-4265-b0f1-9e0fcd5842d8@github.com>

On Tue, 7 May 2024 21:11:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 229:
>> 
>>> 227: 
>>> 228:   const size_t requested_bytes = req.size() * HeapWordSize;
>>> 229:   if (can_promote(requested_bytes)) {
>> 
>> Even though we don't check whether this req.is_promotion(), we know:
>>   if there's room to promote requested_bytes, there is also room to evacuate requested_bytes
>> 
>> A comment could be helpful here.
>
> Done

Why not call it `bool can_allocate(size_t bytes)` and expend with the need for the comment to resolve the reader's confusion from the naming?


  // Check if there is space available to accommodate the allocation.
  if (can_allocate(requested_bytes)) {
    // the allocation might still fail because of alignment constraints
    // or because of fragmentation of available space.
    return true;
  }

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596060717

From ysr at openjdk.org  Fri May 10 00:31:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 10 May 2024 00:31:17 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <RO4KSCyaqwm4LSVpwmVMVJULlcL8BXtJyxXfw08Fuuw=.c66ffbd5-7c4f-4265-b0f1-9e0fcd5842d8@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
 <6aJTFhR071whY5rfEAzfl0MMGY6AAss_dUWBnNKm56A=.4f14bac3-6316-42e3-b7f0-993a8c337de4@github.com>
 <RO4KSCyaqwm4LSVpwmVMVJULlcL8BXtJyxXfw08Fuuw=.c66ffbd5-7c4f-4265-b0f1-9e0fcd5842d8@github.com>
Message-ID: <_cSkGdbZ-0Av1-3gWZEPSGZHibuFbdYj5nCK_hwU2Mw=.2dc8b471-cb77-4282-9cbf-7ec075d3b1b2@github.com>

On Thu, 9 May 2024 22:34:33 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Done
>
> Why not call it `bool can_allocate(size_t bytes)` and expend with the need for the comment to resolve the reader's confusion from the naming?
> 
> 
>   // Check if there is space available to accommodate the allocation.
>   if (can_allocate(requested_bytes)) {
>     // the allocation might still fail because of alignment constraints
>     // or because of fragmentation of available space.
>     return true;
>   }

I'd similarly adjust subsequent comments to not mention promotion specifically, but just call it `allocation_reserve` and `allocation`, instead of `promotion_reserve` and `promotion`.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596110578

From ysr at openjdk.org  Fri May 10 00:31:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 10 May 2024 00:31:17 GMT
Subject: RFR: 8331609: GenShen: Refactor generational mode allocations [v4]
In-Reply-To: <_cSkGdbZ-0Av1-3gWZEPSGZHibuFbdYj5nCK_hwU2Mw=.2dc8b471-cb77-4282-9cbf-7ec075d3b1b2@github.com>
References: <0KT8yOQkkep0zzF7eI6kSijNm4F-BXLZG9p1lI_xqJk=.2d834ba0-cf4d-49b8-9cc5-b27cb084e5fa@github.com>
 <9UFxAzv0sHHa22BBi2Nzv68e63Dql5yjO6wO5Y2FgUg=.5cf8e78a-aa27-4829-824c-fbac2833a623@github.com>
 <6aJTFhR071whY5rfEAzfl0MMGY6AAss_dUWBnNKm56A=.4f14bac3-6316-42e3-b7f0-993a8c337de4@github.com>
 <RO4KSCyaqwm4LSVpwmVMVJULlcL8BXtJyxXfw08Fuuw=.c66ffbd5-7c4f-4265-b0f1-9e0fcd5842d8@github.com>
 <_cSkGdbZ-0Av1-3gWZEPSGZHibuFbdYj5nCK_hwU2Mw=.2dc8b471-cb77-4282-9cbf-7ec075d3b1b2@github.com>
Message-ID: <y8m2XLIlGnB8A-lHN7X83Tn-QzHvn8u3-0S1Z1PA8y0=.58cd7113-36c8-4138-bc61-026215b9ea84@github.com>

On Thu, 9 May 2024 23:58:02 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Why not call it `bool can_allocate(size_t bytes)` and expend with the need for the comment to resolve the reader's confusion from the naming?
>> 
>> 
>>   // Check if there is space available to accommodate the allocation.
>>   if (can_allocate(requested_bytes)) {
>>     // the allocation might still fail because of alignment constraints
>>     // or because of fragmentation of available space.
>>     return true;
>>   }
>
> I'd similarly adjust subsequent comments to not mention promotion specifically, but just call it `allocation_reserve` and `allocation`, instead of `promotion_reserve` and `promotion`.

Upon reading further and understanding how this is working, scratch the above suggestions.

We should leave things as they are here in terms of using `promotion` in the naming of the methods and the fields.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/427#discussion_r1596111449

From stuefe at openjdk.org  Fri May 10 10:21:49 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 10 May 2024 10:21:49 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file
Message-ID: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>

MEMFLAGS, as well as its enum constants, should live in its own include. 

The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.

The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.

---

This patch moves the enum to its new file.

It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 

For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.

I tested (built) on:
- MacOS aarch64, no precompiled headers, fastdebug
- Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal

-------------

Commit messages:
 - Update g1MonotonicArena.hpp
 - Update g1MonotonicArena.hpp
 - NMT-factor-out-memflags

Changes: https://git.openjdk.org/jdk/pull/19172/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19172&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332042
  Stats: 225 lines in 25 files changed: 124 ins; 64 del; 37 mod
  Patch: https://git.openjdk.org/jdk/pull/19172.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19172/head:pull/19172

PR: https://git.openjdk.org/jdk/pull/19172

From stuefe at openjdk.org  Fri May 10 10:27:10 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 10 May 2024 10:27:10 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file
In-Reply-To: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
Message-ID: <CyeeZHHV6ibQslugZiv2ZlBtMNUemV7GbnUsE1haP2s=.27f30903-50a1-41a3-88bb-0471d69eb848@github.com>

On Fri, 10 May 2024 09:06:08 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> MEMFLAGS, as well as its enum constants, should live in its own include. 
> 
> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
> 
> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
> 
> ---
> 
> This patch moves the enum to its new file.
> 
> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
> 
> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
> 
> I tested (built) on:
> - MacOS aarch64, no precompiled headers, fastdebug
> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal

Ping @afshin-zafari @jdksjolen 
@gerard-ziemski

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19172#issuecomment-2104357678

From wkemper at openjdk.org  Fri May 10 14:14:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 10 May 2024 14:14:59 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <11_S1jA9IwlqgfkvNIozaDocP0UwG14v-ci8B-joGBo=.d399d691-a003-4a39-8148-272f828cc682@github.com>

Merges tag jdk-23+22

-------------

Commit messages:
 - 8322753: RISC-V: C2 ReverseBytesV
 - 8331541: [i386] linking with libjvm.so fails after JDK-8283326
 - 8331886: Allow markdown src file overrides
 - 8331939: Add custom hook for TestImage
 - 8330275: Crash in XMark::follow_array
 - 8331941: Make CollectedHeap::parallel_object_iterator public
 - 8331924: Parallel: Remove unused MutableSpace::mangle_unused_area_complete
 - 8331185: Enable compiler memory limits in debug builds
 - 8331208: Memory stress test that checks OutOfMemoryError stack trace fails
 - 8308033: The jcmd thread dump related tests should test virtual threads
 - ... and 165 more: https://git.openjdk.org/shenandoah/compare/87e864bf...964d6089

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/429/files
  Stats: 24376 lines in 1729 files changed: 9641 ins; 8882 del; 5853 mod
  Patch: https://git.openjdk.org/shenandoah/pull/429.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/429/head:pull/429

PR: https://git.openjdk.org/shenandoah/pull/429

From wkemper at openjdk.org  Fri May 10 16:01:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 10 May 2024 16:01:06 GMT
Subject: RFR: 8332081: GenShen: Disentangle generational mode update references
Message-ID: <4BeFZ8yhpfr2vYp0_rtPL7KE7zPYFMvoawNPhlzaACE=.856369bb-75e0-4bde-bd42-13ad51271ff8@github.com>

I believe the test failure on macosx is unrelated to these changes.

-------------

Commit messages:
 - Remove single threaded usages of heap region iterator
 - Fix asserts to handle case when gc is cancelled
 - Move generational version of update refs to generational heap

Changes: https://git.openjdk.org/shenandoah/pull/430/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=430&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332081
  Stats: 438 lines in 6 files changed: 236 ins; 185 del; 17 mod
  Patch: https://git.openjdk.org/shenandoah/pull/430.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/430/head:pull/430

PR: https://git.openjdk.org/shenandoah/pull/430

From rkennke at openjdk.org  Fri May 10 16:20:23 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 10 May 2024 16:20:23 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
Message-ID: <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>

On Tue, 7 May 2024 14:35:33 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Another attempt at improving clarity of comments

I think it's mostly there. Just a few remainging minor nits.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 100:

> 98: }
> 99: 
> 100: ShenandoahRegionPartitions::~ShenandoahRegionPartitions() {

I think there's no need to put this trivial destructor definition in the .cpp file. Just place it with the declaration in the .hpp file.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 144:

> 142:   assert (which_partition < NumPartitions, "selected free partition must be valid");
> 143:   idx_t idx = _rightmosts[int(which_partition)];
> 144:   // _membership[which_partition].is_set(idx) may not be true if we are shrinking the interval

Same here.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 334:

> 332: const char* ShenandoahRegionPartitions::partition_membership_name(idx_t idx) const {
> 333:   ShenandoahFreeSetPartitionId result = membership(idx);
> 334:   return partition_name(result);

Could make it even more succinct by doing `return partition_name(membership(idx));`

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 2:

> 1: /*
> 2:  * Copyright (c) 2016, 2021, Red Hat, Inc. All rights reserved.

The copyright seems wrong. This is entirely new code by you, right? Should only have Amazon copyright notice, then.

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.hpp line 3:

> 1: /*
> 2:  * Copyright (c) 2016, 2019, Red Hat, Inc. All rights reserved.
> 3:  * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.

Same copyright issue.

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.hpp line 92:

> 90:   }
> 91: 
> 92:   inline idx_t alignment() const {

I guess this could even be constexpr.

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.inline.hpp line 2:

> 1: /*
> 2:  * Copyright (c) 2016, 2019, Red Hat, Inc. All rights reserved.

Same copyright issue.

-------------

Changes requested by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-2049787865
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596812726
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596814322
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596840631
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596860599
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596950049
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596949379
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596950394

From rkennke at openjdk.org  Fri May 10 16:20:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 10 May 2024 16:20:24 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v39]
In-Reply-To: <2t0urUU2aUpG3wa9oNtxeaz35ijBJ03tcQ31oWNY1Bc=.6fcc2de4-4af2-4b36-91fc-f7cbea0e914d@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
 <2t0urUU2aUpG3wa9oNtxeaz35ijBJ03tcQ31oWNY1Bc=.6fcc2de4-4af2-4b36-91fc-f7cbea0e914d@github.com>
Message-ID: <1Qe1KGeCTaJ1BZ7I5QbQFBB5IanD6uZzEHxz64iuvQU=.3dc54af9-0453-4a3a-b879-9744129a1e23@github.com>

On Tue, 16 Apr 2024 00:19:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 42:
>> 
>>> 40:     case Mutator: return "Mutator";
>>> 41:     case Collector: return "Collector";
>>> 42:     default: return "Unrecognized";
>> 
>> I believe using an 'enum class' for ShenandoahFreeSetPartitionId means you don't need to have a default.
>
> Compiler still wants the default clause.  Otherwise, "control reaches end of non-void function".  You'd think the compiler could be less strict, but will leave as is, unless we can figure out a way to get around this.

Hmm, ok. So maybe at least put a ShouldNotReachHere(); before the return?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596574706

From rkennke at openjdk.org  Fri May 10 16:20:30 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 10 May 2024 16:20:30 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v14]
In-Reply-To: <PVrVW-I2bKkYzrdBafHAaWF8mw6WtANB3PoOZW5ekRw=.2ab54392-0f5b-4f6f-b5d7-7b7a9b6a0a97@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <DtAJtZcZPNv2FMbF9-FeJIT6T8Oio1TW9yYnw3cceF0=.ed94d2d3-39c2-4f8d-9e00-5dbd81683c93@github.com>
 <PVrVW-I2bKkYzrdBafHAaWF8mw6WtANB3PoOZW5ekRw=.2ab54392-0f5b-4f6f-b5d7-7b7a9b6a0a97@github.com>
Message-ID: <ltMhY13jQ2wsip80DkaQgQsx8hdMp6sD6eAHsc_axpE=.902463b5-004c-435f-bc13-68d3daccd06c@github.com>

On Fri, 1 Mar 2024 17:39:15 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 60 commits:
>> 
>>  - Remove instrumentation and cleanup magic numbers
>>    
>>    Two places, I had used 63 when I should have used
>>    _bits_per_array_element -1.
>>  - Address 32-bit compile issues
>>  - Fix 32-bit size formatting in log messages
>>  - Fix white space
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Two bug fixes for better performance
>>    
>>    1. Collector reserve size is based on memory available within regions
>>       rather than the region size (oops).
>>    
>>    2. If an attempt to allocate within a region fails and the region has
>>       already provided the percentage presumed by ShenandoahEvacWaste,
>>       retire this region.  This is motivated by observations that
>>       otherwise, we end up with large numbers of regions that have only
>>       a small amount of memory within them (e.g. 4k) and every allocation
>>       request has to wade through all of these regions before it eventually
>>       finds a region that has a sufficiently large amount of available
>>       memory.  In the original Shenandoah free-set implementation, the
>>       behavior was to retire the first time an allocation within that
>>       regions fails, regardless of whether the region has already reached
>>       the ShenandoahEvacWaste threshold.
>>  - Fix off-by-one error in is_forward_consecutive_ones()
>>  - Fix whitespace
>>  - Bug fixes and performance improvements
>>    
>>    1. Correct off-b-one-error in count of trailingones
>>    2. Speed up search for contiguous regions (for humongous allocations) by
>>       sliding window instead of initiating new search each time
>>    3. Bias regular region allocations to favor regions that are already
>>       partially consumed
>>    4. Fix bug in move_regions_from_collector_to_mutator which caused some
>>       non-empty regions to be ignored.
>>  - ... and 50 more: https://git.openjdk.org/jdk/compare/be2b92bd...1aa5a3e6
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 319:
> 
>> 317: #else
>> 318:   log_info(gc)("%6s: %10s %10s %10s", "index", "Mutator Bits", "Collector Bits", "NotFree Bits");
>> 319: #endif
> 
> There is no need for `_LP64` here, just print with `%18s` unconditionally.

What Aleksey said.

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 394:
> 
>> 392:     return _max;
>> 393:   } else {
>> 394:     // _membership[which_partition].is_set(idx) may not be true if we are shrinking the interval
> 
> ...and? This comment is really confusing. What does it have to do with `leftmost`?

What Aleksey said. ;-)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596809257
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1596814014

From wkemper at openjdk.org  Fri May 10 16:20:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 10 May 2024 16:20:34 GMT
Subject: RFR: 8332082: Shenandoah: Use SATB active flag for C2 pre-write
 barrier on x86 and PPC
Message-ID: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>

This is consistent with c1 and other platforms.

-------------

Commit messages:
 - Check for satb active flag, rather than gc state

Changes: https://git.openjdk.org/jdk/pull/19180/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19180&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332082
  Stats: 15 lines in 2 files changed: 9 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/19180.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19180/head:pull/19180

PR: https://git.openjdk.org/jdk/pull/19180

From kdnilsen at openjdk.org  Fri May 10 16:55:03 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 10 May 2024 16:55:03 GMT
Subject: RFR: 8332082: Shenandoah: Use SATB active flag for C2 pre-write
 barrier on x86 and PPC
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <vbGWzrCylp8Vc8TI-Es61jvAO0P3Nilefn6LK3Pz3ds=.21644443-4db3-4ade-a139-fb148cc5ce43@github.com>

On Fri, 10 May 2024 16:13:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is consistent with c1 and other platforms.

Marked as reviewed by kdnilsen (no project role).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19180#pullrequestreview-2050483938

From kdnilsen at openjdk.org  Fri May 10 17:57:22 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 10 May 2024 17:57:22 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
 <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
Message-ID: <iw4dViFLUsCidYS5tltDOZCmPLUxLA6mG0G84UVeYG4=.8dabd5b2-0144-4ee5-9833-97d8280e34cd@github.com>

On Fri, 10 May 2024 14:53:13 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Another attempt at improving clarity of comments
>
> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 2:
> 
>> 1: /*
>> 2:  * Copyright (c) 2016, 2021, Red Hat, Inc. All rights reserved.
> 
> The copyright seems wrong. This is entirely new code by you, right? Should only have Amazon copyright notice, then.

Thanks for review.  I'll work through these suggestions.  Question about copyright: I guess this is pretty much "all new", but the code was derived from original code, and much of the public API was borrowed from original.  So I think maybe I should keep Red Hat copyright, and add Amazon copyright.  What do you advise?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597050924

From rkennke at openjdk.org  Fri May 10 18:00:20 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 10 May 2024 18:00:20 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <iw4dViFLUsCidYS5tltDOZCmPLUxLA6mG0G84UVeYG4=.8dabd5b2-0144-4ee5-9833-97d8280e34cd@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
 <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
 <iw4dViFLUsCidYS5tltDOZCmPLUxLA6mG0G84UVeYG4=.8dabd5b2-0144-4ee5-9833-97d8280e34cd@github.com>
Message-ID: <lr3DuV2FfDGvMtYb-tVg517wM_SXs7yBeS-stfPgOGI=.612a7f92-3854-4339-ab7b-adad951f00b7@github.com>

On Fri, 10 May 2024 17:54:20 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 2:
>> 
>>> 1: /*
>>> 2:  * Copyright (c) 2016, 2021, Red Hat, Inc. All rights reserved.
>> 
>> The copyright seems wrong. This is entirely new code by you, right? Should only have Amazon copyright notice, then.
>
> Thanks for review.  I'll work through these suggestions.  Question about copyright: I guess this is pretty much "all new", but the code was derived from original code, and much of the public API was borrowed from original.  So I think maybe I should keep Red Hat copyright, and add Amazon copyright.  What do you advise?

But has it been derived from Red Hat code? My understanding was that it's basically a fork of src/hotspot/share/utilities/bitMap.* ? Which is Oracle-copyrighted. Is that correct?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597055486

From kdnilsen at openjdk.org  Fri May 10 18:04:23 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 10 May 2024 18:04:23 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <lr3DuV2FfDGvMtYb-tVg517wM_SXs7yBeS-stfPgOGI=.612a7f92-3854-4339-ab7b-adad951f00b7@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
 <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
 <iw4dViFLUsCidYS5tltDOZCmPLUxLA6mG0G84UVeYG4=.8dabd5b2-0144-4ee5-9833-97d8280e34cd@github.com>
 <lr3DuV2FfDGvMtYb-tVg517wM_SXs7yBeS-stfPgOGI=.612a7f92-3854-4339-ab7b-adad951f00b7@github.com>
Message-ID: <MYGdv5dYKdwsPW8hr_W-6IztQXqaydl6D_cO2yP7h04=.c8a6cedd-3612-48dd-b9fa-3961f67a0766@github.com>

On Fri, 10 May 2024 17:57:22 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Thanks for review.  I'll work through these suggestions.  Question about copyright: I guess this is pretty much "all new", but the code was derived from original code, and much of the public API was borrowed from original.  So I think maybe I should keep Red Hat copyright, and add Amazon copyright.  What do you advise?
>
> But has it been derived from Red Hat code? My understanding was that it's basically a fork of src/hotspot/share/utilities/bitMap.* ? Which is Oracle-copyrighted. Is that correct?

Sorry.  I didn't look closely.  This was not derived from original code.  I wrote this blind to the original, and then I made changes under guidance of reviewers to harmonize the API with Oracle implementation of bitMap.  So I think your original comment about copyright is appropriate.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597059209

From wkemper at openjdk.org  Fri May 10 23:08:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 10 May 2024 23:08:27 GMT
Subject: RFR: 8332095: GenShen: Move more generational mode members out of
 shHeap
Message-ID: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>

The following members and methods in shHeap are only used in generational mode and can be moved out of common code:
* ShenandoahHeap::cancel_old_gc
* ShenandoahHeap::transfer_old_pointers_from_satb
* ShenandoahHeap::_is_aging_cycle

-------------

Commit messages:
 - Move is_aging_cycle into genshen heap
 - Move 'cancel_old_gc' from shHeap into shOldGeneration
 - Inline forwarding method ShenandoahHeap::transfer_old_pointers_from_satb

Changes: https://git.openjdk.org/shenandoah/pull/431/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=431&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332095
  Stats: 101 lines in 12 files changed: 38 ins; 45 del; 18 mod
  Patch: https://git.openjdk.org/shenandoah/pull/431.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/431/head:pull/431

PR: https://git.openjdk.org/shenandoah/pull/431

From wkemper at openjdk.org  Fri May 10 23:11:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 10 May 2024 23:11:12 GMT
Subject: RFR: 8332094: GenShen: Reuse existing code to verify usage before
 rebuilding free set
Message-ID: <UNGWSTaTeiLk_7qJrm0L3AB34DiydJSKVwcbwfpq3es=.a7500c98-3c8a-449c-84cd-8aa6266b858d@github.com>

We had some code in shHeap that largely duplicated existing functionality in the shVerify. This change just reuses the existing code (which has nicer assert messages).

-------------

Commit messages:
 - Reuse existing generational usage check, cleanup some warning in verifier

Changes: https://git.openjdk.org/shenandoah/pull/432/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=432&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332094
  Stats: 104 lines in 5 files changed: 11 ins; 62 del; 31 mod
  Patch: https://git.openjdk.org/shenandoah/pull/432.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/432/head:pull/432

PR: https://git.openjdk.org/shenandoah/pull/432

From azafari at openjdk.org  Sun May 12 17:54:03 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Sun, 12 May 2024 17:54:03 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file
In-Reply-To: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
Message-ID: <lVp9SLXC_Wad6DqMpMOOHmUzaeMAavs6RmtuNEek3KY=.bfe0d48d-7615-4ece-a15f-f87207e2f2ef@github.com>

On Fri, 10 May 2024 09:06:08 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> MEMFLAGS, as well as its enum constants, should live in its own include. 
> 
> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
> 
> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
> 
> ---
> 
> This patch moves the enum to its new file.
> 
> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
> 
> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
> 
> I tested (built) on:
> - MacOS aarch64, no precompiled headers, fastdebug
> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal

src/hotspot/share/services/mallocLimit.hpp line 3:

> 1: /*
> 2:  * Copyright (c) 2023 SAP SE. All rights reserved.
> 3:  * Copyright (c) 2024, 2024, Oracle and/or its affiliates. All rights reserved.

Maybe 2023, 2024 instead?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1597683285

From dlong at openjdk.org  Sun May 12 22:26:04 2024
From: dlong at openjdk.org (Dean Long)
Date: Sun, 12 May 2024 22:26:04 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
Message-ID: <m6bQAl_exnCb2wAGMQ3PkMms2WhbfgVBXLLnhgZloWM=.ea6fa98e-6c6e-430f-996c-98f0d4fe578d@github.com>

On Fri, 12 Apr 2024 14:40:05 GMT, Sergey Nazarkin <snazarki at openjdk.org> wrote:

> An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 
> 
> Additional testing:
> - [x] MacOS AArch64 server fastdebug *gtets*
> - [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
> - [ ] Benchmarking
> 
> @apangin and @parttimenerd could you please check the patch on your scenarios??

I think there is a sweet-spot middle-ground between the two extremes: full-lazy, ideal for performance, and fine-grained execute-by-default, ideal for security.  I don't think we should change to full-lazy and remove all the guard rails at this time.  I am investigating execute-by-default, and it looks promising.

-------------

Changes requested by dlong (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18762#pullrequestreview-2051465621

From kdnilsen at openjdk.org  Mon May 13 02:56:25 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 02:56:25 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>
 <AF3P7JtFsthuhESocVCkQ1IAfA2_ZunP2isFGjU9e50=.786b05da-d566-4746-807b-148e7fa0a8bd@github.com>
Message-ID: <WIdg4yNqt3ekxxkVgxg2tB10lf7HuneYflrtK91vc3k=.5d9753d2-b550-45c2-9814-de0541485a56@github.com>

On Fri, 3 May 2024 16:06:03 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Merge branch 'openjdk:master' into master
>>  - Remove unnecessary call to update_watermark
>>  - Assert progress on find_next and find_prev
>>  - Simplify partition_membership_name by code reuse
>>  - Simplify by combining implemnetations of shrink_interval functions
>>  - Fix NumPartition type
>>    
>>    Beautify the code by changing type of NumPartitions and adding Int and
>>    UInt forms of NumPartitions.
>>  - Refinements to support zero-build compiles
>>  - Fix whitespace
>>  - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c
>
> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp line 51:
> 
>> 49:   uintx mask = ((uintx) 0 - 1) & ~omit_mask;
>> 50:   if ((element_bits & mask) == mask) {
>> 51:     size_t counted_ones = BitsPerWord - bit_number;
> 
> So this is the case where all relevant bits are set, right? You might want to add a comment that says so, it took me quite a while to figure this out (including the mask-calculation that leads to it).

Thanks.  I have added a comment to clarify.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597812494

From kdnilsen at openjdk.org  Mon May 13 03:04:21 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 03:04:21 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
 <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
Message-ID: <TrxPQkaDmiuEFvNQqnmgGRWmR8tYIvU_Xh6JOfTS6ss=.d47ea2f4-2cd3-4a15-b89e-09462a902228@github.com>

On Fri, 10 May 2024 14:16:56 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Another attempt at improving clarity of comments
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 100:
> 
>> 98: }
>> 99: 
>> 100: ShenandoahRegionPartitions::~ShenandoahRegionPartitions() {
> 
> I think there's no need to put this trivial destructor definition in the .cpp file. Just place it with the declaration in the .hpp file.

Thanks.  Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597816065

From kdnilsen at openjdk.org  Mon May 13 03:04:21 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 03:04:21 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <TrxPQkaDmiuEFvNQqnmgGRWmR8tYIvU_Xh6JOfTS6ss=.d47ea2f4-2cd3-4a15-b89e-09462a902228@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
 <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
 <TrxPQkaDmiuEFvNQqnmgGRWmR8tYIvU_Xh6JOfTS6ss=.d47ea2f4-2cd3-4a15-b89e-09462a902228@github.com>
Message-ID: <dK_D3qIB9HFVhwMtWn9UFlXrRaZEN82o4ca_LBJHnRw=.5d179d25-d51c-4f75-aeaa-00f8b2ab0988@github.com>

On Mon, 13 May 2024 03:01:55 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 100:
>> 
>>> 98: }
>>> 99: 
>>> 100: ShenandoahRegionPartitions::~ShenandoahRegionPartitions() {
>> 
>> I think there's no need to put this trivial destructor definition in the .cpp file. Just place it with the declaration in the .hpp file.
>
> Thanks.  Fixed.

Thanks.  Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597816290

From kdnilsen at openjdk.org  Mon May 13 03:04:21 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 03:04:21 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v14]
In-Reply-To: <ltMhY13jQ2wsip80DkaQgQsx8hdMp6sD6eAHsc_axpE=.902463b5-004c-435f-bc13-68d3daccd06c@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <DtAJtZcZPNv2FMbF9-FeJIT6T8Oio1TW9yYnw3cceF0=.ed94d2d3-39c2-4f8d-9e00-5dbd81683c93@github.com>
 <PVrVW-I2bKkYzrdBafHAaWF8mw6WtANB3PoOZW5ekRw=.2ab54392-0f5b-4f6f-b5d7-7b7a9b6a0a97@github.com>
 <ltMhY13jQ2wsip80DkaQgQsx8hdMp6sD6eAHsc_axpE=.902463b5-004c-435f-bc13-68d3daccd06c@github.com>
Message-ID: <PM9eUd0cKjlXjfzFO8794RNnYHyQISpUR2wurz1MN2s=.b93f1f4a-40d4-48f4-9c6f-cd18be65eb8d@github.com>

On Fri, 10 May 2024 14:13:58 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 319:
>> 
>>> 317: #else
>>> 318:   log_info(gc)("%6s: %10s %10s %10s", "index", "Mutator Bits", "Collector Bits", "NotFree Bits");
>>> 319: #endif
>> 
>> There is no need for `_LP64` here, just print with `%18s` unconditionally.
>
> What Aleksey said.

Sorry.  I missed that comment.  Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597814838

From kdnilsen at openjdk.org  Mon May 13 03:07:25 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 03:07:25 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v14]
In-Reply-To: <PVrVW-I2bKkYzrdBafHAaWF8mw6WtANB3PoOZW5ekRw=.2ab54392-0f5b-4f6f-b5d7-7b7a9b6a0a97@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <DtAJtZcZPNv2FMbF9-FeJIT6T8Oio1TW9yYnw3cceF0=.ed94d2d3-39c2-4f8d-9e00-5dbd81683c93@github.com>
 <PVrVW-I2bKkYzrdBafHAaWF8mw6WtANB3PoOZW5ekRw=.2ab54392-0f5b-4f6f-b5d7-7b7a9b6a0a97@github.com>
Message-ID: <1_UWOl3Zume_tBi3wzmjR6NEWOLPjcF7ELw-cXXVDjY=.0bb67298-620d-4777-8d7c-5052aa5b0c61@github.com>

On Fri, 1 Mar 2024 17:40:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 60 commits:
>> 
>>  - Remove instrumentation and cleanup magic numbers
>>    
>>    Two places, I had used 63 when I should have used
>>    _bits_per_array_element -1.
>>  - Address 32-bit compile issues
>>  - Fix 32-bit size formatting in log messages
>>  - Fix white space
>>  - Merge remote-tracking branch 'origin/master' into restructure-free-set
>>  - Merge branch 'openjdk:master' into master
>>  - Two bug fixes for better performance
>>    
>>    1. Collector reserve size is based on memory available within regions
>>       rather than the region size (oops).
>>    
>>    2. If an attempt to allocate within a region fails and the region has
>>       already provided the percentage presumed by ShenandoahEvacWaste,
>>       retire this region.  This is motivated by observations that
>>       otherwise, we end up with large numbers of regions that have only
>>       a small amount of memory within them (e.g. 4k) and every allocation
>>       request has to wade through all of these regions before it eventually
>>       finds a region that has a sufficiently large amount of available
>>       memory.  In the original Shenandoah free-set implementation, the
>>       behavior was to retire the first time an allocation within that
>>       regions fails, regardless of whether the region has already reached
>>       the ShenandoahEvacWaste threshold.
>>  - Fix off-by-one error in is_forward_consecutive_ones()
>>  - Fix whitespace
>>  - Bug fixes and performance improvements
>>    
>>    1. Correct off-b-one-error in count of trailingones
>>    2. Speed up search for contiguous regions (for humongous allocations) by
>>       sliding window instead of initiating new search each time
>>    3. Bias regular region allocations to favor regions that are already
>>       partially consumed
>>    4. Fix bug in move_regions_from_collector_to_mutator which caused some
>>       non-empty regions to be ignored.
>>  - ... and 50 more: https://git.openjdk.org/jdk/compare/be2b92bd...1aa5a3e6
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 388:
> 
>> 386: }
>> 387: 
>> 388: inline ssize_t ShenandoahRegionPartitions:: leftmost(ShenandoahFreeSetPartitionId which_partition) const {
> 
> Excess space after `::`.

Fixed.  Thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597817083

From kdnilsen at openjdk.org  Mon May 13 03:22:17 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 03:22:17 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
 <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
Message-ID: <T2joFB_Kkp38BogNdKyaZfbizE7z_kFa5qKANLIT2Jg=.10a2282a-f16a-40fc-916c-51986988b979@github.com>

On Fri, 10 May 2024 14:18:19 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Another attempt at improving clarity of comments
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 144:
> 
>> 142:   assert (which_partition < NumPartitions, "selected free partition must be valid");
>> 143:   idx_t idx = _rightmosts[int(which_partition)];
>> 144:   // _membership[which_partition].is_set(idx) may not be true if we are shrinking the interval
> 
> Same here.

Thanks.  I've rewritten both of those comments to clarify what I was thinking...  Hope this is more clear now.

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 334:
> 
>> 332: const char* ShenandoahRegionPartitions::partition_membership_name(idx_t idx) const {
>> 333:   ShenandoahFreeSetPartitionId result = membership(idx);
>> 334:   return partition_name(result);
> 
> Could make it even more succinct by doing `return partition_name(membership(idx));`

Nice improvement.  Thanks. Changed.

> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.hpp line 3:
> 
>> 1: /*
>> 2:  * Copyright (c) 2016, 2019, Red Hat, Inc. All rights reserved.
>> 3:  * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
> 
> Same copyright issue.

Fixed.

> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.hpp line 92:
> 
>> 90:   }
>> 91: 
>> 92:   inline idx_t alignment() const {
> 
> I guess this could even be constexpr.

Thanks.  Fixed.

> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.inline.hpp line 2:
> 
>> 1: /*
>> 2:  * Copyright (c) 2016, 2019, Red Hat, Inc. All rights reserved.
> 
> Same copyright issue.

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597820331
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597820649
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597826251
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597826232
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597826480

From kdnilsen at openjdk.org  Mon May 13 03:22:18 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 03:22:18 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v57]
In-Reply-To: <MYGdv5dYKdwsPW8hr_W-6IztQXqaydl6D_cO2yP7h04=.c8a6cedd-3612-48dd-b9fa-3961f67a0766@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <xBL_Eu_5vcSwtS6V9SGAW5mbgmv4pYK3Y7lNmjjA8RU=.8220d10b-6e61-47d3-accc-d23498bace72@github.com>
 <kov90_tmaFV2mKuJIerMwEpHdexjE7QgVYOiwG3J4aY=.683d5a86-bc1f-4af5-bc65-6a3436d7c375@github.com>
 <iw4dViFLUsCidYS5tltDOZCmPLUxLA6mG0G84UVeYG4=.8dabd5b2-0144-4ee5-9833-97d8280e34cd@github.com>
 <lr3DuV2FfDGvMtYb-tVg517wM_SXs7yBeS-stfPgOGI=.612a7f92-3854-4339-ab7b-adad951f00b7@github.com>
 <MYGdv5dYKdwsPW8hr_W-6IztQXqaydl6D_cO2yP7h04=.c8a6cedd-3612-48dd-b9fa-3961f67a0766@github.com>
Message-ID: <5XqieLIViX0l5KquCMkveGxXL2AibRCdSRi4CbB6Bfk=.35f49810-d6c0-42ec-93ae-590c501a2f9c@github.com>

On Fri, 10 May 2024 18:01:33 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> But has it been derived from Red Hat code? My understanding was that it's basically a fork of src/hotspot/share/utilities/bitMap.* ? Which is Oracle-copyrighted. Is that correct?
>
> Sorry.  I didn't look closely.  This was not derived from original code.  I wrote this blind to the original, and then I made changes under guidance of reviewers to harmonize the API with Oracle implementation of bitMap.  So I think your original comment about copyright is appropriate.

Thanks.  I've fixed copyright notices for ShenandoahSimpleBitMap.?pp

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1597823099

From kdnilsen at openjdk.org  Mon May 13 03:50:08 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 03:50:08 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v58]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <9qOY81SPOhzK8KAN649jgYZusKmrP1nVxI1uShQlreQ=.ae701ce4-671b-441c-8b7f-1d23a68a91e2@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Respond to reviewer feedback

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/bb2a5ac1..ced947f4

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=57
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=56-57

  Stats: 23 lines in 5 files changed: 6 ins; 11 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From stuefe at openjdk.org  Mon May 13 04:55:24 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Mon, 13 May 2024 04:55:24 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
Message-ID: <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>

> MEMFLAGS, as well as its enum constants, should live in its own include. 
> 
> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
> 
> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
> 
> ---
> 
> This patch moves the enum to its new file.
> 
> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
> 
> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
> 
> I tested (built) on:
> - MacOS aarch64, no precompiled headers, fastdebug
> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal

Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:

  Update mallocLimit.hpp

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19172/files
  - new: https://git.openjdk.org/jdk/pull/19172/files/9a27048a..42361558

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19172&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19172&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/19172.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19172/head:pull/19172

PR: https://git.openjdk.org/jdk/pull/19172

From jsjolen at openjdk.org  Mon May 13 06:26:04 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Mon, 13 May 2024 06:26:04 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
Message-ID: <H0hlhOpsDuevZELprEcqLv6Lo0Eo9AT4mzj9kY5K8eg=.2a71b85e-7512-4128-a381-a1e24b47ee1c@github.com>

On Mon, 13 May 2024 04:55:24 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> MEMFLAGS, as well as its enum constants, should live in its own include. 
>> 
>> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
>> 
>> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
>> 
>> ---
>> 
>> This patch moves the enum to its new file.
>> 
>> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
>> 
>> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
>> 
>> I tested (built) on:
>> - MacOS aarch64, no precompiled headers, fastdebug
>> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update mallocLimit.hpp

Seems reasonable and LGTM, thanks.

-------------

Marked as reviewed by jsjolen (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19172#pullrequestreview-2051788348

From stefank at openjdk.org  Mon May 13 10:22:09 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 13 May 2024 10:22:09 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
Message-ID: <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>

On Mon, 13 May 2024 04:55:24 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> MEMFLAGS, as well as its enum constants, should live in its own include. 
>> 
>> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
>> 
>> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
>> 
>> ---
>> 
>> This patch moves the enum to its new file.
>> 
>> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
>> 
>> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
>> 
>> I tested (built) on:
>> - MacOS aarch64, no precompiled headers, fastdebug
>> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update mallocLimit.hpp

Changes requested by stefank (Reviewer).

src/hotspot/share/nmt/mallocTracker.hpp line 29:

> 27: #define SHARE_NMT_MALLOCTRACKER_HPP
> 28: 
> 29: #include "nmt/memflags.hpp"

Should go after mallocHeader.hpp

src/hotspot/share/nmt/memflags.cpp line 27:

> 25: #include "precompiled.hpp"
> 26: 
> 27: #include "nmt/memflags.hpp"

There should be no blankline between precompiled.hpp and the rest of the includes.

src/hotspot/share/nmt/memflags.cpp line 31:

> 29: 
> 30: // Extra insurance that MEMFLAGS truly has the same size as uint8_t.
> 31: STATIC_ASSERT(sizeof(MEMFLAGS) == sizeof(uint8_t));

I think you can remove this entire .cpp file. There's no need to check the size of an enum with a specified base type.

src/hotspot/share/nmt/memflags.hpp line 30:

> 28: #include "utilities/globalDefinitions.hpp"
> 29: 
> 30: #define MEMORY_TYPES_DO(f)                                                           \

Open-ended comment/question: We call it MEMORY_TYPE and mt, but then we call the type MEMFLAGS (with a completely non-standard UPPERCASE style). Maybe it is time to rename MEMFLAGS?

src/hotspot/share/services/mallocLimit.cpp line 28:

> 26: #include "precompiled.hpp"
> 27: 
> 28: #include "nmt/memflags.hpp"

While poking around in the includes, could you remove the blankline on 27. This style inconsistency has slowly crept into the code base.

-------------

PR Review: https://git.openjdk.org/jdk/pull/19172#pullrequestreview-2052269037
PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598224275
PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598225428
PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598226640
PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598229830
PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598231329

From shade at openjdk.org  Mon May 13 11:26:09 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 13 May 2024 11:26:09 GMT
Subject: RFR: 8332082: Shenandoah: Use SATB active flag for C2 pre-write
 barrier on x86 and PPC
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <tUG2eUxdH_f4gIu1T4tarxjbZMmE1BYYLqNsWkhcxJM=.f3a35a8d-ae41-4c4b-82cb-9dea195f717b@github.com>

On Fri, 10 May 2024 16:13:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is consistent with c1 and other platforms.

I agree there is an inconsistency, but I also see it is deeper than just these two.
https://github.com/openjdk/jdk/blob/1484153c1a092cefc20270b35aa1e508280843a4/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp#L696-L698

Before we go hard on either gc-state or SATB "active" flag, let's decide which way we go?

The underlying issue, IIRC, was that hardly any other GC implementation has a GC state flag, so they are forced to use the SATB "active" flags. But I am thinking that testing the gc-state flag is better on these paths, since the gc-state flag is guaranteed to be uncontended and fast-accessible.

-------------

PR Review: https://git.openjdk.org/jdk/pull/19180#pullrequestreview-2052409802

From stuefe at openjdk.org  Mon May 13 14:31:10 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Mon, 13 May 2024 14:31:10 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
Message-ID: <dsHXwakOEbgffTVz2yyJ8aW_K78MijAC1MoLlHA0uDE=.afaef773-a738-4e65-9151-7a25576faadd@github.com>

On Mon, 13 May 2024 10:16:36 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Update mallocLimit.hpp
>
> src/hotspot/share/nmt/memflags.hpp line 30:
> 
>> 28: #include "utilities/globalDefinitions.hpp"
>> 29: 
>> 30: #define MEMORY_TYPES_DO(f)                                                           \
> 
> Open-ended comment/question: We call it MEMORY_TYPE and mt, but then we call the type MEMFLAGS (with a completely non-standard UPPERCASE style). Maybe it is time to rename MEMFLAGS?

I don't feel like starting that particular bike shedding discussion :) But sure, sometime in the future we should do this. Here, I want it to be a simple renaming change.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598580049

From stuefe at openjdk.org  Mon May 13 14:34:04 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Mon, 13 May 2024 14:34:04 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
Message-ID: <pTStg0P42fAhcn47CxmRTyRElZpvEIbnLHTeJP9n5kc=.39b0265d-4610-4f9d-b426-967163267225@github.com>

On Mon, 13 May 2024 10:13:50 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Update mallocLimit.hpp
>
> src/hotspot/share/nmt/memflags.cpp line 31:
> 
>> 29: 
>> 30: // Extra insurance that MEMFLAGS truly has the same size as uint8_t.
>> 31: STATIC_ASSERT(sizeof(MEMFLAGS) == sizeof(uint8_t));
> 
> I think you can remove this entire .cpp file. There's no need to check the size of an enum with a specified base type.

I rather have this explicit check. If MEMFLAGS>1byte, things break, and I would like to make that explicit.

That said, I can move this static assert to the header. I just wanted to avoid including debug.hpp. My original intent was for this cpp file to be the place in the future for any MEMFLAGS related utility functions, e.g. to-and-from-string conversations.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598583814

From stefank at openjdk.org  Mon May 13 14:54:05 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 13 May 2024 14:54:05 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <pTStg0P42fAhcn47CxmRTyRElZpvEIbnLHTeJP9n5kc=.39b0265d-4610-4f9d-b426-967163267225@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
 <pTStg0P42fAhcn47CxmRTyRElZpvEIbnLHTeJP9n5kc=.39b0265d-4610-4f9d-b426-967163267225@github.com>
Message-ID: <mdPiV6VQ-6lZ9b7_B1AduBqe3EkUNH3JdqKPbO8KPpU=.b03dad27-6021-4874-90c2-75ff733eb4ae@github.com>

On Mon, 13 May 2024 14:31:22 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> src/hotspot/share/nmt/memflags.cpp line 31:
>> 
>>> 29: 
>>> 30: // Extra insurance that MEMFLAGS truly has the same size as uint8_t.
>>> 31: STATIC_ASSERT(sizeof(MEMFLAGS) == sizeof(uint8_t));
>> 
>> I think you can remove this entire .cpp file. There's no need to check the size of an enum with a specified base type.
>
> I rather have this explicit check. If MEMFLAGS>1byte, things break, and I would like to make that explicit.
> 
> That said, I can move this static assert to the header. I just wanted to avoid including debug.hpp. My original intent was for this cpp file to be the place in the future for any MEMFLAGS related utility functions, e.g. to-and-from-string conversations.

Could you instead put the static_assert near the code that will break? Right now it looks obscure and weird to have this check when it is obviously correct as long as no one changes the definition. Would it be enough to write a comment in the header that this needs to be 1 byte?

>> src/hotspot/share/nmt/memflags.hpp line 30:
>> 
>>> 28: #include "utilities/globalDefinitions.hpp"
>>> 29: 
>>> 30: #define MEMORY_TYPES_DO(f)                                                           \
>> 
>> Open-ended comment/question: We call it MEMORY_TYPE and mt, but then we call the type MEMFLAGS (with a completely non-standard UPPERCASE style). Maybe it is time to rename MEMFLAGS?
>
> I don't feel like starting that particular bike shedding discussion :) But sure, sometime in the future we should do this. Here, I want it to be a simple renaming change.

Right. That's why I prefixed this with "Open-ended comment/question", trying to make it super clear that it wasn't intended as a request for this PR, but rather a way to at least plant the seed of an idea that we might want to fix this eyesore.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598603277
PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598608110

From dcubed at openjdk.org  Mon May 13 15:30:07 2024
From: dcubed at openjdk.org (Daniel D. Daugherty)
Date: Mon, 13 May 2024 15:30:07 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <mdPiV6VQ-6lZ9b7_B1AduBqe3EkUNH3JdqKPbO8KPpU=.b03dad27-6021-4874-90c2-75ff733eb4ae@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
 <pTStg0P42fAhcn47CxmRTyRElZpvEIbnLHTeJP9n5kc=.39b0265d-4610-4f9d-b426-967163267225@github.com>
 <mdPiV6VQ-6lZ9b7_B1AduBqe3EkUNH3JdqKPbO8KPpU=.b03dad27-6021-4874-90c2-75ff733eb4ae@github.com>
Message-ID: <3E6BcBm0RVHLAGmuNjoYoeNs5-JLcYT5KgKVfmcxYAc=.4537bffa-ea3c-43f2-bc13-16710444b355@github.com>

On Mon, 13 May 2024 14:44:05 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I rather have this explicit check. If MEMFLAGS>1byte, things break, and I would like to make that explicit.
>> 
>> That said, I can move this static assert to the header. I just wanted to avoid including debug.hpp. My original intent was for this cpp file to be the place in the future for any MEMFLAGS related utility functions, e.g. to-and-from-string conversations.
>
> Could you instead put the static_assert near the code that will break? Right now it looks obscure and weird to have this check when it is obviously correct as long as no one changes the definition. Would it be enough to write a comment in the header that this needs to be 1 byte?

To quote @robehn - Why write a comment for a rule if you can enforce it with code instead...

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598665179

From stefank at openjdk.org  Mon May 13 15:39:17 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 13 May 2024 15:39:17 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <3E6BcBm0RVHLAGmuNjoYoeNs5-JLcYT5KgKVfmcxYAc=.4537bffa-ea3c-43f2-bc13-16710444b355@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
 <pTStg0P42fAhcn47CxmRTyRElZpvEIbnLHTeJP9n5kc=.39b0265d-4610-4f9d-b426-967163267225@github.com>
 <mdPiV6VQ-6lZ9b7_B1AduBqe3EkUNH3JdqKPbO8KPpU=.b03dad27-6021-4874-90c2-75ff733eb4ae@github.com>
 <3E6BcBm0RVHLAGmuNjoYoeNs5-JLcYT5KgKVfmcxYAc=.4537bffa-ea3c-43f2-bc13-16710444b355@github.com>
Message-ID: <6qHBMgu-ZgoMPPym-sSJJR9szoVattW-JuHeXaX4JY0=.05710a47-19fe-48ba-920c-218c16473fbb@github.com>

On Mon, 13 May 2024 15:26:18 GMT, Daniel D. Daugherty <dcubed at openjdk.org> wrote:

>> Could you instead put the static_assert near the code that will break? Right now it looks obscure and weird to have this check when it is obviously correct as long as no one changes the definition. Would it be enough to write a comment in the header that this needs to be 1 byte?
>
> To quote @robehn - Why write a comment for a rule if you can enforce it with code instead...

I tend to agree with that. My earlier question still stands is there a better place to put it? Right now the "enforced" code in a stand-alone file doesn't tell me "why" this is important.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598679057

From stefank at openjdk.org  Mon May 13 15:51:11 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 13 May 2024 15:51:11 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <6qHBMgu-ZgoMPPym-sSJJR9szoVattW-JuHeXaX4JY0=.05710a47-19fe-48ba-920c-218c16473fbb@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
 <pTStg0P42fAhcn47CxmRTyRElZpvEIbnLHTeJP9n5kc=.39b0265d-4610-4f9d-b426-967163267225@github.com>
 <mdPiV6VQ-6lZ9b7_B1AduBqe3EkUNH3JdqKPbO8KPpU=.b03dad27-6021-4874-90c2-75ff733eb4ae@github.com>
 <3E6BcBm0RVHLAGmuNjoYoeNs5-JLcYT5KgKVfmcxYAc=.4537bffa-ea3c-43f2-bc13-16710444b355@github.com>
 <6qHBMgu-ZgoMPPym-sSJJR9szoVattW-JuHeXaX4JY0=.05710a47-19fe-48ba-920c-218c16473fbb@github.com>
Message-ID: <jD8jfJeM_GyOvpD4K1VUuVCbaVdcbIZDrINJNoBA_Zw=.2d745cc5-3057-490d-9c03-05cb6c7ee778@github.com>

On Mon, 13 May 2024 15:36:13 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> To quote @robehn - Why write a comment for a rule if you can enforce it with code instead...
>
> I tend to agree with that. My earlier question still stands: is there a better place to put it? Right now the "enforced with code" in a stand-alone file doesn't tell me "why" this rule is important.

If you want to keep the static_assert it in the .cpp file, then I won't block that.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1598695748

From kdnilsen at openjdk.org  Mon May 13 16:24:41 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 16:24:41 GMT
Subject: RFR: 8332094: GenShen: Reuse existing code to verify usage before
 rebuilding free set
In-Reply-To: <UNGWSTaTeiLk_7qJrm0L3AB34DiydJSKVwcbwfpq3es=.a7500c98-3c8a-449c-84cd-8aa6266b858d@github.com>
References: <UNGWSTaTeiLk_7qJrm0L3AB34DiydJSKVwcbwfpq3es=.a7500c98-3c8a-449c-84cd-8aa6266b858d@github.com>
Message-ID: <XlbzrlDNYkKRAuyULjYwpUUX3YN44MF8WqtcdidAOlg=.fcdf3354-fd45-4365-9976-7f3a100a489a@github.com>

On Fri, 10 May 2024 23:06:26 GMT, William Kemper <wkemper at openjdk.org> wrote:

> We had some code in shHeap that largely duplicated existing functionality in the shVerify. This change just reuses the existing code (which has nicer assert messages).

Thanks for this refactoring.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/432#pullrequestreview-2053161263

From kdnilsen at openjdk.org  Mon May 13 16:44:22 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 16:44:22 GMT
Subject: RFR: 8332095: GenShen: Move more generational mode members out of
 shHeap
In-Reply-To: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
References: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
Message-ID: <k8twQLUgA9B7rbFTP6qali2kEXnFft8R_vPuZsBxeqo=.fd9f3b4c-85d9-443b-97cf-ac0106812356@github.com>

On Fri, 10 May 2024 23:02:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The following members and methods in shHeap are only used in generational mode and can be moved out of common code:
> * ShenandoahHeap::cancel_old_gc
> * ShenandoahHeap::transfer_old_pointers_from_satb
> * ShenandoahHeap::_is_aging_cycle

Thanks.

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 1218:

> 1216:   }
> 1217: 
> 1218:   // This needs to see is_aging_cycle

This comment is a bit unclear.  Are we saying we need the heap state to capture whether this is an aging cycle, because that will influence which regions get aged at the end of update refs?

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/431#pullrequestreview-2053181879
PR Review Comment: https://git.openjdk.org/shenandoah/pull/431#discussion_r1598749704

From kbarrett at openjdk.org  Mon May 13 18:30:04 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Mon, 13 May 2024 18:30:04 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
Message-ID: <4m9MF6p8mpiKijVsc-mg0IDBQE9lLvPZQjqzZeV1kQo=.97de93cf-8f20-406a-886b-a7c98bd3ccb1@github.com>

On Mon, 13 May 2024 04:55:24 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> MEMFLAGS, as well as its enum constants, should live in its own include. 
>> 
>> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
>> 
>> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
>> 
>> ---
>> 
>> This patch moves the enum to its new file.
>> 
>> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
>> 
>> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
>> 
>> I tested (built) on:
>> - MacOS aarch64, no precompiled headers, fastdebug
>> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update mallocLimit.hpp

Looks good, subject to addressing the minor issues already reported by others.
Leaving unapproved, but don't wait for my approval once you have others.

-------------

PR Review: https://git.openjdk.org/jdk/pull/19172#pullrequestreview-2053415755

From wkemper at openjdk.org  Mon May 13 18:40:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 18:40:13 GMT
Subject: RFR: 8332082: Shenandoah: Use SATB active flag for C2 pre-write
 barrier on x86 and PPC
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <PfynvA564HWJUPmm9jFzV-kRifs31HElBphfGK5D-IY=.1e2ac504-533a-4473-9602-a9ea4690a04a@github.com>

On Fri, 10 May 2024 16:13:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is consistent with c1 and other platforms.

Both the gc-state and satb-active flags are synchronized to thread local fields on a safepoint. I expect the cost for the barrier to access either of them is similar. However, there is some additional c2 code that looks like it recognizes the barrier IR based on the presence of nodes to load the gc state: (for example, `ShenandoahBarrierSetC2::is_shenandoah_marking_if`).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19180#issuecomment-2108547726

From wkemper at openjdk.org  Mon May 13 18:45:05 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 18:45:05 GMT
Subject: RFR: 8332082: Shenandoah: Use SATB active flag for C2 pre-write
 barrier on x86 and PPC
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <77VGGeS1lFsQ5_Qp5jZkGIqSw5dDgmTTJtH5QmoA4eo=.8d9d1728-0e62-4a31-a109-4268c916416a@github.com>

On Fri, 10 May 2024 16:13:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is consistent with c1 and other platforms.

Also, as pointed out in https://github.com/openjdk/jdk/pull/18148, `ShenandoahBarrierSetC2::verify_gc_barriers` is looking for the IR nodes that check `ShenandoahThreadLocalData::satb_mark_queue_active_offset`, so inconsistencies abound. I've never seen `verify_gc_barriers` raise an `assert` , even though it is `trueInDebug`. Is `verify_gc_barriers` broken?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19180#issuecomment-2108557318

From wkemper at openjdk.org  Mon May 13 18:46:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 18:46:47 GMT
Subject: RFR: 8332095: GenShen: Move more generational mode members out of
 shHeap
In-Reply-To: <k8twQLUgA9B7rbFTP6qali2kEXnFft8R_vPuZsBxeqo=.fd9f3b4c-85d9-443b-97cf-ac0106812356@github.com>
References: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
 <k8twQLUgA9B7rbFTP6qali2kEXnFft8R_vPuZsBxeqo=.fd9f3b4c-85d9-443b-97cf-ac0106812356@github.com>
Message-ID: <l_vzOKiDv-yUIlcQ8aSk-ILYNKLO55WSzjhQXLWpe44=.3a394c18-f1cb-44d0-a354-3b8d5eca38fe@github.com>

On Mon, 13 May 2024 16:30:15 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> The following members and methods in shHeap are only used in generational mode and can be moved out of common code:
>> * ShenandoahHeap::cancel_old_gc
>> * ShenandoahHeap::transfer_old_pointers_from_satb
>> * ShenandoahHeap::_is_aging_cycle
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 1218:
> 
>> 1216:   }
>> 1217: 
>> 1218:   // This needs to see is_aging_cycle
> 
> This comment is a bit unclear.  Are we saying we need the heap state to capture whether this is an aging cycle, because that will influence which regions get aged at the end of update refs?

No, that was more of a 'note to self' to make sure the method calls didn't get re-ordered as part of this refactoring. I'll update the comment.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/431#discussion_r1598896480

From wkemper at openjdk.org  Mon May 13 18:52:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 18:52:36 GMT
Subject: RFR: 8332095: GenShen: Move more generational mode members out of
 shHeap [v2]
In-Reply-To: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
References: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
Message-ID: <Is8k1WDNSLYmavOo736vqSsl77mYmkFEaZesximLZkI=.4e97700f-4c08-4a08-96f2-7a5bb5538e22@github.com>

> The following members and methods in shHeap are only used in generational mode and can be moved out of common code:
> * ShenandoahHeap::cancel_old_gc
> * ShenandoahHeap::transfer_old_pointers_from_satb
> * ShenandoahHeap::_is_aging_cycle

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Update confusing comment

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/431/files
  - new: https://git.openjdk.org/shenandoah/pull/431/files/b875957d..75fdfdbc

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=431&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=431&range=00-01

  Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/431.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/431/head:pull/431

PR: https://git.openjdk.org/shenandoah/pull/431

From kdnilsen at openjdk.org  Mon May 13 19:58:08 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 19:58:08 GMT
Subject: RFR: 8332081: GenShen: Disentangle generational mode update
 references
In-Reply-To: <4BeFZ8yhpfr2vYp0_rtPL7KE7zPYFMvoawNPhlzaACE=.856369bb-75e0-4bde-bd42-13ad51271ff8@github.com>
References: <4BeFZ8yhpfr2vYp0_rtPL7KE7zPYFMvoawNPhlzaACE=.856369bb-75e0-4bde-bd42-13ad51271ff8@github.com>
Message-ID: <8AwQC5YL_0rFlvhr3jWpDA09NQ4UeNHaueb9r6y9Xfc=.b7d4cbd7-3129-4264-9015-da4f8da106a4@github.com>

On Fri, 10 May 2024 15:56:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

> I believe the test failure on macosx is unrelated to these changes.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/430#pullrequestreview-2053656879

From kdnilsen at openjdk.org  Mon May 13 21:20:14 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 21:20:14 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v14]
In-Reply-To: <LSfLue-6QPygFz_XhIZa_kihwsc98z6MOA7z1TXtsP8=.1fb122df-f06a-4dcb-8091-4610ab8a3081@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <DtAJtZcZPNv2FMbF9-FeJIT6T8Oio1TW9yYnw3cceF0=.ed94d2d3-39c2-4f8d-9e00-5dbd81683c93@github.com>
 <PVrVW-I2bKkYzrdBafHAaWF8mw6WtANB3PoOZW5ekRw=.2ab54392-0f5b-4f6f-b5d7-7b7a9b6a0a97@github.com>
 <XzlTwJFYWhjXumZ-mexXz39jhm5-hjqfBnmWayHTVZg=.e5d25e3d-54f0-4478-a1ed-afacaecf9af9@github.com>
 <LSfLue-6QPygFz_XhIZa_kihwsc98z6MOA7z1TXtsP8=.1fb122df-f06a-4dcb-8091-4610ab8a3081@github.com>
Message-ID: <oMTtvas7HnBuxn20E-p_-Wu_u62ZRnfLKNZyo_63nz4=.2d564b4c-9533-4733-99d9-839fac331d5f@github.com>

On Wed, 10 Apr 2024 11:44:12 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> I'm going to add the following comment at the top of shenandoahFreeSet.hpp:
>> 
>> // The API and internal implementation of ShenandoahSimpleBitMap and ShenandoahRegionPartitions use ssize_t to                      
>> // represent index, even though index is "inherently" unsigned.  There are two reasons for this choice:                             
>> //  1. We use -1 as a sentinel value to represent empty partitions.  This same value may be used to represent                       
>> //     failure to find a previous set bit or previous range of set bits.                                                            
>> //  2. Certain loops are written most naturally if the iterator, which may hold the sentinel -1 value, can be                       
>> //     declared as signed and the terminating condition can be < 0.
>
> Please use types, method signatures and keep structure similar to bitMap.hpp, which use idx_t, which is a size_t (see e.g. BitMap::count_one_bits()).

I am using idx_t, defined here to be ssize_t, as discussed in comments.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1599095492

From wkemper at openjdk.org  Mon May 13 21:31:16 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 21:31:16 GMT
Subject: Integrated: 8332094: GenShen: Reuse existing code to verify usage
 before rebuilding free set
In-Reply-To: <UNGWSTaTeiLk_7qJrm0L3AB34DiydJSKVwcbwfpq3es=.a7500c98-3c8a-449c-84cd-8aa6266b858d@github.com>
References: <UNGWSTaTeiLk_7qJrm0L3AB34DiydJSKVwcbwfpq3es=.a7500c98-3c8a-449c-84cd-8aa6266b858d@github.com>
Message-ID: <AFDEf07gS4fC5PFaa-11Tic5SFtTYhMp6E8lsuTiQyg=.c72ccc26-3a09-4b23-bf74-0082e5500204@github.com>

On Fri, 10 May 2024 23:06:26 GMT, William Kemper <wkemper at openjdk.org> wrote:

> We had some code in shHeap that largely duplicated existing functionality in the shVerify. This change just reuses the existing code (which has nicer assert messages).

This pull request has now been integrated.

Changeset: 78836a57
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/78836a579d83a98eab2e1431a1e7a20ea0d79149
Stats:     104 lines in 5 files changed: 11 ins; 62 del; 31 mod

8332094: GenShen: Reuse existing code to verify usage before rebuilding free set

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/432

From kdnilsen at openjdk.org  Mon May 13 21:31:40 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 13 May 2024 21:31:40 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix copyright notice on test file

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/ced947f4..572aa2b3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=58
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=57-58

  Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From wkemper at openjdk.org  Mon May 13 22:35:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 22:35:32 GMT
Subject: RFR: 8332081: GenShen: Disentangle generational mode update
 references [v2]
In-Reply-To: <4BeFZ8yhpfr2vYp0_rtPL7KE7zPYFMvoawNPhlzaACE=.856369bb-75e0-4bde-bd42-13ad51271ff8@github.com>
References: <4BeFZ8yhpfr2vYp0_rtPL7KE7zPYFMvoawNPhlzaACE=.856369bb-75e0-4bde-bd42-13ad51271ff8@github.com>
Message-ID: <ZLEyANlV_4R3_7cUsgypLjqoI3eru5CW34SuJAYCUAU=.323aed1a-50c3-4af9-9830-d2d718d29e14@github.com>

> I believe the test failure on macosx is unrelated to these changes.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits:

 - Merge remote-tracking branch 'shenandoah/master' into isolate-update-references
 - Remove single threaded usages of heap region iterator
 - Fix asserts to handle case when gc is cancelled
 - Move generational version of update refs to generational heap

-------------

Changes: https://git.openjdk.org/shenandoah/pull/430/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=430&range=01
  Stats: 439 lines in 6 files changed: 237 ins; 185 del; 17 mod
  Patch: https://git.openjdk.org/shenandoah/pull/430.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/430/head:pull/430

PR: https://git.openjdk.org/shenandoah/pull/430

From wkemper at openjdk.org  Mon May 13 22:39:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 22:39:31 GMT
Subject: Integrated: 8332081: GenShen: Disentangle generational mode update
 references
In-Reply-To: <4BeFZ8yhpfr2vYp0_rtPL7KE7zPYFMvoawNPhlzaACE=.856369bb-75e0-4bde-bd42-13ad51271ff8@github.com>
References: <4BeFZ8yhpfr2vYp0_rtPL7KE7zPYFMvoawNPhlzaACE=.856369bb-75e0-4bde-bd42-13ad51271ff8@github.com>
Message-ID: <AFvOBbr_GHZFnZPEQOiP69xARfSQK0Yq2sN9VzrZITA=.71c612f3-4165-4589-b4fa-a79b7d2043f4@github.com>

On Fri, 10 May 2024 15:56:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

> I believe the test failure on macosx is unrelated to these changes.

This pull request has now been integrated.

Changeset: def9d3fe
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/def9d3feb212226688c6fd071723adc4e00cc361
Stats:     439 lines in 6 files changed: 237 ins; 185 del; 17 mod

8332081: GenShen: Disentangle generational mode update references

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/430

From wkemper at openjdk.org  Mon May 13 22:53:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 22:53:46 GMT
Subject: RFR: 8332095: GenShen: Move more generational mode members out of
 shHeap [v3]
In-Reply-To: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
References: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
Message-ID: <IHdJwlBqNX4jzBLevClIeG7ii1ICA8DnH1sq6ASw1u4=.afe6cd47-77f4-4eee-86e7-5b113f670dc8@github.com>

> The following members and methods in shHeap are only used in generational mode and can be moved out of common code:
> * ShenandoahHeap::cancel_old_gc
> * ShenandoahHeap::transfer_old_pointers_from_satb
> * ShenandoahHeap::_is_aging_cycle

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits:

 - Merge remote-tracking branch 'shenandoah/master' into move-shenandoah-heap-fields
 - Merge remote-tracking branch 'shenandoah/master' into move-shenandoah-heap-fields
 - Update confusing comment
 - Move is_aging_cycle into genshen heap
 - Move 'cancel_old_gc' from shHeap into shOldGeneration
 - Inline forwarding method ShenandoahHeap::transfer_old_pointers_from_satb

-------------

Changes: https://git.openjdk.org/shenandoah/pull/431/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=431&range=02
  Stats: 101 lines in 11 files changed: 39 ins; 44 del; 18 mod
  Patch: https://git.openjdk.org/shenandoah/pull/431.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/431/head:pull/431

PR: https://git.openjdk.org/shenandoah/pull/431

From wkemper at openjdk.org  Mon May 13 22:53:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 13 May 2024 22:53:46 GMT
Subject: Integrated: 8332095: GenShen: Move more generational mode members out
 of shHeap
In-Reply-To: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
References: <Z4A8W1_JThLN_YTyQLAr_9qba-RJLj8_1RDRzS1t7sA=.8860769b-13b8-4985-972a-71cb5cd596ea@github.com>
Message-ID: <wenwuyL_tGPcOUpDZyC4cLD9Cr5RjGrcDWRKSE62S5U=.af176458-4df0-444b-a4aa-5862b1309bc5@github.com>

On Fri, 10 May 2024 23:02:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The following members and methods in shHeap are only used in generational mode and can be moved out of common code:
> * ShenandoahHeap::cancel_old_gc
> * ShenandoahHeap::transfer_old_pointers_from_satb
> * ShenandoahHeap::_is_aging_cycle

This pull request has now been integrated.

Changeset: d77f426b
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/d77f426b0a7c8b2c7285ee43d978be2cf584a51f
Stats:     101 lines in 11 files changed: 39 ins; 44 del; 18 mod

8332095: GenShen: Move more generational mode members out of shHeap

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/431

From ysr at openjdk.org  Tue May 14 00:06:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 14 May 2024 00:06:17 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
Message-ID: <08-005vvbDWvi6V9Jyjz2sTCq0ryu1E6XifnXC-6uM4=.5d91b5bf-06a2-4359-96e9-95ad7d5676cc@github.com>

On Mon, 13 May 2024 21:31:40 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix copyright notice on test file

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 105:

> 103:   ShenandoahHeapRegion* r = _heap->get_region(idx);
> 104:   return can_allocate_from(r);
> 105: }

Some of this abstraction leakage can be avoided by making

"can_allocate_from(size_t idx)" a method on the `_heap` object, and calling that, letting the heap do the test:

``` 
bool ShenandoahHeap::can_allocate_from(size_t idx) {
  ShenandoahHeapRegion* r = get_region(idx);
  return r->is_empty() || (r->is_trash() && !is_concurrent_weak_root_in_progress());
}


with ShFreeSet merely delegating to its `_heap`.

Probably similar pattern for `alloc_capacity` method below.

(I'll keep leaving single comments, so I don't slow down the review/feedback across potential interruptions. I don't expect there will be many, but still.)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1599225270

From ysr at openjdk.org  Tue May 14 01:11:14 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 14 May 2024 01:11:14 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
Message-ID: <TITws1EOWa6EKiu-HXXLdgNLRsE9eh4YUkmLMHMCyms=.9649906c-97a4-4b85-92f3-3a12090288d2@github.com>

On Mon, 13 May 2024 21:31:40 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix copyright notice on test file

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 93:

> 91:   void dump_bitmap_row(ssize_t idx) const;
> 92:   void dump_bitmap_range(ssize_t start_idx, ssize_t end_idx) const;
> 93:   void dump_bitmap_all() const;

In the first case you can may be say `region_idx` instead of `idx` ?

Also, it seems `dump_bitmap()` would be sufficient (at least for the case of `dump_bitmap_all()`, although in general for all).

However, given that these are private methods that will only be found in the implementation and nowhere else in its clients' code, may be current naming is ok (indeed, it looks like the first two are work functions in service of the last).

PS: I don't see any current usage of `dump_bitmap_all()`. May be you wanted it in some debug tracing code or something, or keeping it for possible future debugging usage? (Or may be my naive web-page search of this review page is naive/faulty.) It's perhaps useful to keep around for debug/tracing though.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 103:

> 101: 
> 102:   // Set the partition id for a particular region without adjusting interval bounds or usage/capacity tallies
> 103:   inline void raw_set_membership(size_t idx, ShenandoahFreeSetPartitionId p) {

I realize that the word `set` can be a little ambiguous here (from the mathematical notion of a set as  a collection of things), although the fact that it has a void return type tells us that `set` is the imperative verb here, not a collection of things. May be `raw_assign_membership(size_t region_idx, ShFSPId p)` if you want to, but I may be splitting hairs here :-)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1599257135
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1599257281

From ysr at openjdk.org  Tue May 14 01:20:14 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 14 May 2024 01:20:14 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
Message-ID: <D7sXU86YZIqa5-pFTLE_OTwOFctb0DAR8SqPtJFu-VQ=.c9257ae8-bb80-4bf5-aaf9-ea3506426e0c@github.com>

On Mon, 13 May 2024 21:31:40 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix copyright notice on test file

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 109:

> 107:   // Set the Mutator intervals, usage, and capacity according to arguments.  Reset the Collector intervals, used, capacity
> 108:   // to represent empty Collector free set.
> 109:   void establish_intervals(ssize_t mutator_leftmost, ssize_t mutator_rightmost,

Why not clarify in name via `establish_mutator_intervals` ? (I noted there isn't a corresponding method for "collector intervals", although there are corresponding fields for its left and rightmosts etc.) A documentation comment somewhere indicating how/when those get established would be useful. I see them being incrementally updated in the incremental manipulation of set memberships, but wanted to get a sense of the reasons for this asymmetry (and for that reason to be documented when we revisit this code many moons in the future).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1599261601

From ysr at openjdk.org  Tue May 14 01:44:13 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 14 May 2024 01:44:13 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v4]
In-Reply-To: <DtF0h45Ec9GtnLdsyCQc7pbrck1x38rnOeQLVOFJV_E=.0fe89b83-844b-4c9f-981f-988905896fc3@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Uw8Ao0F2vQhi35yNeJtfSsoZbSzgYhrW1-ExVDmJTCM=.65b8d60a-d490-4456-984b-40be92a83fb4@github.com>
 <gViARkv0jhXfIL4ekepO4LblHADqn1AvzTRdW3xvCw0=.8113e2a5-9c84-41c5-ad63-dc239931b1ea@github.com>
 <DtF0h45Ec9GtnLdsyCQc7pbrck1x38rnOeQLVOFJV_E=.0fe89b83-844b-4c9f-981f-988905896fc3@github.com>
Message-ID: <LlBI1i97amXBMbIXmW1T3mGp0d5KX_mgxrjWYF2721w=.fd7e38c2-980b-44c0-b94c-a1603e358b54@github.com>

On Wed, 31 Jan 2024 00:45:20 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 401:
>> 
>>> 399: 
>>> 400: HeapWord* ShenandoahFreeSet::allocate_single(ShenandoahAllocRequest& req, bool& in_new_region) {
>>> 401:   shenandoah_assert_heaplocked();
>> 
>> In addition, another precondition for this method appears to be that req.size() <= humongous size threshold. Perhaps that check should also be disposed of here. (Based on the documentation at the previous review comment above.)
>
> Added similar documentation here.  Thanks.

@kdnilsen ... bumping this up. I see that the documentation was added for precondition, but not the check/assert itself here in the implementation. Commenting to unresolve previously resolved comment.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1599273190

From ysr at openjdk.org  Tue May 14 01:44:13 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 14 May 2024 01:44:13 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v4]
In-Reply-To: <LlBI1i97amXBMbIXmW1T3mGp0d5KX_mgxrjWYF2721w=.fd7e38c2-980b-44c0-b94c-a1603e358b54@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Uw8Ao0F2vQhi35yNeJtfSsoZbSzgYhrW1-ExVDmJTCM=.65b8d60a-d490-4456-984b-40be92a83fb4@github.com>
 <gViARkv0jhXfIL4ekepO4LblHADqn1AvzTRdW3xvCw0=.8113e2a5-9c84-41c5-ad63-dc239931b1ea@github.com>
 <DtF0h45Ec9GtnLdsyCQc7pbrck1x38rnOeQLVOFJV_E=.0fe89b83-844b-4c9f-981f-988905896fc3@github.com>
 <LlBI1i97amXBMbIXmW1T3mGp0d5KX_mgxrjWYF2721w=.fd7e38c2-980b-44c0-b94c-a1603e358b54@github.com>
Message-ID: <b5iJKbpY-MbJmyIao_N8rESPsqEO1I2TH_lXTj5poQk=.d08f8aa1-ac7f-4385-9a9b-c3e1ffc5acc2@github.com>

On Tue, 14 May 2024 01:39:57 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Added similar documentation here.  Thanks.
>
> @kdnilsen ... bumping this up. I see that the documentation was added for precondition, but not the check/assert itself here in the implementation. Commenting to unresolve previously resolved comment.

... unless this gets checked somewhere more appropriate in the callee chain. When documented as a precondition, it's often easiest to catch debug if asserted upstream rather than deep downstream where context may be difficult to work backwards to.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1599274274

From stuefe at openjdk.org  Tue May 14 07:01:03 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 14 May 2024 07:01:03 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <jD8jfJeM_GyOvpD4K1VUuVCbaVdcbIZDrINJNoBA_Zw=.2d745cc5-3057-490d-9c03-05cb6c7ee778@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
 <pTStg0P42fAhcn47CxmRTyRElZpvEIbnLHTeJP9n5kc=.39b0265d-4610-4f9d-b426-967163267225@github.com>
 <mdPiV6VQ-6lZ9b7_B1AduBqe3EkUNH3JdqKPbO8KPpU=.b03dad27-6021-4874-90c2-75ff733eb4ae@github.com>
 <3E6BcBm0RVHLAGmuNjoYoeNs5-JLcYT5KgKVfmcxYAc=.4537bffa-ea3c-43f2-bc13-16710444b355@github.com>
 <6qHBMgu-ZgoMPPym-sSJJR9szoVattW-JuHeXaX4JY0=.05710a47-19fe-48ba-920c-218c16473fbb@github.com>
 <jD8jfJeM_GyOvpD4K1VUuVCbaVdcbIZDrINJNoBA_Zw=.2d745cc5-3057-490d-9c03-05cb6c7ee778@github.com>
Message-ID: <AaG7Mp2WFHI0QhM5k9VujYhkgFqWk9mCAAU3opDT4Bc=.5ef7a60e-557f-41c8-99de-4379a081da73@github.com>

On Mon, 13 May 2024 15:48:43 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I tend to agree with that. My earlier question still stands: is there a better place to put it? Right now the "enforced with code" in a stand-alone file doesn't tell me "why" this rule is important.
>
> If you want to keep the static_assert it in the .cpp file, then I won't block that.

> Could you instead put the static_assert near the code that will break?

I like that. I will do that.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1599476559

From stuefe at openjdk.org  Tue May 14 07:01:03 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 14 May 2024 07:01:03 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <mdPiV6VQ-6lZ9b7_B1AduBqe3EkUNH3JdqKPbO8KPpU=.b03dad27-6021-4874-90c2-75ff733eb4ae@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
 <pTStg0P42fAhcn47CxmRTyRElZpvEIbnLHTeJP9n5kc=.39b0265d-4610-4f9d-b426-967163267225@github.com>
 <mdPiV6VQ-6lZ9b7_B1AduBqe3EkUNH3JdqKPbO8KPpU=.b03dad27-6021-4874-90c2-75ff733eb4ae@github.com>
Message-ID: <7X2GeAXsh_ukEC7SeVR_U0VlLveqVb7qOrjrO0cK76U=.8db41eba-86fd-4943-b403-283b467f0392@github.com>

On Mon, 13 May 2024 14:47:18 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I don't feel like starting that particular bike shedding discussion :) But sure, sometime in the future we should do this. Here, I want it to be a simple renaming change.
>
> Right. That's why I prefixed this with "Open-ended comment/question", trying to make it super clear that it wasn't intended as a request for this PR, but rather a way to at least plant the seed of an idea that we might want to fix this eyesore.

I agree with you on the eyesore. 

MEMFLAGS does not follow any established convention, the implied plural is strange (its just one flag, not a set of), etc. We will change it sometime in the future.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1599478327

From stuefe at openjdk.org  Tue May 14 07:19:32 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 14 May 2024 07:19:32 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v3]
In-Reply-To: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
Message-ID: <-CWxZnLMZoA9GeqM-hJ8m2d8-HWDQ7bVRhoWbf80MTE=.e77cd29b-70f9-4732-b04c-12a333bc559c@github.com>

> MEMFLAGS, as well as its enum constants, should live in its own include. 
> 
> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
> 
> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
> 
> ---
> 
> This patch moves the enum to its new file.
> 
> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
> 
> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
> 
> I tested (built) on:
> - MacOS aarch64, no precompiled headers, fastdebug
> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal

Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:

  Feedback StefanK

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19172/files
  - new: https://git.openjdk.org/jdk/pull/19172/files/42361558..2fc98923

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19172&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19172&range=01-02

  Stats: 41 lines in 4 files changed: 7 ins; 34 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/19172.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19172/head:pull/19172

PR: https://git.openjdk.org/jdk/pull/19172

From stuefe at openjdk.org  Tue May 14 07:19:32 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 14 May 2024 07:19:32 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v2]
In-Reply-To: <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <M4GfPaZ2ryq_1Q3rzFl_sWNYRWrLYQyAkkLdGGRLxq0=.1e17a602-ca47-4e49-a309-ad332e001a12@github.com>
 <eGF28oR9hMVbb2fvwFta6H84VKJePxslaq8EMwzsMFs=.3aa42eb7-df12-4cee-be52-0da6a7cbed5d@github.com>
Message-ID: <eVjoZyMfvBKh-F0HGfVbqJWBCw2BLomIu9kgVUhjreM=.6f81a3cc-0cc4-4966-b9ef-ceae6c319ac6@github.com>

On Mon, 13 May 2024 10:17:57 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Update mallocLimit.hpp
>
> Changes requested by stefank (Reviewer).

@stefank New version, hopefully addressed all your remarks. Thanks!

> src/hotspot/share/nmt/memflags.cpp line 27:
> 
>> 25: #include "precompiled.hpp"
>> 26: 
>> 27: #include "nmt/memflags.hpp"
> 
> There should be no blankline between precompiled.hpp and the rest of the includes.

I removed the file.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19172#issuecomment-2109454676
PR Review Comment: https://git.openjdk.org/jdk/pull/19172#discussion_r1599498716

From rkennke at openjdk.org  Tue May 14 12:11:19 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 14 May 2024 12:11:19 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
Message-ID: <bGm-pmXigx90v53nKJfQ6j7gk4PS3jDHGu1e_fCRa4E=.d249eb5e-7711-4c94-b93d-cd9cbbf1a087@github.com>

On Mon, 13 May 2024 21:31:40 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix copyright notice on test file

Changes look ok now, but you've got an assert failure on linux-x64 in GHA that needs to be fixed:
TEST: gc/shenandoah/oom/TestThreadFailure.java
#  Internal Error (/home/runner/work/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp:533), pid=23127, tid=23231
#  assert(beg_off >= leftmost_empty(ShenandoahFreeSetPartitionId::Mutator)) failed: free empty regions before the leftmost: 119, bound 128

-------------

Changes requested by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-2055159959

From stefank at openjdk.org  Tue May 14 14:50:04 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 14 May 2024 14:50:04 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v3]
In-Reply-To: <-CWxZnLMZoA9GeqM-hJ8m2d8-HWDQ7bVRhoWbf80MTE=.e77cd29b-70f9-4732-b04c-12a333bc559c@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <-CWxZnLMZoA9GeqM-hJ8m2d8-HWDQ7bVRhoWbf80MTE=.e77cd29b-70f9-4732-b04c-12a333bc559c@github.com>
Message-ID: <E8NwDd-AJud5fFfsVVE_A_zUOdpUdIipqFRQybZ9qhI=.c0239d49-a220-4a83-a0f4-9973f4d47c0e@github.com>

On Tue, 14 May 2024 07:19:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> MEMFLAGS, as well as its enum constants, should live in its own include. 
>> 
>> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
>> 
>> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
>> 
>> ---
>> 
>> This patch moves the enum to its new file.
>> 
>> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
>> 
>> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
>> 
>> I tested (built) on:
>> - MacOS aarch64, no precompiled headers, fastdebug
>> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Feedback StefanK

Looks good. Thanks!

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19172#pullrequestreview-2055637663

From stuefe at openjdk.org  Tue May 14 15:02:11 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 14 May 2024 15:02:11 GMT
Subject: RFR: 8332042: Move MEMFLAGS to its own include file [v3]
In-Reply-To: <-CWxZnLMZoA9GeqM-hJ8m2d8-HWDQ7bVRhoWbf80MTE=.e77cd29b-70f9-4732-b04c-12a333bc559c@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
 <-CWxZnLMZoA9GeqM-hJ8m2d8-HWDQ7bVRhoWbf80MTE=.e77cd29b-70f9-4732-b04c-12a333bc559c@github.com>
Message-ID: <oy-dhemCFJqtxOvyLdWWIAaVoTnvVwG2PzebbMrxaOY=.627c49e8-6fa2-486b-842d-bb8c17069dd2@github.com>

On Tue, 14 May 2024 07:19:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> MEMFLAGS, as well as its enum constants, should live in its own include. 
>> 
>> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
>> 
>> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
>> 
>> ---
>> 
>> This patch moves the enum to its new file.
>> 
>> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
>> 
>> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
>> 
>> I tested (built) on:
>> - MacOS aarch64, no precompiled headers, fastdebug
>> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Feedback StefanK

Thanks @afshin-zafari @stefank @kimbarrett @jdksjolen

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19172#issuecomment-2110464347

From stuefe at openjdk.org  Tue May 14 15:02:12 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 14 May 2024 15:02:12 GMT
Subject: Integrated: 8332042: Move MEMFLAGS to its own include file
In-Reply-To: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
References: <ucUpx3JpyNGPTvW-sveZpHOE_XaMDrh8ZJSPsdgho_0=.c2c380a7-35df-481a-9d61-b9755716aa1f@github.com>
Message-ID: <q1DYiUx5rALP0EYqbta8TUw1L8yspfzXI4zvtKOHUxQ=.e32dd292-ae8a-4f54-afe2-00a80e00ed59@github.com>

On Fri, 10 May 2024 09:06:08 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> MEMFLAGS, as well as its enum constants, should live in its own include. 
> 
> The constants are used throughout the code base, often without needing the allocation APIs exposed through allocation.hpp.
> 
> The MEMFLAGS enum def is often needed within NMT itself, again often without needing allocation.hpp.
> 
> ---
> 
> This patch moves the enum to its new file.
> 
> It fixes those `allocation.hpp` includes that where only needed to get MEMFLAGS. It does not fix other includes. 
> 
> For backward compatibility, until we straightened out the dependencies (e.g., fixing all places where we rely on indirect includes), I added memflags.hpp to allocation.hpp.
> 
> I tested (built) on:
> - MacOS aarch64, no precompiled headers, fastdebug
> - Linux x64, no precompiled headers, fastdebug, release, fastdebug crossbuild to aarch64, fastdebug minimal

This pull request has now been integrated.

Changeset: 95a60131
Author:    Thomas Stuefe <stuefe at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/95a601316de06b4b0fbf6e3c7777be5d2a1ca978
Stats:     201 lines in 25 files changed: 99 ins; 66 del; 36 mod

8332042: Move MEMFLAGS to its own include file

Reviewed-by: jsjolen, stefank

-------------

PR: https://git.openjdk.org/jdk/pull/19172

From shade at openjdk.org  Tue May 14 18:24:23 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 14 May 2024 18:24:23 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
Message-ID: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>

As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.

This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 

After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.

Additional testing:
 - [x] Performance test reproducer from the bug improves significantly
 - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
 - [ ] Linux AArch64 server fastdebug, `all`
 - [x] Linux x86_64 server fastdebug, `all`

-------------

Commit messages:
 - Avoid double handle-izing on GC critical path
 - Move critical section to a closer scope
 - Comments
 - Fix

Changes: https://git.openjdk.org/jdk/pull/19229/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19229&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331572
  Stats: 80 lines in 8 files changed: 42 ins; 13 del; 25 mod
  Patch: https://git.openjdk.org/jdk/pull/19229.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19229/head:pull/19229

PR: https://git.openjdk.org/jdk/pull/19229

From shade at openjdk.org  Tue May 14 18:24:23 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 14 May 2024 18:24:23 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
In-Reply-To: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
Message-ID: <IR-NL_ZBogPDjW3hAfV_ZN-mXFY25oUzln1lx2r5U34=.fd4dfa25-b9a3-449e-8a26-83c6801d6d32@github.com>

On Tue, 14 May 2024 12:31:08 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
> 
> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
> 
> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
> 
> Additional testing:
>  - [x] Performance test reproducer from the bug improves significantly
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>  - [ ] Linux AArch64 server fastdebug, `all`
>  - [x] Linux x86_64 server fastdebug, `all`

Performance note: there is an intrinsic tradeoff here between the cost of acquiring the critical section vs the concurrency it unblocks for non-STW GCs and the cache improvements on non-GC paths. The critical section overhead is mostly due to the fence in https://github.com/openjdk/jdk/blob/5a4415a6bddb25cbd5b87ff8ad1a06179c2e452e/src/hotspot/share/utilities/globalCounter.inline.hpp#L43

So, the original reproducer (very stressy, with lots of interpreter frames) improves dramatically (73 -> 6ms) with Shenandoah GC, but run with Serial GC reveals there is a slight regression in GC times (74 -> 79 ms). I have not been able to replicate this regression in larger benchmarks.

Anyhow, this very fine-grained regression nearly disappears (74.1 -> 74.3 ms on Serial) if we optimize the other part of this whole path a bit, done in this PR: https://github.com/openjdk/jdk/pull/19229/commits/455687addeba55dc998dbf9ab4b8ec58f0b69ee4. This also improves Shenandoah times further (6.1 -> 5.6 ms).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19229#issuecomment-2110429057

From wkemper at openjdk.org  Tue May 14 21:16:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 14 May 2024 21:16:08 GMT
Subject: RFR: 8332255: Shenandoah: Remove duplicate definition of init mark
 closure
Message-ID: <F4hfXtnntZ8dWpJCAT65avQJqqoMtasIiyzPI9nZxl8=.1c65a32e-a502-4168-9403-d60fb13cf119@github.com>

8332255: Shenandoah: Remove duplicate definition of init mark closure

-------------

Commit messages:
 - Remove duplicate definition of init mark closure

Changes: https://git.openjdk.org/jdk/pull/19238/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19238&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332255
  Stats: 23 lines in 1 file changed: 0 ins; 23 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/19238.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19238/head:pull/19238

PR: https://git.openjdk.org/jdk/pull/19238

From wkemper at openjdk.org  Tue May 14 21:19:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 14 May 2024 21:19:54 GMT
Subject: RFR: 8332254: GenShen: Remove obsolete comments and revert unnecessary
 formatting changes
Message-ID: <lhri2IuadW78hJVRcfpjSk-CmaBkUY4Bw08EORguEY4=.06fd8d33-d21e-47ef-bfd9-239b1783ae39@github.com>

No code changes here.

-------------

Commit messages:
 - Remove out dated comments and needless changes to other comments

Changes: https://git.openjdk.org/shenandoah/pull/433/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=433&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332254
  Stats: 11 lines in 1 file changed: 1 ins; 5 del; 5 mod
  Patch: https://git.openjdk.org/shenandoah/pull/433.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/433/head:pull/433

PR: https://git.openjdk.org/shenandoah/pull/433

From wkemper at openjdk.org  Tue May 14 21:34:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 14 May 2024 21:34:11 GMT
Subject: RFR: 8332256: Shenandoah: Do not visit heap threads during shutdown
Message-ID: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>

In rare cases, this may lead to a crash or hang during shutdown.

-------------

Commit messages:
 - Do not visit heap threads during shutdown

Changes: https://git.openjdk.org/jdk/pull/19239/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19239&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332256
  Stats: 9 lines in 1 file changed: 7 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/19239.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19239/head:pull/19239

PR: https://git.openjdk.org/jdk/pull/19239

From wkemper at openjdk.org  Tue May 14 23:59:09 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 14 May 2024 23:59:09 GMT
Subject: RFR: 8332257: Shenandoah: Move evacuation methods to implementation
 file
Message-ID: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>

This refactoring moves a large, complex function out of an include header. This improves build times and facilitates changes in the generational mode branch.

-------------

Commit messages:
 - Do not inline evacuate_object

Changes: https://git.openjdk.org/jdk/pull/19240/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19240&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332257
  Stats: 155 lines in 3 files changed: 77 ins; 77 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/19240.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19240/head:pull/19240

PR: https://git.openjdk.org/jdk/pull/19240

From kdnilsen at openjdk.org  Wed May 15 01:49:38 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 01:49:38 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v60]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <7DbmIDx8kSa-oU8jDpx4zgYEJOYmEUyOB7bK-mWDOg8=.f4a38759-0c7f-482e-83d0-59a6aa32b74e@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Directly address range arrays when adjusting intervals
  
  Using the accessor methods may cause misbehavior if an accessor method
  falsely believes a range is empty because the ranges have not yet been
  adjusted to represent regions that were newly inserted into a particular
  partition.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/572aa2b3..3ed62eb7

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=59
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=58-59

  Stats: 9 lines in 1 file changed: 0 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From shade at openjdk.org  Wed May 15 08:45:05 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 15 May 2024 08:45:05 GMT
Subject: RFR: 8332256: Shenandoah: Do not visit heap threads during
 shutdown
In-Reply-To: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
References: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
Message-ID: <gkuAx78R5m5SezVzLcHp_yJQBEi1vFGhecyNtHYB2Q8=.b30c8ae5-b8a2-4fce-bba8-a1ed2173648a@github.com>

On Tue, 14 May 2024 21:29:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> In rare cases, this may lead to a crash or hang during shutdown.

This makes sense. 

Do we have any sample how a crash would look like? Would be nice to attach it to the bug.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19239#pullrequestreview-2057315210

From shade at openjdk.org  Wed May 15 08:48:04 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 15 May 2024 08:48:04 GMT
Subject: RFR: 8332255: Shenandoah: Remove duplicate definition of init mark
 closure
In-Reply-To: <F4hfXtnntZ8dWpJCAT65avQJqqoMtasIiyzPI9nZxl8=.1c65a32e-a502-4168-9403-d60fb13cf119@github.com>
References: <F4hfXtnntZ8dWpJCAT65avQJqqoMtasIiyzPI9nZxl8=.1c65a32e-a502-4168-9403-d60fb13cf119@github.com>
Message-ID: <OoMJT2yPfvifsuv7EnmgH-HBXLnKDuJ3T7jUOjm15lQ=.b233594a-74c1-4c45-942a-78144f22142f@github.com>

On Tue, 14 May 2024 21:10:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> 8332255: Shenandoah: Remove duplicate definition of init mark closure

Oh, gotta love the C++ flexibility. Nice catch, looks fine.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19238#pullrequestreview-2057322312

From shade at openjdk.org  Wed May 15 08:51:01 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 15 May 2024 08:51:01 GMT
Subject: RFR: 8332257: Shenandoah: Move evacuation methods to
 implementation file
In-Reply-To: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
References: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
Message-ID: <d2vA01fttSTTzhZ3twVv__O49QfCOuyYOkOdmgEwUA8=.46e2f60c-0052-4d5f-80ec-709bf4541e8e@github.com>

On Tue, 14 May 2024 23:54:15 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This refactoring moves a large, complex function out of an include header. This improves build times and facilitates changes in the generational mode branch.

All right, as long as performance data does not show any regressions in evac times.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19240#pullrequestreview-2057329498

From shade at openjdk.org  Wed May 15 09:13:07 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 15 May 2024 09:13:07 GMT
Subject: RFR: 8332082: Shenandoah: Use SATB active flag for C2 pre-write
 barrier on x86 and PPC
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <hBuLGPZaSbrnKljxb60ZFSIb69uLrmkhSLyI4iK3nJg=.02081603-ebf7-47b6-9232-1f88b77a24f3@github.com>

On Fri, 10 May 2024 16:13:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is consistent with c1 and other platforms.

Hold on. Here is the C2 SATB barrier check, which checks gc-state for `MARKING`:
https://github.com/openjdk/jdk/blob/2f10a316ff0c5a4c124b94f6fabb38fb119d2c82/src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp#L241-L243

What this PR changes is `ShenandoahBarrierSetAssembler::satb_write_barrier_impl`, which is the SATB barrier for generic assembly code. Yes, some of that may be reached from C2.

Looking around, I see that C1 AArch64, RISC-V, x86 `ShenandoahBarrierSetAssembler::generate_c1_pre_barrier_runtime_stub`-s use `gc_state == MARKING` too:
https://github.com/openjdk/jdk/blob/2f10a316ff0c5a4c124b94f6fabb38fb119d2c82/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp#L695-L698
https://github.com/openjdk/jdk/blob/2f10a316ff0c5a4c124b94f6fabb38fb119d2c82/src/hotspot/cpu/riscv/gc/shenandoah/shenandoahBarrierSetAssembler_riscv.cpp#L642-L646
https://github.com/openjdk/jdk/blob/2f10a316ff0c5a4c124b94f6fabb38fb119d2c82/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp#L226-L228

It looks to me the current status is:

* Assembler:
  - AArch64, RISC-V: SATBMarkQueue.isActive
  - x86, PPC: gc_state == MARKING
* C1 (IR):
  - All platforms: SATBMarkQueue.isActive
* C1 (assembler stub):
  - All platforms: gc_state == MARKING
* C2 (IR):
  - All platforms: gc-state == MARKING

So, are we better off going the other way around, towards gc-state == MARKING? This would "only" need to rewrite AArch64, RISC-V parts in assembler, and shared C1 barrier part.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19180#issuecomment-2111971770

From kdnilsen at openjdk.org  Wed May 15 13:55:19 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 13:55:19 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <08-005vvbDWvi6V9Jyjz2sTCq0ryu1E6XifnXC-6uM4=.5d91b5bf-06a2-4359-96e9-95ad7d5676cc@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
 <08-005vvbDWvi6V9Jyjz2sTCq0ryu1E6XifnXC-6uM4=.5d91b5bf-06a2-4359-96e9-95ad7d5676cc@github.com>
Message-ID: <vxehQFTtkOQmF1O8ydKGfemSxBHoZK6w8Nqc-jUSiZk=.147cc5b4-0998-4cf6-a399-5613b714e28a@github.com>

On Tue, 14 May 2024 00:03:47 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix copyright notice on test file
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 105:
> 
>> 103:   ShenandoahHeapRegion* r = _heap->get_region(idx);
>> 104:   return can_allocate_from(r);
>> 105: }
> 
> Some of this abstraction leakage can be avoided by making
> 
> "can_allocate_from(size_t idx)" a method on the `_heap` object, and calling that, letting the heap do the test:
> 
> ``` 
> bool ShenandoahHeap::can_allocate_from(size_t idx) {
>   ShenandoahHeapRegion* r = get_region(idx);
>   return r->is_empty() || (r->is_trash() && !is_concurrent_weak_root_in_progress());
> }
> 
> 
> with ShFreeSet merely delegating to its `_heap`.
> 
> Probably similar pattern for `alloc_capacity` method below.
> 
> (I'll keep leaving single comments, so I don't slow down the review/feedback across potential interruptions. I don't expect there will be many, but still.)

I'm not sure I understand the rationale for these changes.  In my mind, the "notions" of can_allocate_from() and alloc_capacity() as used here within ShenandoahFreeSet are not universal.  These notions are specific to what the free set understands about the life-cycle of regions.  I think moving this functionality into heap actually leaks more encapsulation, in that nobody else cares to use these novel interpretations of can_allocate_from() and alloc_capacity().

Maybe you can elaborate?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1601696376

From kdnilsen at openjdk.org  Wed May 15 14:27:46 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:27:46 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v61]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <pcce6ChOUO6vhVF6X8hx_i1UCS01EOWd7SlXSSrxN2U=.da25c6db-7660-4b76-bcf8-5e58ebeb8622@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Improvements to dump_bitmap_* services

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/3ed62eb7..fe76f493

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=60
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=59-60

  Stats: 16 lines in 2 files changed: 3 ins; 0 del; 13 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Wed May 15 14:27:46 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:27:46 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <TITws1EOWa6EKiu-HXXLdgNLRsE9eh4YUkmLMHMCyms=.9649906c-97a4-4b85-92f3-3a12090288d2@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
 <TITws1EOWa6EKiu-HXXLdgNLRsE9eh4YUkmLMHMCyms=.9649906c-97a4-4b85-92f3-3a12090288d2@github.com>
Message-ID: <2-1OT2h2Te8a-t96pdBd5Vl-R5nn6jau9a1wdATwFLU=.bf3764a7-fd3b-4e08-9f26-fad3094bb061@github.com>

On Tue, 14 May 2024 01:08:01 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix copyright notice on test file
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 93:
> 
>> 91:   void dump_bitmap_row(ssize_t idx) const;
>> 92:   void dump_bitmap_range(ssize_t start_idx, ssize_t end_idx) const;
>> 93:   void dump_bitmap_all() const;
> 
> In the first case you can may be say `region_idx` instead of `idx` ?
> 
> Also, it seems `dump_bitmap()` would be sufficient (at least for the case of `dump_bitmap_all()`, although in general for all).
> 
> However, given that these are private methods that will only be found in the implementation and nowhere else in its clients' code, may be current naming is ok (indeed, it looks like the first two are work functions in service of the last).
> 
> PS: I don't see any current usage of `dump_bitmap_all()`. May be you wanted it in some debug tracing code or something, or keeping it for possible future debugging usage? (Or may be my naive web-page search of this review page is naive/faulty.) It's perhaps useful to keep around for debug/tracing though.

Thanks.  Making these changes to argument names, and conditionalizing this code on #ifndef PRODUCT

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1601749296

From kdnilsen at openjdk.org  Wed May 15 14:31:47 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:31:47 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v62]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <b_j4rM7uUn07TBClIAOVhA18iuiMXnUf7LOAoClMb5A=.35107a63-bf72-47cc-bd69-888ce9be0753@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Rename raw_assign_membership()

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/fe76f493..93761ed9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=61
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=60-61

  Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Wed May 15 14:31:48 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:31:48 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <TITws1EOWa6EKiu-HXXLdgNLRsE9eh4YUkmLMHMCyms=.9649906c-97a4-4b85-92f3-3a12090288d2@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
 <TITws1EOWa6EKiu-HXXLdgNLRsE9eh4YUkmLMHMCyms=.9649906c-97a4-4b85-92f3-3a12090288d2@github.com>
Message-ID: <uEDyBCY8GdsIbmYsS3FsVBTHXZgi8xtOeOUX2gHwS40=.dc5d8ad9-4ddb-481a-b9e5-c1ea8c5b941b@github.com>

On Tue, 14 May 2024 01:08:19 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix copyright notice on test file
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 103:
> 
>> 101: 
>> 102:   // Set the partition id for a particular region without adjusting interval bounds or usage/capacity tallies
>> 103:   inline void raw_set_membership(size_t idx, ShenandoahFreeSetPartitionId p) {
> 
> I realize that the word `set` can be a little ambiguous here (from the mathematical notion of a set as  a collection of things), although the fact that it has a void return type tells us that `set` is the imperative verb here, not a collection of things. May be `raw_assign_membership(size_t region_idx, ShFSPId p)` if you want to, but I may be splitting hairs here :-)

Thanks.  Making this change.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1601756334

From kdnilsen at openjdk.org  Wed May 15 14:39:47 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:39:47 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <D7sXU86YZIqa5-pFTLE_OTwOFctb0DAR8SqPtJFu-VQ=.c9257ae8-bb80-4bf5-aaf9-ea3506426e0c@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
 <D7sXU86YZIqa5-pFTLE_OTwOFctb0DAR8SqPtJFu-VQ=.c9257ae8-bb80-4bf5-aaf9-ea3506426e0c@github.com>
Message-ID: <oxMhwA7kLEVvCy1qRacR6bEZ1oN5t06wA1mdRz0pWJ0=.d5258f7c-1d12-4be9-9bf6-cc84bf73ceb4@github.com>

On Tue, 14 May 2024 01:17:03 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix copyright notice on test file
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 109:
> 
>> 107:   // Set the Mutator intervals, usage, and capacity according to arguments.  Reset the Collector intervals, used, capacity
>> 108:   // to represent empty Collector free set.
>> 109:   void establish_intervals(ssize_t mutator_leftmost, ssize_t mutator_rightmost,
> 
> Why not clarify in name via `establish_mutator_intervals` ? (I noted there isn't a corresponding method for "collector intervals", although there are corresponding fields for its left and rightmosts etc.) A documentation comment somewhere indicating how/when those get established would be useful. I see them being incrementally updated in the incremental manipulation of set memberships, but wanted to get a sense of the reasons for this asymmetry (and for that reason to be documented when we revisit this code many moons in the future).

Thanks.  Making this change and adding a comment to motivate this "special" method for mutator intervals.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1601764093

From kdnilsen at openjdk.org  Wed May 15 14:39:46 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:39:46 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v63]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <ey1Qz9KUm4z0MD4KG4THlTwWCaeaXV1401rK-M-Y5ms=.2f08abff-21fa-4383-80e1-6e6334b66691@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Rename establish_mutator_intervals

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/93761ed9..32191190

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=62
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=61-62

  Stats: 10 lines in 2 files changed: 1 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Wed May 15 14:56:06 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:56:06 GMT
Subject: RFR: 8332255: Shenandoah: Remove duplicate definition of init mark
 closure
In-Reply-To: <F4hfXtnntZ8dWpJCAT65avQJqqoMtasIiyzPI9nZxl8=.1c65a32e-a502-4168-9403-d60fb13cf119@github.com>
References: <F4hfXtnntZ8dWpJCAT65avQJqqoMtasIiyzPI9nZxl8=.1c65a32e-a502-4168-9403-d60fb13cf119@github.com>
Message-ID: <R1Q0wWvZaJkMI6z397zQ5SmhxkDJmlBBp8BGQiSgqNw=.6dd5d50b-b5e0-4871-9cf6-dd70545d497b@github.com>

On Tue, 14 May 2024 21:10:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> 8332255: Shenandoah: Remove duplicate definition of init mark closure

Marked as reviewed by kdnilsen (no project role).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19238#pullrequestreview-2058272453

From kdnilsen at openjdk.org  Wed May 15 14:57:17 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:57:17 GMT
Subject: RFR: 8332256: Shenandoah: Do not visit heap threads during
 shutdown
In-Reply-To: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
References: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
Message-ID: <LLu3stekFkwbFPHsD8Kgi4irpA-fUQDwspOOQZP8OaI=.facb0ffc-cd47-480a-9afd-e830a52e111a@github.com>

On Tue, 14 May 2024 21:29:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> In rare cases, this may lead to a crash or hang during shutdown.

Marked as reviewed by kdnilsen (no project role).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19239#pullrequestreview-2058275787

From kdnilsen at openjdk.org  Wed May 15 14:59:07 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:59:07 GMT
Subject: RFR: 8332257: Shenandoah: Move evacuation methods to
 implementation file
In-Reply-To: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
References: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
Message-ID: <Ezbs9aKYyPr5hbYpHJLn4Ly-xOagHhu0yIIUMD68Z4Q=.3219cb94-1b9f-4495-b4ca-c93abf4a3bf8@github.com>

On Tue, 14 May 2024 23:54:15 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This refactoring moves a large, complex function out of an include header. This improves build times and facilitates changes in the generational mode branch.

Marked as reviewed by kdnilsen (no project role).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19240#pullrequestreview-2058279171

From kdnilsen at openjdk.org  Wed May 15 14:59:48 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 15 May 2024 14:59:48 GMT
Subject: RFR: 8332254: GenShen: Remove obsolete comments and revert
 unnecessary formatting changes
In-Reply-To: <lhri2IuadW78hJVRcfpjSk-CmaBkUY4Bw08EORguEY4=.06fd8d33-d21e-47ef-bfd9-239b1783ae39@github.com>
References: <lhri2IuadW78hJVRcfpjSk-CmaBkUY4Bw08EORguEY4=.06fd8d33-d21e-47ef-bfd9-239b1783ae39@github.com>
Message-ID: <ICNeZ6bg4GoO-2JrEKkDVh5721QSnKjkJVL5PR5qpMg=.2272afa4-1f2b-47cc-b49b-dd508f321ed9@github.com>

On Tue, 14 May 2024 21:14:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

> No code changes here.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/433#pullrequestreview-2058282279

From wkemper at openjdk.org  Wed May 15 16:43:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 16:43:03 GMT
Subject: RFR: 8332082: Shenandoah: Use SATB active flag for C2 pre-write
 barrier on x86 and PPC
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <OKKK5x8Iran9J-hvYGkiXO2jCM_mMZBVBzxUlNhDm00=.c6d19d94-3cbb-4333-9e36-9db175820198@github.com>

On Fri, 10 May 2024 16:13:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is consistent with c1 and other platforms.

`ShenandoahBarrierSetC2::verify_gc_barriers` is also looking for IR node pattern for `SATBMarkQueue.isActive`, so it'll need rework too.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19180#issuecomment-2113004499

From wkemper at openjdk.org  Wed May 15 16:44:19 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 16:44:19 GMT
Subject: Integrated: 8332254: GenShen: Remove obsolete comments and revert
 unnecessary formatting changes
In-Reply-To: <lhri2IuadW78hJVRcfpjSk-CmaBkUY4Bw08EORguEY4=.06fd8d33-d21e-47ef-bfd9-239b1783ae39@github.com>
References: <lhri2IuadW78hJVRcfpjSk-CmaBkUY4Bw08EORguEY4=.06fd8d33-d21e-47ef-bfd9-239b1783ae39@github.com>
Message-ID: <Yhlddp8bOl5i55kCjAeI7IjovhXPhAI16TLuy5T1vD4=.8e1f2284-97ef-4176-a012-cca68d2ae7c5@github.com>

On Tue, 14 May 2024 21:14:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

> No code changes here.

This pull request has now been integrated.

Changeset: dfa78866
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/dfa788667714b4ec7e0e235e9c0c58cc5e3ceab4
Stats:     11 lines in 1 file changed: 1 ins; 5 del; 5 mod

8332254: GenShen: Remove obsolete comments and revert unnecessary formatting changes

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/433

From wkemper at openjdk.org  Wed May 15 16:45:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 16:45:06 GMT
Subject: Integrated: 8332255: Shenandoah: Remove duplicate definition of init
 mark closure
In-Reply-To: <F4hfXtnntZ8dWpJCAT65avQJqqoMtasIiyzPI9nZxl8=.1c65a32e-a502-4168-9403-d60fb13cf119@github.com>
References: <F4hfXtnntZ8dWpJCAT65avQJqqoMtasIiyzPI9nZxl8=.1c65a32e-a502-4168-9403-d60fb13cf119@github.com>
Message-ID: <smTC6BaWVttwLjPScd-Z-3dGRSabLcs7VhYHAOHNaGY=.de9c25fe-c0b5-4b90-b20f-344484d535d4@github.com>

On Tue, 14 May 2024 21:10:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> 8332255: Shenandoah: Remove duplicate definition of init mark closure

This pull request has now been integrated.

Changeset: 9c02c8dd
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/9c02c8dd71023df6338cb94997bca6b00768af6f
Stats:     23 lines in 1 file changed: 0 ins; 23 del; 0 mod

8332255: Shenandoah: Remove duplicate definition of init mark closure

Reviewed-by: shade, kdnilsen

-------------

PR: https://git.openjdk.org/jdk/pull/19238

From wkemper at openjdk.org  Wed May 15 16:56:09 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 16:56:09 GMT
Subject: RFR: 8332256: Shenandoah: Do not visit heap threads during
 shutdown
In-Reply-To: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
References: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
Message-ID: <S3yBzUV_vQJAFeCod4FdPm_6cr7OD-q_tMZaUoa6SWc=.e103fb7c-7f1d-4eff-adc7-7b7655224bc9@github.com>

On Tue, 14 May 2024 21:29:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> In rare cases, this may lead to a crash or hang during shutdown.

Alas, we made this change more than a year ago in the shenandoah repo when we were less disciplined about tickets and other artifacts. I looked for discussions of this change, but couldn't find anything.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19239#issuecomment-2113025391

From wkemper at openjdk.org  Wed May 15 16:56:10 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 16:56:10 GMT
Subject: Integrated: 8332256: Shenandoah: Do not visit heap threads during
 shutdown
In-Reply-To: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
References: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
Message-ID: <MnHaViuC-xxIBHpjWn125I2HnXTl_ofYcE3LZHjJgmU=.7bf0a5fd-e46e-44a6-a4d1-30e69bdac33e@github.com>

On Tue, 14 May 2024 21:29:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> In rare cases, this may lead to a crash or hang during shutdown.

This pull request has now been integrated.

Changeset: 491b3b45
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/491b3b45634fffb0101244f7d491a1681e7e8002
Stats:     9 lines in 1 file changed: 7 ins; 0 del; 2 mod

8332256: Shenandoah: Do not visit heap threads during shutdown

Reviewed-by: shade, kdnilsen

-------------

PR: https://git.openjdk.org/jdk/pull/19239

From ysr at openjdk.org  Wed May 15 17:11:07 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 15 May 2024 17:11:07 GMT
Subject: RFR: 8332256: Shenandoah: Do not visit heap threads during
 shutdown
In-Reply-To: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
References: <T5oJChEf-Gh1Cn2iKA4dFTE9BME3Kbo29-ZkQtSpaSE=.8868503e-3275-43b4-b286-aaa1601f8d87@github.com>
Message-ID: <Q3JfcH_TA2Jc6UTsP9fvIQj68CYgKZA61sAkyQgIE4k=.414e3ade-81d9-478e-9599-ba4eed8b04ac@github.com>

On Tue, 14 May 2024 21:29:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> In rare cases, this may lead to a crash or hang during shutdown.

I agree that what you did makes intuitive sense.

> 
> Alas, we made this change more than a year ago in the shenandoah repo when we were less disciplined about tickets and other artifacts. I looked for discussions of this change, but couldn't find anything.

What do other GCs do?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19239#issuecomment-2113050210

From wkemper at openjdk.org  Wed May 15 18:39:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 18:39:40 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <Royn_2WG7ByIGOjOVabugyyeG3T8MKtYo96aIuTy8qk=.b3f17e9c-60c8-464d-9ab3-de990eaab87d@github.com>
References: <Royn_2WG7ByIGOjOVabugyyeG3T8MKtYo96aIuTy8qk=.b3f17e9c-60c8-464d-9ab3-de990eaab87d@github.com>
Message-ID: <n-AoTYUpVVYnSf7wce8Y_OaNeaQHXemanH299bF1xQE=.52f5b90c-8979-4f9b-8eb5-3d1054e7a7e9@github.com>

> Merges tag jdk-21.0.4+2

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/43/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/43/files/93d091ad..93d091ad

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=43&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=43&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/43.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/43/head:pull/43

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/43

From wkemper at openjdk.org  Wed May 15 18:39:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 18:39:43 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <Royn_2WG7ByIGOjOVabugyyeG3T8MKtYo96aIuTy8qk=.b3f17e9c-60c8-464d-9ab3-de990eaab87d@github.com>
References: <Royn_2WG7ByIGOjOVabugyyeG3T8MKtYo96aIuTy8qk=.b3f17e9c-60c8-464d-9ab3-de990eaab87d@github.com>
Message-ID: <9biI4Yg-wBPqc6PatYzMpgpCy9sOZADYZb_pS6GuSuw=.e70d0bc4-db01-43d6-8e12-0e0ab1f9b600@github.com>

On Thu, 9 May 2024 14:16:31 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.4+2

This pull request has now been integrated.

Changeset: dca6bd5e
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/dca6bd5e31fd806ff589842cb4884d0368e10f43
Stats:     2979 lines in 80 files changed: 1993 ins; 795 del; 191 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/43

From coleenp at openjdk.org  Wed May 15 20:08:04 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Wed, 15 May 2024 20:08:04 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
In-Reply-To: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
Message-ID: <0FHsLntrHofCG7x31n4Worx5TdfoBZ7jGCTkDqJJU8M=.2e89548b-c70a-43e2-a91e-1a80f954188d@github.com>

On Tue, 14 May 2024 12:31:08 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
> 
> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
> 
> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
> 
> Additional testing:
>  - [x] Performance test reproducer from the bug improves significantly
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>  - [x] Linux AArch64 server fastdebug, `all`
>  - [x] Linux x86_64 server fastdebug, `all`

This looks good but one question for ZGC, does ZGC need an OopMapCache::cleanup_old_entries() ?

-------------

Marked as reviewed by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19229#pullrequestreview-2058904373

From coleenp at openjdk.org  Wed May 15 20:11:02 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Wed, 15 May 2024 20:11:02 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
In-Reply-To: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
Message-ID: <pdwUxXXSbzrkCFQnVDWm9iFuISnuW-GmUsij3S3yLN8=.f6546d6a-150b-4646-b13b-1f104cb75abe@github.com>

On Tue, 14 May 2024 12:31:08 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
> 
> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
> 
> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
> 
> Additional testing:
>  - [x] Performance test reproducer from the bug improves significantly
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>  - [x] Linux AArch64 server fastdebug, `all`
>  - [x] Linux x86_64 server fastdebug, `all`

I did have questions (sorry hit approve too soon).

src/hotspot/share/interpreter/oopMapCache.cpp line 545:

> 543: 
> 544:   // First search for an empty slot
> 545:   for (int i = 0; i < _probe_depth; i++) {

Does the GlobalCounter read barrier belong around this too?

src/hotspot/share/interpreter/oopMapCache.cpp line 593:

> 591: bool OopMapCache::has_cleanup_work() {
> 592:   return Atomic::load(&_old_entries) != nullptr;
> 593: }

Does this need to notify the ServiceThread?  Since the ServiceThread is now a timed wait, maybe this is fine.

-------------

Changes requested by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19229#pullrequestreview-2058906401
PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1602184993
PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1602186516

From zgu at openjdk.org  Wed May 15 22:05:01 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Wed, 15 May 2024 22:05:01 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
In-Reply-To: <0FHsLntrHofCG7x31n4Worx5TdfoBZ7jGCTkDqJJU8M=.2e89548b-c70a-43e2-a91e-1a80f954188d@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <0FHsLntrHofCG7x31n4Worx5TdfoBZ7jGCTkDqJJU8M=.2e89548b-c70a-43e2-a91e-1a80f954188d@github.com>
Message-ID: <EJMB-7cL_u-ftSBhR3h1qa9DVIzfoBV76SUVuNcLLs8=.c0573f72-2b57-4df4-9b66-c01f7858f383@github.com>

On Wed, 15 May 2024 20:05:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> This looks good but one question for ZGC, does ZGC need an OopMapCache::cleanup_old_entries() ?

We still call `OopMapCache::cleanup_old_entries() ` after STW pause, but now  concurrent phase also can accumulate old entries, should we unify them? e.g. all depend on service thread to clean up old entries?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19229#issuecomment-2113526362

From wkemper at openjdk.org  Wed May 15 23:10:49 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 23:10:49 GMT
Subject: RFR: 8332330: Shenandoah: Change jcheck configuration for fix version
 to match project repo
Message-ID: <PZHHRRFXXdXltI1Z2SL2zVVzKbj2w17HTetQWp18r5I=.3d397d90-a360-487e-9b02-5a0b92ffe2a3@github.com>

Per https://bugs.openjdk.org/browse/SKARA-2262, we need to configure the correct fixVersion in `.jcheck/conf` in this repo for the bot to interact properly with JBS.

-------------

Commit messages:
 - Change jcheck configuration for fix version to match project repo

Changes: https://git.openjdk.org/shenandoah/pull/434/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=434&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332330
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/434.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/434/head:pull/434

PR: https://git.openjdk.org/shenandoah/pull/434

From wkemper at openjdk.org  Wed May 15 23:20:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 15 May 2024 23:20:36 GMT
Subject: RFR: 8332331: Shenandoah: Change jcheck configuration for fix version
 to match project repo
Message-ID: <_2XrfBh6wTMVAPceovK4lPWaKNIq-D69iA7OE7wW8lc=.5359329f-4853-4447-aa91-7533a6ca77cd@github.com>

Per https://bugs.openjdk.org/browse/SKARA-2262, we need to configure the correct fixVersion in .jcheck/conf in this repo for the bot to interact properly with JBS.

-------------

Commit messages:
 - Change jcheck configuration for fix version to match project repo

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/44/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=44&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332331
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/44.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/44/head:pull/44

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/44

From ysr at openjdk.org  Thu May 16 07:15:21 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 16 May 2024 07:15:21 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v59]
In-Reply-To: <vxehQFTtkOQmF1O8ydKGfemSxBHoZK6w8Nqc-jUSiZk=.147cc5b4-0998-4cf6-a399-5613b714e28a@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Jhna34J5cBCwnymIcLLukPQ2egnTmAklol5bfm94RuI=.12c966b4-9752-42a8-9d1c-9d4281fe7fe6@github.com>
 <08-005vvbDWvi6V9Jyjz2sTCq0ryu1E6XifnXC-6uM4=.5d91b5bf-06a2-4359-96e9-95ad7d5676cc@github.com>
 <vxehQFTtkOQmF1O8ydKGfemSxBHoZK6w8Nqc-jUSiZk=.147cc5b4-0998-4cf6-a399-5613b714e28a@github.com>
Message-ID: <jWxvf_7PaWgqargIthz1gJ7kmpAZd9X_QzHwflcl8GY=.fa70c25c-10f4-4935-bdfa-c76a38f1de1c@github.com>

On Wed, 15 May 2024 13:51:54 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 105:
>> 
>>> 103:   ShenandoahHeapRegion* r = _heap->get_region(idx);
>>> 104:   return can_allocate_from(r);
>>> 105: }
>> 
>> Some of this abstraction leakage can be avoided by making
>> 
>> "can_allocate_from(size_t idx)" a method on the `_heap` object, and calling that, letting the heap do the test:
>> 
>> ``` 
>> bool ShenandoahHeap::can_allocate_from(size_t idx) {
>>   ShenandoahHeapRegion* r = get_region(idx);
>>   return r->is_empty() || (r->is_trash() && !is_concurrent_weak_root_in_progress());
>> }
>> 
>> 
>> with ShFreeSet merely delegating to its `_heap`.
>> 
>> Probably similar pattern for `alloc_capacity` method below.
>> 
>> (I'll keep leaving single comments, so I don't slow down the review/feedback across potential interruptions. I don't expect there will be many, but still.)
>
> I'm not sure I understand the rationale for these changes.  In my mind, the "notions" of can_allocate_from() and alloc_capacity() as used here within ShenandoahFreeSet are not universal.  These notions are specific to what the free set understands about the life-cycle of regions.  I think moving this functionality into heap actually leaks more encapsulation, in that nobody else cares to use these novel interpretations of can_allocate_from() and alloc_capacity().
> 
> Maybe you can elaborate?

I was looking at the structural effect of moving the methods there and failed to see that these were policies encoded in the free set abstraction rather than policies of (or characteristics of) ShenandoahHeap _used_ by the free set (as I was thinking).

No change is needed in that case. Sorry for my confusion.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1602741162

From shade at openjdk.org  Thu May 16 07:25:03 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 16 May 2024 07:25:03 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
In-Reply-To: <pdwUxXXSbzrkCFQnVDWm9iFuISnuW-GmUsij3S3yLN8=.f6546d6a-150b-4646-b13b-1f104cb75abe@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <pdwUxXXSbzrkCFQnVDWm9iFuISnuW-GmUsij3S3yLN8=.f6546d6a-150b-4646-b13b-1f104cb75abe@github.com>
Message-ID: <QuhomStpHuWq_yfyIq4l5FJdS7uaiEPYfFKhoULYhYs=.8e2e7ee6-ec1c-42c6-983a-53565bc631fc@github.com>

On Wed, 15 May 2024 20:06:31 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
>> 
>> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
>> 
>> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
>> 
>> Additional testing:
>>  - [x] Performance test reproducer from the bug improves significantly
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>>  - [x] Linux AArch64 server fastdebug, `all`
>>  - [x] Linux x86_64 server fastdebug, `all`
>
> src/hotspot/share/interpreter/oopMapCache.cpp line 545:
> 
>> 543: 
>> 544:   // First search for an empty slot
>> 545:   for (int i = 0; i < _probe_depth; i++) {
> 
> Does the GlobalCounter read barrier belong around this too?

I don't think so: GlobalCounter guards against the reclamation of `OopMapCacheEntry`-es, so we only need to protect the paths that access their contents. We don't need it for anything else, like just poking into the array slots here. I used to have the critical section that spans this entire method, but reasoned it was excessive.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1602752873

From ysr at openjdk.org  Thu May 16 07:45:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 16 May 2024 07:45:17 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v63]
In-Reply-To: <ey1Qz9KUm4z0MD4KG4THlTwWCaeaXV1401rK-M-Y5ms=.2f08abff-21fa-4383-80e1-6e6334b66691@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <ey1Qz9KUm4z0MD4KG4THlTwWCaeaXV1401rK-M-Y5ms=.2f08abff-21fa-4383-80e1-6e6334b66691@github.com>
Message-ID: <Kqneq3s6TBfpG5gNm1Lx_LGuRLYFI3-op3JYD3XxMzs=.fc709482-48de-4fde-ac92-481373348cbc@github.com>

On Wed, 15 May 2024 14:39:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename establish_mutator_intervals

Reviewed. Note: I only skimmed the bitmap implementation and didn't review it closely. I am relying on the unit tests you wrote, your testing of the code in codepipeline stress testing and on others' reviews to have caught ay issues, so don't feel further deep review of the bitmap implementation by me is going to add additional value.

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-2059855419

From shade at openjdk.org  Thu May 16 07:55:28 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 16 May 2024 07:55:28 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v2]
In-Reply-To: <pdwUxXXSbzrkCFQnVDWm9iFuISnuW-GmUsij3S3yLN8=.f6546d6a-150b-4646-b13b-1f104cb75abe@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <pdwUxXXSbzrkCFQnVDWm9iFuISnuW-GmUsij3S3yLN8=.f6546d6a-150b-4646-b13b-1f104cb75abe@github.com>
Message-ID: <NrgOs9d7uJMpBitNqRk0svH9SdOA1138TfUsuBPCipI=.030948dd-a17f-4830-a104-5ac4dccb7259@github.com>

On Wed, 15 May 2024 20:07:57 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Notify service thread on first enqueue
>
> src/hotspot/share/interpreter/oopMapCache.cpp line 593:
> 
>> 591: bool OopMapCache::has_cleanup_work() {
>> 592:   return Atomic::load(&_old_entries) != nullptr;
>> 593: }
> 
> Does this need to notify the ServiceThread?  Since the ServiceThread is now a timed wait, maybe this is fine.

Right. I have not realized the service thread timed wait addition was recent. So it would become a problem if we pull this patch to the JDK release where service thread needs to be explicitly notified. We need to do this carefully, though, since we don't want to acquire `ServiceLock` all that often or inundate the thread with requests. So I added a simple notification when queue is populated with the first element.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1602795482

From shade at openjdk.org  Thu May 16 07:55:28 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 16 May 2024 07:55:28 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v2]
In-Reply-To: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
Message-ID: <UbV9HnnqdMbPcwkuW3_3SfphuPtONviHnbzVJtoPspo=.e4928199-36be-4154-830f-ccd0d5c3d52c@github.com>

> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
> 
> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
> 
> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
> 
> Additional testing:
>  - [x] Performance test reproducer from the bug improves significantly
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>  - [x] Linux AArch64 server fastdebug, `all`
>  - [x] Linux x86_64 server fastdebug, `all`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Notify service thread on first enqueue

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19229/files
  - new: https://git.openjdk.org/jdk/pull/19229/files/455687ad..29dee418

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19229&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19229&range=00-01

  Stats: 6 lines in 1 file changed: 5 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/19229.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19229/head:pull/19229

PR: https://git.openjdk.org/jdk/pull/19229

From shade at openjdk.org  Thu May 16 07:58:05 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 16 May 2024 07:58:05 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v2]
In-Reply-To: <EJMB-7cL_u-ftSBhR3h1qa9DVIzfoBV76SUVuNcLLs8=.c0573f72-2b57-4df4-9b66-c01f7858f383@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <0FHsLntrHofCG7x31n4Worx5TdfoBZ7jGCTkDqJJU8M=.2e89548b-c70a-43e2-a91e-1a80f954188d@github.com>
 <EJMB-7cL_u-ftSBhR3h1qa9DVIzfoBV76SUVuNcLLs8=.c0573f72-2b57-4df4-9b66-c01f7858f383@github.com>
Message-ID: <4T5P_gVCfrkiSHmRtR7Qew6wmmit9Pa0Z3z2jZwZt0A=.8f190081-bef9-48e0-9c8d-7d1032e0d204@github.com>

On Wed, 15 May 2024 22:02:55 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> > This looks good but one question for ZGC, does ZGC need an OopMapCache::cleanup_old_entries() ?
> 
> We still call `OopMapCache::cleanup_old_entries() ` after STW pause, but now concurrent phase also can accumulate old entries, should we unify them? e.g. all depend on service thread to clean up old entries?

Yes. I think the post-GC cleanup is opportunistic after this patch: it is not necessary, since service thread is supposed to catch up with cleanups, but we might still do it after the phases that we know might generate lots of old entries. This is why I left current calls to `cleanup_old_entries()` in current paths, and we might consider adding those for ZGC paths as well. I don't see a strong pressure to do it here, though.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19229#issuecomment-2114328579

From coleenp at openjdk.org  Thu May 16 14:20:03 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Thu, 16 May 2024 14:20:03 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v2]
In-Reply-To: <UbV9HnnqdMbPcwkuW3_3SfphuPtONviHnbzVJtoPspo=.e4928199-36be-4154-830f-ccd0d5c3d52c@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <UbV9HnnqdMbPcwkuW3_3SfphuPtONviHnbzVJtoPspo=.e4928199-36be-4154-830f-ccd0d5c3d52c@github.com>
Message-ID: <Jeb7spePcHKl_c0EZmBZLI4hABA0TIIzYumNVbGMLDE=.a42129be-cb2e-455b-a5ef-76d2fceaef09@github.com>

On Thu, 16 May 2024 07:55:28 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
>> 
>> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
>> 
>> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
>> 
>> Additional testing:
>>  - [x] Performance test reproducer from the bug improves significantly
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>>  - [x] Linux AArch64 server fastdebug, `all`
>>  - [x] Linux x86_64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Notify service thread on first enqueue

This looks good.

src/hotspot/share/interpreter/oopMapCache.cpp line 584:

> 582:         MutexLocker ml(Service_lock, Mutex::_no_safepoint_check_flag);
> 583:         Service_lock->notify_all();
> 584:       }

Yes, you don't want to do this too frequently.

-------------

Marked as reviewed by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19229#pullrequestreview-2060914518
PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1603439246

From coleenp at openjdk.org  Thu May 16 14:20:04 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Thu, 16 May 2024 14:20:04 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v2]
In-Reply-To: <QuhomStpHuWq_yfyIq4l5FJdS7uaiEPYfFKhoULYhYs=.8e2e7ee6-ec1c-42c6-983a-53565bc631fc@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <pdwUxXXSbzrkCFQnVDWm9iFuISnuW-GmUsij3S3yLN8=.f6546d6a-150b-4646-b13b-1f104cb75abe@github.com>
 <QuhomStpHuWq_yfyIq4l5FJdS7uaiEPYfFKhoULYhYs=.8e2e7ee6-ec1c-42c6-983a-53565bc631fc@github.com>
Message-ID: <4lTTfkHuxga5Lf4mNyVXRaAp5RB1O9bx7Ltptb3ZcDU=.14022077-357f-4073-af77-3c1da8db0172@github.com>

On Thu, 16 May 2024 07:22:20 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> src/hotspot/share/interpreter/oopMapCache.cpp line 545:
>> 
>>> 543: 
>>> 544:   // First search for an empty slot
>>> 545:   for (int i = 0; i < _probe_depth; i++) {
>> 
>> Does the GlobalCounter read barrier belong around this too?
>
> I don't think so: GlobalCounter guards against the reclamation of `OopMapCacheEntry`-es, so we only need to protect the paths that access their contents. We don't need it for anything else, like just poking into the array slots here. I used to have the critical section that spans this entire method, but reasoned it was excessive.

Ok, I concur.  It looks ok since the oopMapCache is an array so the search for the null container won't search through next pointers of entries that could be removed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1603438240

From wkemper at openjdk.org  Thu May 16 14:22:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 16 May 2024 14:22:33 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <W-NWDkIpFxW_bVToTwi6Z-JhYFgvGAx0amT4cFjLhJ8=.becc459b-6d2b-4d49-8bc9-9e6c2ab96d99@github.com>

Merges tag jdk-21.0.4+3

-------------

Commit messages:
 - 8324243: Compilation failures in java.desktop module with gcc 14
 - 8311964: Some jtreg tests failing on x86 with error 'unrecognized VM options' (C2 flags)
 - 8321925: sun/security/mscapi/KeytoolChangeAlias.java fails with "Alias <246810> does not exist"
 - 8320370: NMT: Change MallocMemorySnapshot to simplify code.
 - 8325621: Improve jspawnhelper version checks
 - 8330524: Linux ppc64le compile warning with clang  in os_linux_ppc.cpp
 - 8330523: Reduce runtime and improve efficiency of KeepAliveTest
 - 8330815: Use pattern matching for instanceof in KeepAliveCache

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/45/files
  Stats: 1346 lines in 21 files changed: 140 ins; 1028 del; 178 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/45.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/45/head:pull/45

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/45

From rkennke at openjdk.org  Thu May 16 15:08:25 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 16 May 2024 15:08:25 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v63]
In-Reply-To: <ey1Qz9KUm4z0MD4KG4THlTwWCaeaXV1401rK-M-Y5ms=.2f08abff-21fa-4383-80e1-6e6334b66691@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <ey1Qz9KUm4z0MD4KG4THlTwWCaeaXV1401rK-M-Y5ms=.2f08abff-21fa-4383-80e1-6e6334b66691@github.com>
Message-ID: <4GNfmr6mCjgHNgfFt6pRJFErhi85LVWiMsYw7hesOBI=.cba54d73-b823-4652-8763-f6fd86780781@github.com>

On Wed, 15 May 2024 14:39:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename establish_mutator_intervals

Looks good to me now! Thanks a lot for following through all the review so far!

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-2061108458

From shade at openjdk.org  Thu May 16 15:29:02 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 16 May 2024 15:29:02 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v2]
In-Reply-To: <Jeb7spePcHKl_c0EZmBZLI4hABA0TIIzYumNVbGMLDE=.a42129be-cb2e-455b-a5ef-76d2fceaef09@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <UbV9HnnqdMbPcwkuW3_3SfphuPtONviHnbzVJtoPspo=.e4928199-36be-4154-830f-ccd0d5c3d52c@github.com>
 <Jeb7spePcHKl_c0EZmBZLI4hABA0TIIzYumNVbGMLDE=.a42129be-cb2e-455b-a5ef-76d2fceaef09@github.com>
Message-ID: <tBfc-UOtKJcX9VOHdNtuxogmZV0z2DTp4uUW9iirg5c=.e5f217c1-55c1-41b1-b15e-74f135f9d2bc@github.com>

On Thu, 16 May 2024 14:17:03 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Notify service thread on first enqueue
>
> src/hotspot/share/interpreter/oopMapCache.cpp line 584:
> 
>> 582:         MutexLocker ml(Service_lock, Mutex::_no_safepoint_check_flag);
>> 583:         Service_lock->notify_all();
>> 584:       }
> 
> Yes, you don't want to do this too frequently.

Well, tests pass with this change, but now I am thinking if we would eventually run into any lock ranking problem here. At very least `stackwatermark` is ranked above `service`, so we are safe for concurrent GCs. There are only a few locks that are ranked below `service`, so maybe I am overthinking this?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1603597310

From kdnilsen at openjdk.org  Thu May 16 16:24:16 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 16 May 2024 16:24:16 GMT
Subject: RFR: 8332330: Shenandoah: Change jcheck configuration for fix
 version to match project repo
In-Reply-To: <PZHHRRFXXdXltI1Z2SL2zVVzKbj2w17HTetQWp18r5I=.3d397d90-a360-487e-9b02-5a0b92ffe2a3@github.com>
References: <PZHHRRFXXdXltI1Z2SL2zVVzKbj2w17HTetQWp18r5I=.3d397d90-a360-487e-9b02-5a0b92ffe2a3@github.com>
Message-ID: <szrJM1NOwL0L1RLXCj47gPx8-G2fJ2Q2oTkEs29fTm4=.2b701e56-ac47-4584-9cdc-b9db480c2fc1@github.com>

On Wed, 15 May 2024 23:06:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Per https://bugs.openjdk.org/browse/SKARA-2262, we need to configure the correct fixVersion in `.jcheck/conf` in this repo for the bot to interact properly with JBS.

Marked as reviewed by kdnilsen (Committer).

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/434#pullrequestreview-2061298051
PR Review: https://git.openjdk.org/shenandoah/pull/434#pullrequestreview-2061300805

From kdnilsen at openjdk.org  Thu May 16 16:50:33 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 16 May 2024 16:50:33 GMT
Subject: Integrated: 8324649: Shenandoah: replace implementation of free set
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <wHdwsNIiZB6dtU5XESanOGpfCG5w-AyF52Bt9tVOqvU=.4e12e2bb-6c43-41a7-b88f-492a563ca3a2@github.com>

On Wed, 24 Jan 2024 16:10:02 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

This pull request has now been integrated.

Changeset: dc184f10
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
Committer: Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/dc184f1099e09cef095cd8438d88dd8a1e6f5522
Stats:     2569 lines in 8 files changed: 2185 ins; 189 del; 195 mod

8324649: Shenandoah: replace implementation of free set

Reviewed-by: wkemper, ysr, rkennke

-------------

PR: https://git.openjdk.org/jdk/pull/17561

From wkemper at openjdk.org  Thu May 16 16:59:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 16 May 2024 16:59:14 GMT
Subject: RFR: 8332257: Shenandoah: Move evacuation methods to
 implementation file [v2]
In-Reply-To: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
References: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
Message-ID: <i4ios92AC8AfGQUkD8fbADzQJcc_xTrZaMdbcdHivXo=.15681443-c841-4abb-bdf0-92a71d8c79ed@github.com>

> This refactoring moves a large, complex function out of an include header. This improves build times and facilitates changes in the generational mode branch.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:

 - Merge remote-tracking branch 'jdk/master' into move-evacuation-to-impl
 - Do not inline evacuate_object

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19240/files
  - new: https://git.openjdk.org/jdk/pull/19240/files/c28960c6..60f9111e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19240&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19240&range=00-01

  Stats: 6957 lines in 146 files changed: 5240 ins; 980 del; 737 mod
  Patch: https://git.openjdk.org/jdk/pull/19240.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19240/head:pull/19240

PR: https://git.openjdk.org/jdk/pull/19240

From coleenp at openjdk.org  Thu May 16 17:50:02 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Thu, 16 May 2024 17:50:02 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v2]
In-Reply-To: <tBfc-UOtKJcX9VOHdNtuxogmZV0z2DTp4uUW9iirg5c=.e5f217c1-55c1-41b1-b15e-74f135f9d2bc@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <UbV9HnnqdMbPcwkuW3_3SfphuPtONviHnbzVJtoPspo=.e4928199-36be-4154-830f-ccd0d5c3d52c@github.com>
 <Jeb7spePcHKl_c0EZmBZLI4hABA0TIIzYumNVbGMLDE=.a42129be-cb2e-455b-a5ef-76d2fceaef09@github.com>
 <tBfc-UOtKJcX9VOHdNtuxogmZV0z2DTp4uUW9iirg5c=.e5f217c1-55c1-41b1-b15e-74f135f9d2bc@github.com>
Message-ID: <-tSv8ySxibUCWI0vT1FMkA7zP5iL8AfQSsaQKw_bAMs=.ece60fa5-e7c2-4e53-a9dd-e8066474b3c3@github.com>

On Thu, 16 May 2024 15:26:25 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> src/hotspot/share/interpreter/oopMapCache.cpp line 584:
>> 
>>> 582:         MutexLocker ml(Service_lock, Mutex::_no_safepoint_check_flag);
>>> 583:         Service_lock->notify_all();
>>> 584:       }
>> 
>> Yes, you don't want to do this too frequently.
>
> Well, tests pass with this change, but now I am thinking if we would eventually run into any lock ranking problem here. At very least `stackwatermark` is ranked above `service`, so we are safe for concurrent GCs. There are only a few locks that are ranked below `service`, so maybe I am overthinking this?

It is a low level lock, I think it'll be ok, you could check out some call stacks but the tests should find these lock inversions if they exist (famous last words).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1603789260

From wkemper at openjdk.org  Thu May 16 20:44:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 16 May 2024 20:44:26 GMT
Subject: Integrated: 8332330: Shenandoah: Change jcheck configuration for fix
 version to match project repo
In-Reply-To: <PZHHRRFXXdXltI1Z2SL2zVVzKbj2w17HTetQWp18r5I=.3d397d90-a360-487e-9b02-5a0b92ffe2a3@github.com>
References: <PZHHRRFXXdXltI1Z2SL2zVVzKbj2w17HTetQWp18r5I=.3d397d90-a360-487e-9b02-5a0b92ffe2a3@github.com>
Message-ID: <P7gXvR8K65gJEX-JOskcC6h6IOTRgGwUM-5i1UYQVZQ=.ca8b9852-4566-4d2f-a7fc-9985c70461b0@github.com>

On Wed, 15 May 2024 23:06:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Per https://bugs.openjdk.org/browse/SKARA-2262, we need to configure the correct fixVersion in `.jcheck/conf` in this repo for the bot to interact properly with JBS.

This pull request has now been integrated.

Changeset: 792436de
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/792436deb17e3b9058a671afeb61731e3a139b62
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8332330: Shenandoah: Change jcheck configuration for fix version to match project repo

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/434

From wkemper at openjdk.org  Fri May 17 00:47:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 00:47:06 GMT
Subject: RFR: 8332257: Shenandoah: Move evacuation methods to
 implementation file [v2]
In-Reply-To: <i4ios92AC8AfGQUkD8fbADzQJcc_xTrZaMdbcdHivXo=.15681443-c841-4abb-bdf0-92a71d8c79ed@github.com>
References: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
 <i4ios92AC8AfGQUkD8fbADzQJcc_xTrZaMdbcdHivXo=.15681443-c841-4abb-bdf0-92a71d8c79ed@github.com>
Message-ID: <7KPxCOGUhjgt1rZcOsNjThs2YXRO41zFV8_-1QZBlbM=.79c45a62-e345-45c9-a8e5-c6bab93e7308@github.com>

On Thu, 16 May 2024 16:59:14 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This refactoring moves a large, complex function out of an include header. This improves build times and facilitates changes in the generational mode branch.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'jdk/master' into move-evacuation-to-impl
>  - Do not inline evacuate_object

No regressions in performance tests on x86 or aarch64.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19240#issuecomment-2116433407

From wkemper at openjdk.org  Fri May 17 00:47:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 00:47:06 GMT
Subject: Integrated: 8332257: Shenandoah: Move evacuation methods to
 implementation file
In-Reply-To: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
References: <jdu_QS2d2Eyt0VQVb2Dsz8pGMI6w3dBM4x3icHsuCQc=.8bc86701-f5a4-4b96-ae53-e007740cfd99@github.com>
Message-ID: <zSeRKDY4_cbcaRdpvyhLwVeaNAWSk4AkiAeqF0oRg9o=.1c61cfaa-320e-43eb-89e2-3620d5bd5bb0@github.com>

On Tue, 14 May 2024 23:54:15 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This refactoring moves a large, complex function out of an include header. This improves build times and facilitates changes in the generational mode branch.

This pull request has now been integrated.

Changeset: de57d4b2
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/de57d4b2e0fe3add0ef09945b34ddd0b67bbfa2b
Stats:     155 lines in 3 files changed: 77 ins; 77 del; 1 mod

8332257: Shenandoah: Move evacuation methods to implementation file

Reviewed-by: shade, kdnilsen

-------------

PR: https://git.openjdk.org/jdk/pull/19240

From ysr at openjdk.org  Fri May 17 01:06:35 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 17 May 2024 01:06:35 GMT
Subject: RFR: 8332331: Shenandoah: Change jcheck configuration for fix
 version to match project repo
In-Reply-To: <_2XrfBh6wTMVAPceovK4lPWaKNIq-D69iA7OE7wW8lc=.5359329f-4853-4447-aa91-7533a6ca77cd@github.com>
References: <_2XrfBh6wTMVAPceovK4lPWaKNIq-D69iA7OE7wW8lc=.5359329f-4853-4447-aa91-7533a6ca77cd@github.com>
Message-ID: <Y8UmyrH-UaZa5utJRcz0eR5uauEb92kk5U1puvuLHno=.1e196515-c079-4e00-93e6-7997253f22e3@github.com>

On Wed, 15 May 2024 23:15:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Per https://bugs.openjdk.org/browse/SKARA-2262, we need to configure the correct fixVersion in .jcheck/conf in this repo for the bot to interact properly with JBS.

Thank you!

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/44#pullrequestreview-2062151068

From wkemper at openjdk.org  Fri May 17 01:27:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 01:27:23 GMT
Subject: Integrated: 8332331: Shenandoah: Change jcheck configuration for fix
 version to match project repo
In-Reply-To: <_2XrfBh6wTMVAPceovK4lPWaKNIq-D69iA7OE7wW8lc=.5359329f-4853-4447-aa91-7533a6ca77cd@github.com>
References: <_2XrfBh6wTMVAPceovK4lPWaKNIq-D69iA7OE7wW8lc=.5359329f-4853-4447-aa91-7533a6ca77cd@github.com>
Message-ID: <f_aOOXVy04RpPPoUZsfsb00KkJTDX7z5IzK5SlkalRA=.9d271b8a-75a8-4474-994c-8452ccfde0a0@github.com>

On Wed, 15 May 2024 23:15:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Per https://bugs.openjdk.org/browse/SKARA-2262, we need to configure the correct fixVersion in .jcheck/conf in this repo for the bot to interact properly with JBS.

This pull request has now been integrated.

Changeset: f43e24ae
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/f43e24ae4005c4830f949974740862df20079e5f
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8332331: Shenandoah: Change jcheck configuration for fix version to match project repo

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/44

From shade at openjdk.org  Fri May 17 11:35:37 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 17 May 2024 11:35:37 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v3]
In-Reply-To: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
Message-ID: <08qAsGXFBbttgcQ0tfpfKQmtcuyiP4E2I0Tvi1tJNCE=.3cf52475-8b0e-4f7b-b43d-48305b74e008@github.com>

> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
> 
> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
> 
> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
> 
> Additional testing:
>  - [x] Performance test reproducer from the bug improves significantly
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>  - [x] Linux AArch64 server fastdebug, `all`
>  - [x] Linux x86_64 server fastdebug, `all`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Chicken out: do not notify Service thread from OopMapCache enqueue paths

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19229/files
  - new: https://git.openjdk.org/jdk/pull/19229/files/29dee418..c81a1139

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19229&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19229&range=01-02

  Stats: 22 lines in 5 files changed: 10 ins; 5 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/19229.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19229/head:pull/19229

PR: https://git.openjdk.org/jdk/pull/19229

From shade at openjdk.org  Fri May 17 11:35:37 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 17 May 2024 11:35:37 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v2]
In-Reply-To: <-tSv8ySxibUCWI0vT1FMkA7zP5iL8AfQSsaQKw_bAMs=.ece60fa5-e7c2-4e53-a9dd-e8066474b3c3@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <UbV9HnnqdMbPcwkuW3_3SfphuPtONviHnbzVJtoPspo=.e4928199-36be-4154-830f-ccd0d5c3d52c@github.com>
 <Jeb7spePcHKl_c0EZmBZLI4hABA0TIIzYumNVbGMLDE=.a42129be-cb2e-455b-a5ef-76d2fceaef09@github.com>
 <tBfc-UOtKJcX9VOHdNtuxogmZV0z2DTp4uUW9iirg5c=.e5f217c1-55c1-41b1-b15e-74f135f9d2bc@github.com>
 <-tSv8ySxibUCWI0vT1FMkA7zP5iL8AfQSsaQKw_bAMs=.ece60fa5-e7c2-4e53-a9dd-e8066474b3c3@github.com>
Message-ID: <dNKvNoX_QfE47nPalQAA85azYeUgjdMJyS9D-rOk_Bk=.82ac8678-25f1-4a42-b2c2-dadf0708403f@github.com>

On Thu, 16 May 2024 17:47:16 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Well, tests pass with this change, but now I am thinking if we would eventually run into any lock ranking problem here. At very least `stackwatermark` is ranked above `service`, so we are safe for concurrent GCs. There are only a few locks that are ranked below `service`, so maybe I am overthinking this?
>
> It is a low level lock, I think it'll be ok, you could check out some call stacks but the tests should find these lock inversions if they exist (famous last words).

OK, the "problem" here is that OopMapCache is used from the generic interpreter frame walkers, so I cannot really trace the callers all that well. And it looks to me that capturing the failure in tests would be maddeningly hard: the notification happens very rarely. So we cannot even rely on current tests. 

So I am chickening out from notifying the service thread on this path. Instead, we would rely on service thread timed wait in current mainline. I amended the patch a bit, with the method that can trigger the cleanup. When/if we backport this change to previous releases, we can just hook the call to that method to safepoint cleanup sequence, like we used to have for String/SymbolTable; which would cover the case when service thread is not time-waited yet.

This also likely frees application threads from the cleanup, as cleanup is delegated solely to Service thread now.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19229#discussion_r1604830095

From shade at openjdk.org  Fri May 17 11:52:09 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 17 May 2024 11:52:09 GMT
Subject: RFR: 8331721: Introduce CollectedHeap helper methods for any GC in
 progress [v3]
In-Reply-To: <_yFkn6p3JH9DPML_qw04vIUwu-yOcOLap3zjoKOrM3c=.2d01cca2-2ee5-4c16-9aae-ffd417cc76b3@github.com>
References: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
 <_yFkn6p3JH9DPML_qw04vIUwu-yOcOLap3zjoKOrM3c=.2d01cca2-2ee5-4c16-9aae-ffd417cc76b3@github.com>
Message-ID: <zEpW8NUHW2TDvv_ewZ9HzWImiZ0vmH4IfWqNy_QU4_M=.53c977c4-e2b8-45c0-9a53-4464377faac2@github.com>

On Tue, 7 May 2024 18:38:05 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Continuation of [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573). So far we only have a way to test for STW GCs. We would need these more generic sensing methods for [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
>> 
>> This patch considers that there can be several concurrent GCs in progress. A notable example of that is Generational ZGC, where minor and major concurrent collections can run at the same time. For G1, concurrent mark can interleave with collection pauses. This is why we track the number of GCs in progress as atomic.
>>  
>> Initially, I thought to use `is_gc_active` name freed up by [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573), but then I thought better of it. My concern would be that some backports might be confused about what `is_gc_active` name really means, depending on whether JDK-8331573 and this PR is backported or not. To avoid this confusion, I chose `is_any_gc_active` name instead.
>> 
>> This PR is somewhat open for bikeshedding about the names :)
>> 
>> Additional testing:
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc`
>>  - [ ] Linux AArch64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Cleanup

JDK-8331572 took another turn, so we don't need this. Feel free to reopen if need arises.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19122#issuecomment-2117422979

From shade at openjdk.org  Fri May 17 11:52:10 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 17 May 2024 11:52:10 GMT
Subject: Withdrawn: 8331721: Introduce CollectedHeap helper methods for any GC
 in progress
In-Reply-To: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
References: <kFVOFeC2btksh-f2qEWrupFELFCmYF5aJrHVuDl_9eA=.5563fcb0-2d55-4d37-ac40-e54d4d59af60@github.com>
Message-ID: <knS-cxg1Gf-qKjYfEX5rtviFAGWhmhe_1xYugsrHSX8=.11d94804-12fe-4671-a29f-4af3798e4793@github.com>

On Tue, 7 May 2024 14:33:26 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Continuation of [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573). So far we only have a way to test for STW GCs. We would need these more generic sensing methods for [JDK-8331572](https://bugs.openjdk.org/browse/JDK-8331572).
> 
> This patch considers that there can be several concurrent GCs in progress. A notable example of that is Generational ZGC, where minor and major concurrent collections can run at the same time. For G1, concurrent mark can interleave with collection pauses. This is why we track the number of GCs in progress as atomic.
>  
> Initially, I thought to use `is_gc_active` name freed up by [JDK-8331573](https://bugs.openjdk.org/browse/JDK-8331573), but then I thought better of it. My concern would be that some backports might be confused about what `is_gc_active` name really means, depending on whether JDK-8331573 and this PR is backported or not. To avoid this confusion, I chose `is_any_gc_active` name instead.
> 
> This PR is somewhat open for bikeshedding about the names :)
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc`
>  - [ ] Linux AArch64 server fastdebug, `all`

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/19122

From wkemper at openjdk.org  Fri May 17 14:15:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 14:15:35 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <ChQAUz8pLvlazy6P-Ykj5DHuJzb9EBbyJ99pysR5Rw0=.930db7d4-7c51-4f60-8544-a3a3e48afe7e@github.com>

Merges tag jdk-23+23

-------------

Commit messages:
 - 8302850: Implement C1 clone intrinsic that reuses arraycopy code for primitive arrays
 - 8329839: Cleanup ZPhysicalMemoryBacking trace logging
 - 8332236: javac crashes with module imports and implicitly declared class
 - 8330795: C2: assert((uint)type <= T_CONFLICT && _zero_type[type] != nullptr) failed: bad type with -XX:-UseCompressedClassPointers
 - 8325932: Replace ATTRIBUTE_NORETURN with direct [[noreturn]]
 - 8332248: (fc) java/nio/channels/FileChannel/BlockDeviceSize.java failed with RuntimeException
 - 8322008: Exclude some CDS tests from running with -Xshare:off
 - 8313674: (fc) java/nio/channels/FileChannel/BlockDeviceSize.java should test for more block devices
 - 8332042: Move MEMFLAGS to its own include file
 - 8331858: [nmt] VM.native_memory statistics should work in summary mode
 - ... and 227 more: https://git.openjdk.org/shenandoah/compare/87e864bf...2f10a316

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/435/files
  Stats: 43952 lines in 2060 files changed: 19879 ins; 15301 del; 8772 mod
  Patch: https://git.openjdk.org/shenandoah/pull/435.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/435/head:pull/435

PR: https://git.openjdk.org/shenandoah/pull/435

From coleenp at openjdk.org  Fri May 17 15:01:03 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Fri, 17 May 2024 15:01:03 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v3]
In-Reply-To: <08qAsGXFBbttgcQ0tfpfKQmtcuyiP4E2I0Tvi1tJNCE=.3cf52475-8b0e-4f7b-b43d-48305b74e008@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <08qAsGXFBbttgcQ0tfpfKQmtcuyiP4E2I0Tvi1tJNCE=.3cf52475-8b0e-4f7b-b43d-48305b74e008@github.com>
Message-ID: <TVYTfDt9jId5j4lRFDEPnU1G7iHyt37b1EnFu-3ihys=.d1c962ab-0e74-499f-b6af-87d74ddebd44@github.com>

On Fri, 17 May 2024 11:35:37 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
>> 
>> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
>> 
>> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
>> 
>> Additional testing:
>>  - [x] Performance test reproducer from the bug improves significantly
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>>  - [x] Linux AArch64 server fastdebug, `all`
>>  - [x] Linux x86_64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Chicken out: do not notify Service thread from OopMapCache enqueue paths

This looks safe.  You still get some trigger for cleanup, plus the ServiceThread timeout.  The timer comment is vague enough to cover this case also.

-------------

Marked as reviewed by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19229#pullrequestreview-2063667904

From zgu at openjdk.org  Fri May 17 15:17:02 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Fri, 17 May 2024 15:17:02 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v3]
In-Reply-To: <08qAsGXFBbttgcQ0tfpfKQmtcuyiP4E2I0Tvi1tJNCE=.3cf52475-8b0e-4f7b-b43d-48305b74e008@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <08qAsGXFBbttgcQ0tfpfKQmtcuyiP4E2I0Tvi1tJNCE=.3cf52475-8b0e-4f7b-b43d-48305b74e008@github.com>
Message-ID: <aKJ_C9eMhwuCKQkFysSdpIxVOMnJwWgVVdNgaAIRJf4=.35654bdc-a72e-4ad3-9feb-29416da732a3@github.com>

On Fri, 17 May 2024 11:35:37 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
>> 
>> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
>> 
>> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
>> 
>> Additional testing:
>>  - [x] Performance test reproducer from the bug improves significantly
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>>  - [x] Linux AArch64 server fastdebug, `all`
>>  - [x] Linux x86_64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Chicken out: do not notify Service thread from OopMapCache enqueue paths

I would suggest to move `OopMapCache::trigger_cleanup();` from `VM_ShenandoahReferenceOperation::doit_epilogue()` to `VM_ShenandoahOperation::doit_prologue()` and add the call to ` VM_ZOperation` and ` VM_XOperation`'s `doit_epilogue()`

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19229#issuecomment-2117821424

From shade at openjdk.org  Fri May 17 15:36:09 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 17 May 2024 15:36:09 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v3]
In-Reply-To: <aKJ_C9eMhwuCKQkFysSdpIxVOMnJwWgVVdNgaAIRJf4=.35654bdc-a72e-4ad3-9feb-29416da732a3@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <08qAsGXFBbttgcQ0tfpfKQmtcuyiP4E2I0Tvi1tJNCE=.3cf52475-8b0e-4f7b-b43d-48305b74e008@github.com>
 <aKJ_C9eMhwuCKQkFysSdpIxVOMnJwWgVVdNgaAIRJf4=.35654bdc-a72e-4ad3-9feb-29416da732a3@github.com>
Message-ID: <jYewetHLaR4yE00ujflXWoQjm2lfA7C5BDJSkVc-4wI=.d9526f8a-fba6-4230-8e29-e58958e7b77d@github.com>

On Fri, 17 May 2024 15:14:07 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> I would suggest to move `OopMapCache::trigger_cleanup();` from `VM_ShenandoahReferenceOperation::doit_epilogue()` to `VM_ShenandoahOperation::doit_prologue()` and add the call to ` VM_ZOperation` and ` VM_XOperation`'s `doit_epilogue()`

Right. Surely, it would be better to move it to `VM_ShenandoahOperation::doit_epilogue()` as well? This way we don't risk Service thread waking up during the short GC pause.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19229#issuecomment-2117859377

From shade at openjdk.org  Fri May 17 15:58:32 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 17 May 2024 15:58:32 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v4]
In-Reply-To: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
Message-ID: <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>

> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
> 
> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
> 
> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
> 
> Additional testing:
>  - [x] Performance test reproducer from the bug improves significantly
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>  - [x] Linux AArch64 server fastdebug, `all`
>  - [x] Linux x86_64 server fastdebug, `all`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Add more GC triggers around

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19229/files
  - new: https://git.openjdk.org/jdk/pull/19229/files/c81a1139..ad9f97f3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19229&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19229&range=02-03

  Stats: 16 lines in 3 files changed: 13 ins; 3 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/19229.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19229/head:pull/19229

PR: https://git.openjdk.org/jdk/pull/19229

From ysr at openjdk.org  Fri May 17 16:11:37 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 17 May 2024 16:11:37 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v3]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <wDQSfRdHe1FwUgjxrRuWvJ5_6wojOjDr2IwiZx9h0ZE=.f140d429-946d-45c0-b64c-b128c79b488e@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed. A case of flickering of active_generation() was identified, and found to be benign. An assert now checks for this situation. The code has been made robust wrt the flickering (seen only by mutators executing load barriers).
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 32 additional commits since the last revision:

 - jcheck whitespace
 - Fixed mismerge because of code moving between files in merge from
   master; encapsulate assert & provide more useful debugging info. We
   aren't done yet... It might be the case that we do away with the
   sync/async split of the variable and instead move the relevant state
   into the closure if possible. That might have to wait for another day
   though based on whether it'll work well everywhere or not.
 - Merge branch 'master' into active_generation
 - Greater separation of gc_generation and active_generation fields of
   ShHeap. WIP.
 - Clean ups.
 - jcheck whitespace
 - Banish forcing entirely. Fix one case where full gc sets
   active_generation directly. Might be other cases as well. Will need a
   cleaner and more natural abstraction different from the unseemly
   under-the-covers groveling that we do currently.
 - Relax an assert that is too strong; still iterating execution paths,
   more cleanups planned.
 - Disallow forcing.
   Test.
 - Remove an incorrect gc status setting.
 - ... and 22 more: https://git.openjdk.org/shenandoah/compare/5cefc2d8...61848c82

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/407/files
  - new: https://git.openjdk.org/shenandoah/pull/407/files/a86d3edd..61848c82

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=01-02

  Stats: 588111 lines in 6590 files changed: 100933 ins; 144093 del; 343085 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From ysr at openjdk.org  Fri May 17 16:11:38 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 17 May 2024 16:11:38 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v2]
In-Reply-To: <ypVZMdCjy119ULVp1pMTnlHIlV7czno82A3ruIh9ABE=.3e97d621-c3f1-4fe6-8fff-8cbf77c27164@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
 <ypVZMdCjy119ULVp1pMTnlHIlV7czno82A3ruIh9ABE=.3e97d621-c3f1-4fe6-8fff-8cbf77c27164@github.com>
Message-ID: <inW_4-7cWUe93TyWKlyD44b-IA7fCSlHSpbhjWyUNnU=.685f7bcf-3ac3-490f-90d5-d4c5f4046186@github.com>

On Wed, 20 Mar 2024 10:21:11 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed. A case of flickering of active_generation() was identified, and found to be benign. An assert now checks for this situation. The code has been made robust wrt the flickering (seen only by mutators executing load barriers).
>> 
>> *Testing*:
>> - [x] code pipeline
>> - [x] specjbb
>> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
>> - [x] GHA
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
> 
>  - Refine assertion further & modify comment block.
>  - Merge branch 'master' into active_generation
>  - jcheck cleanup.
>  - Weaken assertion so it passes for now; add comment.
>  - Clean up assertion checking for race.
>  - Read active_generetion() only once in is_is_active_generation() to work
>    around potential flicker; interlocking reads to detect flicker. Need
>    better checks.
>  - Robustify ShenandoahGCSession, and fix a missing use for
>    coalesce_and_fill of old gen.

Publishing the draft for feedback.

Still needs a few cleanups. Not too happy with the minutiae of the mechanics that need to be better abstracted.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/407#issuecomment-2117925118

From zgu at openjdk.org  Fri May 17 16:18:03 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Fri, 17 May 2024 16:18:03 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v4]
In-Reply-To: <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>
Message-ID: <hLwcHzE0Ymlq7-a906EswJ_CRLDVAQMQM2v7s4uA21Y=.61df0353-9b5d-4815-aab8-f39d0f970df8@github.com>

On Fri, 17 May 2024 15:58:32 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
>> 
>> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
>> 
>> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
>> 
>> Additional testing:
>>  - [x] Performance test reproducer from the bug improves significantly
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>>  - [x] Linux AArch64 server fastdebug, `all`
>>  - [x] Linux x86_64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add more GC triggers around

LGTM

-------------

Marked as reviewed by zgu (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19229#pullrequestreview-2063858977

From ysr at openjdk.org  Fri May 17 17:26:29 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 17 May 2024 17:26:29 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v3]
In-Reply-To: <wDQSfRdHe1FwUgjxrRuWvJ5_6wojOjDr2IwiZx9h0ZE=.f140d429-946d-45c0-b64c-b128c79b488e@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
 <wDQSfRdHe1FwUgjxrRuWvJ5_6wojOjDr2IwiZx9h0ZE=.f140d429-946d-45c0-b64c-b128c79b488e@github.com>
Message-ID: <L-pDWdxZYQL15bdgT2UPPCIDU30Xt2lzyjNAsUakCd0=.42803be7-3c39-46d7-b971-c87e918513e7@github.com>

On Fri, 17 May 2024 16:11:37 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> ~ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed. A case of flickering of active_generation() was identified, and found to be benign. An assert now checks for this situation. The code has been made robust wrt the flickering (seen only by mutators executing load barriers).~ To be fixed.
>> 
>> *Testing*:
>> - [x] code pipeline
>> - [x] specjbb
>> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
>> - [x] GHA
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 32 additional commits since the last revision:
> 
>  - jcheck whitespace
>  - Fixed mismerge because of code moving between files in merge from
>    master; encapsulate assert & provide more useful debugging info. We
>    aren't done yet... It might be the case that we do away with the
>    sync/async split of the variable and instead move the relevant state
>    into the closure if possible. That might have to wait for another day
>    though based on whether it'll work well everywhere or not.
>  - Merge branch 'master' into active_generation
>  - Greater separation of gc_generation and active_generation fields of
>    ShHeap. WIP.
>  - Clean ups.
>  - jcheck whitespace
>  - Banish forcing entirely. Fix one case where full gc sets
>    active_generation directly. Might be other cases as well. Will need a
>    cleaner and more natural abstraction different from the unseemly
>    under-the-covers groveling that we do currently.
>  - Relax an assert that is too strong; still iterating execution paths,
>    more cleanups planned.
>  - Disallow forcing.
>    Test.
>  - Remove an incorrect gc status setting.
>  - ... and 22 more: https://git.openjdk.org/shenandoah/compare/4a8ee77b...61848c82

src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp line 108:

> 106:   // Weak reference processing
> 107:   ShenandoahReferenceProcessor* rp = heap->gc_generation()->ref_processor();
> 108:   assert(heap->gc_generation() == heap->active_generation(), "Should be identical at stw phases");

`assert_generations_reconciled();` instead.

src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp line 177:

> 175:   ShenandoahWorkerTimingsTracker timer(phase, ShenandoahPhaseTimings::ParallelMark, worker_id);
> 176:   ShenandoahReferenceProcessor* rp = ShenandoahHeap::heap()->gc_generation()->ref_processor();
> 177:   assert(ShenandoahHeap::heap()->gc_generation() == ShenandoahHeap::heap()->active_generation(),

`assert_generations_reconciled();` instead.

test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 81:

> 79:             ProcessBuilder pb = ProcessTools.createLimitedTestJavaProcessBuilder(
> 80:                     "-Xmx32m",
> 81:                     // "-XX:+UnlockExperimentalVMOptions", "-XX:ShenandoahNoProgressThreshold=12", "-XX:+ShowMessageBoxOnError",

Ignore this. Will be removed; debugging detritus left in place until we believe we are ready.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1605345202
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1605345705
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1605347017

From wkemper at openjdk.org  Fri May 17 23:23:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:23:33 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <11_S1jA9IwlqgfkvNIozaDocP0UwG14v-ci8B-joGBo=.d399d691-a003-4a39-8148-272f828cc682@github.com>
References: <11_S1jA9IwlqgfkvNIozaDocP0UwG14v-ci8B-joGBo=.d399d691-a003-4a39-8148-272f828cc682@github.com>
Message-ID: <fLl-jHXzaG0uv-patpe9uBnQVFeeEo5en8ubQQMka10=.1fc0fdf2-1140-4738-a8cd-6ccc38daacde@github.com>

> Merges tag jdk-23+22

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/429/files
  - new: https://git.openjdk.org/shenandoah/pull/429/files/964d6089..964d6089

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=429&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=429&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/429.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/429/head:pull/429

PR: https://git.openjdk.org/shenandoah/pull/429

From wkemper at openjdk.org  Fri May 17 23:23:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:23:33 GMT
Subject: RFR: Merge openjdk/jdk:master
In-Reply-To: <11_S1jA9IwlqgfkvNIozaDocP0UwG14v-ci8B-joGBo=.d399d691-a003-4a39-8148-272f828cc682@github.com>
References: <11_S1jA9IwlqgfkvNIozaDocP0UwG14v-ci8B-joGBo=.d399d691-a003-4a39-8148-272f828cc682@github.com>
Message-ID: <AFomNQQVVNrVgx73JX-CerW-_v1uinsFQMaaxow4efw=.9192c7a7-f8b8-4e09-a66d-6905c09f47c0@github.com>

On Fri, 10 May 2024 14:10:14 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+22

Superseded by https://github.com/openjdk/shenandoah/pull/435

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/429#issuecomment-2118482195

From wkemper at openjdk.org  Fri May 17 23:23:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:23:33 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <11_S1jA9IwlqgfkvNIozaDocP0UwG14v-ci8B-joGBo=.d399d691-a003-4a39-8148-272f828cc682@github.com>
References: <11_S1jA9IwlqgfkvNIozaDocP0UwG14v-ci8B-joGBo=.d399d691-a003-4a39-8148-272f828cc682@github.com>
Message-ID: <tAg0IJFRg-5oxfVHSqjoGPosFKAoAGLOEAuEDDHcMXc=.b138e04f-013f-4754-be74-f7c5cee20b2a@github.com>

On Fri, 10 May 2024 14:10:14 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+22

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/429

From wkemper at openjdk.org  Fri May 17 23:25:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:25:03 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <ChQAUz8pLvlazy6P-Ykj5DHuJzb9EBbyJ99pysR5Rw0=.930db7d4-7c51-4f60-8544-a3a3e48afe7e@github.com>
References: <ChQAUz8pLvlazy6P-Ykj5DHuJzb9EBbyJ99pysR5Rw0=.930db7d4-7c51-4f60-8544-a3a3e48afe7e@github.com>
Message-ID: <4QeoHkI0LFWxLmE7XH0x8_YhKNMFXRHhq3l8gR636QE=.12e93648-d036-4290-950f-4d938025f968@github.com>

> Merges tag jdk-23+23

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/435/files
  - new: https://git.openjdk.org/shenandoah/pull/435/files/2f10a316..2f10a316

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=435&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=435&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/435.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/435/head:pull/435

PR: https://git.openjdk.org/shenandoah/pull/435

From wkemper at openjdk.org  Fri May 17 23:25:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:25:11 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <H1kUL5WTyt7CUj75YNJq-UjWabUI9q1axiTsqBBCmII=.139432f1-1bf6-4384-8d84-31852979f9ae@github.com>
References: <H1kUL5WTyt7CUj75YNJq-UjWabUI9q1axiTsqBBCmII=.139432f1-1bf6-4384-8d84-31852979f9ae@github.com>
Message-ID: <Hdo6A8kPC3_iDWkAiOG7eh3O94lDhAqSrCGy3gt6YfA=.561cf333-9ff0-4120-9d5e-2978beecb150@github.com>

> Merges tag jdk-23+21

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/428/files
  - new: https://git.openjdk.org/shenandoah/pull/428/files/e833bfc8..e833bfc8

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=428&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=428&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/428.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/428/head:pull/428

PR: https://git.openjdk.org/shenandoah/pull/428

From wkemper at openjdk.org  Fri May 17 23:25:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:25:11 GMT
Subject: RFR: Merge openjdk/jdk:master
In-Reply-To: <H1kUL5WTyt7CUj75YNJq-UjWabUI9q1axiTsqBBCmII=.139432f1-1bf6-4384-8d84-31852979f9ae@github.com>
References: <H1kUL5WTyt7CUj75YNJq-UjWabUI9q1axiTsqBBCmII=.139432f1-1bf6-4384-8d84-31852979f9ae@github.com>
Message-ID: <iCWj0goi7ouDiy3qaxweGFC-aHsGV0CrQQgrNikaX88=.d61a6565-f7c5-463f-ada9-c1a743e06668@github.com>

On Fri, 3 May 2024 14:10:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+21

Superseded by https://github.com/openjdk/shenandoah/pull/435

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/428#issuecomment-2118482425

From duke at openjdk.org  Fri May 17 23:25:11 2024
From: duke at openjdk.org (duke)
Date: Fri, 17 May 2024 23:25:11 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <H1kUL5WTyt7CUj75YNJq-UjWabUI9q1axiTsqBBCmII=.139432f1-1bf6-4384-8d84-31852979f9ae@github.com>
References: <H1kUL5WTyt7CUj75YNJq-UjWabUI9q1axiTsqBBCmII=.139432f1-1bf6-4384-8d84-31852979f9ae@github.com>
Message-ID: <4CbFm4klBWgsKLmYDR-0p9-TKJcCH4O6czI4nqbagG8=.c53e17a9-30c8-4ebf-9401-f048868eb3d4@github.com>

On Fri, 3 May 2024 14:10:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+21

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/428

From wkemper at openjdk.org  Fri May 17 23:25:05 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:25:05 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <ChQAUz8pLvlazy6P-Ykj5DHuJzb9EBbyJ99pysR5Rw0=.930db7d4-7c51-4f60-8544-a3a3e48afe7e@github.com>
References: <ChQAUz8pLvlazy6P-Ykj5DHuJzb9EBbyJ99pysR5Rw0=.930db7d4-7c51-4f60-8544-a3a3e48afe7e@github.com>
Message-ID: <eqqEIRhcb-FeumHygq-jRRkXzyFdL-t1eanbKyIr_bk=.5472ebe9-9d6e-41fb-bd92-33028c4950b7@github.com>

On Fri, 17 May 2024 14:11:04 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+23

This pull request has now been integrated.

Changeset: c4e2cf8c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/c4e2cf8cb2f0f883cc6226604363240406ed9d9a
Stats:     43952 lines in 2060 files changed: 19879 ins; 15301 del; 8772 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/435

From wkemper at openjdk.org  Fri May 17 23:27:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:27:59 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <W-NWDkIpFxW_bVToTwi6Z-JhYFgvGAx0amT4cFjLhJ8=.becc459b-6d2b-4d49-8bc9-9e6c2ab96d99@github.com>
References: <W-NWDkIpFxW_bVToTwi6Z-JhYFgvGAx0amT4cFjLhJ8=.becc459b-6d2b-4d49-8bc9-9e6c2ab96d99@github.com>
Message-ID: <d9pnabybpfetOzP6NPUlmfMcnDTRZQKf_vlnYsMh1Yg=.39eaac02-7135-4c3c-9795-7495917699ca@github.com>

> Merges tag jdk-21.0.4+3

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/45/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/45/files/035e3144..035e3144

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=45&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=45&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/45.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/45/head:pull/45

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/45

From wkemper at openjdk.org  Fri May 17 23:27:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:27:59 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <W-NWDkIpFxW_bVToTwi6Z-JhYFgvGAx0amT4cFjLhJ8=.becc459b-6d2b-4d49-8bc9-9e6c2ab96d99@github.com>
References: <W-NWDkIpFxW_bVToTwi6Z-JhYFgvGAx0amT4cFjLhJ8=.becc459b-6d2b-4d49-8bc9-9e6c2ab96d99@github.com>
Message-ID: <yUi0UcPRpFXmk-kvT1IhTZ-xXpg_ISjnlAxBCTUIF70=.d34194f0-3e81-4675-ab50-1b538d1ffe11@github.com>

On Thu, 16 May 2024 14:17:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.4+3

This pull request has now been integrated.

Changeset: e5e47086
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/e5e4708629eb5c4678caea30f488d733afd54bed
Stats:     1346 lines in 21 files changed: 140 ins; 1028 del; 178 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/45

From wkemper at openjdk.org  Fri May 17 23:44:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:44:37 GMT
Subject: Integrated: 8331609: GenShen: Refactor generational mode allocations
Message-ID: <NuVLVtR8jj2sprMOObo0O2RkzVZHcPkcCDKyI91h-iQ=.d31f7579-f895-4376-8197-4a177e574d97@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 80e69877f807499e811ff0353d7477057fa2f63d

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/46/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=46&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331609
  Stats: 503 lines in 8 files changed: 272 ins; 175 del; 56 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/46.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/46/head:pull/46

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/46

From wkemper at openjdk.org  Fri May 17 23:44:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:44:44 GMT
Subject: RFR: 8332254: GenShen: Remove obsolete comments and revert unnecessary
 formatting changes
Message-ID: <FUilbrVYOs4rYt3ZxMW_yKGmpFvht-p3CrKc4p4FWHE=.524353a7-2336-4418-b9b3-6bcf37379c8c@github.com>

Clean backport

-------------

Commit messages:
 - Backport dfa788667714b4ec7e0e235e9c0c58cc5e3ceab4

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/48/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=48&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332254
  Stats: 11 lines in 1 file changed: 1 ins; 5 del; 5 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/48.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/48/head:pull/48

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/48

From wkemper at openjdk.org  Fri May 17 23:44:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:44:38 GMT
Subject: Integrated: 8331609: GenShen: Refactor generational mode allocations
In-Reply-To: <NuVLVtR8jj2sprMOObo0O2RkzVZHcPkcCDKyI91h-iQ=.d31f7579-f895-4376-8197-4a177e574d97@github.com>
References: <NuVLVtR8jj2sprMOObo0O2RkzVZHcPkcCDKyI91h-iQ=.d31f7579-f895-4376-8197-4a177e574d97@github.com>
Message-ID: <UYtEge6AFoTfQ_e99gtLJju2wNdG6F8a9oqpo3P7w80=.2cab7c7e-af6a-4d99-9e1b-c8947a813fe3@github.com>

On Fri, 17 May 2024 23:37:04 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: 0efff564
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/0efff5647b8db1845f9cb7f2521498544fc7f49b
Stats:     503 lines in 8 files changed: 272 ins; 175 del; 56 mod

8331609: GenShen: Refactor generational mode allocations

Backport-of: 80e69877f807499e811ff0353d7477057fa2f63d

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/46

From wkemper at openjdk.org  Fri May 17 23:44:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:44:54 GMT
Subject: Integrated: 8332094: GenShen: Reuse existing code to verify usage
 before rebuilding free set
Message-ID: <2u7eDJNlJA_V2vGrYGkCvx8PpeYZ89GoSpv-_2dmG3A=.c08ba1b3-ff3c-4a76-8264-adb20d79a8ac@github.com>

Clean backport

-------------

Commit messages:
 - Backport 78836a579d83a98eab2e1431a1e7a20ea0d79149

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/47/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=47&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332094
  Stats: 104 lines in 5 files changed: 11 ins; 62 del; 31 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/47.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/47/head:pull/47

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/47

From wkemper at openjdk.org  Fri May 17 23:44:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:44:54 GMT
Subject: Integrated: 8332094: GenShen: Reuse existing code to verify usage
 before rebuilding free set
In-Reply-To: <2u7eDJNlJA_V2vGrYGkCvx8PpeYZ89GoSpv-_2dmG3A=.c08ba1b3-ff3c-4a76-8264-adb20d79a8ac@github.com>
References: <2u7eDJNlJA_V2vGrYGkCvx8PpeYZ89GoSpv-_2dmG3A=.c08ba1b3-ff3c-4a76-8264-adb20d79a8ac@github.com>
Message-ID: <CaFDu3_O97vThDjjOEh26RyaRbVaS_8FPHQldVp0dY0=.d1999a07-33d0-47d0-a973-133f27f45995@github.com>

On Fri, 17 May 2024 23:37:39 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport

This pull request has now been integrated.

Changeset: dbbc2d28
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/dbbc2d285897a1938b84767d9eefbc45828bfb56
Stats:     104 lines in 5 files changed: 11 ins; 62 del; 31 mod

8332094: GenShen: Reuse existing code to verify usage before rebuilding free set

Backport-of: 78836a579d83a98eab2e1431a1e7a20ea0d79149

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/47

From wkemper at openjdk.org  Fri May 17 23:48:19 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 17 May 2024 23:48:19 GMT
Subject: Integrated: 8332254: GenShen: Remove obsolete comments and revert
 unnecessary formatting changes
In-Reply-To: <FUilbrVYOs4rYt3ZxMW_yKGmpFvht-p3CrKc4p4FWHE=.524353a7-2336-4418-b9b3-6bcf37379c8c@github.com>
References: <FUilbrVYOs4rYt3ZxMW_yKGmpFvht-p3CrKc4p4FWHE=.524353a7-2336-4418-b9b3-6bcf37379c8c@github.com>
Message-ID: <osU5ftfIyq8a1nqCokCP_igPye_AYsBXVYRupOXUeyU=.5c05cb41-f7b9-4d4d-a25e-86cd09a0a5a5@github.com>

On Fri, 17 May 2024 23:39:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport

This pull request has now been integrated.

Changeset: 0719cab2
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/0719cab27076a05d8450f9cc928b67effb8c01d5
Stats:     11 lines in 1 file changed: 1 ins; 5 del; 5 mod

8332254: GenShen: Remove obsolete comments and revert unnecessary formatting changes

Backport-of: dfa788667714b4ec7e0e235e9c0c58cc5e3ceab4

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/48

From wkemper at openjdk.org  Sat May 18 00:00:57 2024
From: wkemper at openjdk.org (William Kemper)
Date: Sat, 18 May 2024 00:00:57 GMT
Subject: Integrated: 8332081: GenShen: Disentangle generational mode update
 references
Message-ID: <rwMv0bVVLvB0Js6GC-ZlJbvUoGFXRFDsdXKVfPQASa8=.33025eb2-2d5b-4973-89d8-b5e4a0a6b9b5@github.com>

No conflicts after rebasing on recent PRs.

-------------

Commit messages:
 - Backport def9d3feb212226688c6fd071723adc4e00cc361

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/49/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=49&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332081
  Stats: 439 lines in 6 files changed: 237 ins; 185 del; 17 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/49.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/49/head:pull/49

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/49

From wkemper at openjdk.org  Sat May 18 00:00:57 2024
From: wkemper at openjdk.org (William Kemper)
Date: Sat, 18 May 2024 00:00:57 GMT
Subject: Integrated: 8332081: GenShen: Disentangle generational mode update
 references
In-Reply-To: <rwMv0bVVLvB0Js6GC-ZlJbvUoGFXRFDsdXKVfPQASa8=.33025eb2-2d5b-4973-89d8-b5e4a0a6b9b5@github.com>
References: <rwMv0bVVLvB0Js6GC-ZlJbvUoGFXRFDsdXKVfPQASa8=.33025eb2-2d5b-4973-89d8-b5e4a0a6b9b5@github.com>
Message-ID: <er-eq2a8o7inOaC_z0bQUQ5Xt6lYgf7JshIXDoILaFI=.2bc757b3-787d-4152-8f71-c7b396cc674f@github.com>

On Fri, 17 May 2024 23:54:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

> No conflicts after rebasing on recent PRs.

This pull request has now been integrated.

Changeset: c7cae752
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/c7cae7522c90dfe8cb897346b305f364781ddeee
Stats:     439 lines in 6 files changed: 237 ins; 185 del; 17 mod

8332081: GenShen: Disentangle generational mode update references

Backport-of: def9d3feb212226688c6fd071723adc4e00cc361

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/49

From wkemper at openjdk.org  Sat May 18 00:01:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Sat, 18 May 2024 00:01:23 GMT
Subject: RFR: 8332095: GenShen: Move more generational mode members out of
 shHeap
Message-ID: <NaZxQ44AjWTspzw1xZ20TABvB6HNYC3Z-GluCUJU1bU=.daaa504d-26a2-44f0-a2de-7fb3fabd1e5d@github.com>

No conflicts after rebasing on other recent PRs

-------------

Commit messages:
 - Backport d77f426b0a7c8b2c7285ee43d978be2cf584a51f

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/50/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=50&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332095
  Stats: 101 lines in 11 files changed: 39 ins; 44 del; 18 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/50.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/50/head:pull/50

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/50

From wkemper at openjdk.org  Sat May 18 00:07:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Sat, 18 May 2024 00:07:14 GMT
Subject: Integrated: 8332095: GenShen: Move more generational mode members out
 of shHeap
In-Reply-To: <NaZxQ44AjWTspzw1xZ20TABvB6HNYC3Z-GluCUJU1bU=.daaa504d-26a2-44f0-a2de-7fb3fabd1e5d@github.com>
References: <NaZxQ44AjWTspzw1xZ20TABvB6HNYC3Z-GluCUJU1bU=.daaa504d-26a2-44f0-a2de-7fb3fabd1e5d@github.com>
Message-ID: <SpAHZI7bA4f02VfmkXit3G-hhW667Xm0PWUmz6xCWa8=.791f7ced-edd3-479c-9b58-be50beb599e2@github.com>

On Fri, 17 May 2024 23:56:09 GMT, William Kemper <wkemper at openjdk.org> wrote:

> No conflicts after rebasing on other recent PRs

This pull request has now been integrated.

Changeset: d0f69c97
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/d0f69c97b383dbc2043bcbe5ce6b1a8d99e6cba5
Stats:     101 lines in 11 files changed: 39 ins; 44 del; 18 mod

8332095: GenShen: Move more generational mode members out of shHeap

Backport-of: d77f426b0a7c8b2c7285ee43d978be2cf584a51f

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/50

From wkemper at openjdk.org  Sat May 18 00:16:19 2024
From: wkemper at openjdk.org (William Kemper)
Date: Sat, 18 May 2024 00:16:19 GMT
Subject: RFR: 8332082: Shenandoah: Use consistent tests to determine when
 pre-write barrier is active [v2]
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <NTmSqXSNO1ZK0X2KAR2vmVvVmG1rzA5T3no1dZIVgAc=.5d6834dc-30b1-4ffa-9a7c-d37108f2cdcb@github.com>

> This is consistent with c1 and other platforms.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Make all the barriers and verifiers use gc_state
   
   Still need to fix verifier for c2
 - Revert "Check for satb active flag, rather than gc state"
   
   This reverts commit 2769c97cbf5313c5c0f1336060ec39cb66584e3c.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19180/files
  - new: https://git.openjdk.org/jdk/pull/19180/files/2769c97c..3271fecf

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19180&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19180&range=00-01

  Stats: 73 lines in 7 files changed: 17 ins; 31 del; 25 mod
  Patch: https://git.openjdk.org/jdk/pull/19180.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19180/head:pull/19180

PR: https://git.openjdk.org/jdk/pull/19180

From wkemper at openjdk.org  Mon May 20 16:59:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 20 May 2024 16:59:25 GMT
Subject: RFR: 8332082: Shenandoah: Use consistent tests to determine when
 pre-write barrier is active [v3]
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>

> This is consistent with c1 and other platforms.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix typo

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19180/files
  - new: https://git.openjdk.org/jdk/pull/19180/files/3271fecf..cff7dc0c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19180&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19180&range=01-02

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/19180.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19180/head:pull/19180

PR: https://git.openjdk.org/jdk/pull/19180

From wkemper at openjdk.org  Mon May 20 22:20:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 20 May 2024 22:20:38 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
Message-ID: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>

Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.

-------------

Commit messages:
 - Cleanup IDE artifacts
 - Move some methods into shGenerationalHeap
 - Reduce generational mode checks

Changes: https://git.openjdk.org/shenandoah/pull/436/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=436&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332548
  Stats: 323 lines in 6 files changed: 124 ins; 157 del; 42 mod
  Patch: https://git.openjdk.org/shenandoah/pull/436.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/436/head:pull/436

PR: https://git.openjdk.org/shenandoah/pull/436

From ysr at openjdk.org  Tue May 21 01:23:32 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 21 May 2024 01:23:32 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v4]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:

  Small clean-ups.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/407/files
  - new: https://git.openjdk.org/shenandoah/pull/407/files/61848c82..b609d78b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=02-03

  Stats: 7 lines in 2 files changed: 1 ins; 4 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From shade at openjdk.org  Tue May 21 08:01:07 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 21 May 2024 08:01:07 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v4]
In-Reply-To: <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>
Message-ID: <5oqqqssGKeq67XEpuNL1T7g0U-X75igjYYXVWQb0Vq8=.73a6e149-bc83-46c9-8c0b-7c16059af533@github.com>

On Fri, 17 May 2024 15:58:32 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
>> 
>> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
>> 
>> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
>> 
>> Additional testing:
>>  - [x] Performance test reproducer from the bug improves significantly
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>>  - [x] Linux AArch64 server fastdebug, `all`
>>  - [x] Linux x86_64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add more GC triggers around

Thanks for reviews! @stefank, I assume you are fine with the way we (lightly) touched ZGC code?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19229#issuecomment-2121997820

From stefank at openjdk.org  Tue May 21 08:37:05 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 21 May 2024 08:37:05 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v4]
In-Reply-To: <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>
Message-ID: <dAYIR_hjjAWx-3QW3fP7XPKANrNfA7xEZWmwA6g33q8=.5845472f-0ef7-4c8a-b5ba-aed321b22a60@github.com>

On Fri, 17 May 2024 15:58:32 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
>> 
>> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
>> 
>> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
>> 
>> Additional testing:
>>  - [x] Performance test reproducer from the bug improves significantly
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>>  - [x] Linux AArch64 server fastdebug, `all`
>>  - [x] Linux x86_64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add more GC triggers around

Yes, ZGC code looks fine. Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19229#issuecomment-2122074293

From shade at openjdk.org  Tue May 21 15:00:08 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 21 May 2024 15:00:08 GMT
Subject: RFR: 8331572: Allow using OopMapCache outside of STW GC phases
 [v4]
In-Reply-To: <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
 <_mFVw8VmpUzTscas3PU4wFHW63mgIrEPlbGPo3iTMrM=.81b20124-b3dd-4264-9d23-e4fbfc79fc78@github.com>
Message-ID: <XZfsxMnBJEVaPX3XxoWp2FP4rdfV2f7q8NJfPW-P5Y8=.6e470c87-4e96-40b2-af12-2400356b2520@github.com>

On Fri, 17 May 2024 15:58:32 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
>> 
>> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
>> 
>> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
>> 
>> Additional testing:
>>  - [x] Performance test reproducer from the bug improves significantly
>>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>>  - [x] Linux AArch64 server fastdebug, `all`
>>  - [x] Linux x86_64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add more GC triggers around

All right, thank you all. Test re-run passes, so I am integrating.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19229#issuecomment-2122826846

From shade at openjdk.org  Tue May 21 15:00:09 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 21 May 2024 15:00:09 GMT
Subject: Integrated: 8331572: Allow using OopMapCache outside of STW GC phases
In-Reply-To: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
References: <Jhae5VCAxKNjByrzuq6FZJez3v9-1OD9hhj64RJeA0A=.9ce534db-e9d7-4d20-ac0d-c1820d799ddb@github.com>
Message-ID: <74gZNddHCA9MbyV0SP4Tha8ywNntRbFfNZC1lIHNkw0=.f76fd1a8-4f43-4382-ad36-31c539531e61@github.com>

On Tue, 14 May 2024 12:31:08 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> As the reproducer in the issue shows, we would also like to use the `OopMapCache` during the concurrent GC phases. Zhengyu mentions there is also a production problem for stack walking that would benefit from letting `OopMapCache` be used without looking at GC at all.
> 
> This PR unblocks `OopMapCache` uses for everything. Cleanups are nominally done by service thread. But, still appreciating that majority of use cases would be from GCs, we leave the proactive cleanups from the GC ops here as well. It requires the synchronization between readers that might be copying out the entries out of the hashmap and the concurrent reclamation. Handily, `GlobalCounter` can be used for that purpose. 
> 
> After this lands, I think we can go over `OopMapCache::compute_one_oop_map` uses and see if they would instead like to use the cached `lookup` to benefit from this cache too. I think those paths are for OSR and deopts, so their performance is unlikely to be critical. This PR already covers the concurrent GC paths well.
> 
> Additional testing:
>  - [x] Performance test reproducer from the bug improves significantly
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah` (10x)
>  - [x] Linux AArch64 server fastdebug, `all`
>  - [x] Linux x86_64 server fastdebug, `all`

This pull request has now been integrated.

Changeset: d999b81e
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/d999b81e7110751be402012e1ed41b3256f5895e
Stats:     105 lines in 10 files changed: 63 ins; 14 del; 28 mod

8331572: Allow using OopMapCache outside of STW GC phases

Co-authored-by: Zhengyu Gu <zgu at openjdk.org>
Reviewed-by: coleenp, zgu

-------------

PR: https://git.openjdk.org/jdk/pull/19229

From ysr at openjdk.org  Tue May 21 17:59:24 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 21 May 2024 17:59:24 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
In-Reply-To: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
Message-ID: <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>

On Mon, 20 May 2024 22:16:16 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.

Changes look mostly good to me. Just some very minor comments and a check about brackets in diff in one place.

Approving it, since any changes there should not require my rereview.

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 789:

> 787:   JavaThread* const jt = JavaThread::cast(thread);
> 788:   StackWatermarkSet::finish_processing(jt, _oops, StackWatermarkKind::gc);
> 789:   if (ShenandoahHeap::heap()->mode()->is_generational()) {

For patterns such as this, I wonder if the closure might just be enhanced with a `_generational_mode` boolean (or templatized, but that is usually ugly and makes debugging a pain) to do these kinds of checks, and whether it makes a difference from a readability or performance perspective. Not important but thought a uniform approach might make it easier. I personally prefer a const value embedded in closures nicer, but it would show up as a diff in legacy Shenandoah (which might still be acceptable to reviewers, I think). Anyway, nothing to actually do now, but leaving this comment here for people to think/dicsuss about from a maintainability and readability perspective for the future....

src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 175:

> 173:         // and this degenerated cycle. These pointers need to be included the 'read' table
> 174:         // used to scan the remembered set during the STW mark which follows here.
> 175:         _generation->merge_write_table();

Interesting. I am trying to understand this diff (the change in nesting looks curious because I didn't see anything change above. As a result, this specifically generational work(?) may have been pulled into the `else {` arm of the if-then-else at lines 145-149, introducing a bug?

Or is this intentional and I am missing some subtle reasoning here?

src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 286:

> 284: 
> 285:       if (heap->mode()->is_generational()) {
> 286:         ShenandoahGenerationalHeap::heap()->complete_degenerated_cycle();

Not a change that you made, but I wondered about the semantics of "complete" in the name "op_cleanup_complete". It appears to use a phase timer around its work. Was the idea to include the "complete_degenerated_cycle" work inside that phase for timers associated with whatever `ShenandoahPhaseTimings::degen_gc_cleanup_complete` wanted to track, or is it good as is to exclude that generational work which may be is separately tracked?

src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.hpp line 111:

> 109:   TransferResult balance_generations();
> 110: 
> 111:   // Balance generations, coalesce and fill old regions if necessary

To keep with surrounding documentation style, use the form "Balances generations, coalesces and fills ... etc."

The same comment might apply at line 53 further up too, but doesn't matter since there are no other comments nearby wrt which it would look inconsistent (in either the prescriptive or descriptive style).

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/436#pullrequestreview-2069167961
PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608710998
PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608694281
PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608683130
PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608662149

From ysr at openjdk.org  Tue May 21 17:59:24 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 21 May 2024 17:59:24 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
In-Reply-To: <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
 <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
Message-ID: <9xvnVmYdSoYszmUOXK7jrObOuaXStn8YJk1Tz3DBrbE=.e4e02dd0-a23f-4c81-90bd-9764de81f200@github.com>

On Tue, 21 May 2024 17:29:19 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.
>
> src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 175:
> 
>> 173:         // and this degenerated cycle. These pointers need to be included the 'read' table
>> 174:         // used to scan the remembered set during the STW mark which follows here.
>> 175:         _generation->merge_write_table();
> 
> Interesting. I am trying to understand this diff (the change in nesting looks curious because I didn't see anything change above. As a result, this specifically generational work(?) may have been pulled into the `else {` arm of the if-then-else at lines 145-149, introducing a bug?
> 
> Or is this intentional and I am missing some subtle reasoning here?

Or may be just an issue with how this displays in the diffs. Worth a quick check.

> src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 286:
> 
>> 284: 
>> 285:       if (heap->mode()->is_generational()) {
>> 286:         ShenandoahGenerationalHeap::heap()->complete_degenerated_cycle();
> 
> Not a change that you made, but I wondered about the semantics of "complete" in the name "op_cleanup_complete". It appears to use a phase timer around its work. Was the idea to include the "complete_degenerated_cycle" work inside that phase for timers associated with whatever `ShenandoahPhaseTimings::degen_gc_cleanup_complete` wanted to track, or is it good as is to exclude that generational work which may be is separately tracked?

(fwiw, I didn't see any obvious phase timers around the work in the generational `complete_degenerated_cycle`. Should there be any?)

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608723088
PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608685213

From wkemper at openjdk.org  Tue May 21 21:11:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 21 May 2024 21:11:33 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
In-Reply-To: <9xvnVmYdSoYszmUOXK7jrObOuaXStn8YJk1Tz3DBrbE=.e4e02dd0-a23f-4c81-90bd-9764de81f200@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
 <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
 <9xvnVmYdSoYszmUOXK7jrObOuaXStn8YJk1Tz3DBrbE=.e4e02dd0-a23f-4c81-90bd-9764de81f200@github.com>
Message-ID: <PEI-fsj8wZlEabaa_R3in7zbgFB_sx1DWfP5Qjvd1jk=.a05f47a4-66a8-451a-90b2-bcd5338d7bfb@github.com>

On Tue, 21 May 2024 17:21:13 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 286:
>> 
>>> 284: 
>>> 285:       if (heap->mode()->is_generational()) {
>>> 286:         ShenandoahGenerationalHeap::heap()->complete_degenerated_cycle();
>> 
>> Not a change that you made, but I wondered about the semantics of "complete" in the name "op_cleanup_complete". It appears to use a phase timer around its work. Was the idea to include the "complete_degenerated_cycle" work inside that phase for timers associated with whatever `ShenandoahPhaseTimings::degen_gc_cleanup_complete` wanted to track, or is it good as is to exclude that generational work which may be is separately tracked?
>
> (fwiw, I didn't see any obvious phase timers around the work in the generational `complete_degenerated_cycle`. Should there be any?)

We have a phase timer for coalesce and fill:

    ShenandoahGCPhase phase(ShenandoahPhaseTimings::degen_gc_coalesce_and_fill);

We could introduce a timer around this extracted method, but I expect coalesce and fill will be the long pole.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608953899

From wkemper at openjdk.org  Tue May 21 21:16:24 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 21 May 2024 21:16:24 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
In-Reply-To: <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
 <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
Message-ID: <dJAO4xRu2J5dxtE0hQhSjoLvvqmKvuxnnvqGJt-qmnA=.d3c5acf9-e44b-499f-b764-f76caa3dac49@github.com>

On Tue, 21 May 2024 17:44:16 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 789:
> 
>> 787:   JavaThread* const jt = JavaThread::cast(thread);
>> 788:   StackWatermarkSet::finish_processing(jt, _oops, StackWatermarkKind::gc);
>> 789:   if (ShenandoahHeap::heap()->mode()->is_generational()) {
> 
> For patterns such as this, I wonder if the closure might just be enhanced with a `_generational_mode` boolean (or templatized, but that is usually ugly and makes debugging a pain) to do these kinds of checks, and whether it makes a difference from a readability or performance perspective. Not important but thought a uniform approach might make it easier. I personally prefer a const value embedded in closures nicer, but it would show up as a diff in legacy Shenandoah (which might still be acceptable to reviewers, I think). Anyway, nothing to actually do now, but leaving this comment here for people to think/dicsuss about from a maintainability and readability perspective for the future....

Yes, there are many cases where the singleton is accessed directly. I also do not prefer this approach because it makes it difficult to run such classes in a unit test. I will refactor this.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608958237

From wkemper at openjdk.org  Tue May 21 21:16:24 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 21 May 2024 21:16:24 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
In-Reply-To: <9xvnVmYdSoYszmUOXK7jrObOuaXStn8YJk1Tz3DBrbE=.e4e02dd0-a23f-4c81-90bd-9764de81f200@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
 <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
 <9xvnVmYdSoYszmUOXK7jrObOuaXStn8YJk1Tz3DBrbE=.e4e02dd0-a23f-4c81-90bd-9764de81f200@github.com>
Message-ID: <SHAxX_g7mEk5ULzCNWXWGclI9plMveziC5k6GoWH66U=.029ed25b-0c0a-4f65-afd2-d3951ac5a33f@github.com>

On Tue, 21 May 2024 17:55:04 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 175:
>> 
>>> 173:         // and this degenerated cycle. These pointers need to be included the 'read' table
>>> 174:         // used to scan the remembered set during the STW mark which follows here.
>>> 175:         _generation->merge_write_table();
>> 
>> Interesting. I am trying to understand this diff (the change in nesting looks curious because I didn't see anything change above. As a result, this specifically generational work(?) may have been pulled into the `else {` arm of the if-then-else at lines 145-149, introducing a bug?
>> 
>> Or is this intentional and I am missing some subtle reasoning here?
>
> Or may be just an issue with how this displays in the diffs. Worth a quick check.

Good eye. I pulled this into the preceding `else` block because `merge_write_table` really means `merge_write_card_table`. It's only for the generational mode, so we don't need to execute it in non-generational modes as was done before this change.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608956514

From wkemper at openjdk.org  Tue May 21 21:29:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 21 May 2024 21:29:21 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
In-Reply-To: <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
 <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
Message-ID: <I8J2LaZvpw9hS3p-PFHbJ8lY2K5SSbCg6WcVgALVOyc=.04f9dfb7-419e-4d71-9ce9-6ba68dbf7621@github.com>

On Tue, 21 May 2024 17:00:30 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.hpp line 111:
> 
>> 109:   TransferResult balance_generations();
>> 110: 
>> 111:   // Balance generations, coalesce and fill old regions if necessary
> 
> To keep with surrounding documentation style, use the form "Balances generations, coalesces and fills ... etc."
> 
> The same comment might apply at line 53 further up too, but doesn't matter since there are no other comments nearby wrt which it would look inconsistent (in either the prescriptive or descriptive style).

Done.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1608971843

From wkemper at openjdk.org  Tue May 21 22:10:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 21 May 2024 22:10:41 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
 [v2]
In-Reply-To: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
Message-ID: <-FssnbnFYogREiSttyrixm3EQLsK6tZbHKyoSKwNZ7g=.34a2580c-1dfb-44d5-8bf5-4c352cea4fae@github.com>

> Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Templatize evact update task and closure to prevent excessive mode checks
 - Use descriptive style in comments for consistency

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/436/files
  - new: https://git.openjdk.org/shenandoah/pull/436/files/31ffa8f6..3c5cf2e1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=436&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=436&range=00-01

  Stats: 30 lines in 2 files changed: 7 ins; 6 del; 17 mod
  Patch: https://git.openjdk.org/shenandoah/pull/436.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/436/head:pull/436

PR: https://git.openjdk.org/shenandoah/pull/436

From ysr at openjdk.org  Tue May 21 22:18:23 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 21 May 2024 22:18:23 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
 [v2]
In-Reply-To: <SHAxX_g7mEk5ULzCNWXWGclI9plMveziC5k6GoWH66U=.029ed25b-0c0a-4f65-afd2-d3951ac5a33f@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
 <VLzi-3EZqi2jhxFmiez6_vYBbCdIxYwW5AfrIak09nM=.925bb9f0-c5da-4e5f-a4d7-30002f220490@github.com>
 <9xvnVmYdSoYszmUOXK7jrObOuaXStn8YJk1Tz3DBrbE=.e4e02dd0-a23f-4c81-90bd-9764de81f200@github.com>
 <SHAxX_g7mEk5ULzCNWXWGclI9plMveziC5k6GoWH66U=.029ed25b-0c0a-4f65-afd2-d3951ac5a33f@github.com>
Message-ID: <rLr9ujwJ6tZh_pKi0VhQHot4FUrTvy7rJ1iq-V7fxb8=.9a883735-c557-4c6a-919f-6870cb45672f@github.com>

On Tue, 21 May 2024 21:11:37 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Or may be just an issue with how this displays in the diffs. Worth a quick check.
>
> Good eye. I pulled this into the preceding `else` block because `merge_write_table` really means `merge_write_card_table`. It's only for the generational mode, so we don't need to execute it in non-generational modes as was done before this change.

ah, ok. I see that now. The diff had looked weird to me. Looks good with yr help above :-)

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/436#discussion_r1609008154

From ysr at openjdk.org  Tue May 21 22:21:23 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 21 May 2024 22:21:23 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
 [v2]
In-Reply-To: <-FssnbnFYogREiSttyrixm3EQLsK6tZbHKyoSKwNZ7g=.34a2580c-1dfb-44d5-8bf5-4c352cea4fae@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
 <-FssnbnFYogREiSttyrixm3EQLsK6tZbHKyoSKwNZ7g=.34a2580c-1dfb-44d5-8bf5-4c352cea4fae@github.com>
Message-ID: <4DV381Now8X89l1ajDA2i2oebhhMQIZLbvm3Q16EUsE=.ed51e7c7-efba-4937-8f2f-cf51eb3188a6@github.com>

On Tue, 21 May 2024 22:10:41 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Templatize evact update task and closure to prevent excessive mode checks
>  - Use descriptive style in comments for consistency

LGTM; thanks for the changes!

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/436#pullrequestreview-2069720912

From wkemper at openjdk.org  Tue May 21 22:56:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 21 May 2024 22:56:20 GMT
Subject: RFR: 8332548: GenShen: Factor generational mode out of gc helpers
 [v2]
In-Reply-To: <-FssnbnFYogREiSttyrixm3EQLsK6tZbHKyoSKwNZ7g=.34a2580c-1dfb-44d5-8bf5-4c352cea4fae@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
 <-FssnbnFYogREiSttyrixm3EQLsK6tZbHKyoSKwNZ7g=.34a2580c-1dfb-44d5-8bf5-4c352cea4fae@github.com>
Message-ID: <Y40x8a-KSnZkm3Kg9wUaAPS9Sq00PWUz_x6Zdic4BzE=.7ffe040b-d4ec-4081-b1f5-aaebb655782e@github.com>

On Tue, 21 May 2024 22:10:41 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Templatize evact update task and closure to prevent excessive mode checks
>  - Use descriptive style in comments for consistency

Thanks for the review!

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/436#issuecomment-2123559651

From wkemper at openjdk.org  Tue May 21 22:56:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 21 May 2024 22:56:20 GMT
Subject: Integrated: 8332548: GenShen: Factor generational mode out of gc
 helpers
In-Reply-To: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
References: <BrKMUultS2-E0a6u1wpVS-FE2y7G3iVf9ZqibLmtDWs=.33eb5619-f2e4-423b-9736-b0757e7fd278@github.com>
Message-ID: <uGU_E6sGePCVYTvXIExwL0kWh_z7YzxixU9s9s1yvNc=.4462f478-30ed-40fc-a869-07da4fd50cf4@github.com>

On Mon, 20 May 2024 22:16:16 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Some of the generational mode support in `shenandoahConcurrentGC.cpp` and `shenandoahDegeneratedGC.cpp` can be factored into generational mode specific files and coalesced under fewer generational mode checks.

This pull request has now been integrated.

Changeset: 8f808276
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/8f808276edfb163c14223f642f12dc6faddc49c6
Stats:     340 lines in 6 files changed: 124 ins; 156 del; 60 mod

8332548: GenShen: Factor generational mode out of gc helpers

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/436

From azafari at openjdk.org  Wed May 22 08:34:25 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 22 May 2024 08:34:25 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API
Message-ID: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>

This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
  1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.

  2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT region-list and when a new reserve happens at that regions, NMT complains by raising an exception.

-------------

Commit messages:
 - 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument to family of os::reserve/commit/uncommit memory API

Changes: https://git.openjdk.org/jdk/pull/19343/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19343&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331539
  Stats: 609 lines in 62 files changed: 78 ins; 148 del; 383 mod
  Patch: https://git.openjdk.org/jdk/pull/19343.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19343/head:pull/19343

PR: https://git.openjdk.org/jdk/pull/19343

From shade at openjdk.org  Wed May 22 10:20:06 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 22 May 2024 10:20:06 GMT
Subject: RFR: 8332082: Shenandoah: Use consistent tests to determine when
 pre-write barrier is active [v3]
In-Reply-To: <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
 <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>
Message-ID: <nyTNAUJHT1SL0VHtVh_z4RqJ3zsEI7abxrn12Cw4CKE=.7eabe67f-e3b6-4b61-8de3-f67551401366@github.com>

On Mon, 20 May 2024 16:59:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This is consistent with c1 and other platforms.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix typo

Dang. So we are regressing C1 performance a little here, by changing the single load-cmp to load-and-cmp. That is unfortunate, but might be acceptable?

src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp line 75:

> 73: 
> 74:   // Create a mask to test if the marking bit is set.
> 75:   // TODO: can we directly test if bit is set?

No, we cannot: C1 LIR does not have a corresponding operation.

-------------

PR Review: https://git.openjdk.org/jdk/pull/19180#pullrequestreview-2070736549
PR Review Comment: https://git.openjdk.org/jdk/pull/19180#discussion_r1609666303

From azafari at openjdk.org  Wed May 22 12:09:14 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 22 May 2024 12:09:14 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
Message-ID: <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>

> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
> 
>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT region-list and when a new reserve happens at that regions, NMT complains by raising an exception.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  fixed the missing parts of shenandoahHeap.cpp

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19343/files
  - new: https://git.openjdk.org/jdk/pull/19343/files/6b6e2e12..86ae1e37

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19343&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19343&range=00-01

  Stats: 122 lines in 1 file changed: 94 ins; 23 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/19343.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19343/head:pull/19343

PR: https://git.openjdk.org/jdk/pull/19343

From dcubed at openjdk.org  Wed May 22 16:11:03 2024
From: dcubed at openjdk.org (Daniel D. Daugherty)
Date: Wed, 22 May 2024 16:11:03 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
Message-ID: <k_yZcgEoXpaDS5atXt4TL2yT-dseOXXOTe5ToyodYsQ=.02c727e5-3794-4810-a3c8-0f777478a353@github.com>

On Wed, 22 May 2024 12:09:14 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
>> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
>> 
>>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT region-list and when a new reserve happens at that regions, NMT complains by raising an exception.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fixed the missing parts of shenandoahHeap.cpp

Please document the testing that has been done on this PR.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19343#issuecomment-2125172737

From wkemper at openjdk.org  Wed May 22 17:34:04 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 22 May 2024 17:34:04 GMT
Subject: RFR: 8332082: Shenandoah: Use consistent tests to determine when
 pre-write barrier is active [v3]
In-Reply-To: <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
 <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>
Message-ID: <KGD_T9-teIevY0uVfiZlCbjhYsd2ub3jsZkV_7CfjZU=.dec7474f-25cd-41fe-af20-1bd3cb32d978@github.com>

On Mon, 20 May 2024 16:59:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This is consistent with c1 and other platforms.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix typo

We usually benchmark with tiered compilation disabled. I will run some tests with tiered compilation configured to stop at level 1.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19180#issuecomment-2125387707

From duke at openjdk.org  Thu May 23 11:19:13 2024
From: duke at openjdk.org (Neethu)
Date: Thu, 23 May 2024 11:19:13 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
Message-ID: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>

**Notes**
We are spending significant time on acquiring the per-nmethod as all the
threads are in same nmethod.
Adding double-check lock by calling is_armed before lock acquisition.

**Verification**

Shenendoah

> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms

ZGC

> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms

**Issue**
https://bugs.openjdk.org/browse/JDK-8331911

-------------

Commit messages:
 - 8331911: Fix whitespace
 - 8331911: Reconsider locking for recently disarmed nmethods

Changes: https://git.openjdk.org/jdk/pull/19285/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19285&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331911
  Stats: 25 lines in 3 files changed: 22 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/19285.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19285/head:pull/19285

PR: https://git.openjdk.org/jdk/pull/19285

From shade at openjdk.org  Thu May 23 11:19:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 23 May 2024 11:19:14 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
In-Reply-To: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
Message-ID: <ItPLVmJ_m9lPmaczn0zYcON_6iegkQbYA5SJMS2xtTM=.35ed851f-d76d-431f-859f-778330600ef1@github.com>

On Fri, 17 May 2024 13:51:57 GMT, Neethu <duke at openjdk.org> wrote:

> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
> 
> **Verification**
> 
> Shenendoah
> 
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
> 
> ZGC
> 
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
> 
> **Issue**
> https://bugs.openjdk.org/browse/JDK-8331911

I like this. Some stylistic comments.

src/hotspot/share/gc/shenandoah/shenandoahBarrierSetNMethod.cpp line 2:

> 1: /*
> 2:  * Copyright (c) 2019, 2024, Red Hat, Inc. All rights reserved.

This update is unnecessary.

src/hotspot/share/gc/shenandoah/shenandoahBarrierSetNMethod.cpp line 39:

> 37: 
> 38: bool ShenandoahBarrierSetNMethod::nmethod_entry_barrier(nmethod* nm) {
> 39: 

Here and later: no need for new line at the beginning of the method.

src/hotspot/share/gc/shenandoah/shenandoahBarrierSetNMethod.cpp line 42:

> 40:   if (!is_armed(nm)) {
> 41:     // Some other thread got here first and healed the oops
> 42:     // and disarmed the nmethod.

Suggestion for the comment (here and later):


    // Some other thread got here first and healed the oops
    // and disarmed the nmethod. No need to continue.


...and then later, under the lock:


    // Some other thread managed to complete while we were
    // waiting for lock. No need to continue.

src/hotspot/share/gc/z/zBarrierSetNMethod.cpp line 42:

> 40: 
> 41:   if (!is_armed(nm)) {
> 42:     log_develop_trace(gc, nmethod)("nmethod: " PTR_FORMAT " visited by entry (disarmed)", p2i(nm));

Should it be "(disarmed before lock)" to disambiguate against "(disarmed)" later?

-------------

PR Review: https://git.openjdk.org/jdk/pull/19285#pullrequestreview-2070558538
PR Review Comment: https://git.openjdk.org/jdk/pull/19285#discussion_r1609570657
PR Review Comment: https://git.openjdk.org/jdk/pull/19285#discussion_r1609563104
PR Review Comment: https://git.openjdk.org/jdk/pull/19285#discussion_r1609578850
PR Review Comment: https://git.openjdk.org/jdk/pull/19285#discussion_r1609571647

From stefank at openjdk.org  Thu May 23 13:08:07 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 23 May 2024 13:08:07 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
Message-ID: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>

On Wed, 22 May 2024 12:09:14 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
>> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
>> 
>>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
>> 
>> Tests:
>> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fixed the missing parts of shenandoahHeap.cpp

I've left an initial set of comments.

src/hotspot/os/posix/os_posix.cpp line 386:

> 384:   if (begin_offset > 0) {
> 385:     if (os::release_memory(extra_base, begin_offset))
> 386:     {

The `{` should be moved to the line above.

src/hotspot/os/posix/os_posix.cpp line 387:

> 385:     if (os::release_memory(extra_base, begin_offset))
> 386:     {
> 387:       ThreadCritical tc;

In many of the functions we put the `ThreadCritical` inside the `MemTracker` after the `enabled()` check, but we don't do it here. Why is that? Shouldn't the `ThreadCritical` usage be hidden inside `MemTracker`?

src/hotspot/share/cds/metaspaceShared.cpp line 1088:

> 1086: #endif // ASSERT
> 1087: 
> 1088:     if (archive_space_rs.is_reserved()) {

We've already asserted that this should be true, so this if should not be needed.

src/hotspot/share/cds/metaspaceShared.cpp line 1092:

> 1090:                     p2i(archive_space_rs.base()), p2i(archive_space_rs.end()), archive_space_rs.size());
> 1091:     }
> 1092:     if (class_space_rs.is_reserved()) {

`class_space_rs.is_reserved()` is asserted if `if (Metaspace::using_class_space())` is taken. I think this could be changed to:
Suggestion:

    if (Metaspace::using_class_space()) {

src/hotspot/share/cds/metaspaceShared.cpp line 1341:

> 1339:   } else {
> 1340:     if (use_archive_base_addr && base_address != nullptr) {
> 1341:       total_space_rs = ReservedSpace(total_range_size, archive_space_alignment,

Can you explain why you changed this?

It's also interesting that after this change we only use `base_address_alignment` in asserts. I think this indicates that something should be cleaned up / fixed here.

src/hotspot/share/cds/metaspaceShared.cpp line 1370:

> 1368:                                                      ccs_begin_offset, mtClassShared, mtClass);
> 1369:   }
> 1370:   assert(archive_space_rs.is_reserved(), "Archive space is not reserved.");

Something is dubious about the code above:

    archive_space_rs = total_space_rs.first_part(ccs_begin_offset,
                                                 (size_t)archive_space_alignment);
    class_space_rs = total_space_rs.last_part(ccs_begin_offset);
    MemTracker::record_virtual_memory_split_reserved(total_space_rs.base(), total_space_rs.size(),
                                                     ccs_begin_offset, mtClassShared, mtClass);


In one path `total_space_rs` gets initialized with `mtClass` and in another path it gets initialized with `mtClassShared`. This means that we always get the wrong flag in one of `archive_space_rs` and `class_space_rs`.

src/hotspot/share/memory/virtualspace.hpp line 63:

> 61:   //                       it should not change after.
> 62:   //  * _alignment       - Not to be changed after initialization
> 63:   //  * _executable      - Not to be changed after initialization

I think this would be a good change to do in the future, but currently this isn't true. `clear_members` do clear these fields, so I think you should remove these two lines.

src/hotspot/share/memory/virtualspace.hpp line 76:

> 74: 
> 75:   MEMFLAGS nmt_flag() const { assert(is_reserved(), "Memory region is not reserved."); assert(_flag != mtNone, "Memory flag is not set."); return _flag; }
> 76: 

Looking at this again, and realize that this function should probably be moved to the other accessors below.

src/hotspot/share/memory/virtualspace.hpp line 98:

> 96:   bool   special()         const { assert(is_reserved(), "Memory region is not reserved."); return _special;   }
> 97:   bool   executable()      const { assert(is_reserved(), "Memory region is not reserved."); return _executable;   }
> 98:   size_t noaccess_prefix() const { assert(is_reserved(), "Memory region is not reserved."); return _noaccess_prefix;   }

FWIW, this change comes from one of my debugging sessions. I think it is good to have these asserts, I just wish they could says something like `assert(is_initialized(), ...)` to more clearly convey why we are doing this check.

We are considering if there are ways to split ReservedSpace into two classes, one that handles reserving of memory and one that is a plain view of already reserved memory. If/when we do such a change we could consider updating these asserts to be more legible.

In the meantime, it would be nice to change the string to "Fields not initialized" (and get rid of the `.`).

src/hotspot/share/nmt/virtualMemoryTracker.cpp line 506:

> 504:       return true;
> 505:     assert(reserved_rgn->end() == rgn.end() || reserved_rgn->base() == rgn.base(), "extra memory should be at either end of the region.");
> 506:   }

This seems like an extreme hack. I understand that this just follows the tradition of the rest of the hacks in this file, but can't this be better handled in the CDS layer above?

-------------

Changes requested by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19343#pullrequestreview-2073757670
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611620811
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611624784
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611566535
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611567731
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611582321
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611608926
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611629459
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611630735
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611641554
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1611657743

From wkemper at openjdk.org  Thu May 23 13:35:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 23 May 2024 13:35:44 GMT
Subject: RFR: 8332824: GenShen: Temporarily revert recent changes to GC helpers
Message-ID: <Zl2OWOsnIALCm688Nam2M3I-yKF1XhgOO8RJ9ftYNm0=.5f332cdc-70d0-4ec8-8f1f-a425e5d087d3@github.com>

This reverts commit 8f808276edfb163c14223f642f12dc6faddc49c6. This commit is breaking in stress test pipelines. I'm backing it out while it is debugged.

-------------

Commit messages:
 - Revert "8332548: GenShen: Factor generational mode out of gc helpers"

Changes: https://git.openjdk.org/shenandoah/pull/437/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=437&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332824
  Stats: 342 lines in 6 files changed: 158 ins; 126 del; 58 mod
  Patch: https://git.openjdk.org/shenandoah/pull/437.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/437/head:pull/437

PR: https://git.openjdk.org/shenandoah/pull/437

From wkemper at openjdk.org  Thu May 23 14:21:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 23 May 2024 14:21:32 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <6bFbVDMMrxaSSTuQpNNE0BAjzmu2HG3dbhU63x8wDC8=.b5920329-aa3c-4de7-82f5-d437852fff6e@github.com>

Merges tag jdk-21.0.4+4

-------------

Commit messages:
 - 8321107: Add more test cases for JDK-8319372
 - 8316138: Add GlobalSign 2 TLS root certificates
 - 8320715: Improve the tests of test/hotspot/jtreg/compiler/intrinsics/float16
 - 8331298: avoid alignment checks in UBSAN enabled build
 - 8312218: Print additional debug information when hitting assert(in_hash)
 - 8308144: Uncontrolled memory consumption in SSLFlowDelegate.Reader
 - 8299023: TestPLABResize.java and TestPLABPromotion.java are failing intermittently
 - 8329663: hs_err file event log entry for thread adding/removing should print current thread
 - 8330615: avoid signed integer overflows in zip_util.c readCen / hashN
 - 8331167: UBSan enabled build fails in adlc on macOS
 - ... and 5 more: https://git.openjdk.org/shenandoah-jdk21u/compare/035e3144...c44d4e93

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/51/files
  Stats: 1020 lines in 27 files changed: 947 ins; 0 del; 73 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/51.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/51/head:pull/51

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/51

From kdnilsen at openjdk.org  Thu May 23 15:44:37 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 23 May 2024 15:44:37 GMT
Subject: RFR: 8332824: GenShen: Temporarily revert recent changes to GC
 helpers
In-Reply-To: <Zl2OWOsnIALCm688Nam2M3I-yKF1XhgOO8RJ9ftYNm0=.5f332cdc-70d0-4ec8-8f1f-a425e5d087d3@github.com>
References: <Zl2OWOsnIALCm688Nam2M3I-yKF1XhgOO8RJ9ftYNm0=.5f332cdc-70d0-4ec8-8f1f-a425e5d087d3@github.com>
Message-ID: <hCJXXs4peprNodLt70naBXVOfSByX-DlaWyO8vUXzqg=.def7e5ae-37a0-4880-b9a7-dfa809ebfcf1@github.com>

On Thu, 23 May 2024 13:31:46 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This reverts commit 8f808276edfb163c14223f642f12dc6faddc49c6. This commit is breaking in stress test pipelines. I'm backing it out while it is debugged.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/437#pullrequestreview-2074366011

From wkemper at openjdk.org  Thu May 23 17:47:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 23 May 2024 17:47:03 GMT
Subject: RFR: 8332082: Shenandoah: Use consistent tests to determine when
 pre-write barrier is active [v3]
In-Reply-To: <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
 <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>
Message-ID: <4G16f8wLsBeOVAGQthPsdtG4UcdwykXyEFYy1w2BiFk=.b5d697f2-0455-4807-af8e-3de60b397f65@github.com>

On Mon, 20 May 2024 16:59:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This is consistent with c1 and other platforms.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix typo

Total time here is defined as the sum of 10 measurement runs for each of the 20 working benchmarks in the dacapo suite. The total time was taken for `jdk:master` and this PR with `-XX:TieredStopAtLevel=1` to keep all benchmarks running in C1.

jdk:master:
  Category |  Count |         Total |      GeoMean |      Average |     Trim 0.1 |       StdDev |      Minimum |      Maximum
    result |    200 |   2110857.000 |     7656.397 |    10554.285 |     9063.875 |     8894.464 |     1159.000 |    36836.000

This PR:
  Category |  Count |         Total |      GeoMean |      Average |     Trim 0.1 |       StdDev |      Minimum |      Maximum
    result |    200 |   2139852.000 |     7711.781 |    10699.260 |     9129.150 |     9166.330 |     1167.000 |    37336.000

This is a 1.4% increase in total time when running with just C1. The increase at trimmed mean 10% is just 0.7%. I'm okay with this.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19180#issuecomment-2127724163

From duke at openjdk.org  Thu May 23 18:06:13 2024
From: duke at openjdk.org (Neethu)
Date: Thu, 23 May 2024 18:06:13 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
 [v2]
In-Reply-To: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
Message-ID: <oMBF1BM4AzkwUW-_JR5SidmcZG7kqUkG2Visz3iRDQY=.ce83a05a-3ff5-4426-af6d-037528b79f3a@github.com>

> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
> 
> **Verification**
> 
> Shenendoah
> 
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
> 
> ZGC
> 
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
> 
> **Issue**
> https://bugs.openjdk.org/browse/JDK-8331911

Neethu has updated the pull request incrementally with one additional commit since the last revision:

  Address feedback on whitespace and comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19285/files
  - new: https://git.openjdk.org/jdk/pull/19285/files/5a0d5562..8c5bad2e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19285&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19285&range=00-01

  Stats: 16 lines in 3 files changed: 0 ins; 3 del; 13 mod
  Patch: https://git.openjdk.org/jdk/pull/19285.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19285/head:pull/19285

PR: https://git.openjdk.org/jdk/pull/19285

From shade at openjdk.org  Thu May 23 18:10:02 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 23 May 2024 18:10:02 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
 [v2]
In-Reply-To: <oMBF1BM4AzkwUW-_JR5SidmcZG7kqUkG2Visz3iRDQY=.ce83a05a-3ff5-4426-af6d-037528b79f3a@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
 <oMBF1BM4AzkwUW-_JR5SidmcZG7kqUkG2Visz3iRDQY=.ce83a05a-3ff5-4426-af6d-037528b79f3a@github.com>
Message-ID: <hEJvygGsZ4NqMnwXRf1auvlbNo9f5Svc8PGEhAzPs2o=.2fb3df48-c43f-4f1a-82fb-9984059ecb9e@github.com>

On Thu, 23 May 2024 18:06:13 GMT, Neethu <duke at openjdk.org> wrote:

>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>> 
>> **Verification**
>> 
>> Shenendoah
>> 
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>> 
>> ZGC
>> 
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>> 
>> **Issue**
>> https://bugs.openjdk.org/browse/JDK-8331911
>
> Neethu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address feedback on whitespace and comments

I think this is good. 

Yes, we could have restructured the code so that `nmethod_entry_barrier` was not called when nmethod is already disarmed. There are already some places where we check it externally, but the reproducer in the bug shows that it is easy to miss. So checking right here in the method looks appropriate.

@fisk might want to take a look as well.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19285#pullrequestreview-2074693680

From shade at openjdk.org  Thu May 23 18:18:02 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 23 May 2024 18:18:02 GMT
Subject: RFR: 8332082: Shenandoah: Use consistent tests to determine when
 pre-write barrier is active [v3]
In-Reply-To: <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
 <7YitGep10T35vf9lzitE2Oz3A9XwZywdDpgeiQoMXho=.7bb368d9-ea10-447d-ad29-6429f8ef6631@github.com>
Message-ID: <CSeiydrW8mRtYYIoxeDtK20hI7gQY5IbGfGlCgh7ppw=.27037fcd-443b-4505-b23f-912127a66af9@github.com>

On Mon, 20 May 2024 16:59:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This is consistent with c1 and other platforms.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix typo

Agreed, this is acceptable. We could resolve it in future by expanding gc-state to several bytes.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19180#pullrequestreview-2074711115

From wkemper at openjdk.org  Thu May 23 19:06:17 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 23 May 2024 19:06:17 GMT
Subject: RFR: 8332841: GenShen: Pull shared members from control thread into
 common base class
Message-ID: <TokKZVWFQvTcDmf7ZaYzexaHM9sI5wmU94kX5B7gWUc=.618edc2f-e95a-4476-9ebb-d6ac7a8387b3@github.com>

This change facilitates further extensions from this base class in the forthcoming generational mode for Shenandoah.

-------------

Commit messages:
 - Pull shared members from control thread into common base class

Changes: https://git.openjdk.org/jdk/pull/19372/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19372&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332841
  Stats: 365 lines in 5 files changed: 224 ins; 129 del; 12 mod
  Patch: https://git.openjdk.org/jdk/pull/19372.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19372/head:pull/19372

PR: https://git.openjdk.org/jdk/pull/19372

From wkemper at openjdk.org  Thu May 23 21:43:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 23 May 2024 21:43:23 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v4]
In-Reply-To: <DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
 <DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>
Message-ID: <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>

On Tue, 21 May 2024 01:23:32 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>> 
>> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
>> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
>> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>> 
>> Asserts check the protocol for setting and clearing the variables.
>> 
>> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>> 
>> *Testing*:
>> - [x] code pipeline
>> - [x] specjbb testing
>> - [ ] specjbb performance
>> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
>> - [x] GHA
>
> Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Small clean-ups.

Left a few minor coding suggestings. On a higher level, it's not clear to me when and which threads should use `gc_generation` or `active_generation`. I gather that active_generation is set on the safepoint and `gc_generation` is set by control thread, does that mean any code before `init-mark` needs to use `gc_generation`?

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 931:

> 929:   if (!CompressedOops::is_null(obj)) {
> 930:     if (!_mark_context->is_marked(obj)) {
> 931:       _heap->assert_generations_reconciled();

Could put this in `shenandoahAsserts.hpp` as macro: `shenandoah_assert_generations_reconciled()`.

src/hotspot/share/gc/shenandoah/shenandoahController.cpp line 31:

> 29: #include "gc/shenandoah/shenandoahHeapRegion.inline.hpp"
> 30: 
> 31: Thread* ShenandoahController::_thread = nullptr;

I don't think this is necessary. We already have `ShenandoahHeap::control_thread()`, which is the same thing. That is `ShenandoahControlThread::this == Thread::current() == ShenandoahHeap::_control_thread`.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalEvacuationTask.cpp line 134:

> 132:   {
> 133:     const size_t old_garbage_threshold = (ShenandoahHeapRegion::region_size_bytes() * ShenandoahOldGarbageThreshold) / 100;
> 134:     assert(_heap->gc_generation()->is_mark_complete(), "sanity");

Could use `_heap->gc_generation()->complete_marking_context()` on line 129, which just asserts that `is_mark_complete` before returning marking context.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1578:

> 1576: void ShenandoahHeap::set_gc_generation(ShenandoahGeneration* generation) {
> 1577:   _gc_generation = generation;
> 1578:   assert(Thread::current() == ShenandoahController::thread() ||

Could use `shenandoah_assert_control_or_vm_thread` from `shenandoahAsserts.hpp` here?

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 376:

> 374: 
> 375:   assert(is_in(obj), "only check if is in active generation for objects (" PTR_FORMAT ") in heap", p2i(obj));
> 376:   assert(gen == (ShenandoahGeneration*)old_generation() ||

These casts shouldn't be necessary any more.

-------------

Changes requested by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/407#pullrequestreview-2074963704
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1612282304
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1612299050
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1612286279
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1612289975
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1612294430

From ysr at openjdk.org  Thu May 23 22:34:03 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 23 May 2024 22:34:03 GMT
Subject: RFR: 8332841: GenShen: Pull shared members from control thread
 into common base class
In-Reply-To: <TokKZVWFQvTcDmf7ZaYzexaHM9sI5wmU94kX5B7gWUc=.618edc2f-e95a-4476-9ebb-d6ac7a8387b3@github.com>
References: <TokKZVWFQvTcDmf7ZaYzexaHM9sI5wmU94kX5B7gWUc=.618edc2f-e95a-4476-9ebb-d6ac7a8387b3@github.com>
Message-ID: <qQW8Yf-i2rbESbje1wITGaJr4v9CTi66W1MEFir8tqU=.084d13a3-21cb-4279-8eac-9f83e3855984@github.com>

On Thu, 23 May 2024 19:01:49 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change facilitates further extensions from this base class in the forthcoming generational mode for Shenandoah.

LGTM!

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19372#pullrequestreview-2075245929

From wkemper at openjdk.org  Thu May 23 22:36:07 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 23 May 2024 22:36:07 GMT
Subject: Integrated: 8332082: Shenandoah: Use consistent tests to determine
 when pre-write barrier is active
In-Reply-To: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
References: <LNGlPYvOZEADPwdxNhax-1i5TnWU7hwvZXDsyCxVkOQ=.df9859a5-4e11-428c-a883-60876906c468@github.com>
Message-ID: <pdsqqCGzB27oZi8wx--4lSF0Czb7N7wxpYWnzGAA3_M=.3fec9ef7-71f7-4f82-bc16-9751b9f15922@github.com>

On Fri, 10 May 2024 16:13:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is consistent with c1 and other platforms.

This pull request has now been integrated.

Changeset: ddd73b45
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/ddd73b458355bffeaa8e0e5017c27d6c6af2dc94
Stats:     57 lines in 6 files changed: 17 ins; 22 del; 18 mod

8332082: Shenandoah: Use consistent tests to determine when pre-write barrier is active

Reviewed-by: kdnilsen, shade

-------------

PR: https://git.openjdk.org/jdk/pull/19180

From wkemper at openjdk.org  Thu May 23 23:35:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 23 May 2024 23:35:23 GMT
Subject: Integrated: 8332824: GenShen: Temporarily revert recent changes to GC
 helpers
In-Reply-To: <Zl2OWOsnIALCm688Nam2M3I-yKF1XhgOO8RJ9ftYNm0=.5f332cdc-70d0-4ec8-8f1f-a425e5d087d3@github.com>
References: <Zl2OWOsnIALCm688Nam2M3I-yKF1XhgOO8RJ9ftYNm0=.5f332cdc-70d0-4ec8-8f1f-a425e5d087d3@github.com>
Message-ID: <84QjlzJy-koR_pEQagfCWXpAdBa0UgxZRYUjK7qyU4U=.367aa70c-4824-427b-83b3-353fa1c3843d@github.com>

On Thu, 23 May 2024 13:31:46 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This reverts commit 8f808276edfb163c14223f642f12dc6faddc49c6. This commit is breaking in stress test pipelines. I'm backing it out while it is debugged.

This pull request has now been integrated.

Changeset: 894aefc0
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/894aefc077d495b3cd892d339e9ae9120cf3be54
Stats:     342 lines in 6 files changed: 158 ins; 126 del; 58 mod

8332824: GenShen: Temporarily revert recent changes to GC helpers

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/437

From wkemper at openjdk.org  Fri May 24 00:26:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 00:26:08 GMT
Subject: RFR: 8332851: Shenandoah: Update refs iterator does not need to be a
 member of ShenandoahHeap
Message-ID: <omhHCW7qtlOIMBsx28ln-d9tylNZU8RFx1oSuJYfL0w=.fed3a379-9a64-4936-a88e-d57e026e0b49@github.com>

The iterator is used outside of Shenandoah heap to support an assert, and must be 'reset' for correctness. These iterators are typically used as a local variable, and we can do that with this iterator as well to simplify the code.

-------------

Commit messages:
 - Use local variable for update refs iterator

Changes: https://git.openjdk.org/jdk/pull/19376/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19376&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332851
  Stats: 9 lines in 3 files changed: 1 ins; 5 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/19376.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19376/head:pull/19376

PR: https://git.openjdk.org/jdk/pull/19376

From ysr at openjdk.org  Fri May 24 02:08:01 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 24 May 2024 02:08:01 GMT
Subject: RFR: 8332851: Shenandoah: Update refs iterator does not need to be
 a member of ShenandoahHeap
In-Reply-To: <omhHCW7qtlOIMBsx28ln-d9tylNZU8RFx1oSuJYfL0w=.fed3a379-9a64-4936-a88e-d57e026e0b49@github.com>
References: <omhHCW7qtlOIMBsx28ln-d9tylNZU8RFx1oSuJYfL0w=.fed3a379-9a64-4936-a88e-d57e026e0b49@github.com>
Message-ID: <MRacRXeGjOckE23JDeUhdwiQjItINGktT1QKd7swhmY=.375016d9-421c-4c2d-8d04-8823553c4de2@github.com>

On Fri, 24 May 2024 00:21:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The iterator is used outside of Shenandoah heap to support an assert, and must be 'reset' for correctness. These iterators are typically used as a local variable, and we can do that with this iterator as well to simplify the code.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2230:

> 2228:     workers()->run_task(&task);
> 2229:   }
> 2230:   assert(cancelled_gc() || !update_refs_iterator.has_next(), "Should have finished update references");

Help me out here... Can a degenerate GC never pick up in the midst of an update_refs phase, continuing work left from the point of interruption without having to start from scratch?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19376#discussion_r1612594755

From azafari at openjdk.org  Fri May 24 08:54:08 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 08:54:08 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <ukaaXP_qGKL4gO4YWEzIhyxrpi-eTFIoF-CYjWt_SgI=.4829588d-ccbc-49a1-8ccb-e87e3c014634@github.com>

On Thu, 23 May 2024 12:38:40 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/os/posix/os_posix.cpp line 386:
> 
>> 384:   if (begin_offset > 0) {
>> 385:     if (os::release_memory(extra_base, begin_offset))
>> 386:     {
> 
> The `{` should be moved to the line above.

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613115512

From azafari at openjdk.org  Fri May 24 09:01:02 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 09:01:02 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>

On Thu, 23 May 2024 12:41:25 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/os/posix/os_posix.cpp line 387:
> 
>> 385:     if (os::release_memory(extra_base, begin_offset))
>> 386:     {
>> 387:       ThreadCritical tc;
> 
> In many of the functions we put the `ThreadCritical` inside the `MemTracker` after the `enabled()` check, but we don't do it here. Why is that? Shouldn't the `ThreadCritical` usage be hidden inside `MemTracker`?

I have already tried to move `ThreadCritical` into the `MemTracker` (in another PR), but it failed. AFAIR, the unmapping/releasing the memory should be in critical section too. The current implementation follows this order: 1) create critical section 2) unmap/release 3) if successful, call MemTracker. The step 2) should be in critical section.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613124698

From jsjolen at openjdk.org  Fri May 24 09:06:04 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Fri, 24 May 2024 09:06:04 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
Message-ID: <fBrpDqASHn4JauBNu1190W0rZkIL0_JTuJqCtUZZ8hs=.a46b3c69-ae0b-4f18-b6b4-2ad1f152deb5@github.com>

On Wed, 22 May 2024 12:09:14 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
>> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
>> 
>>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
>> 
>> Tests:
>> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fixed the missing parts of shenandoahHeap.cpp

>2- In order to have adjacent regions with different flags, CDS reserves a (large) region R and then splits it into sub regions R1 and R2 (R == <---R1---><--R2-->). At release time, NMT tracks only R and ignores releasing R1 and R2. This ignoring is problematic when a requested region R is size-aligned to R1---R---R2 first and then the R1 and R2 are released (chop_extra_memory function is called for this). In this case, NMT ignores tracking R1 and R2 with false assumption that a containing R will be released. Therefore, R1 and R2 remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.

Thank you for the in-depth explanation, I think I understand it. What was the fix for this issue?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19343#issuecomment-2129009945

From azafari at openjdk.org  Fri May 24 09:17:01 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 09:17:01 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <ROPDGpH9tEq5-ONWNKtUkx2TZ-WwNlZVAVzVZZhPkbU=.c2308643-6b1b-4ba7-b482-c52f2de8d08f@github.com>

On Thu, 23 May 2024 11:57:17 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/share/cds/metaspaceShared.cpp line 1088:
> 
>> 1086: #endif // ASSERT
>> 1087: 
>> 1088:     if (archive_space_rs.is_reserved()) {
> 
> We've already asserted that this should be true, so this if should not be needed.

I had to add these, since the `log_info(cds)` calls caused the assertions in the `ReservedSpace` getters raised (IIRC, even during jdk-build).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613148798

From azafari at openjdk.org  Fri May 24 09:23:01 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 09:23:01 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <S9ZX9xUv7-ZChXowzxnUPOgtqG18tCANeozxqeA8wco=.bf3b665c-ca7d-46f0-b765-9adce1ea39c9@github.com>

On Thu, 23 May 2024 12:10:17 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/share/cds/metaspaceShared.cpp line 1341:
> 
>> 1339:   } else {
>> 1340:     if (use_archive_base_addr && base_address != nullptr) {
>> 1341:       total_space_rs = ReservedSpace(total_range_size, archive_space_alignment,
> 
> Can you explain why you changed this?
> 
> It's also interesting that after this change we only use `base_address_alignment` in asserts. I think this indicates that something should be cleaned up / fixed here.

That comes after merge with mainline.
I trusted the tests in tiers 1-5 for the correctness of this change.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613163280

From azafari at openjdk.org  Fri May 24 09:29:02 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 09:29:02 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <RJJ_2cVpBzHoSyY_Oif-8SpRyUJKwIRibe2iRt3h_MM=.95da0348-625b-4d90-8c05-a03148b81bc4@github.com>

On Thu, 23 May 2024 12:30:41 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/share/cds/metaspaceShared.cpp line 1370:
> 
>> 1368:                                                      ccs_begin_offset, mtClassShared, mtClass);
>> 1369:   }
>> 1370:   assert(archive_space_rs.is_reserved(), "Archive space is not reserved.");
> 
> Something is dubious about the code above:
> 
>     archive_space_rs = total_space_rs.first_part(ccs_begin_offset,
>                                                  (size_t)archive_space_alignment);
>     class_space_rs = total_space_rs.last_part(ccs_begin_offset);
>     MemTracker::record_virtual_memory_split_reserved(total_space_rs.base(), total_space_rs.size(),
>                                                      ccs_begin_offset, mtClassShared, mtClass);
> 
> 
> In one path `total_space_rs` gets initialized with `mtClass` and in another path it gets initialized with `mtClassShared`. This means that we always get the wrong flag in one of `archive_space_rs` and `class_space_rs`.

The logic is that, a large region `total_space_rs` is reserved and then is split into two sub regions. It doesn't matter what is the flag for `total_space_rs`. At split time the flags are set correctly for sub regions.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613171792

From azafari at openjdk.org  Fri May 24 09:32:07 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 09:32:07 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <xLdnsC9FsNbd37KHYRQPnjZQWOKrXwVuXe95veTH2HI=.1fcfd94a-e598-459b-9fc1-88fc2eb752ff@github.com>

On Thu, 23 May 2024 12:44:42 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/share/memory/virtualspace.hpp line 63:
> 
>> 61:   //                       it should not change after.
>> 62:   //  * _alignment       - Not to be changed after initialization
>> 63:   //  * _executable      - Not to be changed after initialization
> 
> I think this would be a good change to do in the future, but currently this isn't true. `clear_members` do clear these fields, so I think you should remove these two lines.

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613175723

From azafari at openjdk.org  Fri May 24 09:39:02 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 09:39:02 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <w4pX9XFKZobnte-zpejDMVDZEpZTLYSWQkVrInsR3E0=.0581d6c1-a476-4d39-8233-a49b5156fa72@github.com>

On Thu, 23 May 2024 12:45:37 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/share/memory/virtualspace.hpp line 76:
> 
>> 74: 
>> 75:   MEMFLAGS nmt_flag() const { assert(is_reserved(), "Memory region is not reserved."); assert(_flag != mtNone, "Memory flag is not set."); return _flag; }
>> 76: 
> 
> Looking at this again, and realize that this function should probably be moved to the other accessors below.

Done.

> src/hotspot/share/memory/virtualspace.hpp line 98:
> 
>> 96:   bool   special()         const { assert(is_reserved(), "Memory region is not reserved."); return _special;   }
>> 97:   bool   executable()      const { assert(is_reserved(), "Memory region is not reserved."); return _executable;   }
>> 98:   size_t noaccess_prefix() const { assert(is_reserved(), "Memory region is not reserved."); return _noaccess_prefix;   }
> 
> FWIW, this change comes from one of my debugging sessions. I think it is good to have these asserts, I just wish they could says something like `assert(is_initialized(), ...)` to more clearly convey why we are doing this check.
> 
> We are considering if there are ways to split ReservedSpace into two classes, one that handles reserving of memory and one that is a plain view of already reserved memory. If/when we do such a change we could consider updating these asserts to be more legible.
> 
> In the meantime, it would be nice to change the string to "Fields not initialized" (and get rid of the `.`).

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613185089
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613185323

From azafari at openjdk.org  Fri May 24 09:54:03 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 09:54:03 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <OTBI8Dq9UyxSU-wkCAdcn7tCpwMLBLpSY713eXCTrak=.6d70daff-e9ab-4fdd-964a-3a42e9f051bd@github.com>

On Thu, 23 May 2024 13:04:24 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/share/nmt/virtualMemoryTracker.cpp line 506:
> 
>> 504:       return true;
>> 505:     assert(reserved_rgn->end() == rgn.end() || reserved_rgn->base() == rgn.base(), "extra memory should be at either end of the region.");
>> 506:   }
> 
> This seems like an extreme hack. I understand that this just follows the tradition of the rest of the hacks in this file, but can't this be better handled in the CDS layer above?

TBH, I don't like it too. Unfortunately, chopping extra memory is done at `os:xxx` layer and reporting the case back to CDS would need to pass all the chopping info up to CDS. In addition, it is valid in CDS that a region is partitioned into sub regions and releasing sub regions can be silently and correctly ignored.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613202175

From azafari at openjdk.org  Fri May 24 09:57:01 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 09:57:01 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
Message-ID: <jWkbEaMvozVdlp3O88uxb9toh_2d1qS-4sXpgnE-q5g=.030d685f-e4c8-496b-bcb7-77d4a1427e11@github.com>

On Thu, 23 May 2024 11:58:15 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/share/cds/metaspaceShared.cpp line 1092:
> 
>> 1090:                     p2i(archive_space_rs.base()), p2i(archive_space_rs.end()), archive_space_rs.size());
>> 1091:     }
>> 1092:     if (class_space_rs.is_reserved()) {
> 
> `class_space_rs.is_reserved()` is asserted if `if (Metaspace::using_class_space())` is taken. I think this could be changed to:
> Suggestion:
> 
>     if (Metaspace::using_class_space()) {

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613208902

From azafari at openjdk.org  Fri May 24 10:03:27 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 10:03:27 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v3]
In-Reply-To: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
Message-ID: <qWWq1J7dW-o7Gk7uPjecgqVEhia7qsnGowDbyWrEjCw=.6f241eac-b822-4aae-8cad-3d4df1d23950@github.com>

> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
> 
>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
> 
> Tests:
> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  applied review comments.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19343/files
  - new: https://git.openjdk.org/jdk/pull/19343/files/86ae1e37..c7ff3867

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19343&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19343&range=01-02

  Stats: 16 lines in 3 files changed: 2 ins; 5 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/19343.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19343/head:pull/19343

PR: https://git.openjdk.org/jdk/pull/19343

From azafari at openjdk.org  Fri May 24 10:03:27 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 10:03:27 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
Message-ID: <zJpWmirgbGZXMH9C0Y8EjYJBFzOM0eLHxq4fnhYfPMw=.f31b0d75-2629-4b0b-bb63-d566804bd6a0@github.com>

On Wed, 22 May 2024 12:09:14 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
>> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
>> 
>>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
>> 
>> Tests:
>> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fixed the missing parts of shenandoahHeap.cpp

Thank @stefank for your comments. Many of them applied, a few remained open for discussion.

-------------

PR Review: https://git.openjdk.org/jdk/pull/19343#pullrequestreview-2076445838

From stuefe at openjdk.org  Fri May 24 10:17:02 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 24 May 2024 10:17:02 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
Message-ID: <rgcIaHG19lNRzHO55HH8kbRI67pWJPhp9BxVfmszWNs=.fa0b4b21-ccc5-40bc-8f57-12228b4a3ffb@github.com>

On Wed, 22 May 2024 12:09:14 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
>> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
>> 
>>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
>> 
>> Tests:
>> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fixed the missing parts of shenandoahHeap.cpp

Good analysis, @afshin-zafari. 

However, I keep thinking we are stop-gap fixing holes in a badly designed system.

The problem is two-fold:

1) NMT assumes reserves and commits to be different layers and, e.g., for committed regions to be fully contained in a reserved region. This is wrong and does not reflect the realities of mmap. We can overlay and overlap any reservation/committing/uncommitting/releasing in any way we want.

The right way to track virtual memory regions is what we do now in https://github.com/openjdk/jdk/pull/18289 with src/hotspot/share/nmt/vmatree.hpp. See [1] for the (simple) theory behind it. Not only would this be a lot faster and simpler, but it would also be less error-prone since it does not assume any kind of layering between reservations and committing memory. With the VMATree, releasing a whole region would remove all containing regions automatically.

With Johan's PR we now will do this for ZGC memory file allocations. However, we should also use the same technique to track VirtualMemory in NMT. Then, errors like this will disappear.

2) Another problem is that ReservedSpace assumes ownership of the underlying memory. On Windows, we cannot split regions allocated with VirtualAlloc. So ReservedRegions are assigned a MEMFLAG at construction, and we can never split up the region because Windows. Therefore, to have a contiguous region with different regions and different flags in NMT, NMT forces us to allocate them in two steps, side by side, with all that can go wrong. This is suboptimal. NMT is a simple tracker; it should not dictate how we allocate memory but be able to accommodate any way we want.

I don't have a good solution in my head for (2). It is also the less urgent problem, I think.


[1] https://gist.github.com/tstuefe/d9682b7f11b3375da27faa100f45e621

src/hotspot/share/cds/metaspaceShared.cpp line 1169:

> 1167:           // Set up compressed Klass pointer encoding: the encoding range must
> 1168:           //  cover both archive and class space.
> 1169:           assert(class_space_rs.is_reserved(), "Memory region should be reserved.");

Not necessary. Checked in reserve_address_space_for_archives, and in Metaspace::initialize_class_space

-------------

PR Review: https://git.openjdk.org/jdk/pull/19343#pullrequestreview-2076383403
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613180561

From azafari at openjdk.org  Fri May 24 10:17:03 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 10:17:03 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <fBrpDqASHn4JauBNu1190W0rZkIL0_JTuJqCtUZZ8hs=.a46b3c69-ae0b-4f18-b6b4-2ad1f152deb5@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <fBrpDqASHn4JauBNu1190W0rZkIL0_JTuJqCtUZZ8hs=.a46b3c69-ae0b-4f18-b6b4-2ad1f152deb5@github.com>
Message-ID: <EBeW_gtHCP2b3OU6ewGDuBe5UGqwnBVOZD4C-2vOgEE=.30551f20-10fc-46a8-b7b8-b1c12c572dc6@github.com>

On Fri, 24 May 2024 09:03:07 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> Thank you for the in-depth explanation, I think I understand it. What was the fix for this issue?

As can be seen in `os_posix.cpp::chop_extra_memory`, the exceptional case (of releasing sub regions due to extra memory) is notified to `MemTracker` for handling it. The optional `bool extra_memory` arg of `MemTracker::record_virtual_memory_release()` is used for addressing the case.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19343#issuecomment-2129155345

From stefank at openjdk.org  Fri May 24 10:17:04 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 24 May 2024 10:17:04 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <ROPDGpH9tEq5-ONWNKtUkx2TZ-WwNlZVAVzVZZhPkbU=.c2308643-6b1b-4ba7-b482-c52f2de8d08f@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <ROPDGpH9tEq5-ONWNKtUkx2TZ-WwNlZVAVzVZZhPkbU=.c2308643-6b1b-4ba7-b482-c52f2de8d08f@github.com>
Message-ID: <oeqfaH30ZTvSeQ_u2FNpEqzgNMvDEai9rexXmladSBU=.962d96bf-e963-4917-9184-5f9241ffff71@github.com>

On Fri, 24 May 2024 09:12:16 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/cds/metaspaceShared.cpp line 1088:
>> 
>>> 1086: #endif // ASSERT
>>> 1087: 
>>> 1088:     if (archive_space_rs.is_reserved()) {
>> 
>> We've already asserted that this should be true, so this if should not be needed.
>
> I had to add these, since the `log_info(cds)` calls caused the assertions in the `ReservedSpace` getters raised (IIRC, even during jdk-build).

Can you show the error message?

>> src/hotspot/share/cds/metaspaceShared.cpp line 1370:
>> 
>>> 1368:                                                      ccs_begin_offset, mtClassShared, mtClass);
>>> 1369:   }
>>> 1370:   assert(archive_space_rs.is_reserved(), "Archive space is not reserved.");
>> 
>> Something is dubious about the code above:
>> 
>>     archive_space_rs = total_space_rs.first_part(ccs_begin_offset,
>>                                                  (size_t)archive_space_alignment);
>>     class_space_rs = total_space_rs.last_part(ccs_begin_offset);
>>     MemTracker::record_virtual_memory_split_reserved(total_space_rs.base(), total_space_rs.size(),
>>                                                      ccs_begin_offset, mtClassShared, mtClass);
>> 
>> 
>> In one path `total_space_rs` gets initialized with `mtClass` and in another path it gets initialized with `mtClassShared`. This means that we always get the wrong flag in one of `archive_space_rs` and `class_space_rs`.
>
> The logic is that, a large region `total_space_rs` is reserved and then is split into two sub regions. It doesn't matter what is the flag for `total_space_rs`. At split time the flags are set correctly for sub regions.

The flags sent to the NMT subsystem is correct, but the flags recorded in the ReservedSpaces will be wrong, AFAIKT. You can probably verify that by adding asserts.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613235966
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613237903

From stuefe at openjdk.org  Fri May 24 10:17:05 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 24 May 2024 10:17:05 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <S9ZX9xUv7-ZChXowzxnUPOgtqG18tCANeozxqeA8wco=.bf3b665c-ca7d-46f0-b765-9adce1ea39c9@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <S9ZX9xUv7-ZChXowzxnUPOgtqG18tCANeozxqeA8wco=.bf3b665c-ca7d-46f0-b765-9adce1ea39c9@github.com>
Message-ID: <LrUcO28yMABy2IkthvPAFsUHPftyXB5pqlPirR41Zk0=.0aec0646-38be-4995-8f02-abf322832ab6@github.com>

On Fri, 24 May 2024 09:20:13 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/cds/metaspaceShared.cpp line 1341:
>> 
>>> 1339:   } else {
>>> 1340:     if (use_archive_base_addr && base_address != nullptr) {
>>> 1341:       total_space_rs = ReservedSpace(total_range_size, archive_space_alignment,
>> 
>> Can you explain why you changed this?
>> 
>> It's also interesting that after this change we only use `base_address_alignment` in asserts. I think this indicates that something should be cleaned up / fixed here.
>
> That comes after merge with mainline.
> I trusted the tests in tiers 1-5 for the correctness of this change.

No, I think this is wrong. I changed it with https://github.com/openjdk/jdk/pull/19152.

Please be careful, this part is rather tricky, and a lot of thought went into this. And I am pretty sure we don't cover all possible code paths in tests.

Please also note that I am working on adding no-access zones for the Klass Encoding range (see https://github.com/openjdk/jdk/pull/19290), which may impact these regions too.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613191595

From stefank at openjdk.org  Fri May 24 10:38:03 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 24 May 2024 10:38:03 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>
Message-ID: <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>

On Fri, 24 May 2024 08:57:57 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/os/posix/os_posix.cpp line 387:
>> 
>>> 385:     if (os::release_memory(extra_base, begin_offset))
>>> 386:     {
>>> 387:       ThreadCritical tc;
>> 
>> In many of the functions we put the `ThreadCritical` inside the `MemTracker` after the `enabled()` check, but we don't do it here. Why is that? Shouldn't the `ThreadCritical` usage be hidden inside `MemTracker`?
>
> I have already tried to move `ThreadCritical` into the `MemTracker` (in another PR), but it failed. AFAIR, the unmapping/releasing the memory should be in critical section too. The current implementation follows this order: 1) create critical section 2) unmap/release 3) if successful, call MemTracker. The step 2) should be in critical section.

Hmm. os::release_memory also calls `record_virtual_memory_release`, and then this code calls it again with a second ThreadCritical, but then it is called again with `extra_memory`. I still find this addition of `extra_memory` highly dubious.

>> src/hotspot/share/nmt/virtualMemoryTracker.cpp line 506:
>> 
>>> 504:       return true;
>>> 505:     assert(reserved_rgn->end() == rgn.end() || reserved_rgn->base() == rgn.base(), "extra memory should be at either end of the region.");
>>> 506:   }
>> 
>> This seems like an extreme hack. I understand that this just follows the tradition of the rest of the hacks in this file, but can't this be better handled in the CDS layer above?
>
> TBH, I don't like it too. Unfortunately, chopping extra memory is done at `os:xxx` layer and reporting the case back to CDS would need to pass all the chopping info up to CDS. In addition, it is valid in CDS that a region is partitioned into sub regions and releasing sub regions can be silently and correctly ignored.

I'd like to take an extra look at that before this PR gets integrated.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613259695
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613262335

From stefank at openjdk.org  Fri May 24 10:38:02 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 24 May 2024 10:38:02 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v3]
In-Reply-To: <qWWq1J7dW-o7Gk7uPjecgqVEhia7qsnGowDbyWrEjCw=.6f241eac-b822-4aae-8cad-3d4df1d23950@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <qWWq1J7dW-o7Gk7uPjecgqVEhia7qsnGowDbyWrEjCw=.6f241eac-b822-4aae-8cad-3d4df1d23950@github.com>
Message-ID: <vMWv4Nca2Mhrp8gITaTKh5UfxtCR8mtHQKC-5TUq2G4=.917dcd75-cb6f-40a8-b34b-234eb3b0eaef@github.com>

On Fri, 24 May 2024 10:03:27 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
>> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
>> 
>>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
>> 
>> Tests:
>> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   applied review comments.

Changes requested by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19343#pullrequestreview-2076525088

From azafari at openjdk.org  Fri May 24 11:51:02 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 11:51:02 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>
 <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>
Message-ID: <a4Qk5XdNcGz-gcsdR-CvY0MwnQv1tZN9pvX-k5ctLkk=.a28c804d-dd76-4bc1-b215-3c8b4f5eca68@github.com>

On Fri, 24 May 2024 10:32:17 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I have already tried to move `ThreadCritical` into the `MemTracker` (in another PR), but it failed. AFAIR, the unmapping/releasing the memory should be in critical section too. The current implementation follows this order: 1) create critical section 2) unmap/release 3) if successful, call MemTracker. The step 2) should be in critical section.
>
> Hmm. os::release_memory also calls `record_virtual_memory_release`, and then this code calls it again with a second ThreadCritical, but then it is called again with `extra_memory`. I still find this addition of `extra_memory` highly dubious.

Some facts: 
- `MemTracker::record_virtual_memory_release()` has no `ThreadCritical` internally and therefore should be called inside a critical section.
- When `os::release_memory()` returns, the `ThreadCritical` that is created there is destroyed and a new one should be created again here.
- Releasing a sub-region that flagged for CDS and is contained in a larger CDS region is ignored at `MemTracker::record_virtual_memory_release()`. It is a valid case due to the way that CDS reserves and/or releases regions.
- This exceptional case is notified to `MemTracker` by passing `true` as `extra_memory`.
- Inside `MemTracker`, the `extra_memory == true` is used in the places where the exceptional case should/would be addressed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613352945

From azafari at openjdk.org  Fri May 24 12:01:02 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 12:01:02 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>
 <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>
Message-ID: <ZFpacAsyZ2bGIXUVB2INsM4az1ik25aRowS1Q6oVpDE=.6b34e833-3d05-47fd-80e4-4db4e57b49c5@github.com>

On Fri, 24 May 2024 10:34:56 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> TBH, I don't like it too. Unfortunately, chopping extra memory is done at `os:xxx` layer and reporting the case back to CDS would need to pass all the chopping info up to CDS. In addition, it is valid in CDS that a region is partitioned into sub regions and releasing sub regions can be silently and correctly ignored.
>
> I'd like to take an extra look at that before this PR gets integrated.

I agree and will work on it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613367242

From azafari at openjdk.org  Fri May 24 13:13:04 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 13:13:04 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <oeqfaH30ZTvSeQ_u2FNpEqzgNMvDEai9rexXmladSBU=.962d96bf-e963-4917-9184-5f9241ffff71@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <ROPDGpH9tEq5-ONWNKtUkx2TZ-WwNlZVAVzVZZhPkbU=.c2308643-6b1b-4ba7-b482-c52f2de8d08f@github.com>
 <oeqfaH30ZTvSeQ_u2FNpEqzgNMvDEai9rexXmladSBU=.962d96bf-e963-4917-9184-5f9241ffff71@github.com>
Message-ID: <J7ZCTukLZdInwe3Qqe3prB7H5FPOmPhfRsf4CbvBtB8=.a7f7a910-c09f-4c65-8330-a4430ab8b5d8@github.com>

On Fri, 24 May 2024 10:14:41 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> The logic is that, a large region `total_space_rs` is reserved and then is split into two sub regions. It doesn't matter what is the flag for `total_space_rs`. At split time the flags are set correctly for sub regions.
>
> The flags sent to the NMT subsystem is correct, but the flags recorded in the ReservedSpaces will be wrong, AFAIKT. You can probably verify that by adding asserts.

If your comment refers only to these lines of code, they are already verified. Since, inside the split function, the sub-regions get the new flags and all the reserved and committed amounts are moved from the large region to the new ones. So, the accounting of memory is correct.

FWIW, if we trace down the call at line 1346 of `total_space_rs = Metaspace::reserve_address_space_for_compressed_classes(total_range_size, false /* optimize_for_zero_base */);` the region may get different flags of `mtClass` or `mtMetaspace` based on the checked criteria down there.
 
If you comment on all such cases, then I will double check for them and add assertion for.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613462000

From azafari at openjdk.org  Fri May 24 13:46:15 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 13:46:15 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
Message-ID: <VuFm0lU78YLRVyTMOvNd1rofkOkF6VyvzhAePmQMFJc=.d8e6ca76-c405-4148-9fe9-007f0a3e616d@github.com>

> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
> 
>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
> 
> Tests:
> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  more fixes.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19343/files
  - new: https://git.openjdk.org/jdk/pull/19343/files/c7ff3867..302e35ea

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19343&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19343&range=02-03

  Stats: 6 lines in 1 file changed: 0 ins; 3 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/19343.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19343/head:pull/19343

PR: https://git.openjdk.org/jdk/pull/19343

From azafari at openjdk.org  Fri May 24 13:46:15 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 13:46:15 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <oeqfaH30ZTvSeQ_u2FNpEqzgNMvDEai9rexXmladSBU=.962d96bf-e963-4917-9184-5f9241ffff71@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <ROPDGpH9tEq5-ONWNKtUkx2TZ-WwNlZVAVzVZZhPkbU=.c2308643-6b1b-4ba7-b482-c52f2de8d08f@github.com>
 <oeqfaH30ZTvSeQ_u2FNpEqzgNMvDEai9rexXmladSBU=.962d96bf-e963-4917-9184-5f9241ffff71@github.com>
Message-ID: <APn-aZnsck6puRlakkmS4F5kTN8eh7hJ9zKooTkoveA=.ac06fedf-97bc-4513-b2ac-8a23e5bb2529@github.com>

On Fri, 24 May 2024 10:12:53 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I had to add these, since the `log_info(cds)` calls caused the assertions in the `ReservedSpace` getters raised (IIRC, even during jdk-build).
>
> Can you show the error message?

I could not reproduce the error.
Removed the check.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613507173

From azafari at openjdk.org  Fri May 24 13:46:15 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 13:46:15 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <rgcIaHG19lNRzHO55HH8kbRI67pWJPhp9BxVfmszWNs=.fa0b4b21-ccc5-40bc-8f57-12228b4a3ffb@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <rgcIaHG19lNRzHO55HH8kbRI67pWJPhp9BxVfmszWNs=.fa0b4b21-ccc5-40bc-8f57-12228b4a3ffb@github.com>
Message-ID: <z-_ddfMVyvankHmPFvg6JeY_mQODuhDJ-DQcT9GlyOM=.cc73eb1e-1354-45af-9575-1b8557c574e4@github.com>

On Fri, 24 May 2024 09:33:06 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed the missing parts of shenandoahHeap.cpp
>
> src/hotspot/share/cds/metaspaceShared.cpp line 1169:
> 
>> 1167:           // Set up compressed Klass pointer encoding: the encoding range must
>> 1168:           //  cover both archive and class space.
>> 1169:           assert(class_space_rs.is_reserved(), "Memory region should be reserved.");
> 
> Not necessary. Checked in reserve_address_space_for_archives, and in Metaspace::initialize_class_space

Removed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613503505

From azafari at openjdk.org  Fri May 24 13:46:15 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 13:46:15 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <LrUcO28yMABy2IkthvPAFsUHPftyXB5pqlPirR41Zk0=.0aec0646-38be-4995-8f02-abf322832ab6@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <S9ZX9xUv7-ZChXowzxnUPOgtqG18tCANeozxqeA8wco=.bf3b665c-ca7d-46f0-b765-9adce1ea39c9@github.com>
 <LrUcO28yMABy2IkthvPAFsUHPftyXB5pqlPirR41Zk0=.0aec0646-38be-4995-8f02-abf322832ab6@github.com>
Message-ID: <XIYkXtLTJotebjF0hoWajfLFbGClgcwGGQ1wRbuw7tY=.4595ce39-d751-40c5-998f-ea9c397104bf@github.com>

On Fri, 24 May 2024 09:41:25 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> That comes after merge with mainline.
>> I trusted the tests in tiers 1-5 for the correctness of this change.
>
> No, I think this is wrong. I changed it with https://github.com/openjdk/jdk/pull/19152.
> 
> Please be careful, this part is rather tricky, and a lot of thought went into this. And I am pretty sure we don't cover all possible code paths in tests.
> 
> Please also note that I am working on adding no-access zones for the Klass Encoding range (see https://github.com/openjdk/jdk/pull/19290), which may impact these regions too.

Overlooked in merge. 
Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1613504094

From azafari at openjdk.org  Fri May 24 13:59:03 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 24 May 2024 13:59:03 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <rgcIaHG19lNRzHO55HH8kbRI67pWJPhp9BxVfmszWNs=.fa0b4b21-ccc5-40bc-8f57-12228b4a3ffb@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <rgcIaHG19lNRzHO55HH8kbRI67pWJPhp9BxVfmszWNs=.fa0b4b21-ccc5-40bc-8f57-12228b4a3ffb@github.com>
Message-ID: <oRiGPyU2-kd8YhZCgnGhzWGI5O89zOfXsWtHx80p6WI=.8d07204f-32b8-46a5-86db-c54f22b7e330@github.com>

On Fri, 24 May 2024 10:10:08 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> 1. NMT assumes reserves and commits to be different layers and, e.g., for committed regions to be fully contained in a reserved region. This is wrong and does not reflect the realities of mmap. We can overlay and overlap any reservation/committing/uncommitting/releasing in any way we want.

On Windows, a commit without reserve is not allowed. ([reference](https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualalloc#:~:text=MEM_COMMIT%20%7C%20MEM_RESERVE.-,Attempting%20to%20commit%20a%20specific%20address%20range%20by%20specifying%20MEM_COMMIT%20without%20MEM_RESERVE%20and%20a%20non%2DNULL%20lpAddress%20fails%20unless%20the%20entire%20range%20has%20already%20been%20reserved.%20The%20resulting%20error%20code%20is%20ERROR_INVALID_ADDRESS.,-An%20attempt%20to))

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19343#issuecomment-2129603425

From wkemper at openjdk.org  Fri May 24 14:16:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 14:16:41 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <hKShEAF4YkXn29eJigBB3RVLjRQ4UKdgJq-mIhMRpB4=.22cc2608-ea19-45a2-a966-8401b9bf63ad@github.com>

Merges tag jdk-23+24

-------------

Commit messages:
 - 8307193: Several Swing jtreg tests use class.forName on L&F classes
 - 8332490: JMH org.openjdk.bench.java.util.zip.InflaterInputStreams.inflaterInputStreamRead OOM
 - 8332739: Problemlist compiler/codecache/CheckLargePages until JDK-8332654 is fixed
 - 8332463: Byte conditional pattern case element dominates short constant case element
 - 8329718: Incorrect `@since` tags in elements in jdk.compiler and java.compiler
 - 8332740: [BACKOUT] JDK-8331081 'internal proprietary API' diagnostics if --system is configured to an earlier JDK version
 - 8332096: hotspot-ide-project fails with this-escape
 - 8331081: 'internal proprietary API' diagnostics if --system is configured to an earlier JDK version
 - 8332071: Convert package.html files in `java.management.rmi` to package-info.java
 - 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic
 - ... and 82 more: https://git.openjdk.org/shenandoah/compare/2f10a316...9d332e65

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=438&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/438/files
  Stats: 47193 lines in 669 files changed: 38304 ins; 6420 del; 2469 mod
  Patch: https://git.openjdk.org/shenandoah/pull/438.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/438/head:pull/438

PR: https://git.openjdk.org/shenandoah/pull/438

From wkemper at openjdk.org  Fri May 24 14:43:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 14:43:06 GMT
Subject: RFR: 8332851: Shenandoah: Update refs iterator does not need to be
 a member of ShenandoahHeap
In-Reply-To: <omhHCW7qtlOIMBsx28ln-d9tylNZU8RFx1oSuJYfL0w=.fed3a379-9a64-4936-a88e-d57e026e0b49@github.com>
References: <omhHCW7qtlOIMBsx28ln-d9tylNZU8RFx1oSuJYfL0w=.fed3a379-9a64-4936-a88e-d57e026e0b49@github.com>
Message-ID: <lh2YGXuP9WOVVtUlEoUTJmWYYH3-HfMShbs9TrJpx94=.397c63f1-1d40-41ac-a26e-3f851f6c8336@github.com>

On Fri, 24 May 2024 00:21:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The iterator is used outside of Shenandoah heap to support an assert, and must be 'reset' for correctness. These iterators are typically used as a local variable, and we can do that with this iterator as well to simplify the code.

Missed that `_update_refs_iterator` is shared between concurrent and degenerated cycles. Abandoning this PR.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19376#issuecomment-2129700274

From wkemper at openjdk.org  Fri May 24 14:43:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 14:43:06 GMT
Subject: RFR: 8332851: Shenandoah: Update refs iterator does not need to be
 a member of ShenandoahHeap
In-Reply-To: <MRacRXeGjOckE23JDeUhdwiQjItINGktT1QKd7swhmY=.375016d9-421c-4c2d-8d04-8823553c4de2@github.com>
References: <omhHCW7qtlOIMBsx28ln-d9tylNZU8RFx1oSuJYfL0w=.fed3a379-9a64-4936-a88e-d57e026e0b49@github.com>
 <MRacRXeGjOckE23JDeUhdwiQjItINGktT1QKd7swhmY=.375016d9-421c-4c2d-8d04-8823553c4de2@github.com>
Message-ID: <HLL8JYhg3FzMj-n5-m_m4HzLWGuNeh9aaqN695cIi24=.d4547887-d029-4e21-bb90-6b417fa66cbd@github.com>

On Fri, 24 May 2024 02:05:07 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> The iterator is used outside of Shenandoah heap to support an assert, and must be 'reset' for correctness. These iterators are typically used as a local variable, and we can do that with this iterator as well to simplify the code.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2230:
> 
>> 2228:     workers()->run_task(&task);
>> 2229:   }
>> 2230:   assert(cancelled_gc() || !update_refs_iterator.has_next(), "Should have finished update references");
> 
> Help me out here... Can a degenerate GC never pick up in the midst of an update_refs phase, continuing work left from the point of interruption without having to start from scratch?

Yes! It can! Good catch! I'll abandon this PR and fix up `shenandoah:master`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19376#discussion_r1613588878

From wkemper at openjdk.org  Fri May 24 14:43:07 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 14:43:07 GMT
Subject: Withdrawn: 8332851: Shenandoah: Update refs iterator does not need to
 be a member of ShenandoahHeap
In-Reply-To: <omhHCW7qtlOIMBsx28ln-d9tylNZU8RFx1oSuJYfL0w=.fed3a379-9a64-4936-a88e-d57e026e0b49@github.com>
References: <omhHCW7qtlOIMBsx28ln-d9tylNZU8RFx1oSuJYfL0w=.fed3a379-9a64-4936-a88e-d57e026e0b49@github.com>
Message-ID: <q2Q2lob2VBlz6D_nk-iIGiZoQLTpHBkZIPugN5jaGC4=.fd3666c5-9212-4a03-8e7c-a78c33361e5b@github.com>

On Fri, 24 May 2024 00:21:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The iterator is used outside of Shenandoah heap to support an assert, and must be 'reset' for correctness. These iterators are typically used as a local variable, and we can do that with this iterator as well to simplify the code.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/19376

From wkemper at openjdk.org  Fri May 24 17:59:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 17:59:45 GMT
Subject: RFR: 8332913: GenShen: Restore shared update refs iterator
Message-ID: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>

Turns out, this iterator is shared between concurrent and degenerated GC cycles. It should not have been made into a local variable.

-------------

Commit messages:
 - Restore shared update refs iterator

Changes: https://git.openjdk.org/shenandoah/pull/439/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=439&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332913
  Stats: 23 lines in 4 files changed: 10 ins; 5 del; 8 mod
  Patch: https://git.openjdk.org/shenandoah/pull/439.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/439/head:pull/439

PR: https://git.openjdk.org/shenandoah/pull/439

From wkemper at openjdk.org  Fri May 24 18:13:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 18:13:08 GMT
Subject: Integrated: 8332841: GenShen: Pull shared members from control thread
 into common base class
In-Reply-To: <TokKZVWFQvTcDmf7ZaYzexaHM9sI5wmU94kX5B7gWUc=.618edc2f-e95a-4476-9ebb-d6ac7a8387b3@github.com>
References: <TokKZVWFQvTcDmf7ZaYzexaHM9sI5wmU94kX5B7gWUc=.618edc2f-e95a-4476-9ebb-d6ac7a8387b3@github.com>
Message-ID: <0q63-YDIt9rnHNDSaq1OjAKjF5MPVGqNI0XvORZpAdc=.dfbd1e8a-35dc-4512-9f6b-0a709ad5c64f@github.com>

On Thu, 23 May 2024 19:01:49 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change facilitates further extensions from this base class in the forthcoming generational mode for Shenandoah.

This pull request has now been integrated.

Changeset: ebc520e8
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/ebc520e83f503eeb4e5af6d5aef62df9227af4f7
Stats:     365 lines in 5 files changed: 224 ins; 129 del; 12 mod

8332841: GenShen: Pull shared members from control thread into common base class

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/jdk/pull/19372

From ysr at openjdk.org  Fri May 24 19:10:19 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 24 May 2024 19:10:19 GMT
Subject: RFR: 8332913: GenShen: Restore shared update refs iterator
In-Reply-To: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
References: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
Message-ID: <Nre5riSupPy935AWUw7P7YiIqjeBS3eWVOlAz9VwJY4=.4a5bbb72-54c0-4cef-b827-6834d556d221@github.com>

On Fri, 24 May 2024 17:55:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Turns out, this iterator is shared between concurrent and degenerated GC cycles. It should not have been made into a local variable.

LGTM. Minor comments on an assert.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2384:

> 2382:     workers()->run_task(&task);
> 2383:   }
> 2384:   assert(cancelled_gc() || !update_refs_iterator.has_next(), "Should have finished update references");

Should one keep this assert, but have it refer to the `_update_refs_iterator` field that you restored in shHeap?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/439#pullrequestreview-2077654097
PR Review Comment: https://git.openjdk.org/shenandoah/pull/439#discussion_r1613900742

From ysr at openjdk.org  Fri May 24 19:10:19 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 24 May 2024 19:10:19 GMT
Subject: RFR: 8332913: GenShen: Restore shared update refs iterator
In-Reply-To: <Nre5riSupPy935AWUw7P7YiIqjeBS3eWVOlAz9VwJY4=.4a5bbb72-54c0-4cef-b827-6834d556d221@github.com>
References: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
 <Nre5riSupPy935AWUw7P7YiIqjeBS3eWVOlAz9VwJY4=.4a5bbb72-54c0-4cef-b827-6834d556d221@github.com>
Message-ID: <qK3sBwASkj3Yf6ifsi5sYmw7Y7DNShbaz8HoZtMTECI=.6c8d9a0a-8fbb-4382-b968-b555b5f37389@github.com>

On Fri, 24 May 2024 19:03:38 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Turns out, this iterator is shared between concurrent and degenerated GC cycles. It should not have been made into a local variable.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2384:
> 
>> 2382:     workers()->run_task(&task);
>> 2383:   }
>> 2384:   assert(cancelled_gc() || !update_refs_iterator.has_next(), "Should have finished update references");
> 
> Should one keep this assert, but have it refer to the `_update_refs_iterator` field that you restored in shHeap?

I do see that the assert in `op_final_updaterefs()` checks this at the start of the next phase, so may be we are adequately covered already.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/439#discussion_r1613903745

From kdnilsen at openjdk.org  Fri May 24 19:25:28 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 24 May 2024 19:25:28 GMT
Subject: RFR: 8332913: GenShen: Restore shared update refs iterator
In-Reply-To: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
References: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
Message-ID: <H1qMUAJ7pXfe_0FiIRwjR7rNcTHedwCCXcyTh1k1Bcg=.03082836-c53b-429f-91d1-2a8190fcd826@github.com>

On Fri, 24 May 2024 17:55:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Turns out, this iterator is shared between concurrent and degenerated GC cycles. It should not have been made into a local variable.

Good teamwork figuring this out...  Thanks.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/439#pullrequestreview-2077694536

From kdnilsen at openjdk.org  Fri May 24 19:33:20 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 24 May 2024 19:33:20 GMT
Subject: RFR: 8332913: GenShen: Restore shared update refs iterator
In-Reply-To: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
References: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
Message-ID: <aUcp7464QNZjKLu2lKMKOVjCOplIFOsYP2RfJp6VlgU=.40085309-2387-4747-b576-a2fb997a5859@github.com>

On Fri, 24 May 2024 17:55:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Turns out, this iterator is shared between concurrent and degenerated GC cycles. It should not have been made into a local variable.

We should understand the GHA failures before integrating.  Lots of them...  ???

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/439#issuecomment-2130234096

From wkemper at openjdk.org  Fri May 24 20:00:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 20:00:44 GMT
Subject: RFR: 8332913: GenShen: Restore shared update refs iterator [v2]
In-Reply-To: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
References: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
Message-ID: <a9EwMTp4_j54vMD8Wk0VIBSOVPfOJSVGXLRj8ONUwDI=.7202432d-6161-4b72-b357-79d472722c94@github.com>

> Turns out, this iterator is shared between concurrent and degenerated GC cycles. It should not have been made into a local variable.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Restore card stats enable check

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/439/files
  - new: https://git.openjdk.org/shenandoah/pull/439/files/6fdc1691..643edc23

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=439&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=439&range=00-01

  Stats: 5 lines in 1 file changed: 3 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/439.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/439/head:pull/439

PR: https://git.openjdk.org/shenandoah/pull/439

From wkemper at openjdk.org  Fri May 24 20:55:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 20:55:14 GMT
Subject: RFR: 8332913: GenShen: Restore shared update refs iterator [v2]
In-Reply-To: <a9EwMTp4_j54vMD8Wk0VIBSOVPfOJSVGXLRj8ONUwDI=.7202432d-6161-4b72-b357-79d472722c94@github.com>
References: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
 <a9EwMTp4_j54vMD8Wk0VIBSOVPfOJSVGXLRj8ONUwDI=.7202432d-6161-4b72-b357-79d472722c94@github.com>
Message-ID: <sDiHhZzDJ6BVolBDVeHmlc4pSRgBAtiltstDokYrkV4=.ca0eba98-4f94-4e80-9765-a04afaf63a15@github.com>

On Fri, 24 May 2024 20:00:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Turns out, this iterator is shared between concurrent and degenerated GC cycles. It should not have been made into a local variable.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Restore card stats enable check

GHA failures are from a slip of the mouse when I was restoring the original code.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/439#issuecomment-2130340312

From wkemper at openjdk.org  Fri May 24 20:55:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 24 May 2024 20:55:14 GMT
Subject: RFR: 8332913: GenShen: Restore shared update refs iterator [v2]
In-Reply-To: <qK3sBwASkj3Yf6ifsi5sYmw7Y7DNShbaz8HoZtMTECI=.6c8d9a0a-8fbb-4382-b968-b555b5f37389@github.com>
References: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
 <Nre5riSupPy935AWUw7P7YiIqjeBS3eWVOlAz9VwJY4=.4a5bbb72-54c0-4cef-b827-6834d556d221@github.com>
 <qK3sBwASkj3Yf6ifsi5sYmw7Y7DNShbaz8HoZtMTECI=.6c8d9a0a-8fbb-4382-b968-b555b5f37389@github.com>
Message-ID: <0ojVOC3WO40QRgyT178dvuh1ZbQwhWjeM9QmF7G95tY=.74eb7b2b-5876-4d66-9518-9634d8174855@github.com>

On Fri, 24 May 2024 19:07:24 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2384:
>> 
>>> 2382:     workers()->run_task(&task);
>>> 2383:   }
>>> 2384:   assert(cancelled_gc() || !update_refs_iterator.has_next(), "Should have finished update references");
>> 
>> Should one keep this assert, but have it refer to the `_update_refs_iterator` field that you restored in shHeap?
>
> I do see that the assert in `op_final_updaterefs()` checks this at the start of the next phase, so may be we are adequately covered already.

Right, the assert is moved to `op_final_updaterefs` and covers the case for concurrent and degenerated cycles.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/439#discussion_r1614010859

From stuefe at openjdk.org  Sat May 25 06:18:01 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Sat, 25 May 2024 06:18:01 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <oRiGPyU2-kd8YhZCgnGhzWGI5O89zOfXsWtHx80p6WI=.8d07204f-32b8-46a5-86db-c54f22b7e330@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <rgcIaHG19lNRzHO55HH8kbRI67pWJPhp9BxVfmszWNs=.fa0b4b21-ccc5-40bc-8f57-12228b4a3ffb@github.com>
 <oRiGPyU2-kd8YhZCgnGhzWGI5O89zOfXsWtHx80p6WI=.8d07204f-32b8-46a5-86db-c54f22b7e330@github.com>
Message-ID: <eUxnm-f-BprJ0EVE9BMe5Gh2kO1rVZ2Zvbf3j_l163c=.fb515cab-fd7c-4c01-85f3-6a1394caf2ac@github.com>

On Fri, 24 May 2024 13:55:58 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

> > 1. NMT assumes reserves and commits to be different layers and, e.g., for committed regions to be fully contained in a reserved region. This is wrong and does not reflect the realities of mmap. We can overlay and overlap any reservation/committing/uncommitting/releasing in any way we want.
> 
> On Windows, a commit without reserve is not allowed. ([reference](https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualalloc#:~:text=MEM_COMMIT%20%7C%20MEM_RESERVE.-,Attempting%20to%20commit%20a%20specific%20address%20range%20by%20specifying%20MEM_COMMIT%20without%20MEM_RESERVE%20and%20a%20non%2DNULL%20lpAddress%20fails%20unless%20the%20entire%20range%20has%20already%20been%20reserved.%20The%20resulting%20error%20code%20is%20ERROR_INVALID_ADDRESS.,-An%20attempt%20to))

Sorry, but why does NMT have to enforce that? NMT should allow what the most flexible of OSes allows.

On mmap on Posix is a lot more flexible than VirtualAlloc on Windows. We want to be able to use the APIs that the OS gives us to the fullest. 

If the OS allows us to reserve a memory range with a single system call, then to treat that range as disjunct entities that I can commit/release independently from another, the JVM should be able to do just that without having to worry about NMT. NMT should cope with that.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19343#issuecomment-2130872202

From azafari at openjdk.org  Mon May 27 09:28:04 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 27 May 2024 09:28:04 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <VuFm0lU78YLRVyTMOvNd1rofkOkF6VyvzhAePmQMFJc=.d8e6ca76-c405-4148-9fe9-007f0a3e616d@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <VuFm0lU78YLRVyTMOvNd1rofkOkF6VyvzhAePmQMFJc=.d8e6ca76-c405-4148-9fe9-007f0a3e616d@github.com>
Message-ID: <_ca1EJiY646IzwhzyaYcFalLhCVmQuvICmvHnOL2cUk=.884fcf4b-7cf6-4d24-9cf8-0b423add835c@github.com>

On Fri, 24 May 2024 13:46:15 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
>> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
>> 
>>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
>> 
>> Tests:
>> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   more fixes.

fixes and further discussions are added.

-------------

PR Review: https://git.openjdk.org/jdk/pull/19343#pullrequestreview-2080464778

From stefank at openjdk.org  Mon May 27 13:22:04 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 27 May 2024 13:22:04 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <VuFm0lU78YLRVyTMOvNd1rofkOkF6VyvzhAePmQMFJc=.d8e6ca76-c405-4148-9fe9-007f0a3e616d@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <VuFm0lU78YLRVyTMOvNd1rofkOkF6VyvzhAePmQMFJc=.d8e6ca76-c405-4148-9fe9-007f0a3e616d@github.com>
Message-ID: <vZOZAc5DcwcDqtXmx3N2Au8N_Evzvc4cTCwAYPSrX2o=.fe9cad57-96d2-434d-b27d-02403df661ed@github.com>

On Fri, 24 May 2024 13:46:15 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> This PR fixes the problems existed in the original PR (https://github.com/openjdk/jdk/pull/18745).  There are two main fixes here:
>>   1- `ReservedSpace` class is changed so that the `_flag` member never changes after it is set in ctor. Since reserving memory regions may go thru a try and fail sequence of reserve-release pairs, changing the `_flag` member at failed releases would lead to incorrect flags in subsequent reserves.
>> Also, some assertion are added to the getters of a `ReservedSpace` to check if the region is successfully reserved.
>> 
>>   2- In order to have adjacent regions with different flags, CDS reserves a (large) region `R` and then splits it into sub regions `R1` and `R2` (`R == <---R1---><--R2-->`). At release time, NMT tracks only `R` and ignores releasing `R1` and `R2`. This ignoring is problematic when a requested region `R` is size-aligned to `R1---R---R2` first and then the `R1` and `R2` are released (`chop_extra_memory` function is called for this). In this case, NMT ignores tracking `R1` and `R2` with false assumption that a containing `R` will be released. Therefore, `R1` and `R2` remain in the NMT reserved-regions-list and when a new reserve happens at that regions, NMT complains by raising an exception.
>> 
>> Tests:
>> mach5 tiers 1-5, {linux-x64, macosx-aarch64, windows-x64, linux-aarch64 } x {debug, non-debug}
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   more fixes.

Changes requested by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19343#pullrequestreview-2080884905

From stefank at openjdk.org  Mon May 27 13:22:04 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 27 May 2024 13:22:04 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <a4Qk5XdNcGz-gcsdR-CvY0MwnQv1tZN9pvX-k5ctLkk=.a28c804d-dd76-4bc1-b215-3c8b4f5eca68@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>
 <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>
 <a4Qk5XdNcGz-gcsdR-CvY0MwnQv1tZN9pvX-k5ctLkk=.a28c804d-dd76-4bc1-b215-3c8b4f5eca68@github.com>
Message-ID: <8_W3gPFqX8RC7V2QvFSmKAOTEK4z6uHOf4NnA0RDp7A=.428d32d1-0624-48de-ac7b-0f1acc6a0a14@github.com>

On Fri, 24 May 2024 11:48:37 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> Hmm. os::release_memory also calls `record_virtual_memory_release`, and then this code calls it again with a second ThreadCritical, but then it is called again with `extra_memory`. I still find this addition of `extra_memory` highly dubious.
>
> Some facts: 
> - `MemTracker::record_virtual_memory_release()` has no `ThreadCritical` internally and therefore should be called inside a critical section.
> - When `os::release_memory()` returns, the `ThreadCritical` that is created there is destroyed and a new one should be created again here.
> - Releasing a sub-region that flagged for CDS and is contained in a larger CDS region is ignored at `MemTracker::record_virtual_memory_release()`. It is a valid case due to the way that CDS reserves and/or releases regions.
> - This exceptional case is notified to `MemTracker` by passing `true` as `extra_memory`.
> - Inside `MemTracker`, the `extra_memory == true` is used in the places where the exceptional case should/would be addressed.

There are a couple of "therefore", "should", and "is a valid case", above that makes it sound like this is the only way to implement things. My point is that I don't think it is, and I would like to see a step back where we try to think about a way to write this without adding these extra layers of code.

With that said, Thomas is thinking about ways to change how we keep track of the NMT flags, so I hope that if we make those changes we could skip having to add the code here.

>> The flags sent to the NMT subsystem is correct, but the flags recorded in the ReservedSpaces will be wrong, AFAIKT. You can probably verify that by adding asserts.
>
> If your comment refers only to these lines of code, they are already verified. Since, inside the split function, the sub-regions get the new flags and all the reserved and committed amounts are moved from the large region to the new ones. So, the accounting of memory is correct.
> 
> FWIW, if we trace down the call at line 1346 of `total_space_rs = Metaspace::reserve_address_space_for_compressed_classes(total_range_size, false /* optimize_for_zero_base */);` the region may get different flags of `mtClass` or `mtMetaspace` based on the checked criteria down there.
>  
> If you comment on all such cases, then I will double check for them and add assertion for.

My point is that the `archive_space_rs` and `class_space_rs` can get the wrong flags assigned to them. The split functions don't change them. Right?

I would like to see the code run through our testing with these checks:

  assert(archive_space_rs.nmt_flag() == mtClassShared, "Sanity");
  assert(class_space_rs.nmt_flag() == mtClass, "Sanity");

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1616042193
PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1616028639

From azafari at openjdk.org  Mon May 27 21:28:01 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 27 May 2024 21:28:01 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <8_W3gPFqX8RC7V2QvFSmKAOTEK4z6uHOf4NnA0RDp7A=.428d32d1-0624-48de-ac7b-0f1acc6a0a14@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>
 <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>
 <a4Qk5XdNcGz-gcsdR-CvY0MwnQv1tZN9pvX-k5ctLkk=.a28c804d-dd76-4bc1-b215-3c8b4f5eca68@github.com>
 <8_W3gPFqX8RC7V2QvFSmKAOTEK4z6uHOf4NnA0RDp7A=.428d32d1-0624-48de-ac7b-0f1acc6a0a14@github.com>
Message-ID: <aki1kBw2AQvmAYEnLLEEvs2e8H7ZDG_V2eSTIHquIJE=.c03ba99d-18c2-4c86-a8d4-520d4f912d18@github.com>

On Mon, 27 May 2024 13:04:32 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> If your comment refers only to these lines of code, they are already verified. Since, inside the split function, the sub-regions get the new flags and all the reserved and committed amounts are moved from the large region to the new ones. So, the accounting of memory is correct.
>> 
>> FWIW, if we trace down the call at line 1346 of `total_space_rs = Metaspace::reserve_address_space_for_compressed_classes(total_range_size, false /* optimize_for_zero_base */);` the region may get different flags of `mtClass` or `mtMetaspace` based on the checked criteria down there.
>>  
>> If you comment on all such cases, then I will double check for them and add assertion for.
>
> My point is that the `archive_space_rs` and `class_space_rs` can get the wrong flags assigned to them. The split functions don't change them. Right?
> 
> I would like to see the code run through our testing with these checks:
> 
>   assert(archive_space_rs.nmt_flag() == mtClassShared, "Sanity");
>   assert(class_space_rs.nmt_flag() == mtClass, "Sanity");

The call to `MemTracker::record_virtual_memory_split_reserved` at line 1364, takes two flags for the split parts. The corresponding regions in NMT take that flags.
The sanity assertions will be added anyway.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1616398251

From ysr at openjdk.org  Tue May 28 13:05:24 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 13:05:24 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v4]
In-Reply-To: <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
 <DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>
 <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
Message-ID: <-tMFqjG1UF42dNRGpTuMkXz23qoKmPCXvh8ijxK-iZg=.dc040b68-0ea9-4723-ab46-072a246bdf61@github.com>

On Thu, 23 May 2024 21:40:38 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Left a few minor coding suggestings. On a higher level, it's not clear to me when and which threads should use `gc_generation` or `active_generation`. I gather that active_generation is set on the safepoint and `gc_generation` is set by control thread, does that mean any code before `init-mark` needs to use `gc_generation`?

Thanks for the review. I'll make the suggested changes and respond individually to the feedback.

As regards, `gc_generation` vs `active_generation`, the intention is that only the load-reference-barrier and the mutator threads use `active_generation` and the GC worker threads use `gc_generation` (unless executing the load-reference-barrier). The idea is that the LRB's notion of an active generation cannot be updated safely except at a safepoint. I suspect this splitting of roles still presents a problem when evacuations occur during mixed collections. So there is likely more work to do to clean up the two roles.

More generally, `gc_generation` is tightly controlled by scopes by the coordinator thread, and indicate the generation that GC worker threads are dealing with at any time, except during evacuation/update-refs when the load-reference-barrier uses the active_generation.

I suspect there is still some badness or mix-ups in these LRB paths that is leading to issues that need to be resolved (there are some failures since a recent merge that I am yet to resolve).

Once I am done with those, I hope to have a cleaner solution.

Meanwhile, I am going to go over your suggestions and make the changes stemming from them.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/407#issuecomment-2135165180

From ysr at openjdk.org  Tue May 28 13:35:29 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 13:35:29 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v4]
In-Reply-To: <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
 <DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>
 <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
Message-ID: <msm_EF3FND1BEykJma3Buvfba0SEbFKbgJ-OBPDExL8=.8ce7434e-b789-442f-a0f3-1a5c28d0b6c5@github.com>

On Thu, 23 May 2024 20:44:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Small clean-ups.
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 931:
> 
>> 929:   if (!CompressedOops::is_null(obj)) {
>> 930:     if (!_mark_context->is_marked(obj)) {
>> 931:       _heap->assert_generations_reconciled();
> 
> Could put this in `shenandoahAsserts.hpp` as macro: `shenandoah_assert_generations_reconciled()`.

Done.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1617256289

From ysr at openjdk.org  Tue May 28 13:47:31 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 13:47:31 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v5]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <mhTMsCpd_i8LfnrdDhDhOw2toDC94Vu-TJYYOBfHZiw=.f8b51242-0e15-4c78-aa9b-3aba6ddecf2a@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:

  shenandoah_assert_generations_reconciled() macro instead of a method in
  ShenandoahHeap, bringing it in line with shenandoah assertion check
  idiom.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/407/files
  - new: https://git.openjdk.org/shenandoah/pull/407/files/b609d78b..70014a9d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=04
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=03-04

  Stats: 47 lines in 11 files changed: 12 ins; 15 del; 20 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From ysr at openjdk.org  Tue May 28 13:58:18 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 13:58:18 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v4]
In-Reply-To: <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
 <DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>
 <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
Message-ID: <kGr-wnJqOTRJxPiUZPtoKZ8gqzCrE4KGIwUaiPPy-r0=.57d5a135-7304-468e-8c58-fafadc12629e@github.com>

On Thu, 23 May 2024 20:52:02 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Small clean-ups.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1578:
> 
>> 1576: void ShenandoahHeap::set_gc_generation(ShenandoahGeneration* generation) {
>> 1577:   _gc_generation = generation;
>> 1578:   assert(Thread::current() == ShenandoahController::thread() ||
> 
> Could use `shenandoah_assert_control_or_vm_thread` from `shenandoahAsserts.hpp` here?

Done (defined and used a stronger version of the assertion that checks that we are at a safepoint if vm thread).

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1617292646

From ysr at openjdk.org  Tue May 28 14:06:39 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 14:06:39 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v6]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <6-ld_WuLG3YB8aV1V401vlsVF7us5hajJsiHqWibltw=.e38c28e3-80c0-4e2b-92dc-6f7604edaecf@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:

  Macro-ize previously inlined shenandoah assert.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/407/files
  - new: https://git.openjdk.org/shenandoah/pull/407/files/70014a9d..982dcd22

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=05
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=04-05

  Stats: 9 lines in 2 files changed: 6 ins; 3 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From ysr at openjdk.org  Tue May 28 14:24:35 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 14:24:35 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v7]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <EW_V1LveZ7LOfUv18rNzzeUE0Qaey4WtzbFEOeRDom4=.c51fc547-1ae4-4b53-9792-6b10737fe0f3@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:

  Remove static _thread field introduced in ShenandoahController, as its
  role is already served by the _control_thread field in ShenandoahHeap.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/407/files
  - new: https://git.openjdk.org/shenandoah/pull/407/files/982dcd22..9f1e1d6b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=06
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=05-06

  Stats: 21 lines in 5 files changed: 0 ins; 20 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From ysr at openjdk.org  Tue May 28 14:24:36 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 14:24:36 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v4]
In-Reply-To: <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
 <DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>
 <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
Message-ID: <t4aK-CLTwpP2q3M-mNHAxZ6JPOpq3XRNRlsXIpGl6WY=.4b8f6cd1-c4e7-4365-9e01-3d2006153e29@github.com>

On Thu, 23 May 2024 20:59:19 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Small clean-ups.
>
> src/hotspot/share/gc/shenandoah/shenandoahController.cpp line 31:
> 
>> 29: #include "gc/shenandoah/shenandoahHeapRegion.inline.hpp"
>> 30: 
>> 31: Thread* ShenandoahController::_thread = nullptr;
> 
> I don't think this is necessary. We already have `ShenandoahHeap::control_thread()`, which is the same thing. That is `ShenandoahControlThread::this == Thread::current() == ShenandoahHeap::_control_thread`.

Thanks, removed & replaced usage.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1617337449

From ysr at openjdk.org  Tue May 28 14:36:34 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 14:36:34 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v8]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <VYxpWUbPJFrgkL2r-YKLw46TU6JxI14gvhP_-lq_DEc=.34f4849c-ce60-4846-8ec8-68e984bacb60@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request incrementally with two additional commits since the last revision:

 - cosmetic
 - jcheck white-space

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/407/files
  - new: https://git.openjdk.org/shenandoah/pull/407/files/9f1e1d6b..0541a99c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=07
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=06-07

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From ysr at openjdk.org  Tue May 28 14:49:47 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 14:49:47 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v9]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <U-_tGX6KX7OIfbkU_ghpeD5dtfKOYbU9qewGMMif1GU=.6d58a413-d0a9-4b9b-a715-38e226b44a07@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:

  Remove vestigial thread() method in ShenandoahController.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/407/files
  - new: https://git.openjdk.org/shenandoah/pull/407/files/0541a99c..56e35ac3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=08
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=07-08

  Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From ysr at openjdk.org  Tue May 28 15:43:36 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 28 May 2024 15:43:36 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v10]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <y3VuJKOki6j6MkvFOEMK60-_g73ZevwHk6F21IvwKJk=.d89945d2-849d-4e8e-a934-71e1e58ef9fd@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits:

 - Merge branch 'master' into active_generation
 - Remove vestigial thread() method in ShenandoahController.
 - cosmetic
 - jcheck white-space
 - Remove static _thread field introduced in ShenandoahController, as its
   role is already served by the _control_thread field in ShenandoahHeap.
 - Macro-ize previously inlined shenandoah assert.
 - shenandoah_assert_generations_reconciled() macro instead of a method in
   ShenandoahHeap, bringing it in line with shenandoah assertion check
   idiom.
 - Small clean-ups.
 - jcheck whitespace
 - Fixed mismerge because of code moving between files in merge from
   master; encapsulate assert & provide more useful debugging info. We
   aren't done yet... It might be the case that we do away with the
   sync/async split of the variable and instead move the relevant state
   into the closure if possible. That might have to wait for another day
   though based on whether it'll work well everywhere or not.
 - ... and 30 more: https://git.openjdk.org/shenandoah/compare/894aefc0...f1f40981

-------------

Changes: https://git.openjdk.org/shenandoah/pull/407/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=09
  Stats: 167 lines in 17 files changed: 132 ins; 1 del; 34 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From wkemper at openjdk.org  Tue May 28 18:06:10 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 28 May 2024 18:06:10 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost for
 degenerated cycle
Message-ID: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>

If Shenandoah does not detect a cancellation after completing concurrent mark, but before completing final mark it will fall through the concurrent cycle without collecting anything. This will shortly lead to a degenerated cycle which will _not_ use the results from the nearly complete concurrent mark. This will result in an unnecessarily longer degenerated cycle.

-------------

Commit messages:
 - Check again for cancellation after final mark

Changes: https://git.openjdk.org/jdk/pull/19434/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19434&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8333105
  Stats: 20 lines in 1 file changed: 20 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/19434.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19434/head:pull/19434

PR: https://git.openjdk.org/jdk/pull/19434

From wkemper at openjdk.org  Tue May 28 18:08:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 28 May 2024 18:08:25 GMT
Subject: Integrated: 8332913: GenShen: Restore shared update refs iterator
In-Reply-To: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
References: <amkImT0_UC3SFeq33iNBJHljknY_fvWIhfcPgRqRM4k=.f4d30d27-0f23-4c4f-b8a3-fcbcdb03773c@github.com>
Message-ID: <cq-3LgBymnnJ7FKNjOWykV-J4IrmqV9V_CcWZzqmV0Q=.e6de0cae-2d1c-45cc-987a-f94105dbfc49@github.com>

On Fri, 24 May 2024 17:55:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Turns out, this iterator is shared between concurrent and degenerated GC cycles. It should not have been made into a local variable.

This pull request has now been integrated.

Changeset: 31bca307
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/31bca3076634a3119a049283587dedd1c3de9f16
Stats:     21 lines in 4 files changed: 11 ins; 3 del; 7 mod

8332913: GenShen: Restore shared update refs iterator

Reviewed-by: ysr, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/439

From wkemper at openjdk.org  Tue May 28 18:19:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 28 May 2024 18:19:47 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <6bFbVDMMrxaSSTuQpNNE0BAjzmu2HG3dbhU63x8wDC8=.b5920329-aa3c-4de7-82f5-d437852fff6e@github.com>
References: <6bFbVDMMrxaSSTuQpNNE0BAjzmu2HG3dbhU63x8wDC8=.b5920329-aa3c-4de7-82f5-d437852fff6e@github.com>
Message-ID: <d8K2wVxVTdvOV3rRasRrMg-ayIw63yVA1KsATOZSMRE=.aa6d2e3f-e948-4c58-aa07-fd0ddd3213e2@github.com>

On Thu, 23 May 2024 14:16:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.4+4

This pull request has now been integrated.

Changeset: 0dda663b
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/0dda663b6291b86888216bab941d35e0eb861a21
Stats:     1020 lines in 27 files changed: 947 ins; 0 del; 73 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/51

From wkemper at openjdk.org  Tue May 28 18:19:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 28 May 2024 18:19:45 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <6bFbVDMMrxaSSTuQpNNE0BAjzmu2HG3dbhU63x8wDC8=.b5920329-aa3c-4de7-82f5-d437852fff6e@github.com>
References: <6bFbVDMMrxaSSTuQpNNE0BAjzmu2HG3dbhU63x8wDC8=.b5920329-aa3c-4de7-82f5-d437852fff6e@github.com>
Message-ID: <YGDwCN38U09gHv1R1S1po5eRCuOv05Mk-nNfSJYTmqg=.002f2065-8dcf-436c-a7c4-5c51e1e5bc57@github.com>

> Merges tag jdk-21.0.4+4

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/51/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/51/files/c44d4e93..c44d4e93

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=51&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=51&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/51.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/51/head:pull/51

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/51

From wkemper at openjdk.org  Tue May 28 18:46:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 28 May 2024 18:46:39 GMT
Subject: RFR: 8333109: GenShen: Factor generational mode out of gc helpers
 (redo)
Message-ID: <25XkcHMeVqnL8huICQhK2vywDNl2pnzANHyn8Ir2swk=.8ea5c82d-f718-4fa0-8f50-7ee09696417d@github.com>

This is the same as https://git.openjdk.org/shenandoah/pull/436, except that does _not_ remove a critical cancellation check. The comments around this cancellation check have been improved.

-------------

Commit messages:
 - Merge remote-tracking branch 'shenandoah/master' into fix-isolate-gc-helpers
 - Improve comments and cancellation check
 - Improve comment regarding cancellation during mark
 - Restore wrong looking cancellation check

Changes: https://git.openjdk.org/shenandoah/pull/441/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=441&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8333109
  Stats: 201 lines in 1 file changed: 33 ins; 109 del; 59 mod
  Patch: https://git.openjdk.org/shenandoah/pull/441.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/441/head:pull/441

PR: https://git.openjdk.org/shenandoah/pull/441

From wkemper at openjdk.org  Tue May 28 19:14:57 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 28 May 2024 19:14:57 GMT
Subject: RFR: 8333109: GenShen: Factor generational mode out of gc helpers
 (redo) [v2]
In-Reply-To: <25XkcHMeVqnL8huICQhK2vywDNl2pnzANHyn8Ir2swk=.8ea5c82d-f718-4fa0-8f50-7ee09696417d@github.com>
References: <25XkcHMeVqnL8huICQhK2vywDNl2pnzANHyn8Ir2swk=.8ea5c82d-f718-4fa0-8f50-7ee09696417d@github.com>
Message-ID: <QD6T97lTq8nfGDsJJLJKJMVXW135O-M1UBkR_Dial2E=.b97e5778-a658-4ed1-9c46-1a2d9762bc83@github.com>

> This is the same as https://git.openjdk.org/shenandoah/pull/436, except that does _not_ remove a critical cancellation check. The comments around this cancellation check have been improved.

William Kemper has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains two new commits since the last revision:

 - Restore cancellation check
 - Restore orginal isolate-gc-helpers commit
   
   This reverts commit 894aefc077d495b3cd892d339e9ae9120cf3be54.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/441/files
  - new: https://git.openjdk.org/shenandoah/pull/441/files/553192c6..13de0bc9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=441&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=441&range=00-01

  Stats: 144 lines in 5 files changed: 99 ins; 31 del; 14 mod
  Patch: https://git.openjdk.org/shenandoah/pull/441.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/441/head:pull/441

PR: https://git.openjdk.org/shenandoah/pull/441

From kdnilsen at openjdk.org  Tue May 28 21:14:01 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 28 May 2024 21:14:01 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
Message-ID: <uAAI4NFMM-pkOdKijX9FcZFBkRPI-Pc8L9Uj3VHFt5Q=.7e940218-b6ec-4bcb-b5b1-949c93c3c98e@github.com>

On Tue, 28 May 2024 18:01:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> If Shenandoah does not detect a cancellation after completing concurrent mark, but before completing final mark it will fall through the concurrent cycle without collecting anything. This will shortly lead to a degenerated cycle which will _not_ use the results from the nearly complete concurrent mark. This will result in an unnecessarily longer degenerated cycle.

Marked as reviewed by kdnilsen (no project role).

-------------

PR Review: https://git.openjdk.org/jdk/pull/19434#pullrequestreview-2083855329

From kdnilsen at openjdk.org  Tue May 28 21:29:22 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 28 May 2024 21:29:22 GMT
Subject: RFR: 8333109: GenShen: Factor generational mode out of gc helpers
 (redo) [v2]
In-Reply-To: <QD6T97lTq8nfGDsJJLJKJMVXW135O-M1UBkR_Dial2E=.b97e5778-a658-4ed1-9c46-1a2d9762bc83@github.com>
References: <25XkcHMeVqnL8huICQhK2vywDNl2pnzANHyn8Ir2swk=.8ea5c82d-f718-4fa0-8f50-7ee09696417d@github.com>
 <QD6T97lTq8nfGDsJJLJKJMVXW135O-M1UBkR_Dial2E=.b97e5778-a658-4ed1-9c46-1a2d9762bc83@github.com>
Message-ID: <quQ8Qd0LwWJ5k81eKVqhBzqRhO5fKe7a63Jo1fDlMdc=.b96284a8-88d5-4485-8e3f-40233368d73c@github.com>

On Tue, 28 May 2024 19:14:57 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This is the same as https://git.openjdk.org/shenandoah/pull/436, except that does _not_ remove a critical cancellation check. The comments around this cancellation check have been improved.
>
> William Kemper has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains two new commits since the last revision:
> 
>  - Restore cancellation check
>  - Restore orginal isolate-gc-helpers commit
>    
>    This reverts commit 894aefc077d495b3cd892d339e9ae9120cf3be54.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/441#pullrequestreview-2083878449

From eosterlund at openjdk.org  Wed May 29 07:37:10 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Wed, 29 May 2024 07:37:10 GMT
Subject: RFR: 8333005: Deadlock when setting or updating the inline cache
Message-ID: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>

In our concurrently class unloading collectors (ZGC, Generational ZGC, Shenandoah), there is a per-nmethod lock. This lock is used to protect the nmethod oops, and is used by nmethod entry barriers and for lazily computing the value of is_unloading, for example. It is also used to protect some other random stuff, and it is also used to protect the state machine of inline caches, which is otherwise completely orthogonal to any of the GC stuff.

Because the lock is used to protect the inline caches (is taken by the CompiledICLocker), you are not allowed to call is_unloading() on *other* nmethods while holding it. Because when we need access to the oops to compute is_unloading, we need to take the nmethod lock. So if two nmethods have inline caches pointing at each other, and calls are resolved at the same time, while concurrent class unloading is going on, we can sometimes get a deadlock.

I accidentally introduced such a bug when I removed the ICStubs (https://bugs.openjdk.org/browse/JDK-8322630), where this has indeed been observed.

The intention with this patch is to make the system less fragile. While it's possible to move around the call to is_unloading() to get rid of the deadlocks, I think I will sleep better at night knowing that you can call is_unloading() anywhere, at least in the shared runtime code, without knowing the GC implementation details. So I'm adding a per-nmethod inline cache lock that protects the completely orthogonal inline cache state for the CompiledICLocker. This way these deadlocks can't happen.

Tested ZGC tests tier1-7, and it looks green. The reproducer that caught the problem, also has stopped reproducing.

-------------

Commit messages:
 - 8333005: Deadlock when setting or updating the inline cache

Changes: https://git.openjdk.org/jdk/pull/19446/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19446&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8333005
  Stats: 85 lines in 15 files changed: 44 ins; 0 del; 41 mod
  Patch: https://git.openjdk.org/jdk/pull/19446.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19446/head:pull/19446

PR: https://git.openjdk.org/jdk/pull/19446

From stefank at openjdk.org  Wed May 29 07:42:02 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 29 May 2024 07:42:02 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <aki1kBw2AQvmAYEnLLEEvs2e8H7ZDG_V2eSTIHquIJE=.c03ba99d-18c2-4c86-a8d4-520d4f912d18@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>
 <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>
 <a4Qk5XdNcGz-gcsdR-CvY0MwnQv1tZN9pvX-k5ctLkk=.a28c804d-dd76-4bc1-b215-3c8b4f5eca68@github.com>
 <8_W3gPFqX8RC7V2QvFSmKAOTEK4z6uHOf4NnA0RDp7A=.428d32d1-0624-48de-ac7b-0f1acc6a0a14@github.com>
 <aki1kBw2AQvmAYEnLLEEvs2e8H7ZDG_V2eSTIHquIJE=.c03ba99d-18c2-4c86-a8d4-520d4f912d18@github.com>
Message-ID: <BqYLFCOOe-mH9kSA-24l905rsg2amUzYfm_dj5pXWVc=.902b8069-7a3a-440e-ae44-88ceac0fc083@github.com>

On Mon, 27 May 2024 21:25:12 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> My point is that the `archive_space_rs` and `class_space_rs` can get the wrong flags assigned to them. The split functions don't change them. Right?
>> 
>> I would like to see the code run through our testing with these checks:
>> 
>>   assert(archive_space_rs.nmt_flag() == mtClassShared, "Sanity");
>>   assert(class_space_rs.nmt_flag() == mtClass, "Sanity");
>
> The call to `MemTracker::record_virtual_memory_split_reserved` at line 1364, takes two flags for the split parts. The corresponding regions in NMT take that flags.
> The sanity assertions will be added anyway.

I think you are still missing my point. `record_virtual_memory_split_reserved` doesn't update the `archive_space_rs` and `class_space_rs` instances with the correct flag.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1618362958

From azafari at openjdk.org  Wed May 29 10:52:01 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 29 May 2024 10:52:01 GMT
Subject: RFR: 8331539: [REDO] NMT: add/make a mandatory MEMFLAGS argument
 to family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <BqYLFCOOe-mH9kSA-24l905rsg2amUzYfm_dj5pXWVc=.902b8069-7a3a-440e-ae44-88ceac0fc083@github.com>
References: <1i0PKv9mCusM6BZqXG8ULe0lRA2Nz2ix4aZHz9otNMM=.b9d2d151-883e-4cb6-be48-4ba45b49ed43@github.com>
 <_M5SvhyN_E_8HUeamhiLJMp37flhjgTVE_X7t8jmPVc=.f86cbb23-9461-4013-83bf-d6b154b96cfd@github.com>
 <9XzKmn3xJvlbw4gz2vK_NZ6yOwfKB9VzHE6CBSz-73E=.dfaa5291-95b6-403d-b363-42131ebf4c4c@github.com>
 <qRmbGaeAykZJkZea6wdHiWDTflWljMBDveGerqH61T0=.175c3073-3648-44fd-b18d-1dc1c108d183@github.com>
 <xdKxLn95N-sbthY7nvVjRbYugH7opKF-iGTNdIyE7wA=.800a3519-6344-466a-a76a-38ec785931ee@github.com>
 <a4Qk5XdNcGz-gcsdR-CvY0MwnQv1tZN9pvX-k5ctLkk=.a28c804d-dd76-4bc1-b215-3c8b4f5eca68@github.com>
 <8_W3gPFqX8RC7V2QvFSmKAOTEK4z6uHOf4NnA0RDp7A=.428d32d1-0624-48de-ac7b-0f1acc6a0a14@github.com>
 <aki1kBw2AQvmAYEnLLEEvs2e8H7ZDG_V2eSTIHquIJE=.c03ba99d-18c2-4c86-a8d4-520d4f912d18@github.com>
 <BqYLFCOOe-mH9kSA-24l905rsg2amUzYfm_dj5pXWVc=.902b8069-7a3a-440e-ae44-88ceac0fc083@github.com>
Message-ID: <lMxFYODwdwGFK1LiKF0Tvopv0xrGx4EJKEHv2mMPw1M=.252de80e-9bd5-401f-a45e-9ebea1dfc68f@github.com>

On Wed, 29 May 2024 07:39:24 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> The call to `MemTracker::record_virtual_memory_split_reserved` at line 1364, takes two flags for the split parts. The corresponding regions in NMT take that flags.
>> The sanity assertions will be added anyway.
>
> I think you are still missing my point. `record_virtual_memory_split_reserved` doesn't update the `archive_space_rs` and `class_space_rs` instances with the correct flag.

Oh yes sorry. I was referring to the regions and flags in NMT. 
Since this PR is going to be closed without merge, no need to do any change in the code. But I remember it for future.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19343#discussion_r1618672203

From wkemper at openjdk.org  Wed May 29 14:12:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 29 May 2024 14:12:41 GMT
Subject: Integrated: 8333109: GenShen: Factor generational mode out of gc
 helpers (redo)
In-Reply-To: <25XkcHMeVqnL8huICQhK2vywDNl2pnzANHyn8Ir2swk=.8ea5c82d-f718-4fa0-8f50-7ee09696417d@github.com>
References: <25XkcHMeVqnL8huICQhK2vywDNl2pnzANHyn8Ir2swk=.8ea5c82d-f718-4fa0-8f50-7ee09696417d@github.com>
Message-ID: <v08Ym225jcPldQ2nVSX65sjxiM-uB6qrYY8hRjDmhI0=.2f75618c-7666-45b0-9085-5be3d9d155a0@github.com>

On Tue, 28 May 2024 18:42:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is the same as https://git.openjdk.org/shenandoah/pull/436, except that does _not_ remove a critical cancellation check. The comments around this cancellation check have been improved.

This pull request has now been integrated.

Changeset: 840c1825
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/840c1825f38a11485243a648b171b1ba5ec78de5
Stats:     345 lines in 6 files changed: 132 ins; 140 del; 73 mod

8333109: GenShen: Factor generational mode out of gc helpers (redo)

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/441

From duke at openjdk.org  Wed May 29 15:35:15 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Wed, 29 May 2024 15:35:15 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
 [v3]
In-Reply-To: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
Message-ID: <ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>

> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
> 
> **Verification**
> 
> Shenendoah
> 
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
> 
> ZGC
> 
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
> 
> **Issue**
> https://bugs.openjdk.org/browse/JDK-8331911

Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:

  Update full name

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19285/files
  - new: https://git.openjdk.org/jdk/pull/19285/files/8c5bad2e..c59034da

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19285&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19285&range=01-02

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/19285.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19285/head:pull/19285

PR: https://git.openjdk.org/jdk/pull/19285

From shade at openjdk.org  Wed May 29 16:00:02 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 29 May 2024 16:00:02 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
 [v3]
In-Reply-To: <ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
 <ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
Message-ID: <UhOht3AsvqgiYoFA2NyzSb1AZl8v5Y1sEICAw2AnR-Q=.68b547b5-b3cb-41ea-9974-35ed6b476d91@github.com>

On Wed, 29 May 2024 15:35:15 GMT, Neethu Prasad <duke at openjdk.org> wrote:

>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>> 
>> **Verification**
>> 
>> Shenendoah
>> 
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>> 
>> ZGC
>> 
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>> 
>> **Issue**
>> https://bugs.openjdk.org/browse/JDK-8331911
>
> Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update full name

@fisk @stefank -- are you good with this for ZGC?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2137755347

From eosterlund at openjdk.org  Wed May 29 16:03:07 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Wed, 29 May 2024 16:03:07 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
 [v3]
In-Reply-To: <ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
 <ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
Message-ID: <uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>

On Wed, 29 May 2024 15:35:15 GMT, Neethu Prasad <duke at openjdk.org> wrote:

>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>> 
>> **Verification**
>> 
>> Shenendoah
>> 
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>> 
>> ZGC
>> 
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>> 
>> **Issue**
>> https://bugs.openjdk.org/browse/JDK-8331911
>
> Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update full name

It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.

-------------

Changes requested by eosterlund (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19285#pullrequestreview-2085826962

From duke at openjdk.org  Wed May 29 17:17:02 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Wed, 29 May 2024 17:17:02 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
 [v3]
In-Reply-To: <uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
 <ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
 <uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>
Message-ID: <-GBU6zDyLGsnEyT3yEMfW_TnA-stpR1tLeqfvfeVcQ0=.c4b13824-35a4-4e22-8132-721a668a0425@github.com>

On Wed, 29 May 2024 16:00:01 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:

> It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.

Thanks for the feedback! Looking around the code, I think there are a few places where we can do more changes.

First, remove check here:
https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/code/nmethod.cpp#L852-L855

This would force us to add the check in super-class implementation here:
https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L83

Second, we can remove the check here:
https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L175-L181

But it does not seem straightforward, because we currently skip cross-modification fence based on?is_armed(...)?check. Unfortunately, we cannot easily know if?nmethod_entry_barrier?acted or not, we only know if method is safe or not.? Can we / should we do these refactoring separately?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2137901687

From shade at openjdk.org  Wed May 29 18:29:02 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 29 May 2024 18:29:02 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
Message-ID: <cb_QhVgFZl7lVLFrrtTQmc0Ef11nifSzouTBxqGgtAM=.a6ed3a61-4b2b-4fb5-afcf-55e4713f0804@github.com>

On Tue, 28 May 2024 18:01:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> If Shenandoah does not detect a cancellation after completing concurrent mark, but before completing final mark it will fall through the concurrent cycle without collecting anything. This will shortly lead to a degenerated cycle which will _not_ use the results from the nearly complete concurrent mark. This will result in an unnecessarily longer degenerated cycle.

OK, this looks reasonable. I don't think we need a verbose comment here. Something like this would suffice:


// Handle the corner case when GC cancellation was detected only during the final mark. 
// Final mark leaves the in-progress flag up in this case, and does not do any evacs.
if (heap->is_concurrent_mark_in_progress()) {
   bool cancelled = check_cancellation_and_abort(ShenandoahDegenPoint::_degenerated_mark);
    assert(cancelled, "GC must have been cancelled between concurrent and final mark");
    return false;
}


How often does this corner case fire anyway?

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19434#pullrequestreview-2086121356

From kbarrett at openjdk.org  Wed May 29 18:57:09 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Wed, 29 May 2024 18:57:09 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort
Message-ID: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>

The "idempotent" argument is removed from that function, with associated
simplifications to the implementation. Callers are updated to remove that
argument. Callers that were providing a false value are unaffected in their
behavior.  The 3 callers that were providing a true value to request the
associated feature are also unaffected (other than by being made faster),
because the arrays involved don't contain any equivalent pairs.

There are also some miscellaneous cleanups, including using the swap utility
and fixing some comments.

Testing: mach5 tier1-3

-------------

Commit messages:
 - remove idempotent

Changes: https://git.openjdk.org/jdk/pull/19464/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19464&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8333133
  Stats: 125 lines in 11 files changed: 3 ins; 95 del; 27 mod
  Patch: https://git.openjdk.org/jdk/pull/19464.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19464/head:pull/19464

PR: https://git.openjdk.org/jdk/pull/19464

From wkemper at openjdk.org  Wed May 29 19:10:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 29 May 2024 19:10:01 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
Message-ID: <7UCkZfX3vYz3suB2qIxOFsq7sGqB1YNL21h-NybfmtM=.ee5eb3ce-6189-424a-9a99-67ee9e9a448e@github.com>

On Tue, 28 May 2024 18:01:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> If Shenandoah does not detect a cancellation after completing concurrent mark, but before completing final mark it will fall through the concurrent cycle without collecting anything. This will shortly lead to a degenerated cycle which will _not_ use the results from the nearly complete concurrent mark. This will result in an unnecessarily longer degenerated cycle.

I'll try to trim it down. I shot myself in the foot after being confused by the original comment and deleting this check. Not having this check is not so serious for non-generational modes, but it is necessary to avoid heap corruption and crashes in the generational mode.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19434#issuecomment-2138084507

From ysr at openjdk.org  Wed May 29 19:50:02 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 29 May 2024 19:50:02 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <6f9RyJck-D4ntL6KZeMmZgJ76-Bxsfmr4jpn75GkDuI=.e0f5e141-1322-4e69-9bbc-5919d411399f@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
 <6f9RyJck-D4ntL6KZeMmZgJ76-Bxsfmr4jpn75GkDuI=.e0f5e141-1322-4e69-9bbc-5919d411399f@github.com>
Message-ID: <LKYkrNAlord7sxvYyqbIWXSvcJ4kC_TPkj8K67IuChE=.23d10375-abf5-498b-a720-63b4a5da72c9@github.com>

On Wed, 29 May 2024 19:29:51 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> If Shenandoah does not detect a cancellation after completing concurrent mark, but before completing final mark it will fall through the concurrent cycle without collecting anything. This will shortly lead to a degenerated cycle which will _not_ use the results from the nearly complete concurrent mark. This will result in an unnecessarily longer degenerated cycle.
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 147:
> 
>> 145:     bool cancelled = check_cancellation_and_abort(ShenandoahDegenPoint::_degenerated_mark);
>> 146:     assert(cancelled, "GC must have been cancelled between concurrent and final mark");
>> 147:     return false;
> 
> Not your change, but I notice that the semantics of the return value hasn't been explicitly documented anywhere in the class hierarchy. Might make sense to place a line of documentation, say in `ShenandoahGC` which first declares the virtual method, even something simple such as:
> 
> 
> // Returns true if the collection was completed successfully, false otherwise.

It also occurs to me that the pattern here is, roughly of the following form:


  do_some_uninterruptible_operation();
  check cancellation status & return if cancelled, else continue to next uninterruptible operation;
  ...


In that case, the checks should not be on the state of the progress of the collection, but on whether the cycle has been cancelled, and the asserts should then check the state of the collection; in other words, the dual of the structure of control in this method.

That said, I do not understand if there's a deeper rationale here in that sometimes we will want to skip a next uninterruptible phase of the collection regardless, because the state machine of the collection explicitly transitions us, under appropriate conditions, into a different phase than the otherwise normal flow, independent of whether a cancellation happened.

For example, what would be the difference between your current test:


  if (concurrent_mark_in_progress()) {


vs, say:


  if (!evacuation_in_progress()) {


vs : 


   if (check_cancellation()) {
     ... return false;
   }
   assert(!concurrent_mark_in_progress(), "Should have completed");
   assert(evacuation_in_progress(), "Next logical phase of collection");


Just wondering if a more uniform idiom might make for easier code patterns and easier, more uniform, reasoning about correctness.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19434#discussion_r1619395782

From ysr at openjdk.org  Wed May 29 19:50:01 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 29 May 2024 19:50:01 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
Message-ID: <6f9RyJck-D4ntL6KZeMmZgJ76-Bxsfmr4jpn75GkDuI=.e0f5e141-1322-4e69-9bbc-5919d411399f@github.com>

On Tue, 28 May 2024 18:01:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> If Shenandoah does not detect a cancellation after completing concurrent mark, but before completing final mark it will fall through the concurrent cycle without collecting anything. This will shortly lead to a degenerated cycle which will _not_ use the results from the nearly complete concurrent mark. This will result in an unnecessarily longer degenerated cycle.

Looks good, but left some comments to think about may be for the future.

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 147:

> 145:     bool cancelled = check_cancellation_and_abort(ShenandoahDegenPoint::_degenerated_mark);
> 146:     assert(cancelled, "GC must have been cancelled between concurrent and final mark");
> 147:     return false;

Not your change, but I notice that the semantics of the return value hasn't been explicitly documented anywhere in the class hierarchy. Might make sense to place a line of documentation, say in `ShenandoahGC` which first declares the virtual method, even something simple such as:


// Returns true if the collection was completed successfully, false otherwise.

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19434#pullrequestreview-2086231537
PR Review Comment: https://git.openjdk.org/jdk/pull/19434#discussion_r1619378762

From ysr at openjdk.org  Wed May 29 19:57:10 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 29 May 2024 19:57:10 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <7UCkZfX3vYz3suB2qIxOFsq7sGqB1YNL21h-NybfmtM=.ee5eb3ce-6189-424a-9a99-67ee9e9a448e@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
 <7UCkZfX3vYz3suB2qIxOFsq7sGqB1YNL21h-NybfmtM=.ee5eb3ce-6189-424a-9a99-67ee9e9a448e@github.com>
Message-ID: <k4ZlXSnkgeHGH4rmaOdMbyX-8uSXm4WdJhLFPT8ka94=.19a42982-5b34-41fa-aa07-8374e1651c42@github.com>

On Wed, 29 May 2024 19:07:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> I'll try to trim it down. I shot myself in the foot after being confused by the original comment and deleting this check. Not having this check is not so serious for non-generational modes, but it is necessary to avoid heap corruption and crashes in the generational mode.

Could you, for the sake of completeness, document this specific _generational pathology_ in the ticket, because it might bring up some interesting thinking about the ways in which the two modes might interact wrt the existing structure of the state machine driving the progress of the collections in the two cases, and the presence of asynchronous interruptions of that normal state machine flow/transitions.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19434#issuecomment-2138158526

From wkemper at openjdk.org  Thu May 30 00:09:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 30 May 2024 00:09:01 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <LKYkrNAlord7sxvYyqbIWXSvcJ4kC_TPkj8K67IuChE=.23d10375-abf5-498b-a720-63b4a5da72c9@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
 <6f9RyJck-D4ntL6KZeMmZgJ76-Bxsfmr4jpn75GkDuI=.e0f5e141-1322-4e69-9bbc-5919d411399f@github.com>
 <LKYkrNAlord7sxvYyqbIWXSvcJ4kC_TPkj8K67IuChE=.23d10375-abf5-498b-a720-63b4a5da72c9@github.com>
Message-ID: <K3d-uWcKHfLhhsKYS2yKEk9-J-pLcX1esZ_1MhjxSyk=.626811b9-2356-454e-88c7-d5756e0bcffe@github.com>

On Wed, 29 May 2024 19:46:15 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 147:
>> 
>>> 145:     bool cancelled = check_cancellation_and_abort(ShenandoahDegenPoint::_degenerated_mark);
>>> 146:     assert(cancelled, "GC must have been cancelled between concurrent and final mark");
>>> 147:     return false;
>> 
>> Not your change, but I notice that the semantics of the return value hasn't been explicitly documented anywhere in the class hierarchy. Might make sense to place a line of documentation, say in `ShenandoahGC` which first declares the virtual method, even something simple such as:
>> 
>> 
>> // Returns true if the collection was completed successfully, false otherwise.
>
> It also occurs to me that the pattern here is, roughly of the following form:
> 
> 
>   do_some_uninterruptible_operation();
>   check cancellation status & return if cancelled, else continue to next uninterruptible operation;
>   ...
> 
> 
> In that case, the checks should not be on the state of the progress of the collection, but on whether the cycle has been cancelled, and the asserts should then check the state of the collection; in other words, the dual of the structure of control in this method.
> 
> That said, I do not understand if there's a deeper rationale here in that sometimes we will want to skip a next uninterruptible phase of the collection regardless, because the state machine of the collection explicitly transitions us, under appropriate conditions, into a different phase than the otherwise normal flow, independent of whether a cancellation happened.
> 
> For example, what would be the difference between your current test:
> 
> 
>   if (concurrent_mark_in_progress()) {
> 
> 
> vs, say:
> 
> 
>   if (!evacuation_in_progress()) {
> 
> 
> vs : 
> 
> 
>    if (check_cancellation()) {
>      ... return false;
>    }
>    assert(!concurrent_mark_in_progress(), "Should have completed");
>    assert(evacuation_in_progress(), "Next logical phase of collection");
> 
> 
> Just wondering if a more uniform idiom might make for easier code patterns and easier, more uniform, reasoning about correctness.

This case doesn't quite follow the `check_cancellation` pattern of the others because it is a little bit different. Here, the uninterruptible operation is the final mark safepoint. This safepoint will itself check if the mark has been cancelled. If final mark detects the cancellation, it will do _nothing_, the concurrent mark will still be in progress and we _must_ resume the degenerated cycle from the marking phase. However, if the final mark safepoint does _not_ detect a cancellation, it will initialize the evacuation phase. If the code simply `checks_cancellation` after final mark it cannot tell if the cancellation request was received just _before_ final mark or just _after_ final mark. These two conditions require different resumption points for the degenerated cycle, so we must check the state of the collection cycle here.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19434#discussion_r1619610654

From wkemper at openjdk.org  Thu May 30 00:27:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 30 May 2024 00:27:13 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle [v2]
In-Reply-To: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
Message-ID: <NWgupf5WmysQSwOOHqkDuGzE5x1MgnPqbLnBKtL6YTo=.108ef8ee-cc8a-4aeb-aecc-497166231919@github.com>

> If Shenandoah does not detect a cancellation after completing concurrent mark, but before completing final mark it will fall through the concurrent cycle without collecting anything. This will shortly lead to a degenerated cycle which will _not_ use the results from the nearly complete concurrent mark. This will result in an unnecessarily longer degenerated cycle.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Improve comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/19434/files
  - new: https://git.openjdk.org/jdk/pull/19434/files/165b3f76..05e8850d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=19434&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=19434&range=00-01

  Stats: 15 lines in 2 files changed: 1 ins; 10 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/19434.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19434/head:pull/19434

PR: https://git.openjdk.org/jdk/pull/19434

From sjohanss at openjdk.org  Thu May 30 08:04:01 2024
From: sjohanss at openjdk.org (Stefan Johansson)
Date: Thu, 30 May 2024 08:04:01 GMT
Subject: RFR: 8333005: Deadlock when setting or updating the inline cache
In-Reply-To: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
References: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
Message-ID: <Vtpy1muuHvy3SVyQbDWkpHJBHoASZfeldv9CVIFQWHk=.81a2a593-8382-45c8-ab8d-0c2a21f66749@github.com>

On Wed, 29 May 2024 07:32:55 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:

> In our concurrently class unloading collectors (ZGC, Generational ZGC, Shenandoah), there is a per-nmethod lock. This lock is used to protect the nmethod oops, and is used by nmethod entry barriers and for lazily computing the value of is_unloading, for example. It is also used to protect some other random stuff, and it is also used to protect the state machine of inline caches, which is otherwise completely orthogonal to any of the GC stuff.
> 
> Because the lock is used to protect the inline caches (is taken by the CompiledICLocker), you are not allowed to call is_unloading() on *other* nmethods while holding it. Because when we need access to the oops to compute is_unloading, we need to take the nmethod lock. So if two nmethods have inline caches pointing at each other, and calls are resolved at the same time, while concurrent class unloading is going on, we can sometimes get a deadlock.
> 
> I accidentally introduced such a bug when I removed the ICStubs (https://bugs.openjdk.org/browse/JDK-8322630), where this has indeed been observed.
> 
> The intention with this patch is to make the system less fragile. While it's possible to move around the call to is_unloading() to get rid of the deadlocks, I think I will sleep better at night knowing that you can call is_unloading() anywhere, at least in the shared runtime code, without knowing the GC implementation details. So I'm adding a per-nmethod inline cache lock that protects the completely orthogonal inline cache state for the CompiledICLocker. This way these deadlocks can't happen.
> 
> Tested ZGC tests tier1-7, and it looks green. The reproducer that caught the problem, also has stopped reproducing.

Looks good.

-------------

Marked as reviewed by sjohanss (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19446#pullrequestreview-2087531270

From shade at openjdk.org  Thu May 30 08:16:05 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 30 May 2024 08:16:05 GMT
Subject: RFR: 8333005: Deadlock when setting or updating the inline cache
In-Reply-To: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
References: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
Message-ID: <q1_ByjeQ8_w4wS4y1JYJIDevKJlU5Xi5JdOMQgxinx4=.b9eb422d-2e52-48b0-b760-31bde6f9e364@github.com>

On Wed, 29 May 2024 07:32:55 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:

> In our concurrently class unloading collectors (ZGC, Generational ZGC, Shenandoah), there is a per-nmethod lock. This lock is used to protect the nmethod oops, and is used by nmethod entry barriers and for lazily computing the value of is_unloading, for example. It is also used to protect some other random stuff, and it is also used to protect the state machine of inline caches, which is otherwise completely orthogonal to any of the GC stuff.
> 
> Because the lock is used to protect the inline caches (is taken by the CompiledICLocker), you are not allowed to call is_unloading() on *other* nmethods while holding it. Because when we need access to the oops to compute is_unloading, we need to take the nmethod lock. So if two nmethods have inline caches pointing at each other, and calls are resolved at the same time, while concurrent class unloading is going on, we can sometimes get a deadlock.
> 
> I accidentally introduced such a bug when I removed the ICStubs (https://bugs.openjdk.org/browse/JDK-8322630), where this has indeed been observed.
> 
> The intention with this patch is to make the system less fragile. While it's possible to move around the call to is_unloading() to get rid of the deadlocks, I think I will sleep better at night knowing that you can call is_unloading() anywhere, at least in the shared runtime code, without knowing the GC implementation details. So I'm adding a per-nmethod inline cache lock that protects the completely orthogonal inline cache state for the CompiledICLocker. This way these deadlocks can't happen.
> 
> Tested ZGC tests tier1-7, and it looks green. The reproducer that caught the problem, also has stopped reproducing.

Looks reasonable.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19446#pullrequestreview-2087569740

From shade at openjdk.org  Thu May 30 11:45:02 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 30 May 2024 11:45:02 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort
In-Reply-To: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
Message-ID: <wlhB17Uooeo53p0QW4rcTK_mnsEF45AWMra4hDHgjd0=.01c0cc5f-3572-439f-bf6c-3f161c55e58e@github.com>

On Wed, 29 May 2024 18:52:03 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:

> The "idempotent" argument is removed from that function, with associated
> simplifications to the implementation. Callers are updated to remove that
> argument. Callers that were providing a false value are unaffected in their
> behavior.  The 3 callers that were providing a true value to request the
> associated feature are also unaffected (other than by being made faster),
> because the arrays involved don't contain any equivalent pairs.
> 
> There are also some miscellaneous cleanups, including using the swap utility
> and fixing some comments.
> 
> Testing: mach5 tier1-3

Looks reasonable.

src/hotspot/share/utilities/quickSort.hpp line 75:

> 73:     for ( ; true; ++left_index, --right_index) {
> 74:       for ( ; comparator(array[left_index], pivot_val) < 0; ++left_index) {
> 75:         assert(left_index < (length - 1), "reached end of partition");

Let me see if I understand this change. It makes assert stronger: we do not accept `left_index == length - 1` anymore. I guess that would mean the pivot is at the last element? Which makes the partition is empty, which cannot happen?

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19464#pullrequestreview-2088033775
PR Review Comment: https://git.openjdk.org/jdk/pull/19464#discussion_r1620546735

From wkemper at openjdk.org  Thu May 30 14:23:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 30 May 2024 14:23:28 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <9cJnMe4iAmTOAvT8MT1p57xQq3o4iPNnJm_yYrPKNGY=.76d16fd2-9e6e-40c8-8b2f-af68e25ad741@github.com>

Merges tag jdk-21.0.4+5

-------------

Commit messages:
 - 8320692: Null icon returned for .exe without custom icon
 - 8330275: Crash in XMark::follow_array
 - 8329862: libjli GetApplicationHome cleanups and enhance jli tracing
 - 8323635: Test gc/g1/TestHumongousAllocConcurrentStart.java fails with -XX:TieredStopAtLevel=3
 - 8295111: dpkg appears to have problems resolving symbolically linked native libraries
 - 8331031: unify os::dont_yield and os::naked_yield across Posix platforms
 - 8329223: Parallel: Parallel GC resizes heap even if -Xms = -Xmx
 - 8330464: hserr generic events - add entry for the before_exit calls
 - 8329605: hs errfile generic events - move memory protections and nmethod flushes to separate sections
 - 8321812: Update GC tests to use execute[Limited]TestJava
 - ... and 7 more: https://git.openjdk.org/shenandoah-jdk21u/compare/c44d4e93...2971cb57

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/52/files
  Stats: 1246 lines in 155 files changed: 442 ins; 338 del; 466 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/52.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/52/head:pull/52

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/52

From eosterlund at openjdk.org  Thu May 30 16:07:03 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Thu, 30 May 2024 16:07:03 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
 [v3]
In-Reply-To: <-GBU6zDyLGsnEyT3yEMfW_TnA-stpR1tLeqfvfeVcQ0=.c4b13824-35a4-4e22-8132-721a668a0425@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
 <ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
 <uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>
 <-GBU6zDyLGsnEyT3yEMfW_TnA-stpR1tLeqfvfeVcQ0=.c4b13824-35a4-4e22-8132-721a668a0425@github.com>
Message-ID: <SM6Tj6DXxNSXZbkM-daZA4gB8baJiPtwcua4JKOcaUg=.2bd3dffb-143c-4dc5-b0ef-166536c0d848@github.com>

On Wed, 29 May 2024 17:14:31 GMT, Neethu Prasad <duke at openjdk.org> wrote:

> > It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.
> 
> 
> 
> Thanks for the feedback! Looking around the code, I think there are a few places where we can do more changes.
> 
> 
> 
> First, remove check here:
> 
> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/code/nmethod.cpp#L852-L855
> 
> 
> 
> This would force us to add the check in super-class implementation here:
> 
> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L83
> 
> 
> 
> Second, we can remove the check here:
> 
> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L175-L181
> 
> 
> 
> But it does not seem straightforward, because we currently skip cross-modification fence based on?is_armed(...)?check. Unfortunately, we cannot easily know if?nmethod_entry_barrier?acted or not, we only know if method is safe or not.? Can we / should we do these refactoring separately?

I see your point. However, this PR is refactoring the code to iron out who is responsible for checking is_armed, so I would prefer if we got that right in this PR. We say it should be the backend code doing that, so the callers shouldn't. I agree with all the changes you just listed and if you make them I would be happy.

Regarding the cross modifying fence, I strongly prefer to not try and be clever. Just run the cross modifying fence unconditionally after calling the backend code. We get there because the barrier was armed anyway.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2140077263

From wkemper at openjdk.org  Thu May 30 16:39:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 30 May 2024 16:39:14 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <k4ZlXSnkgeHGH4rmaOdMbyX-8uSXm4WdJhLFPT8ka94=.19a42982-5b34-41fa-aa07-8374e1651c42@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
 <7UCkZfX3vYz3suB2qIxOFsq7sGqB1YNL21h-NybfmtM=.ee5eb3ce-6189-424a-9a99-67ee9e9a448e@github.com>
 <k4ZlXSnkgeHGH4rmaOdMbyX-8uSXm4WdJhLFPT8ka94=.19a42982-5b34-41fa-aa07-8374e1651c42@github.com>
Message-ID: <PnSp0-3OnfktiGGeRuqOJf2WZp4fN_Jk8bCmPkbiG6U=.971d10db-990a-4add-98b5-56529d638678@github.com>

On Wed, 29 May 2024 19:54:05 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> I'll try to trim it down. I shot myself in the foot after being confused by the original comment and deleting this check. Not having this check is not so serious for non-generational modes, but it is necessary to avoid heap corruption and crashes in the generational mode.
>
>> I'll try to trim it down. I shot myself in the foot after being confused by the original comment and deleting this check. Not having this check is not so serious for non-generational modes, but it is necessary to avoid heap corruption and crashes in the generational mode.
> 
> Could you, for the sake of completeness, document this specific _generational pathology_ in the ticket, because it might bring up some interesting thinking about the ways in which the two modes might interact wrt the existing structure of the state machine driving the progress of the collections in the two cases, and the presence of asynchronous interruptions of that normal state machine flow/transitions.

@ysramakrishna , during init mark, the generational mode copies the cards that have been used by the barrier up until init mark (the 'write' card table) onto a read only copy used by the concurrent mark (the 'read' table) and resets the write table. Mutators continue using the fresh write table. However, the concurrent mark workers are also responsible for _rebuilding_ the write table. If the concurrent mark is cancelled, the write card table may not be complete. If the degenerated cycle resets the card table (as it would without this check), portions of the card table may be lost.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19434#issuecomment-2140192556

From wkemper at openjdk.org  Thu May 30 16:39:15 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 30 May 2024 16:39:15 GMT
Subject: Integrated: 8333105: Shenandoah: Results of concurrent mark may be
 lost for degenerated cycle
In-Reply-To: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
Message-ID: <JXrUKl35yyy6fRxUF12rxXfcv2fBqNu58jPJ5wXJs9g=.cd0959d7-2045-4fa8-bdb1-9186f2bdd4ae@github.com>

On Tue, 28 May 2024 18:01:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> If Shenandoah does not detect a cancellation after completing concurrent mark, but before completing final mark it will fall through the concurrent cycle without collecting anything. This will shortly lead to a degenerated cycle which will _not_ use the results from the nearly complete concurrent mark. This will result in an unnecessarily longer degenerated cycle.

This pull request has now been integrated.

Changeset: 32636dcc
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/32636dcc3d6cd7837c22c5cbcb5c7c6576766cf6
Stats:     11 lines in 2 files changed: 11 ins; 0 del; 0 mod

8333105: Shenandoah: Results of concurrent mark may be lost for degenerated cycle

Reviewed-by: kdnilsen, shade, ysr

-------------

PR: https://git.openjdk.org/jdk/pull/19434

From kbarrett at openjdk.org  Thu May 30 17:59:02 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Thu, 30 May 2024 17:59:02 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort
In-Reply-To: <wlhB17Uooeo53p0QW4rcTK_mnsEF45AWMra4hDHgjd0=.01c0cc5f-3572-439f-bf6c-3f161c55e58e@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
 <wlhB17Uooeo53p0QW4rcTK_mnsEF45AWMra4hDHgjd0=.01c0cc5f-3572-439f-bf6c-3f161c55e58e@github.com>
Message-ID: <xd0NFlkRnQHoS-MmoT6wzzt-SV5-zR3_K_9T0DzyKtU=.61203263-f6c4-41af-b1da-321ad3f77674@github.com>

On Thu, 30 May 2024 11:41:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> The "idempotent" argument is removed from that function, with associated
>> simplifications to the implementation. Callers are updated to remove that
>> argument. Callers that were providing a false value are unaffected in their
>> behavior.  The 3 callers that were providing a true value to request the
>> associated feature are also unaffected (other than by being made faster),
>> because the arrays involved don't contain any equivalent pairs.
>> 
>> There are also some miscellaneous cleanups, including using the swap utility
>> and fixing some comments.
>> 
>> Testing: mach5 tier1-3
>
> src/hotspot/share/utilities/quickSort.hpp line 75:
> 
>> 73:     for ( ; true; ++left_index, --right_index) {
>> 74:       for ( ; comparator(array[left_index], pivot_val) < 0; ++left_index) {
>> 75:         assert(left_index < (length - 1), "reached end of partition");
> 
> Let me see if I understand this change. It makes assert stronger: we do not accept `left_index == length - 1` anymore. I guess that would mean the pivot is at the last element? Which makes the partition is empty, which cannot happen?

The reason I looked at the assert carefully in the first place was that `left_index < length` could
pass and yet we could still do `array[length]`, which could be UB.  So the tightened condition must
be correct, else we have a bug.  Proving we don't have a bug is a little bit harder.

We know that the last element in the sequence can't be less than the pivot,
because of the way find_pivot works.  It arranges for the first, middle, and
last values in the sequence to be ordered A[0] <= A[pivot] <= A[length-1].

The tricky case is the sequence [Vi..., P, P], where P is the pivot and all
Vi < P. The from-left scan proceeds until it reaches the left occurrence of P.
The from-right scan immediately stops on the right occurrence of P. The two
P's are swapped. left_index is incremented and right_index is decremented.
The from-left scan is restarted with left_index == length - 1, stops
immediately because P < P is false, and does not execute the assert. Similarly,
the from-right scan is restarted with right_index == length - 2, and also stops
because P > P is false.  And now the outer loop terminates.

A similar argument applies for the from-right scan, where we already assert
right_index > 0.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19464#discussion_r1621217067

From ysr at openjdk.org  Thu May 30 18:16:07 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 30 May 2024 18:16:07 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle
In-Reply-To: <k4ZlXSnkgeHGH4rmaOdMbyX-8uSXm4WdJhLFPT8ka94=.19a42982-5b34-41fa-aa07-8374e1651c42@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
 <7UCkZfX3vYz3suB2qIxOFsq7sGqB1YNL21h-NybfmtM=.ee5eb3ce-6189-424a-9a99-67ee9e9a448e@github.com>
 <k4ZlXSnkgeHGH4rmaOdMbyX-8uSXm4WdJhLFPT8ka94=.19a42982-5b34-41fa-aa07-8374e1651c42@github.com>
Message-ID: <3A7sdZt3f448XtXihNmFpF-gu8eJcUPcyOCuaT96ieQ=.5b2f20b2-c3c8-4f5d-bbcc-5f4785e61b2d@github.com>

On Wed, 29 May 2024 19:54:05 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> I'll try to trim it down. I shot myself in the foot after being confused by the original comment and deleting this check. Not having this check is not so serious for non-generational modes, but it is necessary to avoid heap corruption and crashes in the generational mode.
>
>> I'll try to trim it down. I shot myself in the foot after being confused by the original comment and deleting this check. Not having this check is not so serious for non-generational modes, but it is necessary to avoid heap corruption and crashes in the generational mode.
> 
> Could you, for the sake of completeness, document this specific _generational pathology_ in the ticket, because it might bring up some interesting thinking about the ways in which the two modes might interact wrt the existing structure of the state machine driving the progress of the collections in the two cases, and the presence of asynchronous interruptions of that normal state machine flow/transitions.

> @ysramakrishna , during init mark, the generational mode copies the cards that have been used by the barrier up until init mark (the 'write' card table) onto a read only copy used by the concurrent mark (the 'read' table) and resets the write table. Mutators continue using the fresh write table. However, the concurrent mark workers are also responsible for _rebuilding_ the write table. If the concurrent mark is cancelled, the write card table may not be complete. If the degenerated cycle resets the card table (as it would without this check), portions of the card table may be lost.

Makes sense; thanks for the explanation.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19434#issuecomment-2140531489

From ysr at openjdk.org  Thu May 30 18:28:07 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 30 May 2024 18:28:07 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle [v2]
In-Reply-To: <K3d-uWcKHfLhhsKYS2yKEk9-J-pLcX1esZ_1MhjxSyk=.626811b9-2356-454e-88c7-d5756e0bcffe@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
 <6f9RyJck-D4ntL6KZeMmZgJ76-Bxsfmr4jpn75GkDuI=.e0f5e141-1322-4e69-9bbc-5919d411399f@github.com>
 <LKYkrNAlord7sxvYyqbIWXSvcJ4kC_TPkj8K67IuChE=.23d10375-abf5-498b-a720-63b4a5da72c9@github.com>
 <K3d-uWcKHfLhhsKYS2yKEk9-J-pLcX1esZ_1MhjxSyk=.626811b9-2356-454e-88c7-d5756e0bcffe@github.com>
Message-ID: <w5Q474PHwyWzEIERiCsg5NWCGThqPfjRvwJsW7aCXYI=.c81c3dd1-2026-4dad-9573-345b2efd2362@github.com>

On Thu, 30 May 2024 00:06:23 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> It also occurs to me that the pattern here is, roughly of the following form:
>> 
>> 
>>   do_some_uninterruptible_operation();
>>   check cancellation status & return if cancelled, else continue to next uninterruptible operation;
>>   ...
>> 
>> 
>> In that case, the checks should not be on the state of the progress of the collection, but on whether the cycle has been cancelled, and the asserts should then check the state of the collection; in other words, the dual of the structure of control in this method.
>> 
>> That said, I do not understand if there's a deeper rationale here in that sometimes we will want to skip a next uninterruptible phase of the collection regardless, because the state machine of the collection explicitly transitions us, under appropriate conditions, into a different phase than the otherwise normal flow, independent of whether a cancellation happened.
>> 
>> For example, what would be the difference between your current test:
>> 
>> 
>>   if (concurrent_mark_in_progress()) {
>> 
>> 
>> vs, say:
>> 
>> 
>>   if (!evacuation_in_progress()) {
>> 
>> 
>> vs : 
>> 
>> 
>>    if (check_cancellation()) {
>>      ... return false;
>>    }
>>    assert(!concurrent_mark_in_progress(), "Should have completed");
>>    assert(evacuation_in_progress(), "Next logical phase of collection");
>> 
>> 
>> Just wondering if a more uniform idiom might make for easier code patterns and easier, more uniform, reasoning about correctness.
>
> This case doesn't quite follow the `check_cancellation` pattern of the others because it is a little bit different. Here, the uninterruptible operation is the final mark safepoint. This safepoint will itself check if the mark has been cancelled. If final mark detects the cancellation, it will do _nothing_, the concurrent mark will still be in progress and we _must_ resume the degenerated cycle from the marking phase. However, if the final mark safepoint does _not_ detect a cancellation, it will initialize the evacuation phase. If the code simply `checks_cancellation` after final mark it cannot tell if the cancellation request was received just _before_ final mark or just _after_ final mark. These two conditions require different resumption points for the degenerated cycle, so we must check the state of the collection cycle here.

Thanks for the explanation. I do wonder if there's a one-to-one bijection between the interrupt/cancellation detection point and the resumption point for the degeerated cycle. If that is true, then the resumption point may be identified with the state of the concurrent GC state machine (the degeneration work is in a coupled state machine in the sense of David Harel) and might avoid this more subtle and perhaps a little bit more fragile programming idiom. But we can think about this in the fullness of time.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19434#discussion_r1621250979

From wkemper at openjdk.org  Thu May 30 18:43:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 30 May 2024 18:43:11 GMT
Subject: RFR: 8333105: Shenandoah: Results of concurrent mark may be lost
 for degenerated cycle [v2]
In-Reply-To: <w5Q474PHwyWzEIERiCsg5NWCGThqPfjRvwJsW7aCXYI=.c81c3dd1-2026-4dad-9573-345b2efd2362@github.com>
References: <CuH7sghlPPh_rMLmO9vxWeQJYrLM7N3d9yeVAkqXd0Q=.b107c0ef-4028-4765-9fbd-ff208a201626@github.com>
 <6f9RyJck-D4ntL6KZeMmZgJ76-Bxsfmr4jpn75GkDuI=.e0f5e141-1322-4e69-9bbc-5919d411399f@github.com>
 <LKYkrNAlord7sxvYyqbIWXSvcJ4kC_TPkj8K67IuChE=.23d10375-abf5-498b-a720-63b4a5da72c9@github.com>
 <K3d-uWcKHfLhhsKYS2yKEk9-J-pLcX1esZ_1MhjxSyk=.626811b9-2356-454e-88c7-d5756e0bcffe@github.com>
 <w5Q474PHwyWzEIERiCsg5NWCGThqPfjRvwJsW7aCXYI=.c81c3dd1-2026-4dad-9573-345b2efd2362@github.com>
Message-ID: <ojzt3IIQ9_L8JsydY2FmYkcSgH_eyKOsrFWwEa-RrjU=.81a58cc7-502e-4e56-983e-5fcc039ad1de@github.com>

On Thu, 30 May 2024 18:25:27 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> This case doesn't quite follow the `check_cancellation` pattern of the others because it is a little bit different. Here, the uninterruptible operation is the final mark safepoint. This safepoint will itself check if the mark has been cancelled. If final mark detects the cancellation, it will do _nothing_, the concurrent mark will still be in progress and we _must_ resume the degenerated cycle from the marking phase. However, if the final mark safepoint does _not_ detect a cancellation, it will initialize the evacuation phase. If the code simply `checks_cancellation` after final mark it cannot tell if the cancellation request was received just _before_ final mark or just _after_ final mark. These two conditions require different resumption points for the degenerated cycle, so we must check the state of the collection cycle here.
>
> Thanks for the explanation. I do wonder if there's a one-to-one bijection between the interrupt/cancellation detection point and the resumption point for the degeerated cycle. If that is true, then the resumption point may be identified with the state of the concurrent GC state machine (the degeneration work is in a coupled state machine in the sense of David Harel) and might avoid this more subtle and perhaps a little bit more fragile programming idiom. But we can think about this in the fullness of time.

Yes, it would be great to clean up the state hand-off from concurrent to degenerated cycles. The generational mode also introduced a new `_degenerated_roots` state to avoid double-resetting of the write card table.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19434#discussion_r1621275897

From ysr at openjdk.org  Thu May 30 23:31:42 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 30 May 2024 23:31:42 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v11]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <jBfk3pDGsBkp9fmeFAmFZUiIRPsfdml6B8pBks42dBA=.9005b7de-86c9-4ce0-88d1-08c48239c67c@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:

  Some asserts to catch a tricky race. These assertions may be too strong
  in general but would help with debugging a rare crash.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/407/files
  - new: https://git.openjdk.org/shenandoah/pull/407/files/f1f40981..fead6fd2

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=10
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=09-10

  Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407

From eosterlund at openjdk.org  Fri May 31 12:52:08 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Fri, 31 May 2024 12:52:08 GMT
Subject: RFR: 8333005: Deadlock when setting or updating the inline cache
In-Reply-To: <Vtpy1muuHvy3SVyQbDWkpHJBHoASZfeldv9CVIFQWHk=.81a2a593-8382-45c8-ab8d-0c2a21f66749@github.com>
References: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
 <Vtpy1muuHvy3SVyQbDWkpHJBHoASZfeldv9CVIFQWHk=.81a2a593-8382-45c8-ab8d-0c2a21f66749@github.com>
Message-ID: <4VEqYGV3Bz7iFODEEX_dFmagVrTPDm_vwgaAiZs6-BM=.b7b98682-638f-4a54-91b3-ac1968aa7b68@github.com>

On Thu, 30 May 2024 08:01:42 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

>> In our concurrently class unloading collectors (ZGC, Generational ZGC, Shenandoah), there is a per-nmethod lock. This lock is used to protect the nmethod oops, and is used by nmethod entry barriers and for lazily computing the value of is_unloading, for example. It is also used to protect some other random stuff, and it is also used to protect the state machine of inline caches, which is otherwise completely orthogonal to any of the GC stuff.
>> 
>> Because the lock is used to protect the inline caches (is taken by the CompiledICLocker), you are not allowed to call is_unloading() on *other* nmethods while holding it. Because when we need access to the oops to compute is_unloading, we need to take the nmethod lock. So if two nmethods have inline caches pointing at each other, and calls are resolved at the same time, while concurrent class unloading is going on, we can sometimes get a deadlock.
>> 
>> I accidentally introduced such a bug when I removed the ICStubs (https://bugs.openjdk.org/browse/JDK-8322630), where this has indeed been observed.
>> 
>> The intention with this patch is to make the system less fragile. While it's possible to move around the call to is_unloading() to get rid of the deadlocks, I think I will sleep better at night knowing that you can call is_unloading() anywhere, at least in the shared runtime code, without knowing the GC implementation details. So I'm adding a per-nmethod inline cache lock that protects the completely orthogonal inline cache state for the CompiledICLocker. This way these deadlocks can't happen.
>> 
>> Tested ZGC tests tier1-7, and it looks green. The reproducer that caught the problem, also has stopped reproducing.
>
> Looks good.

Thanks for the reviews @kstefanj and @shipilev!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19446#issuecomment-2142049130

From eosterlund at openjdk.org  Fri May 31 12:52:09 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Fri, 31 May 2024 12:52:09 GMT
Subject: Integrated: 8333005: Deadlock when setting or updating the inline
 cache
In-Reply-To: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
References: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
Message-ID: <xItqi2rW7oe_glN7pvi-HPAa02Pph8X1aSuGAmkb8no=.096085c7-76e6-4194-b904-f5bf770e929b@github.com>

On Wed, 29 May 2024 07:32:55 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:

> In our concurrently class unloading collectors (ZGC, Generational ZGC, Shenandoah), there is a per-nmethod lock. This lock is used to protect the nmethod oops, and is used by nmethod entry barriers and for lazily computing the value of is_unloading, for example. It is also used to protect some other random stuff, and it is also used to protect the state machine of inline caches, which is otherwise completely orthogonal to any of the GC stuff.
> 
> Because the lock is used to protect the inline caches (is taken by the CompiledICLocker), you are not allowed to call is_unloading() on *other* nmethods while holding it. Because when we need access to the oops to compute is_unloading, we need to take the nmethod lock. So if two nmethods have inline caches pointing at each other, and calls are resolved at the same time, while concurrent class unloading is going on, we can sometimes get a deadlock.
> 
> I accidentally introduced such a bug when I removed the ICStubs (https://bugs.openjdk.org/browse/JDK-8322630), where this has indeed been observed.
> 
> The intention with this patch is to make the system less fragile. While it's possible to move around the call to is_unloading() to get rid of the deadlocks, I think I will sleep better at night knowing that you can call is_unloading() anywhere, at least in the shared runtime code, without knowing the GC implementation details. So I'm adding a per-nmethod inline cache lock that protects the completely orthogonal inline cache state for the CompiledICLocker. This way these deadlocks can't happen.
> 
> Tested ZGC tests tier1-7, and it looks green. The reproducer that caught the problem, also has stopped reproducing.

This pull request has now been integrated.

Changeset: d4812151
Author:    Erik ?sterlund <eosterlund at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/d48121512658003f21826e037590541464141ed6
Stats:     85 lines in 15 files changed: 44 ins; 0 del; 41 mod

8333005: Deadlock when setting or updating the inline cache

Reviewed-by: sjohanss, shade

-------------

PR: https://git.openjdk.org/jdk/pull/19446

From wkemper at openjdk.org  Fri May 31 14:15:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 31 May 2024 14:15:41 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <cZF_aBjMPKsInzDiq8G9ga7m7onD0OXZL0ZCy_I4P0k=.7d96d8d9-6765-41c9-88bd-20fbefb78b88@github.com>

Merges tag jdk-23+25

-------------

Commit messages:
 - 8333189: Make sure clang on linux uses lld as linker
 - 8332919: SA PointerLocation needs to print a newline after dumping java thread info for JNI Local Ref
 - 8333013: Update vmTestbase/nsk/share/LocalProcess.java to don't use finalization
 - 8332259: JvmtiTrace::safe_get_thread_name fails if current thread is in native state
 - 8333149: ubsan : memset on nullptr target detected in jvmtiEnvBase.cpp get_object_monitor_usage
 - 8320999: RISC-V: C2 RotateLeftV
 - 8321543: Update NSS to version 3.96
 - 8331865: Consolidate size and alignment checks in LayoutPath
 - 8324341: Remove redundant preprocessor #if's checks
 - 8332904: ubsan ppc64le: c1_LIRGenerator_ppc.cpp:581:21: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long int'
 - ... and 179 more: https://git.openjdk.org/shenandoah/compare/2f10a316...789ac8b2

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=442&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/442/files
  Stats: 66229 lines in 1195 files changed: 50142 ins; 10206 del; 5881 mod
  Patch: https://git.openjdk.org/shenandoah/pull/442.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/442/head:pull/442

PR: https://git.openjdk.org/shenandoah/pull/442

From ysr at openjdk.org  Fri May 31 23:37:36 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 31 May 2024 23:37:36 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
 missing use [v12]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <PTYawYzB2GzAcBoJ_zy4Fko9zsr4ARocDXKhuyl23ME=.6bc88756-e58d-4edc-9a27-fbaed8a84724@github.com>

> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
> 
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
> 
> Asserts check the protocol for setting and clearing the variables.
> 
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
> 
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 42 commits:

 - Merge branch 'master' into active_generation
 - Some asserts to catch a tricky race. These assertions may be too strong
   in general but would help with debugging a rare crash.
 - Merge branch 'master' into active_generation
 - Remove vestigial thread() method in ShenandoahController.
 - cosmetic
 - jcheck white-space
 - Remove static _thread field introduced in ShenandoahController, as its
   role is already served by the _control_thread field in ShenandoahHeap.
 - Macro-ize previously inlined shenandoah assert.
 - shenandoah_assert_generations_reconciled() macro instead of a method in
   ShenandoahHeap, bringing it in line with shenandoah assertion check
   idiom.
 - Small clean-ups.
 - ... and 32 more: https://git.openjdk.org/shenandoah/compare/840c1825...62934983

-------------

Changes: https://git.openjdk.org/shenandoah/pull/407/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=11
  Stats: 170 lines in 17 files changed: 135 ins; 1 del; 34 mod
  Patch: https://git.openjdk.org/shenandoah/pull/407.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407

PR: https://git.openjdk.org/shenandoah/pull/407