From fyang at openjdk.org  Mon Apr  1 00:22:31 2024
From: fyang at openjdk.org (Fei Yang)
Date: Mon, 1 Apr 2024 00:22:31 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
In-Reply-To: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
Message-ID: <fV2tjC2mHssR9wsfLolfUUXYEtQRkFohiUmdxsxpyz8=.2828d452-1c47-4dc9-998f-f1334372a2d4@github.com>

On Fri, 29 Mar 2024 19:35:45 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
> 
> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
> 
> I do not see (and not expected) performance difference with these changes.
> 
> Tested tier1-5, xcomp, stress. Running performance testing.
> 
> I need help with testing on platforms which Oracle does not support.

Hi, I also performed some tests (tier1-3 and hotspot:tier4) on linux-riscv64 platform. Result looks good.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2028963874

From shade at openjdk.org  Mon Apr  1 07:36:00 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 1 Apr 2024 07:36:00 GMT
Subject: RFR: 8329134: Reconsider TLAB zapping [v3]
In-Reply-To: <cWkDJ301cKsXnEsx2wDgNV1fzqiYqaYBIXNzrKpvL0o=.5171a404-f1ef-43b7-b879-ff9c514379ad@github.com>
References: <cWkDJ301cKsXnEsx2wDgNV1fzqiYqaYBIXNzrKpvL0o=.5171a404-f1ef-43b7-b879-ff9c514379ad@github.com>
Message-ID: <iR14E5L71QleM0e8qge2Xwk9VL7phY4LHR3mhwPHEhc=.bf5f90eb-485a-4489-89f7-7dbb535fd65b@github.com>

> We zap the entire TLAB on initial allocation (`MemAllocator::mem_allocate_inside_tlab_slow`), and then also rezap the object contents when object is allocated from the TLAB (`ThreadLocalAllocBuffer::allocate`). The second part seems excessive, given the TLAB is already fully zapped.
> 
> There is also no way to disable this zapping, like you would in other places with the relevant Zap* flags.
> 
> Fixing both these issues allows to improve fastdebug tests performance, e.g. in jcstress. 
> 
> It also allows to remove the related Zero kludge.
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `all` tests 
>  - [x] MacOS AArch64 Zero fastdebug, `bootcycle-images` pass

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:

 - Merge branch 'master' into JDK-8329134-tlab-zapping
 - Review comments
 - Touchups
 - Also remove Zero kludge
 - Fix

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18500/files
  - new: https://git.openjdk.org/jdk/pull/18500/files/72bf1e8a..5bf36d05

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18500&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18500&range=01-02

  Stats: 6085 lines in 208 files changed: 2917 ins; 2291 del; 877 mod
  Patch: https://git.openjdk.org/jdk/pull/18500.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18500/head:pull/18500

PR: https://git.openjdk.org/jdk/pull/18500

From amitkumar at openjdk.org  Mon Apr  1 12:07:31 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Mon, 1 Apr 2024 12:07:31 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
In-Reply-To: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
Message-ID: <sYoiBxrL8P1iII2LnIoXhajUVbjdOrAhxIyOdSeRbAg=.f6004f6a-34df-4de7-9ce9-9f8b65344045@github.com>

On Fri, 29 Mar 2024 19:35:45 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
> 
> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
> 
> I do not see (and not expected) performance difference with these changes.
> 
> Tested tier1-5, xcomp, stress. Running performance testing.
> 
> I need help with testing on platforms which Oracle does not support.

I performed the build + testing `{fastdebug, release, slowdebug} X {tier1}`  on `s390x` and result looks fine.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2029655163

From wkemper at openjdk.org  Mon Apr  1 15:29:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 1 Apr 2024 15:29:54 GMT
Subject: RFR: 8329342: GenShen: Synchronize shenandoah-jdk21u:master with
 shenandoah:master
In-Reply-To: <Sge-9K0CA-yVExf4NZXAFtLHVl4wWyuNOowyj3AU0C4=.db32b336-b20d-401a-accb-50d92a4a74a3@github.com>
References: <Sge-9K0CA-yVExf4NZXAFtLHVl4wWyuNOowyj3AU0C4=.db32b336-b20d-401a-accb-50d92a4a74a3@github.com>
Message-ID: <4kyi2NfOoOxw_-oRfEDVjuGRe3HcTQqyc9EKBdwVmDI=.62cae0e4-2011-43d1-8005-6cc0811ce66a@github.com>

On Fri, 29 Mar 2024 22:08:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The delta for this fork has grown too great for individual backport PRs. This change synchronizes the `shenandoah-jdk21u` fork by essentially checking out the `shenandoah` directory from `shenandoah` and reverting the few things we can't take because of changes outside of `shenandoah`.

Zero build and TestHumongous are fixed in upstream PR that will be backported following this PR.

-------------

PR Comment: https://git.openjdk.org/shenandoah-jdk21u/pull/31#issuecomment-2029972832

From wkemper at openjdk.org  Mon Apr  1 15:29:56 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 1 Apr 2024 15:29:56 GMT
Subject: Integrated: 8329342: GenShen: Synchronize shenandoah-jdk21u:master
 with shenandoah:master
In-Reply-To: <Sge-9K0CA-yVExf4NZXAFtLHVl4wWyuNOowyj3AU0C4=.db32b336-b20d-401a-accb-50d92a4a74a3@github.com>
References: <Sge-9K0CA-yVExf4NZXAFtLHVl4wWyuNOowyj3AU0C4=.db32b336-b20d-401a-accb-50d92a4a74a3@github.com>
Message-ID: <-Kyqs_kkBeb7VkY34wb4HmBPHO6wGoeOSGjCP249-4k=.95dfacde-7aac-4833-8b4b-1da9db53cab3@github.com>

On Fri, 29 Mar 2024 22:08:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The delta for this fork has grown too great for individual backport PRs. This change synchronizes the `shenandoah-jdk21u` fork by essentially checking out the `shenandoah` directory from `shenandoah` and reverting the few things we can't take because of changes outside of `shenandoah`.

This pull request has now been integrated.

Changeset: c55b0765
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/c55b076548c40ca4fea6a182818ea9910fb09ba6
Stats:     5656 lines in 76 files changed: 2945 ins; 2220 del; 491 mod

8329342: GenShen: Synchronize shenandoah-jdk21u:master with shenandoah:master

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/31

From ysr at openjdk.org  Mon Apr  1 15:50:43 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 1 Apr 2024 15:50:43 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v5]
In-Reply-To: <BFuizLduIbx5NeR5F_0MVUqDCbZJsCbGGrzVESBsS8w=.5848ac32-fcb0-4a0e-94ce-b76f34fa2629@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <BFuizLduIbx5NeR5F_0MVUqDCbZJsCbGGrzVESBsS8w=.5848ac32-fcb0-4a0e-94ce-b76f34fa2629@github.com>
Message-ID: <_Vk9JRksaktAcBJNgK1sLPR0Q0-7_cuSwbI9UBDFQNA=.ee4a4f87-bf86-4f5a-8273-8d95d552c5e0@github.com>

On Thu, 28 Mar 2024 15:37:28 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
>> 
>> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
>> 
>> 
>>    Control: shenandoah-x86-template
>> Experiment: enable-old-growth-triggers-gh-x86
>> 
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>>                                Genshen/specjbb2015 |                               context_switch_count
>>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>>                             Shenandoah/specjbb2015 |                                      trigger_learn
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
>> 
>> Genshen
>> -------------------------------------------------------------------------------------------------------
>> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>>   Control:      2.500   (+/-  0.68  )         30
>>   Test:        19.625   (+/-  4.79  )         10
>> 
>> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>>   Control:      2.625   (+/-  0.92  )         30
>>   Test:        17.375   (+/-  3.89  )         10
>> 
>> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>>   Control:      9.833   (+/-  3.48  )         30
>>   Test:        32.000   (+/-  2.58  )       ...
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix another typo

Wondering if the new trigger methods belong in ShenandoahOldHeuristics to keep all triggering code in one place and to integrate with existing triggering code?

Will read through the triggering criteria details shortly.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3202:

> 3200: 
> 3201:   if (mode()->is_generational()) {
> 3202:     ShenandoahGenerationalHeap* gen_heap = (ShenandoahGenerationalHeap*) this;

May be `ShenandoahGenerationalHeap::heap()` like was done further above? (subjective nit can be ignored; see another suggestion further below that affects this as well.)

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3202:

> 3200: 
> 3201:   if (mode()->is_generational()) {
> 3202:     ShenandoahGenerationalHeap* gen_heap = (ShenandoahGenerationalHeap*) this;

One other question: would something like this not be under the purview of the appropriate heuristics object associated with the heap? May be ShenandoahOldHeuristics? Just wondering about the correct abstraction boundaries here.

How does the regular (non-generational collector) do this following the rebuild?

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 510:

> 508: }
> 509: 
> 510: void ShenandoahOldGeneration::trigger_collection_if_fragmented(ShenandoahGenerationalHeap* gen_heap, size_t first_old_region,

As pointed out further above, would this more naturally belong here or in ShenandoahOldHeuristics? There are a bunch of triggers & checks there as well. Would appear to make sense to keep all of this in one place rather than scattering them between generation and heuristic for clear abstraction boundaries and maintainability of code.

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.hpp line 228:

> 226:   void set_live_bytes_after_last_mark(size_t new_live);
> 227: 
> 228:   void trigger_collection_if_fragmented(ShenandoahGenerationalHeap* gen_heap, size_t first_old_region, size_t last_old_region,

Move to ShenandoahOldHeuristics?

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/409#pullrequestreview-1971490771
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1546448172
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1546460372
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1546466306
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1546467747

From ysr at openjdk.org  Mon Apr  1 15:50:43 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 1 Apr 2024 15:50:43 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v5]
In-Reply-To: <_Vk9JRksaktAcBJNgK1sLPR0Q0-7_cuSwbI9UBDFQNA=.ee4a4f87-bf86-4f5a-8273-8d95d552c5e0@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <BFuizLduIbx5NeR5F_0MVUqDCbZJsCbGGrzVESBsS8w=.5848ac32-fcb0-4a0e-94ce-b76f34fa2629@github.com>
 <_Vk9JRksaktAcBJNgK1sLPR0Q0-7_cuSwbI9UBDFQNA=.ee4a4f87-bf86-4f5a-8273-8d95d552c5e0@github.com>
Message-ID: <bIDuBxwNPnOKhQqaed3myym2agkFZnB5Mu--StVRg8w=.6039a8cd-2135-4119-97de-3bfd84a34534@github.com>

On Mon, 1 Apr 2024 15:26:49 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix another typo
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3202:
> 
>> 3200: 
>> 3201:   if (mode()->is_generational()) {
>> 3202:     ShenandoahGenerationalHeap* gen_heap = (ShenandoahGenerationalHeap*) this;
> 
> May be `ShenandoahGenerationalHeap::heap()` like was done further above? (subjective nit can be ignored; see another suggestion further below that affects this as well.)

The object on which the method is called could keep a reference to the generational heap to avoid the need to pass it as an argument in the call. (In particular, do the heuristics objets -- if these methods were to move there -- keep a reference to the heap and to the appropriate generations?)

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1546476345

From ysr at openjdk.org  Mon Apr  1 16:09:39 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 1 Apr 2024 16:09:39 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v32]
In-Reply-To: <-g5tscBZPsDgVrjfZnM-UBRy0_6qodnRN5_srPWunXk=.82eda809-141c-4bfb-82aa-e722eecd0469@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Mu9wTnVX6kIS89pKXTUOHQT2CB7P_m_kSWGQHOQU-ko=.77851b17-c53b-4b59-8a84-6d518a19d35b@github.com>
 <HgHiTh_t5gXIEQnpJ8EbuF3mVuuyGOOh0DFsikSmo9E=.906d3f45-9210-4c24-9817-720935389846@github.com>
 <-g5tscBZPsDgVrjfZnM-UBRy0_6qodnRN5_srPWunXk=.82eda809-141c-4bfb-82aa-e722eecd0469@github.com>
Message-ID: <U6tUCW9TF58aCdQCJU1ZQ993BWlojiwj145QCaQALkY=.ed211af5-2493-4f9d-89ed-256a609d14d3@github.com>

On Wed, 20 Mar 2024 19:52:55 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 41:
>> 
>>> 39: 
>>> 40: 
>>> 41: // ShenandoahSimpleBitMap resembles CHeapBitMap but adds missing support for find_next_consecutive_bits() and
>> 
>> Ah I see you have considered using bitMap.* ... but why not 'simply' add the required methods there, instead?
>
> Was trying to avoid impacting beyond the boundaries of Shenandoah.  I can tackle this if we think it preferable.  But also trying to work toward "timely integration".

Like Roman, I'd prefer extending existing code and/or making it adaptable for our purposes to duplicating functionality. Let us read through the differences and see if this should be done now, or as you state allow this duplication for now in the interests of speed and expediency.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1546513139

From wkemper at openjdk.org  Mon Apr  1 17:18:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 1 Apr 2024 17:18:11 GMT
Subject: RFR: 8328626: GenShen: Combine old generation surplus/deficit fields
 into a single balance field
Message-ID: <AgcnGKgPLVLiiMfQ6uic7qypQjVTxkF0NHjZFFF84vk=.c9a37938-f7ce-42d8-af42-8a315a13cec1@github.com>

Backport of 8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field

-------------

Commit messages:
 - 8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/32/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=32&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8328626
  Stats: 276 lines in 10 files changed: 101 ins; 117 del; 58 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/32.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/32/head:pull/32

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/32

From shade at openjdk.org  Mon Apr  1 17:29:45 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 1 Apr 2024 17:29:45 GMT
Subject: RFR: 8329134: Reconsider TLAB zapping [v3]
In-Reply-To: <iR14E5L71QleM0e8qge2Xwk9VL7phY4LHR3mhwPHEhc=.bf5f90eb-485a-4489-89f7-7dbb535fd65b@github.com>
References: <cWkDJ301cKsXnEsx2wDgNV1fzqiYqaYBIXNzrKpvL0o=.5171a404-f1ef-43b7-b879-ff9c514379ad@github.com>
 <iR14E5L71QleM0e8qge2Xwk9VL7phY4LHR3mhwPHEhc=.bf5f90eb-485a-4489-89f7-7dbb535fd65b@github.com>
Message-ID: <hm8hr8vHmfN9bhb0X1AAcf-r1gScfSSg-uJQea11pJ0=.e8d99e46-c6b4-41c4-a502-bab6c0eec37e@github.com>

On Mon, 1 Apr 2024 07:36:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We zap the entire TLAB on initial allocation (`MemAllocator::mem_allocate_inside_tlab_slow`), and then also rezap the object contents when object is allocated from the TLAB (`ThreadLocalAllocBuffer::allocate`). The second part seems excessive, given the TLAB is already fully zapped.
>> 
>> There is also no way to disable this zapping, like you would in other places with the relevant Zap* flags.
>> 
>> Fixing both these issues allows to improve fastdebug tests performance, e.g. in jcstress. 
>> 
>> It also allows to remove the related Zero kludge.
>> 
>> Additional testing:
>>  - [x] Linux AArch64 server fastdebug, `all` tests 
>>  - [x] MacOS AArch64 Zero fastdebug, `bootcycle-images` pass
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8329134-tlab-zapping
>  - Review comments
>  - Touchups
>  - Also remove Zero kludge
>  - Fix

Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18500#issuecomment-2030202957

From shade at openjdk.org  Mon Apr  1 17:29:45 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 1 Apr 2024 17:29:45 GMT
Subject: Integrated: 8329134: Reconsider TLAB zapping
In-Reply-To: <cWkDJ301cKsXnEsx2wDgNV1fzqiYqaYBIXNzrKpvL0o=.5171a404-f1ef-43b7-b879-ff9c514379ad@github.com>
References: <cWkDJ301cKsXnEsx2wDgNV1fzqiYqaYBIXNzrKpvL0o=.5171a404-f1ef-43b7-b879-ff9c514379ad@github.com>
Message-ID: <Ejyru5d8Q7_PmcASzTkXVnN5rGek-gr1CdfC2JlP570=.7a84228a-4f40-4d95-98f1-ed39a9bca48c@github.com>

On Tue, 26 Mar 2024 21:08:16 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> We zap the entire TLAB on initial allocation (`MemAllocator::mem_allocate_inside_tlab_slow`), and then also rezap the object contents when object is allocated from the TLAB (`ThreadLocalAllocBuffer::allocate`). The second part seems excessive, given the TLAB is already fully zapped.
> 
> There is also no way to disable this zapping, like you would in other places with the relevant Zap* flags.
> 
> Fixing both these issues allows to improve fastdebug tests performance, e.g. in jcstress. 
> 
> It also allows to remove the related Zero kludge.
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `all` tests 
>  - [x] MacOS AArch64 Zero fastdebug, `bootcycle-images` pass

This pull request has now been integrated.

Changeset: 5698f7ad
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/5698f7ad29c939b7e52882ace575dd7113bf41de
Stats:     30 lines in 5 files changed: 5 ins; 16 del; 9 mod

8329134: Reconsider TLAB zapping

Reviewed-by: stefank, rkennke

-------------

PR: https://git.openjdk.org/jdk/pull/18500

From wkemper at openjdk.org  Mon Apr  1 17:55:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 1 Apr 2024 17:55:30 GMT
Subject: RFR: 8329350: GenShen: Do not reset mark bitmaps on a safepoint
 [v2]
In-Reply-To: <9D7mi7sbY8lCY9njdei5rIyosleJYr7EW5NbQdvy4oY=.e660695f-7f43-453d-9fc4-6983633035f4@github.com>
References: <9D7mi7sbY8lCY9njdei5rIyosleJYr7EW5NbQdvy4oY=.e660695f-7f43-453d-9fc4-6983633035f4@github.com>
Message-ID: <giOLjCralWs5miiSyyp7qqYbIuhmk-Gr6OjlqAs475A=.fcfb2747-04f6-43d3-b15a-20ac0356ae15@github.com>

> Shenandoah is currently resetting mark bitmaps during the init mark pause. This work should happen during the concurrent reset phase to avoid prolonging the safepoint. Also, free regions need to have the corresponding bitmap region reset or we risk having marked regions with no live data (which violates asserts during final mark).

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix zero build

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/413/files
  - new: https://git.openjdk.org/shenandoah/pull/413/files/fb186480..1ef350b5

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=413&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=413&range=00-01

  Stats: 111 lines in 4 files changed: 72 ins; 39 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/413.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/413/head:pull/413

PR: https://git.openjdk.org/shenandoah/pull/413

From dlong at openjdk.org  Mon Apr  1 18:18:00 2024
From: dlong at openjdk.org (Dean Long)
Date: Mon, 1 Apr 2024 18:18:00 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
In-Reply-To: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
Message-ID: <h-K6s_srneCwfdQEalwLzuBDBhS-qobQet-XVPbg8BM=.677c7770-7b3c-4925-bfad-638e455f44eb@github.com>

On Fri, 29 Mar 2024 19:35:45 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
> 
> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
> 
> I do not see (and not expected) performance difference with these changes.
> 
> Tested tier1-5, xcomp, stress. Running performance testing.
> 
> I need help with testing on platforms which Oracle does not support.

The `not_used` state was introduced for AOT.  It can go away now.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2030282409

From kvn at openjdk.org  Mon Apr  1 19:38:59 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Mon, 1 Apr 2024 19:38:59 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
In-Reply-To: <fV2tjC2mHssR9wsfLolfUUXYEtQRkFohiUmdxsxpyz8=.2828d452-1c47-4dc9-998f-f1334372a2d4@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <fV2tjC2mHssR9wsfLolfUUXYEtQRkFohiUmdxsxpyz8=.2828d452-1c47-4dc9-998f-f1334372a2d4@github.com>
Message-ID: <AJ3suZNm4uB8ia5CX8HFNHVUjxNd6gVhlf-3NC4PdKQ=.c9d8611b-767a-4988-b42c-7a5954fa29b6@github.com>

On Mon, 1 Apr 2024 00:19:32 GMT, Fei Yang <fyang at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Hi, I also performed some tests (tier1-3 and hotspot:tier4) on linux-riscv64 platform. Result looks good.

@RealFYang and @offamitkumar  thank you for testing.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2030425253

From kvn at openjdk.org  Mon Apr  1 19:52:59 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Mon, 1 Apr 2024 19:52:59 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
In-Reply-To: <h-K6s_srneCwfdQEalwLzuBDBhS-qobQet-XVPbg8BM=.677c7770-7b3c-4925-bfad-638e455f44eb@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <h-K6s_srneCwfdQEalwLzuBDBhS-qobQet-XVPbg8BM=.677c7770-7b3c-4925-bfad-638e455f44eb@github.com>
Message-ID: <sLjbfVsKam-tp2xp5IJMlZPP58YZ1kZsMDuE0-LDwzQ=.48b59aa2-89a0-47e4-9d2d-fea699b4babf@github.com>

On Mon, 1 Apr 2024 18:15:43 GMT, Dean Long <dlong at openjdk.org> wrote:

> The `not_used` state was introduced for AOT. It can go away now.

Good catch, Dean.

I want to keep `nmethod::make_not_used()` method because we use it in Leyden to keep AOT code (outside of CodeCache): [nmethod.hpp#L476](https://github.com/openjdk/leyden/blob/premain/src/hotspot/share/code/nmethod.hpp#L476)
It does not use this flag value.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2030448462

From kvn at openjdk.org  Mon Apr  1 21:07:31 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Mon, 1 Apr 2024 21:07:31 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
Message-ID: <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>

> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
> 
> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
> 
> I do not see (and not expected) performance difference with these changes.
> 
> Tested tier1-5, xcomp, stress. Running performance testing.
> 
> I need help with testing on platforms which Oracle does not support.

Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:

  Removed not_used state of nmethod

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18554/files
  - new: https://git.openjdk.org/jdk/pull/18554/files/7635b333..246ff68a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18554&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18554&range=00-01

  Stats: 5 lines in 2 files changed: 0 ins; 3 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/18554.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18554/head:pull/18554

PR: https://git.openjdk.org/jdk/pull/18554

From kvn at openjdk.org  Mon Apr  1 21:12:00 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Mon, 1 Apr 2024 21:12:00 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
Message-ID: <fgr3lz6qjcCs0zfbG-4-llRlAr3zz9RfaizEj58-QBo=.c26d1ba0-c833-4288-b641-d1b372a081bb@github.com>

On Mon, 1 Apr 2024 21:07:31 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removed not_used state of nmethod

I did not change `src/hotspot/share/code//codeHeapState.cpp` code which counts nmethods with `not_used` state by checking `(!nm->is_not_entrant()` after `(nm->is_in_use())`.  Removing `not_used` does not affect it.
The code is complicated and needs separate RFE if we decide to clean it.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2030560338

From ysr at openjdk.org  Mon Apr  1 23:51:25 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 1 Apr 2024 23:51:25 GMT
Subject: RFR: 8329350: GenShen: Do not reset mark bitmaps on a safepoint
 [v2]
In-Reply-To: <giOLjCralWs5miiSyyp7qqYbIuhmk-Gr6OjlqAs475A=.fcfb2747-04f6-43d3-b15a-20ac0356ae15@github.com>
References: <9D7mi7sbY8lCY9njdei5rIyosleJYr7EW5NbQdvy4oY=.e660695f-7f43-453d-9fc4-6983633035f4@github.com>
 <giOLjCralWs5miiSyyp7qqYbIuhmk-Gr6OjlqAs475A=.fcfb2747-04f6-43d3-b15a-20ac0356ae15@github.com>
Message-ID: <pKzSEWGjuHte81LS5thRBw9Kx0RDGmYDhy9xHLIid0s=.7c866c8d-a581-4859-ba2a-375856646331@github.com>

On Mon, 1 Apr 2024 17:55:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Shenandoah is currently resetting mark bitmaps during the init mark pause. This work should happen during the concurrent reset phase to avoid prolonging the safepoint. Also, free regions need to have the corresponding bitmap region reset or we risk having marked regions with no live data (which violates asserts during final mark).
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix zero build

LGTM. Some very minor comments. No need for re-review, irrespective of any subsequent changes.

Would be good to see any comparative performance numbers if available (init mark or general improvement).

src/hotspot/share/gc/shenandoah/shenandoahHeapRegionClosures.hpp line 31:

> 29: #include "gc/shenandoah/shenandoahHeap.hpp"
> 30: #include "gc/shenandoah/shenandoahHeapRegion.inline.hpp"
> 31: 

Please add a 1-line documentation for each of these classes.

```A closure that is applied to all regions with this affiliation```

and 

```A closure that is applied to all regions outside this affiliation```

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 249:

> 247: 
> 248: void ShenandoahOldGeneration::parallel_heap_region_iterate(ShenandoahHeapRegionClosure* cl) {
> 249:   ShenandoahIncludeRegionClosure<OLD_GENERATION> old_regions(cl);

Subjective nit: Not something you introduced, but I'd name the variable `old_region_cl` rather than `old_regions` just to avoid dimensional mismatch (to borrow a term from physics) in the naming. It makes code easier to read if one avoids such naming type/dimension mismatch.

Ignore this suggestion if this will affect upstream Shenandoah as well (unless the renaming can be pushed separately and avoid an issue with the diffs).

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/413#pullrequestreview-1972263399
PR Review Comment: https://git.openjdk.org/shenandoah/pull/413#discussion_r1546935512
PR Review Comment: https://git.openjdk.org/shenandoah/pull/413#discussion_r1546948525

From coleenp at openjdk.org  Tue Apr  2 13:23:16 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 13:23:16 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags
Message-ID: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>

Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.

Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.

-------------

Commit messages:
 - Missed one.
 - 8236736: Change notproduct JVM flags to develop flags

Changes: https://git.openjdk.org/jdk/pull/18541/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18541&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8236736
  Stats: 239 lines in 39 files changed: 1 ins; 89 del; 149 mod
  Patch: https://git.openjdk.org/jdk/pull/18541.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18541/head:pull/18541

PR: https://git.openjdk.org/jdk/pull/18541

From kdnilsen at openjdk.org  Tue Apr  2 14:32:44 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 2 Apr 2024 14:32:44 GMT
Subject: RFR: 8327097: GenShen: Align PLAB sizes down rather than up
Message-ID: <O73joEb5qI_KS0wPmhS9Zms0dmteT2pUTxoxsQzb_aw=.a299359f-c77d-43f8-bcaf-8c079b046125@github.com>

Hi all,

This pull request contains a backport of commit f1d98490 from the openjdk/shenandoah repository.

The commit being backported was authored by Kelvin Nilsen on 27 Mar 2024 and was reviewed by Y. Srinivas Ramakrishna and William Kemper.

Thanks!

-------------

Commit messages:
 - Backport f1d98490f4e2fe94f2ed6db961934812c53395ae

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/33/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=33&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8327097
  Stats: 155 lines in 12 files changed: 73 ins; 35 del; 47 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/33.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/33/head:pull/33

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/33

From kdnilsen at openjdk.org  Tue Apr  2 14:54:14 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 2 Apr 2024 14:54:14 GMT
Subject: Integrated: 8327097: GenShen: Align PLAB sizes down rather than up
In-Reply-To: <O73joEb5qI_KS0wPmhS9Zms0dmteT2pUTxoxsQzb_aw=.a299359f-c77d-43f8-bcaf-8c079b046125@github.com>
References: <O73joEb5qI_KS0wPmhS9Zms0dmteT2pUTxoxsQzb_aw=.a299359f-c77d-43f8-bcaf-8c079b046125@github.com>
Message-ID: <ftL1NNs3227L_9ASgLeqJzex0vvs2g03NJQ9NuleT6g=.13d226a4-0081-4e84-8fdb-273c10a460cc@github.com>

On Tue, 2 Apr 2024 14:27:36 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Hi all,
> 
> This pull request contains a backport of commit f1d98490 from the openjdk/shenandoah repository.
> 
> The commit being backported was authored by Kelvin Nilsen on 27 Mar 2024 and was reviewed by Y. Srinivas Ramakrishna and William Kemper.
> 
> Thanks!

This pull request has now been integrated.

Changeset: 7ddbe4df
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/7ddbe4df1f7a8ef48589f553bfcdb2c36ed71c72
Stats:     155 lines in 12 files changed: 73 ins; 35 del; 47 mod

8327097: GenShen: Align PLAB sizes down rather than up

Backport-of: f1d98490f4e2fe94f2ed6db961934812c53395ae

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/33

From iklam at openjdk.org  Tue Apr  2 16:01:12 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Tue, 2 Apr 2024 16:01:12 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags
In-Reply-To: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
Message-ID: <C_Ko3VfolSjPu5xrlr_7dXvRd4hYwHD_o0BsiOKOb2s=.7782e567-3f62-4ed3-be15-a975924e37fb@github.com>

On Thu, 28 Mar 2024 22:53:22 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
> 
> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.

LGTM.

For the past 15 years, "notproduct" flags haven't been working as they claim to be in globals.hpp. That doesn't seem to have bothered anyone. This definitely looks like a design that no one needs and should be removed for simplcity.

There are some references to "notproduct" in test/hotspot/jtreg/runtime/CommandLine that need to be removed.

-------------

Changes requested by iklam (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18541#pullrequestreview-1974267126

From coleenp at openjdk.org  Tue Apr  2 16:24:19 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 16:24:19 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v2]
In-Reply-To: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
Message-ID: <QgftUnVOGk6RVVe-HTPuzxmafkCtcKKenFRyseU-71g=.aaad76be-1e0b-4293-ae8e-9e046b614fe7@github.com>

> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
> 
> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.

Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:

  Clean up notproduct from tests.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18541/files
  - new: https://git.openjdk.org/jdk/pull/18541/files/c3d9a1c8..19b8f6b6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18541&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18541&range=00-01

  Stats: 37 lines in 4 files changed: 0 ins; 8 del; 29 mod
  Patch: https://git.openjdk.org/jdk/pull/18541.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18541/head:pull/18541

PR: https://git.openjdk.org/jdk/pull/18541

From iklam at openjdk.org  Tue Apr  2 16:24:19 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Tue, 2 Apr 2024 16:24:19 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v2]
In-Reply-To: <QgftUnVOGk6RVVe-HTPuzxmafkCtcKKenFRyseU-71g=.aaad76be-1e0b-4293-ae8e-9e046b614fe7@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <QgftUnVOGk6RVVe-HTPuzxmafkCtcKKenFRyseU-71g=.aaad76be-1e0b-4293-ae8e-9e046b614fe7@github.com>
Message-ID: <cuFS3fdwlONF-rGc1InsnkE4GoGSwEx7E4jh0t1jzNY=.603eb76b-a10e-4831-bb87-02a8c8669165@github.com>

On Tue, 2 Apr 2024 16:21:15 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Clean up notproduct from tests.

LGTM

-------------

Marked as reviewed by iklam (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18541#pullrequestreview-1974322553

From coleenp at openjdk.org  Tue Apr  2 16:24:20 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 16:24:20 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags
In-Reply-To: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
Message-ID: <nPWRExv8E_LuD15xKu5pNVGHnWYl70fl82YKPOe7EmA=.c28d2bfe-c0ce-4d1b-b9d6-f9a3acfccbf4@github.com>

On Thu, 28 Mar 2024 22:53:22 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
> 
> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.

Thank you for pointing out that I missed cleaning up the tests.  One failed but I didn't see it.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2032498785

From kdnilsen at openjdk.org  Tue Apr  2 16:55:18 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 2 Apr 2024 16:55:18 GMT
Subject: RFR: 8328626: GenShen: Combine old generation surplus/deficit
 fields into a single balance field
In-Reply-To: <AgcnGKgPLVLiiMfQ6uic7qypQjVTxkF0NHjZFFF84vk=.c9a37938-f7ce-42d8-af42-8a315a13cec1@github.com>
References: <AgcnGKgPLVLiiMfQ6uic7qypQjVTxkF0NHjZFFF84vk=.c9a37938-f7ce-42d8-af42-8a315a13cec1@github.com>
Message-ID: <iQtcDnG9DGS29f51OkgwHJSMwJDDQEnbFJX5ZX-EBMI=.52c6841e-dafa-4a26-b386-63dd09134a2a@github.com>

On Mon, 1 Apr 2024 17:13:07 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Backport of 8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/32#pullrequestreview-1974406491

From wkemper at openjdk.org  Tue Apr  2 16:56:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 2 Apr 2024 16:56:42 GMT
Subject: RFR: 8329350: GenShen: Do not reset mark bitmaps on a safepoint
 [v3]
In-Reply-To: <9D7mi7sbY8lCY9njdei5rIyosleJYr7EW5NbQdvy4oY=.e660695f-7f43-453d-9fc4-6983633035f4@github.com>
References: <9D7mi7sbY8lCY9njdei5rIyosleJYr7EW5NbQdvy4oY=.e660695f-7f43-453d-9fc4-6983633035f4@github.com>
Message-ID: <OruuxGUVrC_n_1n4HuINcPOoaM7yx9IoSaQRaeFBH2U=.45ebea7c-bf8e-431e-9947-81378106302d@github.com>

> Shenandoah is currently resetting mark bitmaps during the init mark pause. This work should happen during the concurrent reset phase to avoid prolonging the safepoint. Also, free regions need to have the corresponding bitmap region reset or we risk having marked regions with no live data (which violates asserts during final mark).

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Improve comments and variable names per review feedback

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/413/files
  - new: https://git.openjdk.org/shenandoah/pull/413/files/1ef350b5..1d553587

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=413&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=413&range=01-02

  Stats: 14 lines in 3 files changed: 2 ins; 0 del; 12 mod
  Patch: https://git.openjdk.org/shenandoah/pull/413.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/413/head:pull/413

PR: https://git.openjdk.org/shenandoah/pull/413

From stefank at openjdk.org  Tue Apr  2 16:59:13 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 2 Apr 2024 16:59:13 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v2]
In-Reply-To: <QgftUnVOGk6RVVe-HTPuzxmafkCtcKKenFRyseU-71g=.aaad76be-1e0b-4293-ae8e-9e046b614fe7@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <QgftUnVOGk6RVVe-HTPuzxmafkCtcKKenFRyseU-71g=.aaad76be-1e0b-4293-ae8e-9e046b614fe7@github.com>
Message-ID: <fL1uGxDkFjBE0ZX5B6ms0995C9yl5AMrQ1z-DEHZuho=.c8e60fab-14cb-4eb4-8a85-aa2e2a1fab79@github.com>

On Tue, 2 Apr 2024 16:24:19 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Clean up notproduct from tests.

src/hotspot/share/runtime/arguments.cpp line 3420:

> 3418: static void apply_debugger_ergo() {
> 3419: #ifndef PRODUCT
> 3420:   // UseDebuggerErgo is notproduct

Now that the flag has been changed to a develop flag, it seems wrong that these are guarded by "#ifndef PRODUCT". Shouldn't this be changed to check for ASSERT instead?

src/hotspot/share/runtime/flags/jvmFlag.hpp line 118:

> 116:     EXPERIMENTAL_FLAG_BUT_LOCKED,
> 117:     DEVELOPER_FLAG_BUT_PRODUCT_BUILD,
> 118:     NOTPRODUCT_FLAG_BUT_PRODUCT_BUILD

Should the ',' on the previous line be removed?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18541#discussion_r1548236362
PR Review Comment: https://git.openjdk.org/jdk/pull/18541#discussion_r1548239130

From coleenp at openjdk.org  Tue Apr  2 17:16:01 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 17:16:01 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v2]
In-Reply-To: <QgftUnVOGk6RVVe-HTPuzxmafkCtcKKenFRyseU-71g=.aaad76be-1e0b-4293-ae8e-9e046b614fe7@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <QgftUnVOGk6RVVe-HTPuzxmafkCtcKKenFRyseU-71g=.aaad76be-1e0b-4293-ae8e-9e046b614fe7@github.com>
Message-ID: <1SplyimCOCzBkP_A15DW-Q_NcUg8d7qmrcNfBU3GJSk=.11aa3fa0-3e3d-4587-922c-ac89d984478d@github.com>

On Tue, 2 Apr 2024 16:24:19 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Clean up notproduct from tests.

Thanks for looking through the changes, Stefan.

-------------

PR Review: https://git.openjdk.org/jdk/pull/18541#pullrequestreview-1974442423

From coleenp at openjdk.org  Tue Apr  2 17:16:02 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 17:16:02 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v2]
In-Reply-To: <fL1uGxDkFjBE0ZX5B6ms0995C9yl5AMrQ1z-DEHZuho=.c8e60fab-14cb-4eb4-8a85-aa2e2a1fab79@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <QgftUnVOGk6RVVe-HTPuzxmafkCtcKKenFRyseU-71g=.aaad76be-1e0b-4293-ae8e-9e046b614fe7@github.com>
 <fL1uGxDkFjBE0ZX5B6ms0995C9yl5AMrQ1z-DEHZuho=.c8e60fab-14cb-4eb4-8a85-aa2e2a1fab79@github.com>
Message-ID: <2XgKLmehivQ4frz5mofTSXn9LDFShIwprD6J2GUS_Is=.ecc3c80d-ea3f-447e-a951-9fbbb5c24a59@github.com>

On Tue, 2 Apr 2024 16:49:19 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Clean up notproduct from tests.
>
> src/hotspot/share/runtime/arguments.cpp line 3420:
> 
>> 3418: static void apply_debugger_ergo() {
>> 3419: #ifndef PRODUCT
>> 3420:   // UseDebuggerErgo is notproduct
> 
> Now that the flag has been changed to a develop flag, it seems wrong that these are guarded by "#ifndef PRODUCT". Shouldn't this be changed to check for ASSERT instead?

Yes, ifdef ASSERT is more appropriate.

> src/hotspot/share/runtime/flags/jvmFlag.hpp line 118:
> 
>> 116:     EXPERIMENTAL_FLAG_BUT_LOCKED,
>> 117:     DEVELOPER_FLAG_BUT_PRODUCT_BUILD,
>> 118:     NOTPRODUCT_FLAG_BUT_PRODUCT_BUILD
> 
> Should the ',' on the previous line be removed?

Yes, I guess our compilers don't complain about that anymore.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18541#discussion_r1548261310
PR Review Comment: https://git.openjdk.org/jdk/pull/18541#discussion_r1548269993

From coleenp at openjdk.org  Tue Apr  2 17:25:12 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 17:25:12 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
Message-ID: <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>

> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
> 
> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.

Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:

  Fix a couple issues pointed out by Stefank.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18541/files
  - new: https://git.openjdk.org/jdk/pull/18541/files/19b8f6b6..00a241d3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18541&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18541&range=01-02

  Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/18541.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18541/head:pull/18541

PR: https://git.openjdk.org/jdk/pull/18541

From wkemper at openjdk.org  Tue Apr  2 17:26:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 2 Apr 2024 17:26:43 GMT
Subject: RFR: 8328626: GenShen: Combine old generation surplus/deficit
 fields into a single balance field [v2]
In-Reply-To: <AgcnGKgPLVLiiMfQ6uic7qypQjVTxkF0NHjZFFF84vk=.c9a37938-f7ce-42d8-af42-8a315a13cec1@github.com>
References: <AgcnGKgPLVLiiMfQ6uic7qypQjVTxkF0NHjZFFF84vk=.c9a37938-f7ce-42d8-af42-8a315a13cec1@github.com>
Message-ID: <ydlozG3qrmEd-R-wNer5_syfG3zHAdn3UGUSl3rJfQ0=.6b47868c-4ff2-4e5c-874d-967252106355@github.com>

> Backport of 8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits:

 - Merge remote-tracking branch 'shenandoah-21/master' into backport-earthling-amzn-573618e6
 - 8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field
   
   Reviewed-by: kdnilsen, ysr

-------------

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/32/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=32&range=01
  Stats: 274 lines in 9 files changed: 99 ins; 117 del; 58 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/32.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/32/head:pull/32

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/32

From wkemper at openjdk.org  Tue Apr  2 17:28:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 2 Apr 2024 17:28:25 GMT
Subject: Integrated: 8329350: GenShen: Do not reset mark bitmaps on a safepoint
In-Reply-To: <9D7mi7sbY8lCY9njdei5rIyosleJYr7EW5NbQdvy4oY=.e660695f-7f43-453d-9fc4-6983633035f4@github.com>
References: <9D7mi7sbY8lCY9njdei5rIyosleJYr7EW5NbQdvy4oY=.e660695f-7f43-453d-9fc4-6983633035f4@github.com>
Message-ID: <cn9cFMlh0V_JusUCz5cJ1O-3EAIcINXpwU8tSJxnPmw=.aae4e624-1d50-420a-958a-5deb3a84f201@github.com>

On Fri, 29 Mar 2024 21:16:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah is currently resetting mark bitmaps during the init mark pause. This work should happen during the concurrent reset phase to avoid prolonging the safepoint. Also, free regions need to have the corresponding bitmap region reset or we risk having marked regions with no live data (which violates asserts during final mark).

This pull request has now been integrated.

Changeset: f43034d8
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/f43034d8d1dc571390f357dc3617f376cafa085f
Stats:     173 lines in 12 files changed: 101 ins; 57 del; 15 mod

8329350: GenShen: Do not reset mark bitmaps on a safepoint

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/413

From kvn at openjdk.org  Tue Apr  2 17:37:10 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Tue, 2 Apr 2024 17:37:10 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
Message-ID: <lPip93C81Id4N8s8RQwpuFuVH67uyTpiqUMhWAYoeQs=.a56d49b4-bcaa-4d3a-99ce-10349df9c68b@github.com>

On Tue, 2 Apr 2024 17:25:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix a couple issues pointed out by Stefank.

So you finally decided to take on [JDK-8183288](https://bugs.openjdk.org/browse/JDK-8183288)

Essentially you are removing "optimized" VM build with these changes. In this case you need to change make files.

All Statistics flags should be product (which will increase product VM size) - it is important. May be need build's variable `--enable-jvm-feature-statistcs` to include statistics code on demand.

-------------

Changes requested by kvn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18541#pullrequestreview-1974500625

From ysr at openjdk.org  Tue Apr  2 17:44:26 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 2 Apr 2024 17:44:26 GMT
Subject: RFR: 8328626: GenShen: Combine old generation surplus/deficit
 fields into a single balance field [v2]
In-Reply-To: <ydlozG3qrmEd-R-wNer5_syfG3zHAdn3UGUSl3rJfQ0=.6b47868c-4ff2-4e5c-874d-967252106355@github.com>
References: <AgcnGKgPLVLiiMfQ6uic7qypQjVTxkF0NHjZFFF84vk=.c9a37938-f7ce-42d8-af42-8a315a13cec1@github.com>
 <ydlozG3qrmEd-R-wNer5_syfG3zHAdn3UGUSl3rJfQ0=.6b47868c-4ff2-4e5c-874d-967252106355@github.com>
Message-ID: <a-dpfH6-OjU6lJJOWdlqDc_98hdFEWCIQcW3fUeSlsw=.9a77a065-ae3c-43b2-b921-e9555d6f651b@github.com>

On Tue, 2 Apr 2024 17:26:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Backport of 8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits:
> 
>  - Merge remote-tracking branch 'shenandoah-21/master' into backport-earthling-amzn-573618e6
>  - 8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field
>    
>    Reviewed-by: kdnilsen, ysr

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/32#pullrequestreview-1974515062

From coleenp at openjdk.org  Tue Apr  2 18:01:10 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 18:01:10 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
Message-ID: <QfTgke7CJrAB0F_IUfeEbuKBuVPk6b6VX3gfcWrn_Tg=.0a99e5dc-b5e4-4e6d-bd0a-b10e82bdb460@github.com>

On Tue, 2 Apr 2024 17:25:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix a couple issues pointed out by Stefank.

The optimized build still works as before (actually surprised it still builds). Since for a long time the notproduct options acted like develop options, they still do just the same for the optimized build.

For optimized, all the develop and notproduct options are materialized.   Now just develop, not distinguishing notproduct from that.  The same code enabled in PRODUCT is still enabled.  I haven't looked at that removing optimized bug in a while.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2032704606

From kvn at openjdk.org  Tue Apr  2 18:42:02 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Tue, 2 Apr 2024 18:42:02 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
Message-ID: <pTHIYUU5KswvwIQoCAjLkqNOww0f6C_M3g78ubP-UoQ=.52462db6-335c-4569-8c64-d47a4fe25666@github.com>

On Tue, 2 Apr 2024 17:25:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix a couple issues pointed out by Stefank.

Good.

-------------

Marked as reviewed by kvn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18541#pullrequestreview-1974635400

From kvn at openjdk.org  Tue Apr  2 18:42:02 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Tue, 2 Apr 2024 18:42:02 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <QfTgke7CJrAB0F_IUfeEbuKBuVPk6b6VX3gfcWrn_Tg=.0a99e5dc-b5e4-4e6d-bd0a-b10e82bdb460@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
 <QfTgke7CJrAB0F_IUfeEbuKBuVPk6b6VX3gfcWrn_Tg=.0a99e5dc-b5e4-4e6d-bd0a-b10e82bdb460@github.com>
Message-ID: <RRbjUV-p1gEfd4HXhepLmtrU7gCXuBHPj30SexzHehw=.3881fa59-7bab-4dc2-84fc-e6e879f18c45@github.com>

On Tue, 2 Apr 2024 17:58:47 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> For optimized, all the develop and notproduct options are materialized.

Okay, I see what you did here. You touched only flags declaration and did not `#ifndef PRODUCT` which guards statistics code, for example. Optimized VM build will get that code but will not include DEBUG_ONLY and `#ifdef ASSERT` guarded code.

So we still need to be careful when we use `#ifndef PRODUCT` and `#ifdef ASSERT`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2032789810

From coleenp at openjdk.org  Tue Apr  2 18:45:01 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 18:45:01 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
Message-ID: <8j655T3fzB8EZT8JdQZMXPgArVeDOvkuY2n0Uwxz1Gk=.e5a2bf28-6ce2-49f2-95cd-f19f92b271df@github.com>

On Tue, 2 Apr 2024 17:25:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix a couple issues pointed out by Stefank.

Yes, I had to remember what optimized did.  It gets all the options, but builds with optimization and doesn't turn on asserts.  I only removed the notproduct flag distinction since it hasn't been distinct for years and it's confusing what we actually wanted it to do.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2032794955

From kvn at openjdk.org  Tue Apr  2 19:02:10 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Tue, 2 Apr 2024 19:02:10 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
Message-ID: <QRE7Qh5c_GGflvWZaVCfp3AxSY4DjCtk4q_JwVsaOCc=.e4af2999-b4e8-4ad1-baca-7600fced3689@github.com>

On Tue, 2 Apr 2024 17:25:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix a couple issues pointed out by Stefank.

Long ago, before [JDK-8024545](https://bugs.openjdk.org/browse/JDK-8024545) we did not have not_product flags declared in product build. Only debug flags were declared as constant. We relied on that change since then.
That is why you may see the issue with not materialized flags in product build.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2032835463

From kbarrett at openjdk.org  Tue Apr  2 19:08:01 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Tue, 2 Apr 2024 19:08:01 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
Message-ID: <-Di8CpUXEjVtG3uu2r_djrEuUDpnQF833TWGkDHmZM4=.176b6bd0-e84d-40f1-999e-4fe41c750d8a@github.com>

On Tue, 2 Apr 2024 17:25:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix a couple issues pointed out by Stefank.

Looks good.  One minor nit.

Also, it seems that develop and nonproduct (before this change) flags have
associated JVMFlag objects even in product builds. (The function
JVMFlag::is_constant_in_binary is evidence for this. I didn't dig through all
the code to verify it.) Probably if one were going to retain nonproduct
options and make them work "properly", they would only have JVMFlag objects in
non-product builds. But it's not obvious to me why we would want such objects
for either category in product builds. I think any change along that line
should be a separate followup.

test/hotspot/jtreg/runtime/CommandLine/VMOptionWarning.java line 64:

> 62:         output = new OutputAnalyzer(pb.start());
> 63:         output.shouldNotHaveExitValue(0);
> 64:         output.shouldContain("Error: VM option 'CheckCompressedOops' is develop and is available only in debug version of VM.");

Seems like we don't need this test of the develop option CheckCompressedOops at all, since we have
the immediately preceding test of the develop option VerifyStack.

-------------

Marked as reviewed by kbarrett (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18541#pullrequestreview-1974549362
PR Review Comment: https://git.openjdk.org/jdk/pull/18541#discussion_r1548328730

From coleenp at openjdk.org  Tue Apr  2 19:29:09 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 19:29:09 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v3]
In-Reply-To: <-Di8CpUXEjVtG3uu2r_djrEuUDpnQF833TWGkDHmZM4=.176b6bd0-e84d-40f1-999e-4fe41c750d8a@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <jPjA4A6tDlOKKK-ui_6RJPQ4x5OIW_tL1AALv6YzW1E=.d161aa1a-de9e-445a-8a88-6d5c644d0880@github.com>
 <-Di8CpUXEjVtG3uu2r_djrEuUDpnQF833TWGkDHmZM4=.176b6bd0-e84d-40f1-999e-4fe41c750d8a@github.com>
Message-ID: <Ko1s3hmbLgDu5ulMH5WJCv8B49-uBsLUjcvmEO7jPZE=.8e7d2fa7-f830-4f6c-81ea-3f89122cddce@github.com>

On Tue, 2 Apr 2024 17:58:16 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:

>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix a couple issues pointed out by Stefank.
>
> test/hotspot/jtreg/runtime/CommandLine/VMOptionWarning.java line 64:
> 
>> 62:         output = new OutputAnalyzer(pb.start());
>> 63:         output.shouldNotHaveExitValue(0);
>> 64:         output.shouldContain("Error: VM option 'CheckCompressedOops' is develop and is available only in debug version of VM.");
> 
> Seems like we don't need this test of the develop option CheckCompressedOops at all, since we have
> the immediately preceding test of the develop option VerifyStack.

You're right, we've already tested this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18541#discussion_r1548488160

From wkemper at openjdk.org  Tue Apr  2 19:33:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 2 Apr 2024 19:33:26 GMT
Subject: Integrated: 8328626: GenShen: Combine old generation surplus/deficit
 fields into a single balance field
In-Reply-To: <AgcnGKgPLVLiiMfQ6uic7qypQjVTxkF0NHjZFFF84vk=.c9a37938-f7ce-42d8-af42-8a315a13cec1@github.com>
References: <AgcnGKgPLVLiiMfQ6uic7qypQjVTxkF0NHjZFFF84vk=.c9a37938-f7ce-42d8-af42-8a315a13cec1@github.com>
Message-ID: <k4nlrCnGAElTk1EsbbXjLZEsoRNDlNJYlMC1IjPAXtY=.193baf20-aefa-4b90-906c-bfa2af7a62c3@github.com>

On Mon, 1 Apr 2024 17:13:07 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Backport of 8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field

This pull request has now been integrated.

Changeset: 7b4d8661
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/7b4d86619d7aca038f13d5b57ddf5eba9a2ac6f9
Stats:     274 lines in 9 files changed: 99 ins; 117 del; 58 mod

8328626: GenShen: Combine old generation surplus/deficit fields into a single balance field

Reviewed-by: kdnilsen, ysr
Backport-of: 573618e60e4c5201d3cbce2dbb9c0112ce852b4a

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/32

From coleenp at openjdk.org  Tue Apr  2 19:47:23 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 19:47:23 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v4]
In-Reply-To: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
Message-ID: <vD-L2x7FsbI73-fnKI9gis14awM8xL-IPeXzDfMYUqE=.c5c01067-f446-4538-8b66-b522a3cdbd2f@github.com>

> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
> 
> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.

Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:

  Remove redundant test case.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18541/files
  - new: https://git.openjdk.org/jdk/pull/18541/files/00a241d3..3b5002d8

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18541&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18541&range=02-03

  Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/18541.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18541/head:pull/18541

PR: https://git.openjdk.org/jdk/pull/18541

From coleenp at openjdk.org  Tue Apr  2 19:54:01 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Tue, 2 Apr 2024 19:54:01 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v4]
In-Reply-To: <vD-L2x7FsbI73-fnKI9gis14awM8xL-IPeXzDfMYUqE=.c5c01067-f446-4538-8b66-b522a3cdbd2f@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <vD-L2x7FsbI73-fnKI9gis14awM8xL-IPeXzDfMYUqE=.c5c01067-f446-4538-8b66-b522a3cdbd2f@github.com>
Message-ID: <8dFwjhziEAnv-PG3YM6FuPLbB6HlyO4BApqmuLwc3xo=.cf4ae09a-cf37-4f30-bcad-1f0582de82d1@github.com>

On Tue, 2 Apr 2024 19:47:23 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove redundant test case.

Thanks for reviewing, Kim.  Is your suggestion to not have a JVMFlag object for develop flags in PRODUCT builds? Presumably to save some footprint?  I'm not sure we would win fighting the macros to accomplish this.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2032981431

From kbarrett at openjdk.org  Tue Apr  2 20:21:11 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Tue, 2 Apr 2024 20:21:11 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v4]
In-Reply-To: <8dFwjhziEAnv-PG3YM6FuPLbB6HlyO4BApqmuLwc3xo=.cf4ae09a-cf37-4f30-bcad-1f0582de82d1@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <vD-L2x7FsbI73-fnKI9gis14awM8xL-IPeXzDfMYUqE=.c5c01067-f446-4538-8b66-b522a3cdbd2f@github.com>
 <8dFwjhziEAnv-PG3YM6FuPLbB6HlyO4BApqmuLwc3xo=.cf4ae09a-cf37-4f30-bcad-1f0582de82d1@github.com>
Message-ID: <uMoZjOdPh_ay7jJL9cMQDEF8xTL0YtkZv4quW9Kn7zk=.9767c201-4b36-49fe-ba37-7c471c2f93fe@github.com>

On Tue, 2 Apr 2024 19:51:03 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> Thanks for reviewing, Kim. Is your suggestion to not have a JVMFlag object for develop flags in PRODUCT builds? Presumably to save some footprint? I'm not sure we would win fighting the macros to accomplish this.

Yes, that's the suggestion and the rationale for it.  It should also remove the need for is_constant_in_binary.  I don't
know how hard it would actually be to accomplish this.  I agree it might not be worth the effort, but we won't know
until someone looks, which I haven't done.  It might even be easy.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2033020865

From iklam at openjdk.org  Tue Apr  2 22:08:10 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Tue, 2 Apr 2024 22:08:10 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v4]
In-Reply-To: <uMoZjOdPh_ay7jJL9cMQDEF8xTL0YtkZv4quW9Kn7zk=.9767c201-4b36-49fe-ba37-7c471c2f93fe@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <vD-L2x7FsbI73-fnKI9gis14awM8xL-IPeXzDfMYUqE=.c5c01067-f446-4538-8b66-b522a3cdbd2f@github.com>
 <8dFwjhziEAnv-PG3YM6FuPLbB6HlyO4BApqmuLwc3xo=.cf4ae09a-cf37-4f30-bcad-1f0582de82d1@github.com>
 <uMoZjOdPh_ay7jJL9cMQDEF8xTL0YtkZv4quW9Kn7zk=.9767c201-4b36-49fe-ba37-7c471c2f93fe@github.com>
Message-ID: <JcPhv31s2Uow_HF2dpnUDrDeqxFwRu7sfVTKOpAbxXo=.47cfa929-9f85-4fa4-aa0b-7e0f74bc6139@github.com>

On Tue, 2 Apr 2024 20:18:34 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:

> > Thanks for reviewing, Kim. Is your suggestion to not have a JVMFlag object for develop flags in PRODUCT builds? Presumably to save some footprint? I'm not sure we would win fighting the macros to accomplish this.
> 
> Yes, that's the suggestion and the rationale for it. It should also remove the need for is_constant_in_binary. I don't know how hard it would actually be to accomplish this. I agree it might not be worth the effort, but we won't know until someone looks, which I haven't done. It might even be easy.

Currently the VM prints an error message for non-product flags, so we need to keep some information about them. We can probably skip the type information, etc, to save a little space, but the space saving would be minimal.


$ java -XX:+LoomDeoptAfterThaw --version
Error: VM option 'LoomDeoptAfterThaw' is develop and is available only in debug version of VM.
Improperly specified VM option 'LoomDeoptAfterThaw'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2033183718

From vlivanov at openjdk.org  Wed Apr  3 02:58:00 2024
From: vlivanov at openjdk.org (Vladimir Ivanov)
Date: Wed, 3 Apr 2024 02:58:00 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
Message-ID: <86hdErNCggxb7O-j9AYmcR9IV7M15p1Hnrowo4nDk_U=.1b2b07cd-209b-4181-bc97-58d1a8fac674@github.com>

On Mon, 1 Apr 2024 21:07:31 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removed not_used state of nmethod

Nice cleanup! Overall, looks very good.

What about `CompiledMethod_lock`? There's no `CompiledMethod` anymore, but the lock name still refers to it.

-------------

Marked as reviewed by vlivanov (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18554#pullrequestreview-1975392018

From coleenp at openjdk.org  Wed Apr  3 12:25:15 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Wed, 3 Apr 2024 12:25:15 GMT
Subject: RFR: 8236736: Change notproduct JVM flags to develop flags [v4]
In-Reply-To: <vD-L2x7FsbI73-fnKI9gis14awM8xL-IPeXzDfMYUqE=.c5c01067-f446-4538-8b66-b522a3cdbd2f@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
 <vD-L2x7FsbI73-fnKI9gis14awM8xL-IPeXzDfMYUqE=.c5c01067-f446-4538-8b66-b522a3cdbd2f@github.com>
Message-ID: <RKR7PwZ9Jy5DEdM9_X1YvCwZFUSQlB7c9pIVqVQXMek=.9b9ec1e6-d84e-4758-9427-0966ac0ab7de@github.com>

On Tue, 2 Apr 2024 19:47:23 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
>> 
>> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove redundant test case.

Thanks for the reviews, Ioi, Vladimir, Kim and Stefan.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18541#issuecomment-2034433313

From coleenp at openjdk.org  Wed Apr  3 12:25:15 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Wed, 3 Apr 2024 12:25:15 GMT
Subject: Integrated: 8236736: Change notproduct JVM flags to develop flags
In-Reply-To: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
References: <25c1XDQrzxvG0AuxlRjQyznnTdLzD1-J4kebuXzj-Zc=.0f5d28e1-7672-40a9-97fe-04a77fda65d9@github.com>
Message-ID: <Wce-Ho-45jQ48BQrSNonpRwYaVfZkD4Lc9xFGQXCA28=.6fca587d-5154-495d-91fb-240d28efb5d5@github.com>

On Thu, 28 Mar 2024 22:53:22 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> Remove the notproduct distinction for command line options, rather than trying to wrestle the macros to fix the bug that they've been treated as develop options for some time now.  This simplifies the command line option macros.
> 
> Tested with tier1-4, tier1 on Oracle platforms.  Also built shenandoah.

This pull request has now been integrated.

Changeset: bea493bc
Author:    Coleen Phillimore <coleenp at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/bea493bcb86370dc3fb00d86c545f01fc614e000
Stats:     282 lines in 43 files changed: 1 ins; 102 del; 179 mod

8236736: Change notproduct JVM flags to develop flags

Reviewed-by: iklam, kvn, kbarrett

-------------

PR: https://git.openjdk.org/jdk/pull/18541

From stefank at openjdk.org  Wed Apr  3 16:03:14 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 3 Apr 2024 16:03:14 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
Message-ID: <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>

On Mon, 1 Apr 2024 21:07:31 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removed not_used state of nmethod

Nice!

We've wanted to clean up some interfaces between the CodeCache and the GC code by using nmethod closures instead of CodeBlob closures. This change (and the Sweeper removal) makes it possible to do those cleanups.

I've made a superficial pass over the patch to and left a few comments. Most of those comments are things that would be nice to fix, but could also be left as follow-up RFEs (if they are deemed to be worthy ideas to pursue).

src/hotspot/os/posix/signals_posix.cpp line 27:

> 25: #include "precompiled.hpp"
> 26: #include "code/codeCache.hpp"
> 27: #include "code/nmethod.hpp"

The include line needs to move down.

src/hotspot/share/code/codeBlob.hpp line 77:

> 75: //   - data space
> 76: 
> 77: enum CodeBlobKind : u1 {

It will probably be safer to change this to an enum class, so that the compiler will help us if we mess up with the argument order when this is used in function calls. I see that this patch switches the parameter order of some functions, so I think it could be worth trying out.

src/hotspot/share/code/codeBlob.hpp line 409:

> 407: 
> 408:   // GC/Verification support
> 409:   virtual void preserve_callee_argument_oops(frame fr, const RegisterMap *reg_map, OopClosure* f) override { /* nothing to do */ }

In the GC code we usually have either virtual OR override, but not both. Could we skip `virtual` here? Or does the compiler code usually use both?

src/hotspot/share/code/codeBlob.hpp line 429:

> 427:    SingletonBlob(
> 428:      const char* name,
> 429:      CodeBlobKind kind,

There's an alignment issue after this change.

src/hotspot/share/code/codeCache.cpp line 1009:

> 1007: int CodeCache::nmethod_count() {
> 1008:   int count = 0;
> 1009:   for (GrowableArrayIterator<CodeHeap*> heap = _nmethod_heaps->begin(); heap != _nmethod_heaps->end(); ++heap) {

Is there a reason why FOR_ALL_NMETHOD_HEAPS wasn't good fit here? I'm wondering since the similar `CodeCache::blob_count()` still uses one of these macros.

src/hotspot/share/code/nmethod.cpp line 812:

> 810:     // By calling this nmethod entry barrier, it plays along and acts
> 811:     // like any other nmethod found on the stack of a thread (fewer surprises).
> 812:     nmethod* nm = as_nmethod_or_null();

Calling as_nmethod_or_null() from within functions in the nmethod class is suspicious. Shouldn't all such usages be removed? (I'm fine with doing that as a separate change)

src/hotspot/share/code/nmethod.cpp line 1009:

> 1007: // Fill in default values for various flag fields
> 1008: void nmethod::init_defaults() {
> 1009:   { // avoid uninitialized fields, even for short time periods

Should these curly braces be removed?

src/hotspot/share/code/nmethod.cpp line 2164:

> 2162:   DTRACE_METHOD_UNLOAD_PROBE(method());
> 2163: 
> 2164:   // If a JVMTI agent has enabled the nmethod Unload event then

I think the event is still called CompiledMethodUnload, so this line should probably be reverted.

src/hotspot/share/code/nmethod.hpp line 50:

> 48: class ScopeDesc;
> 49: class CompiledIC;
> 50: class MetadataClosure;

Maybe merge (and sort) this together with the other forward declarations?

src/hotspot/share/code/nmethod.hpp line 905:

> 903: 
> 904:   // printing support
> 905:   void print()                          const override;

Here and a few other places you only use override and not also virtual. This is inconsistent with other functions in this class. (FWIW, I prefer this style with only the override qualifier).

src/hotspot/share/code/nmethod.inline.hpp line 60:

> 58: // (b) it is a deopt PC
> 59: 
> 60: inline address nmethod::get_deopt_original_pc(const frame* fr) {

While reading this PR I wonder if this really belongs in the `nmethod` class or if it would make more sense to have it as a member function in the `frame` class. It is a static function, which uses `fr` sort-of like a `this` pointer. Maybe something to consider for a separate RFE.

src/hotspot/share/code/relocInfo.hpp line 39:

> 37: class nmethod;
> 38: class CodeBlob;
> 39: class nmethod;

You already have a class nmethod forward declaration above.

src/hotspot/share/compiler/compileBroker.cpp line 1379:

> 1377:   if (osr_bci == InvocationEntryBci) {
> 1378:     // standard compilation
> 1379:     nmethod* method_code = method->code();

Isn't the `method_code->is_nmethod()` redundant now?

src/hotspot/share/compiler/compileBroker.cpp line 1484:

> 1482:   // We accept a higher level osr method
> 1483:   if (osr_bci == InvocationEntryBci) {
> 1484:     nmethod* code = method->code();

Cast below is redundant.

src/hotspot/share/gc/g1/g1HeapRegion.cpp line 339:

> 337: 
> 338:   void do_code_blob(CodeBlob* cb) {
> 339:     nmethod* nm = (cb == nullptr) ? nullptr : cb->as_nmethod_or_null();

After this change I'd like to see if we can change this and similar GC interfaces to work directly against `nmethod` instead of `CodeBlob`.

src/hotspot/share/gc/shared/parallelCleaning.cpp line 54:

> 52: void CodeCacheUnloadingTask::claim_nmethods(nmethod** claimed_nmethods, int *num_claimed_nmethods) {
> 53:   nmethod* first;
> 54:   NMethodIterator last(NMethodIterator::all_blobs);

FWIW, `all_blobs` is slightly confusing name when nmethods are a subset of all "code blobs". We might want to consider renaming this to `NMethodIterator::all` (in a separate RFE).

src/hotspot/share/gc/x/xUnload.cpp line 78:

> 76: class XIsUnloadingBehaviour : public IsUnloadingBehaviour {
> 77: public:
> 78:   virtual bool has_dead_oop(nmethod* method) const {

This now takes an `nmethod` argument, but still calls as_nmethod(). I think that should be removed from this, and all similar functions here in the GC code. If you want, I can do that as a follow-up RFE.

src/hotspot/share/jvmci/jvmciRuntime.cpp line 271:

> 269: 
> 270:   nm = CodeCache::find_nmethod(pc);
> 271:   assert(nm != nullptr, "this is not a compiled method");

Unrelated to this patch, but might be worth mentioning because I think it would be good to think about this in a follow-up RFE. `CodeCache::find_blob` returns null if it can't find a matching blob, but `CodeCache::find_nmethod` asserts that it did find one. The latter makes the assert redundant, but it also begs to question if `find_blob` and `find_nmethod` realy should be different in this regard? Should we have `find_blob_or_null` and `find_nmethod_or_null`? Alt. `find_blob_not_null` and `find_nmethod_not_null`.

src/hotspot/share/prims/whitebox.cpp line 772:

> 770:             if (_make_not_entrant) {
> 771:                 nmethod* nm = CodeCache::find_nmethod(f->pc());
> 772:                 assert(nm != nullptr, "sanity check");

This assert is now redundant.

src/hotspot/share/prims/whitebox.cpp line 1100:

> 1098:   // Check code again because compilation may be finished before Compile_lock is acquired.
> 1099:   if (bci == InvocationEntryBci) {
> 1100:     nmethod* code = mh->code();

`as_nmethod_or_null()` below should be redundant.

src/hotspot/share/runtime/continuationEntry.hpp line 35:

> 33: #include CPU_HEADER(continuationEntry)
> 34: 
> 35: class nmethod;

Maybe keep the forward declarations sorted?

src/hotspot/share/runtime/continuationEntry.hpp line 59:

> 57: public:
> 58:   static int _return_pc_offset; // friend gen_continuation_enter
> 59:   static void set_enter_code(nmethod* cm, int interpreted_entry_offset);

cm => nm?

src/hotspot/share/runtime/frame.cpp line 208:

> 206: address frame::raw_pc() const {
> 207:   if (is_deoptimized_frame()) {
> 208:     nmethod* nm = cb()->as_nmethod_or_null();

Prexisting: It's weird that this code is using the `_or_null()` version when the code below does not null check the returned value.

src/hotspot/share/runtime/vframe.cpp line 75:

> 73:   if (cb != nullptr) {
> 74:     if (cb->is_nmethod()) {
> 75:       nmethod* nm = cb->as_nmethod();;

There's two `;`s on this line.

-------------

PR Review: https://git.openjdk.org/jdk/pull/18554#pullrequestreview-1977027234
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549873079
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549892107
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549895707
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549917406
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549925499
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549949611
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549954407
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549955841
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549958927
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549968764
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549974539
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549975722
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549977276
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549978007
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549979750
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549983251
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549985971
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549992086
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1549999765
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550002167
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550005072
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550006055
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550013866
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550025081

From kvn at openjdk.org  Wed Apr  3 16:18:12 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 16:18:12 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <86hdErNCggxb7O-j9AYmcR9IV7M15p1Hnrowo4nDk_U=.1b2b07cd-209b-4181-bc97-58d1a8fac674@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <86hdErNCggxb7O-j9AYmcR9IV7M15p1Hnrowo4nDk_U=.1b2b07cd-209b-4181-bc97-58d1a8fac674@github.com>
Message-ID: <8Eb7EtxdXdDbbgICKduteW4cHuINHpbQevdPJ7XBvYY=.594146f6-87f3-41ed-86a7-3e6541d67286@github.com>

On Wed, 3 Apr 2024 02:55:52 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> What about `CompiledMethod_lock`? There's no `CompiledMethod` anymore, but the lock name still refers to it.

It was different changes [JDK-8226705](https://bugs.openjdk.org/browse/JDK-8226705). Renaming it will complicate these changes more than I wanted. I can do it in separate RFE.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2035031413

From kvn at openjdk.org  Wed Apr  3 16:32:12 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 16:32:12 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
Message-ID: <XLp_PiTfgwtq8j7xWRuhEPSChBsUcBYgkzrJbHpG1nU=.b29ceb12-830e-43e8-ab9e-d587c6dffd03@github.com>

On Wed, 3 Apr 2024 14:44:03 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removed not_used state of nmethod
>
> src/hotspot/share/code/codeBlob.hpp line 409:
> 
>> 407: 
>> 408:   // GC/Verification support
>> 409:   virtual void preserve_callee_argument_oops(frame fr, const RegisterMap *reg_map, OopClosure* f) override { /* nothing to do */ }
> 
> In the GC code we usually have either virtual OR override, but not both. Could we skip `virtual` here? Or does the compiler code usually use both?

No special rules here. I simply want to see all `virtual` methods explicitly and `override` is required by C++.
I would like to keep it this way in these changes. I am investigating possibility to convert all these virtual methods to normal one to remove virtual table and virtual pointer (8 bytes) from CodeBlob class.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550071713

From kvn at openjdk.org  Wed Apr  3 16:40:59 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 16:40:59 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
Message-ID: <krSXrZzWLlgmaN_nvIgBhdbR75PQqpRcU3H4EBCdtP4=.b378adba-bff1-424f-b07d-d80c641e6608@github.com>

On Wed, 3 Apr 2024 15:01:22 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removed not_used state of nmethod
>
> src/hotspot/share/code/codeCache.cpp line 1009:
> 
>> 1007: int CodeCache::nmethod_count() {
>> 1008:   int count = 0;
>> 1009:   for (GrowableArrayIterator<CodeHeap*> heap = _nmethod_heaps->begin(); heap != _nmethod_heaps->end(); ++heap) {
> 
> Is there a reason why FOR_ALL_NMETHOD_HEAPS wasn't good fit here? I'm wondering since the similar `CodeCache::blob_count()` still uses one of these macros.

No,  `CodeCache::blob_count()` uses different macro `FOR_ALL_HEAPS(heap)` because it looks for all code blobs, not only nmethods.

`CodeCache::nmethod_count()` is the only place where `FOR_ALL_NMETHOD_HEAPS ` was used. So I decided to remove the macro.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550087255

From kvn at openjdk.org  Wed Apr  3 17:01:10 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 17:01:10 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
Message-ID: <mlT7PVOHFFZO7Jwl2QfwOiezYsYFdu7u0w_nzokVe4A=.3cb7886c-3d09-44aa-a06f-d4288b833fda@github.com>

On Wed, 3 Apr 2024 15:12:31 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removed not_used state of nmethod
>
> src/hotspot/share/code/nmethod.cpp line 812:
> 
>> 810:     // By calling this nmethod entry barrier, it plays along and acts
>> 811:     // like any other nmethod found on the stack of a thread (fewer surprises).
>> 812:     nmethod* nm = as_nmethod_or_null();
> 
> Calling as_nmethod_or_null() from within functions in the nmethod class is suspicious. Shouldn't all such usages be removed? (I'm fine with doing that as a separate change)

Good catch! The code was moved from CompiledMethod where it made sense but now it is not needed. Here the change I will make:

     // like any other nmethod found on the stack of a thread (fewer surprises).
-    nmethod* nm = as_nmethod_or_null();
-    if (nm != nullptr && bs_nm->is_armed(nm)) {
+    nmethod* nm = this;
+    if (bs_nm->is_armed(nm)) {
       bool alive = bs_nm->nmethod_entry_barrier(nm);

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550118967

From kvn at openjdk.org  Wed Apr  3 17:38:01 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 17:38:01 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
Message-ID: <T6RrkmW6BhQbPekt4q1p0SgspZS5SjR_4fGtu36E0f8=.5d406e4a-8c9c-4bff-9bf8-06fcd0e5b804@github.com>

On Wed, 3 Apr 2024 15:30:00 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removed not_used state of nmethod
>
> src/hotspot/share/compiler/compileBroker.cpp line 1379:
> 
>> 1377:   if (osr_bci == InvocationEntryBci) {
>> 1378:     // standard compilation
>> 1379:     nmethod* method_code = method->code();
> 
> Isn't the `method_code->is_nmethod()` redundant now?

An other good catch! It leads me to chase all redundant `is_nmethod()` and `as_nmethod_*()` calls.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550187397

From stefank at openjdk.org  Wed Apr  3 17:53:01 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 3 Apr 2024 17:53:01 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <XLp_PiTfgwtq8j7xWRuhEPSChBsUcBYgkzrJbHpG1nU=.b29ceb12-830e-43e8-ab9e-d587c6dffd03@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
 <XLp_PiTfgwtq8j7xWRuhEPSChBsUcBYgkzrJbHpG1nU=.b29ceb12-830e-43e8-ab9e-d587c6dffd03@github.com>
Message-ID: <5V3m6SFRs8kib1w_Oyyrvr-Yu904MuuPQglr-uei_So=.60542853-0800-4113-b4f0-57d328c5d866@github.com>

On Wed, 3 Apr 2024 16:29:03 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> src/hotspot/share/code/codeBlob.hpp line 409:
>> 
>>> 407: 
>>> 408:   // GC/Verification support
>>> 409:   virtual void preserve_callee_argument_oops(frame fr, const RegisterMap *reg_map, OopClosure* f) override { /* nothing to do */ }
>> 
>> In the GC code we usually have either virtual OR override, but not both. Could we skip `virtual` here? Or does the compiler code usually use both?
>
> No special rules here. I simply want to see all `virtual` methods explicitly and `override` is required by C++.
> I would like to keep it this way in these changes. I am investigating possibility to convert all these virtual methods to normal one to remove virtual table and virtual pointer (8 bytes) from CodeBlob class.

`override` is not required by C++. You do however mark all virtual methods with `override` if any of the functions are marked with `override`. I think it would be good to have a HotSpot code style discussion about this (but not in this RFE).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550206804

From stefank at openjdk.org  Wed Apr  3 17:59:09 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 3 Apr 2024 17:59:09 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <krSXrZzWLlgmaN_nvIgBhdbR75PQqpRcU3H4EBCdtP4=.b378adba-bff1-424f-b07d-d80c641e6608@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
 <krSXrZzWLlgmaN_nvIgBhdbR75PQqpRcU3H4EBCdtP4=.b378adba-bff1-424f-b07d-d80c641e6608@github.com>
Message-ID: <mtLiuPqDagi21lvXlRjGpoLdxbEYEDPd8fgN06a0qB8=.ab0137a7-de30-49a2-b9a2-477b24454873@github.com>

On Wed, 3 Apr 2024 16:38:13 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> src/hotspot/share/code/codeCache.cpp line 1009:
>> 
>>> 1007: int CodeCache::nmethod_count() {
>>> 1008:   int count = 0;
>>> 1009:   for (GrowableArrayIterator<CodeHeap*> heap = _nmethod_heaps->begin(); heap != _nmethod_heaps->end(); ++heap) {
>> 
>> Is there a reason why FOR_ALL_NMETHOD_HEAPS wasn't good fit here? I'm wondering since the similar `CodeCache::blob_count()` still uses one of these macros.
>
> No,  `CodeCache::blob_count()` uses different macro `FOR_ALL_HEAPS(heap)` because it looks for all code blobs, not only nmethods.
> 
> `CodeCache::nmethod_count()` is the only place where `FOR_ALL_NMETHOD_HEAPS ` was used. So I decided to remove the macro.

I didn't say that blob_count used `FOR_ALL_NMETHODS_HEAP`. I wrote "one of these macros". I still think this adds an inconsistency to the code that I don't think is beneficial.

With that said, can't this be written as:

for (CodeHeap* heap : *_nmethod_heaps) {


Maybe yet another opportunity for cleanups.

>> src/hotspot/share/code/nmethod.cpp line 812:
>> 
>>> 810:     // By calling this nmethod entry barrier, it plays along and acts
>>> 811:     // like any other nmethod found on the stack of a thread (fewer surprises).
>>> 812:     nmethod* nm = as_nmethod_or_null();
>> 
>> Calling as_nmethod_or_null() from within functions in the nmethod class is suspicious. Shouldn't all such usages be removed? (I'm fine with doing that as a separate change)
>
> Good catch! The code was moved from CompiledMethod where it made sense but now it is not needed. Here the change I will make:
> 
>      // like any other nmethod found on the stack of a thread (fewer surprises).
> -    nmethod* nm = as_nmethod_or_null();
> -    if (nm != nullptr && bs_nm->is_armed(nm)) {
> +    nmethod* nm = this;
> +    if (bs_nm->is_armed(nm)) {
>        bool alive = bs_nm->nmethod_entry_barrier(nm);

Sounds good.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550216628
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550217712

From kvn at openjdk.org  Wed Apr  3 18:20:01 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 18:20:01 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
Message-ID: <om30iD1Xl17sMGyyPunnpH_FYnBNSgmV6xzM80Q1oYI=.9e35b1dc-c0d6-4047-8a5a-2979c43810e1@github.com>

On Wed, 3 Apr 2024 15:49:00 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removed not_used state of nmethod
>
> src/hotspot/share/runtime/frame.cpp line 208:
> 
>> 206: address frame::raw_pc() const {
>> 207:   if (is_deoptimized_frame()) {
>> 208:     nmethod* nm = cb()->as_nmethod_or_null();
> 
> Prexisting: It's weird that this code is using the `_or_null()` version when the code below does not null check the returned value.

Before [JDK-6921352](https://bugs.openjdk.org/browse/JDK-6921352) it was:

  return ((nmethod*) cb())->deopt_handler_begin() - pc_return_offset;


I will add assert with check for null. We definitely expect here only nmethod.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550243895

From kvn at openjdk.org  Wed Apr  3 18:50:10 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 18:50:10 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
Message-ID: <VBPGA16tOGTIOxwdSD4rN7S24OXwI4EN4dMK6ug13g0=.3a3ff6c6-15f0-473e-bff2-88a6dc70bcbb@github.com>

On Wed, 3 Apr 2024 15:35:49 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removed not_used state of nmethod
>
> src/hotspot/share/gc/x/xUnload.cpp line 78:
> 
>> 76: class XIsUnloadingBehaviour : public IsUnloadingBehaviour {
>> 77: public:
>> 78:   virtual bool has_dead_oop(nmethod* method) const {
> 
> This now takes an `nmethod` argument, but still calls as_nmethod(). I think that should be removed from this, and all similar functions here in the GC code. If you want, I can do that as a follow-up RFE.

I decided to fix it in these changes.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550285238

From kvn at openjdk.org  Wed Apr  3 18:55:00 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 18:55:00 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
Message-ID: <FTJNC_vTQA0djakJ_JFp-Q1h7byHju5jHlcmcxltmhM=.af2f0076-0804-4db1-b2ec-3ae6f5b5e630@github.com>

On Wed, 3 Apr 2024 16:00:01 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removed not_used state of nmethod
>
> Nice!
> 
> We've wanted to clean up some interfaces between the CodeCache and the GC code by using nmethod closures instead of CodeBlob closures. This change (and the Sweeper removal) makes it possible to do those cleanups.
> 
> I've made a superficial pass over the patch to and left a few comments. Most of those comments are things that would be nice to fix, but could also be left as follow-up RFEs (if they are deemed to be worthy ideas to pursue).

Thank you, @stefank, for great review. I addressed all your comments locally and with run testing in mach5 before pushing it.

Except your suggestion about `find_blob_not_null()` - should be separate RFE.
The same for suggestion "GC interfaces to work directly against nmethod instead of CodeBlob".

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2035354740

From wkemper at openjdk.org  Wed Apr  3 19:30:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 3 Apr 2024 19:30:12 GMT
Subject: RFR: 8324995: Shenandoah: Skip to full gc for humongous allocation
 failures [v3]
In-Reply-To: <QgXuMhrOQPIV_tADOre-PRtDNJYWXnuuHzMzuZnovqw=.6a5ebb07-a076-4b3f-be35-be4c77ca84f6@github.com>
References: <i_w8n_vKP0Ek0pqUIx1v0S4y_N3Cb5XFwYFDTbIFpYw=.10eaac81-1d30-4bc1-84e7-a18aee872ba2@github.com>
 <QgXuMhrOQPIV_tADOre-PRtDNJYWXnuuHzMzuZnovqw=.6a5ebb07-a076-4b3f-be35-be4c77ca84f6@github.com>
Message-ID: <H0tiev-P27rxTxa2rkPnrKJnmg2ijzaOw39dmIaHZRA=.e711fc9e-dfd5-4742-9b48-64ba356019b5@github.com>

On Wed, 31 Jan 2024 21:50:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Shenandoah degenerated cycles do not compact regions. When a humongous allocation fails, it is likely due to fragmentation which is better addressed by a full gc.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix typo in comment

Abandoning this PR while we conduct further testing.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17638#issuecomment-2035416675

From wkemper at openjdk.org  Wed Apr  3 19:30:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 3 Apr 2024 19:30:12 GMT
Subject: Withdrawn: 8324995: Shenandoah: Skip to full gc for humongous
 allocation failures
In-Reply-To: <i_w8n_vKP0Ek0pqUIx1v0S4y_N3Cb5XFwYFDTbIFpYw=.10eaac81-1d30-4bc1-84e7-a18aee872ba2@github.com>
References: <i_w8n_vKP0Ek0pqUIx1v0S4y_N3Cb5XFwYFDTbIFpYw=.10eaac81-1d30-4bc1-84e7-a18aee872ba2@github.com>
Message-ID: <NA20eWOkUO28XODyMmjOvTexfPAUAAxtnbni-ntkEzw=.65d0a9af-377a-4a44-a75c-727df7327ede@github.com>

On Tue, 30 Jan 2024 19:37:46 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah degenerated cycles do not compact regions. When a humongous allocation fails, it is likely due to fragmentation which is better addressed by a full gc.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/17638

From kvn at openjdk.org  Wed Apr  3 19:32:59 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 19:32:59 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
Message-ID: <jIu9pS_jI5cDQY5iDE_t6P7DHvdux6mDeNib5mWRqDE=.8b15297a-8b1e-4cd6-a292-10969394ff26@github.com>

On Mon, 1 Apr 2024 21:07:31 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removed not_used state of nmethod

I filed RFEs:
[JDK-8329628](https://bugs.openjdk.org/browse/JDK-8329628): Additional changes after JDK-8329332
[JDK-8329629](https://bugs.openjdk.org/browse/JDK-8329629): GC interfaces should work directly against nmethod instead of CodeBlob

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2035421134

From kvn at openjdk.org  Wed Apr  3 19:59:01 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 19:59:01 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <5V3m6SFRs8kib1w_Oyyrvr-Yu904MuuPQglr-uei_So=.60542853-0800-4113-b4f0-57d328c5d866@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
 <XLp_PiTfgwtq8j7xWRuhEPSChBsUcBYgkzrJbHpG1nU=.b29ceb12-830e-43e8-ab9e-d587c6dffd03@github.com>
 <5V3m6SFRs8kib1w_Oyyrvr-Yu904MuuPQglr-uei_So=.60542853-0800-4113-b4f0-57d328c5d866@github.com>
Message-ID: <TsLkhgOLr2NkwL1GXLkb2yq_z8rCrYEyE2MptLsiODU=.9c12c963-a742-4e0b-a27f-3d3c62b7312b@github.com>

On Wed, 3 Apr 2024 17:50:15 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> No special rules here. I simply want to see all `virtual` methods explicitly and `override` is required by C++.
>> I would like to keep it this way in these changes. I am investigating possibility to convert all these virtual methods to normal one to remove virtual table and virtual pointer (8 bytes) from CodeBlob class.
>
> `override` is not required by C++. You do however mark all virtual methods with `override` if any of the functions are marked with `override`. I think it would be good to have a HotSpot code style discussion about this (but not in this RFE).

I put `virtual/override` cleanup in CodeBlob as additional suggestion in followup RFE JDK-8329628.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550370826

From kvn at openjdk.org  Wed Apr  3 20:03:11 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 3 Apr 2024 20:03:11 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v2]
In-Reply-To: <mtLiuPqDagi21lvXlRjGpoLdxbEYEDPd8fgN06a0qB8=.ab0137a7-de30-49a2-b9a2-477b24454873@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <eMUTt2ic_OBuS168USvarGBRDCquzLb9AxrSpog1TZw=.d9ae808b-ed03-4971-bd13-db48794a1024@github.com>
 <2MTKEcGChvwWqFFuPs-5TR8GJnLwrnrgGKqdIV4NX70=.d0a707d4-5dfc-467c-992d-d410f94a7dca@github.com>
 <krSXrZzWLlgmaN_nvIgBhdbR75PQqpRcU3H4EBCdtP4=.b378adba-bff1-424f-b07d-d80c641e6608@github.com>
 <mtLiuPqDagi21lvXlRjGpoLdxbEYEDPd8fgN06a0qB8=.ab0137a7-de30-49a2-b9a2-477b24454873@github.com>
Message-ID: <AOdJarQeO7v1yqhDaRD7jt1UAmxWs7keOOjFmT0TAlM=.b76bf8a7-00b4-4c15-9da2-fb7016cd0fde@github.com>

On Wed, 3 Apr 2024 17:55:38 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> No,  `CodeCache::blob_count()` uses different macro `FOR_ALL_HEAPS(heap)` because it looks for all code blobs, not only nmethods.
>> 
>> `CodeCache::nmethod_count()` is the only place where `FOR_ALL_NMETHOD_HEAPS ` was used. So I decided to remove the macro.
>
> I didn't say that blob_count used `FOR_ALL_NMETHODS_HEAP`. I wrote "one of these macros". I still think this adds an inconsistency to the code that I don't think is beneficial.
> 
> With that said, can't this be written as:
> 
> for (CodeHeap* heap : *_nmethod_heaps) {
> 
> 
> Maybe yet another opportunity for cleanups.

I like it and will do it in JDK-8329628.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1550378690

From kvn at openjdk.org  Thu Apr  4 00:05:20 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 00:05:20 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
Message-ID: <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>

> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
> 
> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
> 
> I do not see (and not expected) performance difference with these changes.
> 
> Tested tier1-5, xcomp, stress. Running performance testing.
> 
> I need help with testing on platforms which Oracle does not support.

Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:

 - Address comments
 - Merge branch 'master' into 8329332
 - Removed not_used state of nmethod
 - remove trailing whitespace
 - 8329332: Remove CompiledMethod and CodeBlobLayout classes

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18554/files
  - new: https://git.openjdk.org/jdk/pull/18554/files/246ff68a..33768fb2

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18554&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18554&range=01-02

  Stats: 9283 lines in 197 files changed: 3058 ins; 4514 del; 1711 mod
  Patch: https://git.openjdk.org/jdk/pull/18554.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18554/head:pull/18554

PR: https://git.openjdk.org/jdk/pull/18554

From kvn at openjdk.org  Thu Apr  4 00:40:03 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 00:40:03 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
Message-ID: <ksNybPkksl9L8hZALhOyEh2yGV5n2DlQ0W_cUyXAtdc=.5be23eee-dcf9-4cdb-a153-e5a9bec4fd01@github.com>

On Thu, 4 Apr 2024 00:05:20 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Address comments
>  - Merge branch 'master' into 8329332
>  - Removed not_used state of nmethod
>  - remove trailing whitespace
>  - 8329332: Remove CompiledMethod and CodeBlobLayout classes

GHA `linux-x64-hs-minimal` failure is not related to changes:

2024-04-04T00:07:46.9654262Z ##[warning]Failed to download action 'https://api.github.com/repos/actions/github-script/tarball/60a0d83039c74a4aee543508d2ffcb1c3799cdea'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
2024-04-04T00:07:46.9656929Z ##[warning]Back off 22.252 seconds before retry.
2024-04-04T00:08:52.1252710Z ##[error]The SSL connection could not be established, see inner exception.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2035859221

From aboldtch at openjdk.org  Thu Apr  4 06:57:03 2024
From: aboldtch at openjdk.org (Axel Boldt-Christmas)
Date: Thu, 4 Apr 2024 06:57:03 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
Message-ID: <5Mj_wuhYdBmtFIJAD0qrBMlrX1TmTzutO7hLN--mvec=.913fd656-7155-4744-ae8a-0d5266e76cca@github.com>

On Thu, 4 Apr 2024 00:05:20 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Address comments
>  - Merge branch 'master' into 8329332
>  - Removed not_used state of nmethod
>  - remove trailing whitespace
>  - 8329332: Remove CompiledMethod and CodeBlobLayout classes

There is a stale comment in `test/jdk/com/sun/jdi/EATests.java:1288`


-// (See CompiledMethod::is_at_poll_return())
+// (See nmethod::is_at_poll_return())

-------------

PR Review: https://git.openjdk.org/jdk/pull/18554#pullrequestreview-1978884423

From ysr at openjdk.org  Thu Apr  4 07:49:46 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 4 Apr 2024 07:49:46 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial [v2]
In-Reply-To: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
Message-ID: <z_klPYdZP-NeMkrWuf-MQEKUCc7Od7ZHPfHmIXbErKo=.f49699ff-a3a4-45eb-ae71-b60be094259d@github.com>

> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
> 
> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
> 
> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
> 
> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
> 
> **Testing**:
> - [x] GHA : there are crashes unrelated to this change that are being addressed in other PRs
> - [ ] Code pipeline testing : there is a crash, unrelated to this change, that needs investigation
> - [x] SPECjbb
> 
> **Performance**:
> - [x] SPECjbb : the variance in tests appears to fail significance under 2-tailed Mann-Whitney
> - [ ] SPECjbb (patched for wk ref issue) : TBD
> - [ ] Extremem : in progress

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 14 commits:

 - Merge branch 'master' into clear_census
 - Remove local_reset of age_census object inadvertently left behind in the
   GLOBAL gen concurrent mark, which was triggering the newly added
   reasonableness assert (yay!).
 - Avoid divide-by-zero.
 - Merge branch 'master' into clear_census
 - Fix word and byte size unit confusion in comparison/assert.
 - Loose verification of "reasonable completeness" of census.
 - Fix release build.
 - Support for verification of census' reasonableness, but no verification
   itself yet.
 - ShenandoahGlobalGeneration also resets local age census data in
   prepare_gc() for the generational case.
 - Fix/elaborate some more documentation. Move age census reset to young
   gen's prepare_gc() method. Still TBD whether global needs this or not.
   Seems safer not to do this for global case, but may extend to global if
   it turns out that global leaves objects in their own generation or uses
   tenuring threshold to make promotion decisions.
 - ... and 4 more: https://git.openjdk.org/shenandoah/compare/c6d39d0a...d6edca1e

-------------

Changes: https://git.openjdk.org/shenandoah/pull/403/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=403&range=01
  Stats: 167 lines in 11 files changed: 122 ins; 27 del; 18 mod
  Patch: https://git.openjdk.org/shenandoah/pull/403.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/403/head:pull/403

PR: https://git.openjdk.org/shenandoah/pull/403

From stefank at openjdk.org  Thu Apr  4 07:57:12 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 4 Apr 2024 07:57:12 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
Message-ID: <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>

On Thu, 4 Apr 2024 00:05:20 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Address comments
>  - Merge branch 'master' into 8329332
>  - Removed not_used state of nmethod
>  - remove trailing whitespace
>  - 8329332: Remove CompiledMethod and CodeBlobLayout classes

I took a second pass over the changes. I've given a few suggestions below. None of them should require respinning of tests (except for making sure that this still builds).

src/hotspot/share/code/codeBlob.hpp line 168:

> 166:   bool is_vtable_blob() const                 { return _kind == CodeBlobKind::Blob_Vtable; }
> 167:   bool is_method_handles_adapter_blob() const { return _kind == CodeBlobKind::Blob_MH_Adapter; }
> 168:   bool is_upcall_stub() const                 { return _kind == CodeBlobKind::Blob_Upcall; }

The `Blob_` prefix is now redundant since we always have to prefix with CodeBlobKind::. Just a suggestion if you want to shorten these.

src/hotspot/share/gc/shared/gcBehaviours.hpp line 31:

> 29: #include "oops/oopsHierarchy.hpp"
> 30: 
> 31: // This is the behaviour for checking if a nmethod is unloading

Maybe this should be *an* nmethod?

src/hotspot/share/gc/shenandoah/shenandoahUnload.cpp line 81:

> 79: class ShenandoahIsUnloadingBehaviour : public IsUnloadingBehaviour {
> 80: public:
> 81:   virtual bool has_dead_oop(nmethod* const nm) const {

Is there a reason why this was changed to `nmethod* const nm` instead of `nmethod* nm`? IsUnloadingBehviour::has_dead_oop uses `nmethod* nm`. This question applies to the other changes in this file as well.

src/hotspot/share/gc/x/xUnload.cpp line 78:

> 76: class XIsUnloadingBehaviour : public IsUnloadingBehaviour {
> 77: public:
> 78:   virtual bool has_dead_oop(nmethod* const nm) const {

`nmethod* const nm` => `nmethod* nm`. (ZGC's style is to use const for local variables, but not for variables in the parameter list). The same applies to the rest of the changes to this file.

src/hotspot/share/gc/z/zUnload.cpp line 77:

> 75: class ZIsUnloadingBehaviour : public IsUnloadingBehaviour {
> 76: public:
> 77:   virtual bool has_dead_oop(nmethod* const nm) const {

`nmethod* const nm` => `nmethod* nm`. (ZGC's style is to use const for local variables, but not for variables in the parameter list). The same applies to the rest of the changes to this file.

src/hotspot/share/runtime/javaThread.hpp line 123:

> 121:   DeoptResourceMark*  _deopt_mark;               // Holds special ResourceMark for deoptimization
> 122: 
> 123:   nmethod*       _deopt_nmethod;                 // nmethod that is currently being deoptimized

The alignment is (and was) weird here.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18554#pullrequestreview-1978954058
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1551107567
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1551073461
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1551077362
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1551080101
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1551080280
PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1551100400

From ysr at openjdk.org  Thu Apr  4 07:57:41 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 4 Apr 2024 07:57:41 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial [v3]
In-Reply-To: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
Message-ID: <1-4_A2bh5rjdObTYBdBXvQrp4sspJM1peFb4ObS-Ij0=.eb667cb7-6315-4bf5-85c3-19a4dc4b660d@github.com>

> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
> 
> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
> 
> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
> 
> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
> 
> **Testing**:
> - [x] GHA : there are crashes unrelated to this change that are being addressed in other PRs
> - [ ] Code pipeline testing : there is a crash, unrelated to this change, that needs investigation
> - [x] SPECjbb
> 
> **Performance**:
> - [x] SPECjbb : the variance in tests appears to fail significance under 2-tailed Mann-Whitney
> - [ ] SPECjbb (patched for wk ref issue) : TBD
> - [ ] Extremem : in progress

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits:

 - Merge branch 'master' into clear_census
 - Merge branch 'master' into clear_census
 - Remove local_reset of age_census object inadvertently left behind in the
   GLOBAL gen concurrent mark, which was triggering the newly added
   reasonableness assert (yay!).
 - Avoid divide-by-zero.
 - Merge branch 'master' into clear_census
 - Fix word and byte size unit confusion in comparison/assert.
 - Loose verification of "reasonable completeness" of census.
 - Fix release build.
 - Support for verification of census' reasonableness, but no verification
   itself yet.
 - ShenandoahGlobalGeneration also resets local age census data in
   prepare_gc() for the generational case.
 - ... and 5 more: https://git.openjdk.org/shenandoah/compare/f43034d8...7cc4cbc5

-------------

Changes: https://git.openjdk.org/shenandoah/pull/403/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=403&range=02
  Stats: 167 lines in 11 files changed: 122 ins; 27 del; 18 mod
  Patch: https://git.openjdk.org/shenandoah/pull/403.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/403/head:pull/403

PR: https://git.openjdk.org/shenandoah/pull/403

From kdnilsen at openjdk.org  Thu Apr  4 15:32:05 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 4 Apr 2024 15:32:05 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v6]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <EceXsAE6gdlyE7-w00aQxM7rmZ0Fkt9PBOD2QIXdNw0=.7a66381f-3f1b-42a9-b5ec-d5d819bedb76@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Move trigger code to ShenandoahOldHeuristics from ShenandoahOldGeneration

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/409/files
  - new: https://git.openjdk.org/shenandoah/pull/409/files/e9fcdc16..b333414d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=05
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=04-05

  Stats: 115 lines in 5 files changed: 56 ins; 56 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From kdnilsen at openjdk.org  Thu Apr  4 15:32:07 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 4 Apr 2024 15:32:07 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v5]
In-Reply-To: <_Vk9JRksaktAcBJNgK1sLPR0Q0-7_cuSwbI9UBDFQNA=.ee4a4f87-bf86-4f5a-8273-8d95d552c5e0@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <BFuizLduIbx5NeR5F_0MVUqDCbZJsCbGGrzVESBsS8w=.5848ac32-fcb0-4a0e-94ce-b76f34fa2629@github.com>
 <_Vk9JRksaktAcBJNgK1sLPR0Q0-7_cuSwbI9UBDFQNA=.ee4a4f87-bf86-4f5a-8273-8d95d552c5e0@github.com>
Message-ID: <wUxMlHFEYG7GoigGdppXZqxnQQjHmK53oCsVcICA1Ac=.39901d96-0b22-4393-9770-6194445cd0b6@github.com>

On Mon, 1 Apr 2024 15:43:14 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix another typo
>
> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.hpp line 228:
> 
>> 226:   void set_live_bytes_after_last_mark(size_t new_live);
>> 227: 
>> 228:   void trigger_collection_if_fragmented(ShenandoahGenerationalHeap* gen_heap, size_t first_old_region, size_t last_old_region,
> 
> Move to ShenandoahOldHeuristics?

Thanks.  I've made this change.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1551917964

From kvn at openjdk.org  Thu Apr  4 15:59:12 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 15:59:12 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
 <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
Message-ID: <Xf-Cp5vIv_E5-s2t0EA0teRwEIZRLQZKAdMhDL3q41I=.640dd40f-a7df-45f1-afc5-183799e60268@github.com>

On Thu, 4 Apr 2024 07:26:21 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Address comments
>>  - Merge branch 'master' into 8329332
>>  - Removed not_used state of nmethod
>>  - remove trailing whitespace
>>  - 8329332: Remove CompiledMethod and CodeBlobLayout classes
>
> src/hotspot/share/gc/shared/gcBehaviours.hpp line 31:
> 
>> 29: #include "oops/oopsHierarchy.hpp"
>> 30: 
>> 31: // This is the behaviour for checking if a nmethod is unloading
> 
> Maybe this should be *an* nmethod?

Quote: "an" goes before words that begin with vowels.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1551989120

From kvn at openjdk.org  Thu Apr  4 16:02:12 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 16:02:12 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
 <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
Message-ID: <SI7n85EeSzaow8NeTQwHFAdV1OS8F7go4oMnO7PgRvM=.ff194cd7-a116-4128-8584-706596151ef0@github.com>

On Thu, 4 Apr 2024 07:31:24 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Address comments
>>  - Merge branch 'master' into 8329332
>>  - Removed not_used state of nmethod
>>  - remove trailing whitespace
>>  - 8329332: Remove CompiledMethod and CodeBlobLayout classes
>
> src/hotspot/share/gc/x/xUnload.cpp line 78:
> 
>> 76: class XIsUnloadingBehaviour : public IsUnloadingBehaviour {
>> 77: public:
>> 78:   virtual bool has_dead_oop(nmethod* const nm) const {
> 
> `nmethod* const nm` => `nmethod* nm`. (ZGC's style is to use const for local variables, but not for variables in the parameter list). The same applies to the rest of the changes to this file.

Okay. I did not know that it is only used for locals. I will update code.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1551997411

From stefank at openjdk.org  Thu Apr  4 16:06:14 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 4 Apr 2024 16:06:14 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <Xf-Cp5vIv_E5-s2t0EA0teRwEIZRLQZKAdMhDL3q41I=.640dd40f-a7df-45f1-afc5-183799e60268@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
 <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
 <Xf-Cp5vIv_E5-s2t0EA0teRwEIZRLQZKAdMhDL3q41I=.640dd40f-a7df-45f1-afc5-183799e60268@github.com>
Message-ID: <GgtzJ6gh2esPs1V2Q5tv4MWk4sSoSGxl7dFNXAk6MdU=.88bb7310-9994-4ddc-96fb-00b2947ddd6c@github.com>

On Thu, 4 Apr 2024 15:56:34 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> src/hotspot/share/gc/shared/gcBehaviours.hpp line 31:
>> 
>>> 29: #include "oops/oopsHierarchy.hpp"
>>> 30: 
>>> 31: // This is the behaviour for checking if a nmethod is unloading
>> 
>> Maybe this should be *an* nmethod?
>
> Quote: "an" goes before words that begin with vowels.

I don't think that holds if the 'n' is pronounced the way nmethod is pronounced.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1552005187

From kvn at openjdk.org  Thu Apr  4 16:09:02 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 16:09:02 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
 <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
Message-ID: <v4khYgI53YgwsNpMUaDWBzR7HAUUBly0tY400kpY1uQ=.32d6b2a2-94a5-4084-9c6c-27276b116e67@github.com>

On Thu, 4 Apr 2024 07:51:47 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Address comments
>>  - Merge branch 'master' into 8329332
>>  - Removed not_used state of nmethod
>>  - remove trailing whitespace
>>  - 8329332: Remove CompiledMethod and CodeBlobLayout classes
>
> src/hotspot/share/code/codeBlob.hpp line 168:
> 
>> 166:   bool is_vtable_blob() const                 { return _kind == CodeBlobKind::Blob_Vtable; }
>> 167:   bool is_method_handles_adapter_blob() const { return _kind == CodeBlobKind::Blob_MH_Adapter; }
>> 168:   bool is_upcall_stub() const                 { return _kind == CodeBlobKind::Blob_Upcall; }
> 
> The `Blob_` prefix is now redundant since we always have to prefix with CodeBlobKind::. Just a suggestion if you want to shorten these.

Good suggestion

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1552009581

From kvn at openjdk.org  Thu Apr  4 16:20:10 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 16:20:10 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <9Lk2-DK1nYNPIyXGbhqsr2DfsaR8mQsD9qEevogrW-U=.036ec57b-5fd4-4711-a781-6139f58d419f@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
 <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
 <Xf-Cp5vIv_E5-s2t0EA0teRwEIZRLQZKAdMhDL3q41I=.640dd40f-a7df-45f1-afc5-183799e60268@github.com>
 <GgtzJ6gh2esPs1V2Q5tv4MWk4sSoSGxl7dFNXAk6MdU=.88bb7310-9994-4ddc-96fb-00b2947ddd6c@github.com>
 <9Lk2-DK1nYNPIyXGbhqsr2DfsaR8mQsD9qEevogrW-U=.036ec57b-5fd4-4711-a781-6139f58d419f@github.com>
Message-ID: <KOrVRmdv0Wj_fgPPoqmOqI6GCxtZS0QFQe8_pflBlSs=.5f2b2b0b-fae8-4494-bf2b-4595319e1018@github.com>

On Thu, 4 Apr 2024 16:16:41 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> I don't think that holds if the 'n' is pronounced the way nmethod is pronounced.
>
> `grep` shows that we have both cases but `an nmethod` is used more.

I will fix it here as you suggested but I am not touching other places.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1552025626

From kvn at openjdk.org  Thu Apr  4 16:20:10 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 16:20:10 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <GgtzJ6gh2esPs1V2Q5tv4MWk4sSoSGxl7dFNXAk6MdU=.88bb7310-9994-4ddc-96fb-00b2947ddd6c@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
 <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
 <Xf-Cp5vIv_E5-s2t0EA0teRwEIZRLQZKAdMhDL3q41I=.640dd40f-a7df-45f1-afc5-183799e60268@github.com>
 <GgtzJ6gh2esPs1V2Q5tv4MWk4sSoSGxl7dFNXAk6MdU=.88bb7310-9994-4ddc-96fb-00b2947ddd6c@github.com>
Message-ID: <9Lk2-DK1nYNPIyXGbhqsr2DfsaR8mQsD9qEevogrW-U=.036ec57b-5fd4-4711-a781-6139f58d419f@github.com>

On Thu, 4 Apr 2024 16:03:12 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Quote: "an" goes before words that begin with vowels.
>
> I don't think that holds if the 'n' is pronounced the way nmethod is pronounced.

`grep` shows that we have both cases but `an nmethod` is used more.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18554#discussion_r1552024067

From kvn at openjdk.org  Thu Apr  4 17:04:30 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 17:04:30 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v4]
In-Reply-To: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
Message-ID: <2t_Et7WG-YB8Jvu9c3JIByOUM59BUo3DhSORCYFBZbY=.f808def0-15f1-4caa-aa3d-2b9b998b459f@github.com>

> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
> 
> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
> 
> I do not see (and not expected) performance difference with these changes.
> 
> Tested tier1-5, xcomp, stress. Running performance testing.
> 
> I need help with testing on platforms which Oracle does not support.

Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:

  Address next round of comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18554/files
  - new: https://git.openjdk.org/jdk/pull/18554/files/33768fb2..0c18ff17

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18554&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18554&range=02-03

  Stats: 80 lines in 13 files changed: 0 ins; 0 del; 80 mod
  Patch: https://git.openjdk.org/jdk/pull/18554.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18554/head:pull/18554

PR: https://git.openjdk.org/jdk/pull/18554

From kvn at openjdk.org  Thu Apr  4 17:04:31 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 17:04:31 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v3]
In-Reply-To: <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2sg6I-HBI12rc2LoWYX-A1S5vfMfDyj_5xoykANrZ8g=.6d0e5daa-30e4-45df-990e-c45b63477182@github.com>
 <KGnuaPm9C_Z4xttuvnu0xmyhxK36q4JNO7Z6Ju7BNW4=.29a0b379-3ece-473f-b255-8ec9b22bf680@github.com>
Message-ID: <3IuWX7uxBB3NeXD5nJiBZbh7jr4cxE6pkPr0TXemwag=.b4ad78d3-02eb-435e-a92e-13662779618e@github.com>

On Thu, 4 Apr 2024 07:54:16 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Address comments
>>  - Merge branch 'master' into 8329332
>>  - Removed not_used state of nmethod
>>  - remove trailing whitespace
>>  - 8329332: Remove CompiledMethod and CodeBlobLayout classes
>
> I took a second pass over the changes. I've given a few suggestions below. None of them should require respinning of tests (except for making sure that this still builds).

Thank you, @stefank , for second round of review. I addressed all your comments.
I also removed `virtual` from `virtual method() override;` as you suggested before since I touched these files again.
I will wait result of new GHA and tier1 in mach5 before integration.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2037731476

From stefank at openjdk.org  Thu Apr  4 17:34:10 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 4 Apr 2024 17:34:10 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v4]
In-Reply-To: <2t_Et7WG-YB8Jvu9c3JIByOUM59BUo3DhSORCYFBZbY=.f808def0-15f1-4caa-aa3d-2b9b998b459f@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2t_Et7WG-YB8Jvu9c3JIByOUM59BUo3DhSORCYFBZbY=.f808def0-15f1-4caa-aa3d-2b9b998b459f@github.com>
Message-ID: <wwlsEVmTe9YMETXRsjC0h7zpNtzek1sdUUEJmAJ6ytY=.6ba8b111-8f48-4e4e-9080-983ab12c1a8e@github.com>

On Thu, 4 Apr 2024 17:04:30 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
>> 
>> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
>> 
>> I do not see (and not expected) performance difference with these changes.
>> 
>> Tested tier1-5, xcomp, stress. Running performance testing.
>> 
>> I need help with testing on platforms which Oracle does not support.
>
> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address next round of comments

Looks good.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18554#pullrequestreview-1980693606

From wkemper at openjdk.org  Thu Apr  4 18:20:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 4 Apr 2024 18:20:43 GMT
Subject: RFR: 8329699: GenShen: Move promotion logic out of shHeap and
 shHeapRegion
Message-ID: <G9ZJV5hPgMHKOgnZjpXYbkMfdeSGA3nabJOCQh_4-Fo=.62da6e6e-16bd-44b2-b4fb-f10a5a4eb676@github.com>

The code to handle promotions during evacuation has been moved into a new class in its own files: `shGenerationalEvacuationTask`

The methods for managing `plabs` have been moved into `shGenerationalHeap`. `plabs` are now also only created when running the generational mode.


-  inline HeapWord* allocate_from_plab(Thread* thread, size_t size, bool is_promotion);
-  HeapWord* allocate_from_plab_slow(Thread* thread, size_t size, bool is_promotion);
-  HeapWord* allocate_new_plab(size_t min_size, size_t word_size, size_t* actual_size);
-  void retire_plab(PLAB* plab);
-  void retire_plab(PLAB* plab, Thread* thread);


The code to handle promotion in place has been moved from `shHeapRegion` into `shGenerationalEvacuationTask`:

-  void promote_humongous();
-  void promote_in_place();


Unused methods related to global coalesce and fill have been deleted from `shHeapRegion`:

-  void global_oop_iterate_and_fill_dead(OopIterateClosure* cl);
-  void global_oop_iterate_objects_and_fill_dead(OopIterateClosure* cl);
-  void oop_iterate_humongous(OopIterateClosure* cl);
-  void oop_iterate_humongous(OopIterateClosure* cl, HeapWord* start, size_t words);


Additionally, these methods in `shHeap` which just delegated to old or young generation instances have more or less been inlined at the usage site (many `includes` have been fixed):

-  ShenandoahOldHeuristics* old_heuristics();
-  ShenandoahYoungHeuristics* young_heuristics();
-
-  bool doing_mixed_evacuations();
-  bool is_old_bitmap_stable() const;
-  bool is_gc_generation_young() const;


The boolean `is_promotion` that was passed through all of the allocation calls has been moved into `shAllocRequest` to reduce the upstream delta for `shHeap`.

-------------

Commit messages:
 - Restore lost copyright comment in new file
 - Merge remote-tracking branch 'shenandoah/master' into shenandoah-heap-isolation
 - Restore preserved marks before rebuilding free set
 - Make evacuate_object virtual
 - Only create PLABs for generational mode
 - Move over plab retirement methods
 - Fix zero build
 - Move plab allocations out of shenandoahHeap
 - Move region promotion out of shenandoahHeapRegion
 - Separate generational evacuation/promotion methods
 - ... and 13 more: https://git.openjdk.org/shenandoah/compare/f43034d8...e398ce38

Changes: https://git.openjdk.org/shenandoah/pull/415/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=415&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329699
  Stats: 1875 lines in 36 files changed: 968 ins; 791 del; 116 mod
  Patch: https://git.openjdk.org/shenandoah/pull/415.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/415/head:pull/415

PR: https://git.openjdk.org/shenandoah/pull/415

From wkemper at openjdk.org  Thu Apr  4 18:40:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 4 Apr 2024 18:40:25 GMT
Subject: RFR: 8329699: GenShen: Move promotion logic out of shHeap and
 shHeapRegion [v2]
In-Reply-To: <G9ZJV5hPgMHKOgnZjpXYbkMfdeSGA3nabJOCQh_4-Fo=.62da6e6e-16bd-44b2-b4fb-f10a5a4eb676@github.com>
References: <G9ZJV5hPgMHKOgnZjpXYbkMfdeSGA3nabJOCQh_4-Fo=.62da6e6e-16bd-44b2-b4fb-f10a5a4eb676@github.com>
Message-ID: <9GiLUn4uHDiTw4ZeLPh8R5TtV5pWVywDzlVaQ7XdKq8=.e90a858a-28f2-4075-a659-a5b219d785d6@github.com>

> The code to handle promotions during evacuation has been moved into a new class in its own files: `shGenerationalEvacuationTask`
> 
> The methods for managing `plabs` have been moved into `shGenerationalHeap`. `plabs` are now also only created when running the generational mode.
> 
> 
> -  inline HeapWord* allocate_from_plab(Thread* thread, size_t size, bool is_promotion);
> -  HeapWord* allocate_from_plab_slow(Thread* thread, size_t size, bool is_promotion);
> -  HeapWord* allocate_new_plab(size_t min_size, size_t word_size, size_t* actual_size);
> -  void retire_plab(PLAB* plab);
> -  void retire_plab(PLAB* plab, Thread* thread);
> 
> 
> The code to handle promotion in place has been moved from `shHeapRegion` into `shGenerationalEvacuationTask`:
> 
> -  void promote_humongous();
> -  void promote_in_place();
> 
> 
> Unused methods related to global coalesce and fill have been deleted from `shHeapRegion`:
> 
> -  void global_oop_iterate_and_fill_dead(OopIterateClosure* cl);
> -  void global_oop_iterate_objects_and_fill_dead(OopIterateClosure* cl);
> -  void oop_iterate_humongous(OopIterateClosure* cl);
> -  void oop_iterate_humongous(OopIterateClosure* cl, HeapWord* start, size_t words);
> 
> 
> Additionally, these methods in `shHeap` which just delegated to old or young generation instances have more or less been inlined at the usage site (many `includes` have been fixed):
> 
> -  ShenandoahOldHeuristics* old_heuristics();
> -  ShenandoahYoungHeuristics* young_heuristics();
> -
> -  bool doing_mixed_evacuations();
> -  bool is_old_bitmap_stable() const;
> -  bool is_gc_generation_young() const;
> 
> 
> The boolean `is_promotion` that was passed through all of the allocation calls has been moved into `shAllocRequest` to reduce the upstream delta for `shHeap`.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix mac build

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/415/files
  - new: https://git.openjdk.org/shenandoah/pull/415/files/e398ce38..8140ce9b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=415&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=415&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/415.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/415/head:pull/415

PR: https://git.openjdk.org/shenandoah/pull/415

From kvn at openjdk.org  Thu Apr  4 19:26:11 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 19:26:11 GMT
Subject: RFR: 8329332: Remove CompiledMethod and CodeBlobLayout classes
 [v4]
In-Reply-To: <wwlsEVmTe9YMETXRsjC0h7zpNtzek1sdUUEJmAJ6ytY=.6ba8b111-8f48-4e4e-9080-983ab12c1a8e@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
 <2t_Et7WG-YB8Jvu9c3JIByOUM59BUo3DhSORCYFBZbY=.f808def0-15f1-4caa-aa3d-2b9b998b459f@github.com>
 <wwlsEVmTe9YMETXRsjC0h7zpNtzek1sdUUEJmAJ6ytY=.6ba8b111-8f48-4e4e-9080-983ab12c1a8e@github.com>
Message-ID: <K_-SZ1cVwrt8x-E8kWWOyJTXGWqoaycR5Afl-DJmZJ8=.c787220a-cba8-41c5-ac5c-bfdab54b09a2@github.com>

On Thu, 4 Apr 2024 17:31:53 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Address next round of comments
>
> Looks good.

Thank you, @stefank, @iwanowww, @dean-long  and @xmas92 for reviews and @RealFYang and @offamitkumar  for testing.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18554#issuecomment-2038043339

From wkemper at openjdk.org  Thu Apr  4 19:30:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 4 Apr 2024 19:30:45 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <SVEFHy342xY7V0EteTxnF5m-D_tOu83VpQnG9FgPW_4=.7e8c45ed-bf24-4e17-ab06-291385ac301a@github.com>

Resolved build issue with changes to `ageTable` API: https://github.com/openjdk/shenandoah/pull/416/files#diff-bccc00e8e28e415d1eaeac169ebb325ff962d7adc171e821e537c2c7f88f848d

-------------

Commit messages:
 - Merge tag 'jdk-23+17' into merge-jdk-23-17
 - 8328786: [AIX] move some important warnings/errors from trcVerbose to UL
 - 8327110: Refactor create_bool_from_template_assertion_predicate() to separate class and fix identical cloning cases used for Loop Unswitching and Split If
 - 8328702: C2: Crash during parsing because sub type check is not folded
 - 8326962: C2 SuperWord: cache VPointer
 - 8328938: C2 SuperWord: disable vectorization for large stride and scale
 - 8329494: Serial: Merge GenMarkSweep into MarkSweep
 - 8329470: Remove obsolete CDS SharedStrings tests
 - 8329564: [JVMCI] TranslatedException::debugPrintStackTrace does not work in the libjvmci compiler.
 - 8328957: Update PKCS11Test.java to not use hardcoded path
 - ... and 66 more: https://git.openjdk.org/shenandoah/compare/f43034d8...48bcc36e

The webrevs contain the adjustments done while merging with regards to each parent branch:
 - master: https://webrevs.openjdk.org/?repo=shenandoah&pr=416&range=00.0
 - openjdk/jdk:master: https://webrevs.openjdk.org/?repo=shenandoah&pr=416&range=00.1

Changes: https://git.openjdk.org/shenandoah/pull/416/files
  Stats: 12378 lines in 259 files changed: 4730 ins; 5658 del; 1990 mod
  Patch: https://git.openjdk.org/shenandoah/pull/416.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/416/head:pull/416

PR: https://git.openjdk.org/shenandoah/pull/416

From kvn at openjdk.org  Thu Apr  4 19:52:16 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 4 Apr 2024 19:52:16 GMT
Subject: Integrated: 8329332: Remove CompiledMethod and CodeBlobLayout classes
In-Reply-To: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
References: <ImQruaXQMD_OppiLZDYuic6PsiEeXofKOzFAE1XrD0s=.c9d0a2ee-ef41-4e4e-82eb-135405111962@github.com>
Message-ID: <M799FP-Rh1GHLXV8bsMJUqYC-7vjmlLw1HfkX1rkhZM=.02a0218d-952a-4546-9e5d-42aeaea051c6@github.com>

On Fri, 29 Mar 2024 19:35:45 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

> Revert [JDK-8152664](https://bugs.openjdk.org/browse/JDK-8152664) RFE [changes](https://github.com/openjdk/jdk/commit/b853eb7f5ca24eeeda18acbb14287f706499c365) which was used for AOT [JEP 295](https://openjdk.org/jeps/295) implementation in JDK 9. The code was left in HotSpot assuming it will help in a future. But during work on Leyden we decided to not use it. In Leyden cached compiled code will be restored in CodeCache as normal nmethods: no need to change VM's runtime and GC code to process them.
> 
> I may work on optimizing `CodeBlob` and `nmethod` fields layout to reduce header size in separate changes. In these changes I did simple fields reordering to keep small (1 byte) fields together.
> 
> I do not see (and not expected) performance difference with these changes.
> 
> Tested tier1-5, xcomp, stress. Running performance testing.
> 
> I need help with testing on platforms which Oracle does not support.

This pull request has now been integrated.

Changeset: 83eba863
Author:    Vladimir Kozlov <kvn at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/83eba863fec5ee7e30c4f9b11122ad1deed3d2ec
Stats:     3941 lines in 119 files changed: 1287 ins; 1753 del; 901 mod

8329332: Remove CompiledMethod and CodeBlobLayout classes

Reviewed-by: vlivanov, stefank

-------------

PR: https://git.openjdk.org/jdk/pull/18554

From cslucas at openjdk.org  Thu Apr  4 23:27:13 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 4 Apr 2024 23:27:13 GMT
Subject: RFR: JDK-8241503: C2: Share MacroAssembler between mach nodes
 during code emission [v11]
In-Reply-To: <zDAtg_h3PMKma0jA6Phr7_Cu-xbERmW8QMzmsbh6_WI=.7ac7920b-7288-4d0a-817a-d3184a47ab08@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <dQZjFk-E_tYaTTg7CzDj_bFhPb4RHsI6EKlga4vsNbQ=.9339a5c7-72c5-425c-b90b-6261d53f5a58@github.com>
 <zDAtg_h3PMKma0jA6Phr7_Cu-xbERmW8QMzmsbh6_WI=.7ac7920b-7288-4d0a-817a-d3184a47ab08@github.com>
Message-ID: <kIJgaJB9Z0Oh1R95WrBBiEMR6aXFuZmNzxDq2CV8Tsg=.7d6b17a1-8aaf-4648-82b5-19db300db1e8@github.com>

On Wed, 27 Mar 2024 16:12:26 GMT, Boris Ulasevich <bulasevich at openjdk.org> wrote:

>> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into reuse-macroasm
>>  - Fix AArch64 build & improve comment about InstructionMark
>>  - Catching up with changes in master
>>  - Catching up with origin/master
>>  - Catch up with origin/master
>>  - Merge with origin/master
>>  - Fix build, copyright dates, m4 files.
>>  - Fix merge
>>  - Catch up with master branch.
>>    
>>    Merge remote-tracking branch 'origin/master' into reuse-macroasm
>>  - Some inst_mark fixes; Catch up with master.
>>  - ... and 2 more: https://git.openjdk.org/jdk/compare/89e0889a...b4d73c98
>
> FYI. Something goes wrong with the change on ARM32.
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/ws/workspace/jdk-dev/label/linux-arm/type/b11/jdk/src/hotspot/share/asm/codeBuffer.hpp:163), pid=10782, tid=10796
> #  assert(_mark != nullptr) failed: not an offset
> #
> # JRE version: OpenJDK Runtime Environment (23.0) (fastdebug build 23-commit8fc9097b-adhoc.re.jdk)
> # Java VM: OpenJDK Server VM (fastdebug 23-commit8fc9097b-adhoc.re.jdk, mixed mode, g1 gc, linux-arm)
> # Problematic frame:
> # V  [libjvm.so+0x136ccc]  emit_call_reloc(C2_MacroAssembler*, MachCallNode const*, MachOper*, RelocationHolder const&)+0x2ac
> #
> # Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P %E" (or dumping to /ws/workspace/jdk-dev-jtreg/label/linux-arm/suite/jdk-tier1/type/t11/core.10782)
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> ---------------  S U M M A R Y ------------
> 
> Command Line:
> 
> Host: vm-ubuntu-16v4-aarch64-1, ARM, 4 cores, 7G, Ubuntu 16.04.7 LTS
> Time: Wed Mar 27 07:16:41 2024 UTC elapsed time: 0.097440 seconds (0d 0h 0m 0s)
> 
> ---------------  T H R E A D  ---------------
> 
> Current thread (0xb120cd10):  JavaThread "C2 CompilerThread0" daemon [_thread_in_vm, id=10796, stack(0xb1090000,0xb1110000) (512K)]
> 
> Stack: [0xb1090000,0xb1110000],  sp=0xb110d200,  free space=500k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V  [libjvm.so+0x136ccc]  emit_call_reloc(C2_MacroAssembler*, MachCallNode const*, MachOper*, RelocationHolder const&)+0x2ac  (codeBuffer.hpp:163)
> V  [libjvm.so+0x14ca28]  CallRuntimeDirectNode::emit(C2_MacroAssembler*, PhaseRegAlloc*) const+0x80
> V  [libjvm.so+0x10ed850]  PhaseOutput::scratch_emit_size(Node const*)+0x37c
> V  [libjvm.so+0x10e5f64]  PhaseOutput::shorten_branches(unsigned int*)+0x274
> V  [libjvm.so+0x10f6dcc]  PhaseOutput::Output()+0x488
> V  [libjvm.so+0x699b54]  Compile::Code_Gen()+0x424
> V  [libjvm.so+0x69a87c]  Compile::Compile(ciEnv*, TypeFunc const* (*)(), unsigned char*, char const*, int, bool, bool, DirectiveSet*)+0xb3c
> V  [libjvm.so+0x122e73c]  OptoRuntime::generate_stub(ciEnv*, TypeFunc const* (*)(), unsigned char*, char const*, int, bool, bool)+0xb4
> V  [libjvm.so+0x122ec68]  OptoRuntime::generate(ciEnv*)+0x50
> V  [libjvm.so+0x4994cc]  C2Compiler::init_c2_runtime()+0x104
> V  [libjvm.so+0x4996dc]  C2Compiler::initialize()+0x9c
> V  [libjvm.so+...

@bulasevich - Is the test that failed one of JDK jtreg tests? Did you include any additional JVM parameter to run the test?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16484#issuecomment-2038435787

From bulasevich at openjdk.org  Fri Apr  5 02:11:13 2024
From: bulasevich at openjdk.org (Boris Ulasevich)
Date: Fri, 5 Apr 2024 02:11:13 GMT
Subject: RFR: JDK-8241503: C2: Share MacroAssembler between mach nodes
 during code emission [v11]
In-Reply-To: <zDAtg_h3PMKma0jA6Phr7_Cu-xbERmW8QMzmsbh6_WI=.7ac7920b-7288-4d0a-817a-d3184a47ab08@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <dQZjFk-E_tYaTTg7CzDj_bFhPb4RHsI6EKlga4vsNbQ=.9339a5c7-72c5-425c-b90b-6261d53f5a58@github.com>
 <zDAtg_h3PMKma0jA6Phr7_Cu-xbERmW8QMzmsbh6_WI=.7ac7920b-7288-4d0a-817a-d3184a47ab08@github.com>
Message-ID: <h9CaNBUHEw4uttMJH2CyNXSCK_tcR3S9lZM7KjFp8oI=.79b41ee1-cf2e-4dd5-88ad-a5250fef81a2@github.com>

On Wed, 27 Mar 2024 16:12:26 GMT, Boris Ulasevich <bulasevich at openjdk.org> wrote:

>> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into reuse-macroasm
>>  - Fix AArch64 build & improve comment about InstructionMark
>>  - Catching up with changes in master
>>  - Catching up with origin/master
>>  - Catch up with origin/master
>>  - Merge with origin/master
>>  - Fix build, copyright dates, m4 files.
>>  - Fix merge
>>  - Catch up with master branch.
>>    
>>    Merge remote-tracking branch 'origin/master' into reuse-macroasm
>>  - Some inst_mark fixes; Catch up with master.
>>  - ... and 2 more: https://git.openjdk.org/jdk/compare/89e0889a...b4d73c98
>
> FYI. Something goes wrong with the change on ARM32.
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/ws/workspace/jdk-dev/label/linux-arm/type/b11/jdk/src/hotspot/share/asm/codeBuffer.hpp:163), pid=10782, tid=10796
> #  assert(_mark != nullptr) failed: not an offset
> #
> # JRE version: OpenJDK Runtime Environment (23.0) (fastdebug build 23-commit8fc9097b-adhoc.re.jdk)
> # Java VM: OpenJDK Server VM (fastdebug 23-commit8fc9097b-adhoc.re.jdk, mixed mode, g1 gc, linux-arm)
> # Problematic frame:
> # V  [libjvm.so+0x136ccc]  emit_call_reloc(C2_MacroAssembler*, MachCallNode const*, MachOper*, RelocationHolder const&)+0x2ac
> #
> # Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P %E" (or dumping to /ws/workspace/jdk-dev-jtreg/label/linux-arm/suite/jdk-tier1/type/t11/core.10782)
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> ---------------  S U M M A R Y ------------
> 
> Command Line:
> 
> Host: vm-ubuntu-16v4-aarch64-1, ARM, 4 cores, 7G, Ubuntu 16.04.7 LTS
> Time: Wed Mar 27 07:16:41 2024 UTC elapsed time: 0.097440 seconds (0d 0h 0m 0s)
> 
> ---------------  T H R E A D  ---------------
> 
> Current thread (0xb120cd10):  JavaThread "C2 CompilerThread0" daemon [_thread_in_vm, id=10796, stack(0xb1090000,0xb1110000) (512K)]
> 
> Stack: [0xb1090000,0xb1110000],  sp=0xb110d200,  free space=500k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V  [libjvm.so+0x136ccc]  emit_call_reloc(C2_MacroAssembler*, MachCallNode const*, MachOper*, RelocationHolder const&)+0x2ac  (codeBuffer.hpp:163)
> V  [libjvm.so+0x14ca28]  CallRuntimeDirectNode::emit(C2_MacroAssembler*, PhaseRegAlloc*) const+0x80
> V  [libjvm.so+0x10ed850]  PhaseOutput::scratch_emit_size(Node const*)+0x37c
> V  [libjvm.so+0x10e5f64]  PhaseOutput::shorten_branches(unsigned int*)+0x274
> V  [libjvm.so+0x10f6dcc]  PhaseOutput::Output()+0x488
> V  [libjvm.so+0x699b54]  Compile::Code_Gen()+0x424
> V  [libjvm.so+0x69a87c]  Compile::Compile(ciEnv*, TypeFunc const* (*)(), unsigned char*, char const*, int, bool, bool, DirectiveSet*)+0xb3c
> V  [libjvm.so+0x122e73c]  OptoRuntime::generate_stub(ciEnv*, TypeFunc const* (*)(), unsigned char*, char const*, int, bool, bool)+0xb4
> V  [libjvm.so+0x122ec68]  OptoRuntime::generate(ciEnv*)+0x50
> V  [libjvm.so+0x4994cc]  C2Compiler::init_c2_runtime()+0x104
> V  [libjvm.so+0x4996dc]  C2Compiler::initialize()+0x9c
> V  [libjvm.so+...

> @bulasevich - Is the test that failed one of JDK jtreg tests? Did you include any additional JVM parameter to run the test?

This happens on VM startup with empty params (no test).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16484#issuecomment-2038631560

From stefank at openjdk.org  Fri Apr  5 12:37:38 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 5 Apr 2024 12:37:38 GMT
Subject: RFR: 8329629: GC interfaces should work directly against nmethod
 instead of CodeBlob
Message-ID: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>

The GCs scan and handles nmethods and ignores CodeBlobs of other kinds. The I propose that we stop sending in CodeBlobs to the GCs and make sure to only give them nmethods.

I removed `void CodeCache::blobs_do(CodeBlobClosure* f)` since there's no more usage of that function. Is this OK?

I also opted to skipped calling the GC verification code from the iterator code:

Universe::heap()->verify_nmethod((nmethod*)cb);

IMHO, I think it is up to the GCs to decide if they want to perform extra nmethod verification. If someone wants to keep this verification in their favorite GC I can add calls to this function where we used to call CodeCache::blobs_do.

I've only done limited testing and will run extensive testing concurrent with the review.

-------------

Commit messages:
 - 8329629: GC interfaces should work directly against nmethod instead of CodeBlob

Changes: https://git.openjdk.org/jdk/pull/18653/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18653&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329629
  Stats: 850 lines in 74 files changed: 238 ins; 318 del; 294 mod
  Patch: https://git.openjdk.org/jdk/pull/18653.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18653/head:pull/18653

PR: https://git.openjdk.org/jdk/pull/18653

From wkemper at openjdk.org  Fri Apr  5 14:16:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 14:16:40 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <Nh0rDofxqylIpeLcW1XlgXhWtTAC-B7yDPLBe-4uh7U=.7e703f72-89be-4fb9-a845-2c9984ae5131@github.com>

Merges tag jdk-23+17

-------------

Commit messages:
 - 8328786: [AIX] move some important warnings/errors from trcVerbose to UL
 - 8327110: Refactor create_bool_from_template_assertion_predicate() to separate class and fix identical cloning cases used for Loop Unswitching and Split If
 - 8328702: C2: Crash during parsing because sub type check is not folded
 - 8326962: C2 SuperWord: cache VPointer
 - 8328938: C2 SuperWord: disable vectorization for large stride and scale
 - 8329494: Serial: Merge GenMarkSweep into MarkSweep
 - 8329470: Remove obsolete CDS SharedStrings tests
 - 8329564: [JVMCI] TranslatedException::debugPrintStackTrace does not work in the libjvmci compiler.
 - 8328957: Update PKCS11Test.java to not use hardcoded path
 - 8327474: Review use of java.io.tmpdir in jdk tests
 - ... and 65 more: https://git.openjdk.org/shenandoah/compare/d580bcf9...8efd7aa6

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=417&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/417/files
  Stats: 12373 lines in 258 files changed: 4730 ins; 5657 del; 1986 mod
  Patch: https://git.openjdk.org/shenandoah/pull/417.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/417/head:pull/417

PR: https://git.openjdk.org/shenandoah/pull/417

From wkemper at openjdk.org  Fri Apr  5 17:49:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 17:49:23 GMT
Subject: RFR: Merge openjdk/jdk:master
In-Reply-To: <Nh0rDofxqylIpeLcW1XlgXhWtTAC-B7yDPLBe-4uh7U=.7e703f72-89be-4fb9-a845-2c9984ae5131@github.com>
References: <Nh0rDofxqylIpeLcW1XlgXhWtTAC-B7yDPLBe-4uh7U=.7e703f72-89be-4fb9-a845-2c9984ae5131@github.com>
Message-ID: <d7CTsLhTHhSeiokGx0Um0wpDThm6VQlohQ8oI4PbY1o=.47a1b8c5-a793-4033-9173-4a178da7bedb@github.com>

On Fri, 5 Apr 2024 14:11:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+17

Abandoning in favor of https://github.com/openjdk/shenandoah/pull/416

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/417#issuecomment-2040327270

From wkemper at openjdk.org  Fri Apr  5 17:49:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 17:49:23 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <Nh0rDofxqylIpeLcW1XlgXhWtTAC-B7yDPLBe-4uh7U=.7e703f72-89be-4fb9-a845-2c9984ae5131@github.com>
References: <Nh0rDofxqylIpeLcW1XlgXhWtTAC-B7yDPLBe-4uh7U=.7e703f72-89be-4fb9-a845-2c9984ae5131@github.com>
Message-ID: <AUu38nWXyAVnZyAembjGF6zWUn-G76RWPFEl4tDhvkw=.704f6503-1a2b-4141-a125-2887143f72d2@github.com>

On Fri, 5 Apr 2024 14:11:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+17

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/417

From wkemper at openjdk.org  Fri Apr  5 17:49:49 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 17:49:49 GMT
Subject: RFR: 8329789: GenShen: Over assertive assert when scanning remembered
 set
Message-ID: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>

When scanning the remembered set for update-refs, it is possible for the scanner to encounter a card that was dirtied by a mutator during the scan. This is by design.

-------------

Commit messages:
 - Soften assertion when using the write card table

Changes: https://git.openjdk.org/shenandoah/pull/418/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=418&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329789
  Stats: 4 lines in 1 file changed: 3 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/418.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/418/head:pull/418

PR: https://git.openjdk.org/shenandoah/pull/418

From wkemper at openjdk.org  Fri Apr  5 21:20:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 21:20:20 GMT
Subject: RFR: 8329807: Shenandoah: Temporarily revert JDK-8324655
Message-ID: <ncI0BXpCxpwJ9ZS9G6KWduQoSLRbZQ0UTpksZdw16MY=.81e2c460-fcba-46b1-a626-a920601edc11@github.com>

No longer seeing crashes locally after backing this change out.

-------------

Commit messages:
 - Revert "8324655: Identify integer minimum and maximum patterns created with if statements"

Changes: https://git.openjdk.org/shenandoah/pull/419/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=419&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329807
  Stats: 708 lines in 7 files changed: 0 ins; 704 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/419.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/419/head:pull/419

PR: https://git.openjdk.org/shenandoah/pull/419

From kdnilsen at openjdk.org  Fri Apr  5 21:30:24 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 5 Apr 2024 21:30:24 GMT
Subject: RFR: 8329807: Shenandoah: Temporarily revert JDK-8324655
In-Reply-To: <ncI0BXpCxpwJ9ZS9G6KWduQoSLRbZQ0UTpksZdw16MY=.81e2c460-fcba-46b1-a626-a920601edc11@github.com>
References: <ncI0BXpCxpwJ9ZS9G6KWduQoSLRbZQ0UTpksZdw16MY=.81e2c460-fcba-46b1-a626-a920601edc11@github.com>
Message-ID: <wEMPAeWYGsZCNseXQ9gYrtEszMhH8SGvutjavuBV544=.cb88a7f3-d7b8-4c86-8f34-54a4e657fe1d@github.com>

On Fri, 5 Apr 2024 21:16:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

> No longer seeing crashes locally after backing this change out.

Thanks for identifying the problematic PR...

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/419#pullrequestreview-1984204925

From wkemper at openjdk.org  Fri Apr  5 21:34:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 21:34:30 GMT
Subject: RFR: 8329807: Shenandoah: Temporarily revert JDK-8324655
In-Reply-To: <ncI0BXpCxpwJ9ZS9G6KWduQoSLRbZQ0UTpksZdw16MY=.81e2c460-fcba-46b1-a626-a920601edc11@github.com>
References: <ncI0BXpCxpwJ9ZS9G6KWduQoSLRbZQ0UTpksZdw16MY=.81e2c460-fcba-46b1-a626-a920601edc11@github.com>
Message-ID: <bd6ZZyn4XUzc5UaiFNFQeXEl4pd6ZVfN9ahJv8HMO5Y=.5f414f21-671b-4f49-88eb-66896895f8ba@github.com>

On Fri, 5 Apr 2024 21:16:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

> No longer seeing crashes locally after backing this change out.

Really, all the credit goes to @caojoshua and @shipilev .

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/419#issuecomment-2040652848

From wkemper at openjdk.org  Fri Apr  5 21:34:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 21:34:30 GMT
Subject: Integrated: 8329807: Shenandoah: Temporarily revert JDK-8324655
In-Reply-To: <ncI0BXpCxpwJ9ZS9G6KWduQoSLRbZQ0UTpksZdw16MY=.81e2c460-fcba-46b1-a626-a920601edc11@github.com>
References: <ncI0BXpCxpwJ9ZS9G6KWduQoSLRbZQ0UTpksZdw16MY=.81e2c460-fcba-46b1-a626-a920601edc11@github.com>
Message-ID: <uTCj160BbAztAuLkOMq6po0Xz3eyGuEs0HHY1iKWWks=.07e081a2-f6c8-429c-a044-4239aad79411@github.com>

On Fri, 5 Apr 2024 21:16:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

> No longer seeing crashes locally after backing this change out.

This pull request has now been integrated.

Changeset: 11295a6a
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/11295a6a92570d7be639ab50a69690d099fc3b58
Stats:     708 lines in 7 files changed: 0 ins; 704 del; 4 mod

8329807: Shenandoah: Temporarily revert JDK-8324655

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/419

From wkemper at openjdk.org  Fri Apr  5 21:47:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 21:47:08 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <SVEFHy342xY7V0EteTxnF5m-D_tOu83VpQnG9FgPW_4=.7e8c45ed-bf24-4e17-ab06-291385ac301a@github.com>
References: <SVEFHy342xY7V0EteTxnF5m-D_tOu83VpQnG9FgPW_4=.7e8c45ed-bf24-4e17-ab06-291385ac301a@github.com>
Message-ID: <GtaI6c6G2DzJRpcqA8gmYG0bYFdGN_nzvzkExOOt8RM=.600bda9a-bc33-4c30-9d0b-de4179aee2dc@github.com>

> Resolved build issue with changes to `ageTable` API: https://github.com/openjdk/shenandoah/pull/416/files#diff-bccc00e8e28e415d1eaeac169ebb325ff962d7adc171e821e537c2c7f88f848d

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 77 commits:

 - Merge remote-tracking branch 'shenandoah/master' into merge-jdk-23-17
 - Merge tag 'jdk-23+17' into merge-jdk-23-17
   
   Added tag jdk-23+17 for changeset 8efd7aa6
 - 8328786: [AIX] move some important warnings/errors from trcVerbose to UL
   
   Reviewed-by: lucy, stuefe
 - 8327110: Refactor create_bool_from_template_assertion_predicate() to separate class and fix identical cloning cases used for Loop Unswitching and Split If
   
   Reviewed-by: epeter, kvn
 - 8328702: C2: Crash during parsing because sub type check is not folded
   
   Reviewed-by: roland, kvn
 - 8326962: C2 SuperWord: cache VPointer
   
   Reviewed-by: chagedorn, kvn
 - 8328938: C2 SuperWord: disable vectorization for large stride and scale
   
   Reviewed-by: chagedorn, kvn
 - 8329494: Serial: Merge GenMarkSweep into MarkSweep
   
   Reviewed-by: ihse, ayang, tschatzl
 - 8329470: Remove obsolete CDS SharedStrings tests
   
   Reviewed-by: ccheung
 - 8329564: [JVMCI] TranslatedException::debugPrintStackTrace does not work in the libjvmci compiler.
   
   Reviewed-by: dnsimon
 - ... and 67 more: https://git.openjdk.org/shenandoah/compare/11295a6a...254699d2

-------------

Changes: https://git.openjdk.org/shenandoah/pull/416/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=416&range=01
  Stats: 12867 lines in 259 files changed: 5219 ins; 5658 del; 1990 mod
  Patch: https://git.openjdk.org/shenandoah/pull/416.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/416/head:pull/416

PR: https://git.openjdk.org/shenandoah/pull/416

From wkemper at openjdk.org  Fri Apr  5 22:10:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 5 Apr 2024 22:10:37 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final mark
 are not made parseable
Message-ID: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>

Pinned regions that are not humongous may be added to the set of _candidates_ for mixed collection. They still may not be added to the collection set for a mixed collection. Regions that are not in this collection of _candidates_ will not be coalesced and filled.

-------------

Commit messages:
 - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
 - Allow pinned/regular regions into mixed collection candidates
 - Little clean up

Changes: https://git.openjdk.org/shenandoah/pull/420/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=420&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329790
  Stats: 60 lines in 3 files changed: 22 ins; 17 del; 21 mod
  Patch: https://git.openjdk.org/shenandoah/pull/420.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/420/head:pull/420

PR: https://git.openjdk.org/shenandoah/pull/420

From bulasevich at openjdk.org  Mon Apr  8 06:12:07 2024
From: bulasevich at openjdk.org (Boris Ulasevich)
Date: Mon, 8 Apr 2024 06:12:07 GMT
Subject: RFR: JDK-8241503: C2: Share MacroAssembler between mach nodes
 during code emission [v11]
In-Reply-To: <dQZjFk-E_tYaTTg7CzDj_bFhPb4RHsI6EKlga4vsNbQ=.9339a5c7-72c5-425c-b90b-6261d53f5a58@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <dQZjFk-E_tYaTTg7CzDj_bFhPb4RHsI6EKlga4vsNbQ=.9339a5c7-72c5-425c-b90b-6261d53f5a58@github.com>
Message-ID: <JOS3FkAoDAb2RHxJSGe4a14GlH60vDLOx3f-4Ugkq6E=.60d72071-60f6-47eb-b449-7f0c9266963a@github.com>

On Tue, 26 Mar 2024 19:02:42 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> # Description
>> 
>> Please review this PR with a patch to re-use the same C2_MacroAssembler object to emit all instructions in the same compilation unit.
>> 
>> Overall, the change is pretty simple. However, due to the renaming of the variable to access C2_MacroAssembler, from `_masm.` to `masm->`, and also some method prototype changes, the patch became quite large.
>> 
>> # Help Needed for Testing
>> 
>> I don't have access to all platforms necessary to test this. I hope some other folks can help with testing on `S390`, `RISC-V` and `PPC`.
>
> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits:
> 
>  - Merge remote-tracking branch 'origin/master' into reuse-macroasm
>  - Fix AArch64 build & improve comment about InstructionMark
>  - Catching up with changes in master
>  - Catching up with origin/master
>  - Catch up with origin/master
>  - Merge with origin/master
>  - Fix build, copyright dates, m4 files.
>  - Fix merge
>  - Catch up with master branch.
>    
>    Merge remote-tracking branch 'origin/master' into reuse-macroasm
>  - Some inst_mark fixes; Catch up with master.
>  - ... and 2 more: https://git.openjdk.org/jdk/compare/89e0889a...b4d73c98

Do you need help understanding the problem? The crash occurred because you removed the line `fprintf(fp, "  cbuf.set_insts_mark();\n");` from the generator of AD nodes ::emit() methods. That is why emit_call_reloc finds cbuf.insts->_mark unitialized.


// Call Runtime Instruction
instruct CallRuntimeDirect(method meth) %{
  match(CallRuntime);
  effect(USE meth);
  ins_cost(CALL_COST);
  format %{ "CALL,runtime" %}
  ins_encode( Java_To_Runtime( meth ),
              call_epilog );
  ins_pipe(simple_call);
%}

-->

void CallRuntimeDirectNode::emit(CodeBuffer& cbuf, PhaseRegAlloc* ra_) const {
  cbuf.set_insts_mark();
  // Start at oper_input_base() and count operands
  unsigned idx0 = 1;
  unsigned idx1 = 1;    //
  {
#line 1217 "/home/boris/jdk-bulasevich/src/hotspot/cpu/arm/arm.ad"
    // CALL directly to the runtime
    emit_call_reloc(cbuf, as_MachCall(), opnd_array(1), runtime_call_Relocation::spec());
#line 999999
  }
  {
#line 1213 "/home/boris/jdk-bulasevich/src/hotspot/cpu/arm/arm.ad"
    // nothing
#line 999999
  }
}

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16484#issuecomment-2041935047

From ayang at openjdk.org  Mon Apr  8 13:21:10 2024
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Mon, 8 Apr 2024 13:21:10 GMT
Subject: RFR: 8329629: GC interfaces should work directly against nmethod
 instead of CodeBlob
In-Reply-To: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
References: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
Message-ID: <SoqzGAfG1GV_P_D1nyQne3ad0iyW-KE2_W7AqHK3J0E=.4e6f96f0-4f5f-4539-a90b-c9d63d7669c5@github.com>

On Fri, 5 Apr 2024 12:32:30 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> The GCs scan and handles nmethods and ignores CodeBlobs of other kinds. The I propose that we stop sending in CodeBlobs to the GCs and make sure to only give them nmethods.
> 
> I removed `void CodeCache::blobs_do(CodeBlobClosure* f)` since there's no more usage of that function. Is this OK?
> 
> I also opted to skipped calling the GC verification code from the iterator code:
> 
> Universe::heap()->verify_nmethod((nmethod*)cb);
> 
> IMHO, I think it is up to the GCs to decide if they want to perform extra nmethod verification. If someone wants to keep this verification in their favorite GC I can add calls to this function where we used to call CodeCache::blobs_do.
> 
> I've only done limited testing and will run extensive testing concurrent with the review.

Marked as reviewed by ayang (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18653#pullrequestreview-1986431388

From cslucas at openjdk.org  Mon Apr  8 16:31:13 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Mon, 8 Apr 2024 16:31:13 GMT
Subject: RFR: JDK-8241503: C2: Share MacroAssembler between mach nodes
 during code emission [v11]
In-Reply-To: <JOS3FkAoDAb2RHxJSGe4a14GlH60vDLOx3f-4Ugkq6E=.60d72071-60f6-47eb-b449-7f0c9266963a@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <dQZjFk-E_tYaTTg7CzDj_bFhPb4RHsI6EKlga4vsNbQ=.9339a5c7-72c5-425c-b90b-6261d53f5a58@github.com>
 <JOS3FkAoDAb2RHxJSGe4a14GlH60vDLOx3f-4Ugkq6E=.60d72071-60f6-47eb-b449-7f0c9266963a@github.com>
Message-ID: <ineZAn4UYqIPjZL-MmKBekoPxKULeuFjknlWZ_PkrLY=.74d5be76-a9d9-409f-8815-2afc8e87a32d@github.com>

On Mon, 8 Apr 2024 06:09:14 GMT, Boris Ulasevich <bulasevich at openjdk.org> wrote:

> Do you need help understanding the problem?

It's hard for me to debug because I don't have direct access to an ARM32. However, I was able to reproduce the problem and I know the reason why it happens (its exactly what you described). I'm working now to find the correct locations to insert the `inst_marks()` _and_ the `clear_inst_marks()`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16484#issuecomment-2043183990

From kdnilsen at openjdk.org  Mon Apr  8 18:51:16 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 8 Apr 2024 18:51:16 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v32]
In-Reply-To: <U6tUCW9TF58aCdQCJU1ZQ993BWlojiwj145QCaQALkY=.ed211af5-2493-4f9d-89ed-256a609d14d3@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Mu9wTnVX6kIS89pKXTUOHQT2CB7P_m_kSWGQHOQU-ko=.77851b17-c53b-4b59-8a84-6d518a19d35b@github.com>
 <HgHiTh_t5gXIEQnpJ8EbuF3mVuuyGOOh0DFsikSmo9E=.906d3f45-9210-4c24-9817-720935389846@github.com>
 <-g5tscBZPsDgVrjfZnM-UBRy0_6qodnRN5_srPWunXk=.82eda809-141c-4bfb-82aa-e722eecd0469@github.com>
 <U6tUCW9TF58aCdQCJU1ZQ993BWlojiwj145QCaQALkY=.ed211af5-2493-4f9d-89ed-256a609d14d3@github.com>
Message-ID: <DgXhJP37I84BIzuXBZ9cItwhOzClFBVphjCPAIn0EWI=.9c18075e-5be5-4ffe-bc05-b57f58175eb8@github.com>

On Mon, 1 Apr 2024 16:06:31 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Was trying to avoid impacting beyond the boundaries of Shenandoah.  I can tackle this if we think it preferable.  But also trying to work toward "timely integration".
>
> Like Roman, I'd prefer extending existing code and/or making it adaptable for our purposes to duplicating functionality. Let us read through the differences and see if this should be done now, or as you state allow this duplication for now in the interests of speed and expediency.

Here's a rough outline of the differences between ShenandoahSimpleBitMap and src/hotspot/share/utilities/bitMap.*
1. ShenandoahSimpleBitMap finds next bit in the forward and backward direction.  BitMap only searches in forward direction.  We use backward searches when we prefer to pack certain types of objects into the high end of the heap.
2. ShenandoahSimpleBitMap also searches for clusters of bits, to facilitate efficient satisfaction of humongous allocation requests.  BitMap only searches for a single bit.  We could add this capability to BitMap.
4. If we were to generalize BitMap to handle reverse searches, this may have far-reaching impact on the API.   The type of results and idx_t (typedef size_t).  When we do backward searches, ShenandoahSimpleBitMap treats -1 as the sentinel NOT_FOUND value.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1556291273

From kdnilsen at openjdk.org  Mon Apr  8 18:51:16 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 8 Apr 2024 18:51:16 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v32]
In-Reply-To: <DgXhJP37I84BIzuXBZ9cItwhOzClFBVphjCPAIn0EWI=.9c18075e-5be5-4ffe-bc05-b57f58175eb8@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Mu9wTnVX6kIS89pKXTUOHQT2CB7P_m_kSWGQHOQU-ko=.77851b17-c53b-4b59-8a84-6d518a19d35b@github.com>
 <HgHiTh_t5gXIEQnpJ8EbuF3mVuuyGOOh0DFsikSmo9E=.906d3f45-9210-4c24-9817-720935389846@github.com>
 <-g5tscBZPsDgVrjfZnM-UBRy0_6qodnRN5_srPWunXk=.82eda809-141c-4bfb-82aa-e722eecd0469@github.com>
 <U6tUCW9TF58aCdQCJU1ZQ993BWlojiwj145QCaQALkY=.ed211af5-2493-4f9d-89ed-256a609d14d3@github.com>
 <DgXhJP37I84BIzuXBZ9cItwhOzClFBVphjCPAIn0EWI=.9c18075e-5be5-4ffe-bc05-b57f58175eb8@github.com>
Message-ID: <TkOEgN0er6u-mWRwXQCEgQStaHXB4TNKgbjrKwFPYmw=.fe570ad1-d01e-4b39-af4d-400bb350df13@github.com>

On Mon, 8 Apr 2024 18:47:08 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Like Roman, I'd prefer extending existing code and/or making it adaptable for our purposes to duplicating functionality. Let us read through the differences and see if this should be done now, or as you state allow this duplication for now in the interests of speed and expediency.
>
> Here's a rough outline of the differences between ShenandoahSimpleBitMap and src/hotspot/share/utilities/bitMap.*
> 1. ShenandoahSimpleBitMap finds next bit in the forward and backward direction.  BitMap only searches in forward direction.  We use backward searches when we prefer to pack certain types of objects into the high end of the heap.
> 2. ShenandoahSimpleBitMap also searches for clusters of bits, to facilitate efficient satisfaction of humongous allocation requests.  BitMap only searches for a single bit.  We could add this capability to BitMap.
> 4. If we were to generalize BitMap to handle reverse searches, this may have far-reaching impact on the API.   The type of results and idx_t (typedef size_t).  When we do backward searches, ShenandoahSimpleBitMap treats -1 as the sentinel NOT_FOUND value.

If consensus is we should not re-invent the wheel, I'll make an effort to make the additional changes required to use BitMap instead of ShenandoahSimpleBitMap.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1556297884

From eosterlund at openjdk.org  Mon Apr  8 20:22:08 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Mon, 8 Apr 2024 20:22:08 GMT
Subject: RFR: 8329629: GC interfaces should work directly against nmethod
 instead of CodeBlob
In-Reply-To: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
References: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
Message-ID: <cIV1fJ8hpgL1G0DJ_iQuekLhG93cfzUuuzsnjEgp7L8=.989808f7-efac-437c-b4d7-ab42bc37e674@github.com>

On Fri, 5 Apr 2024 12:32:30 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> The GCs scan and handles nmethods and ignores CodeBlobs of other kinds. The I propose that we stop sending in CodeBlobs to the GCs and make sure to only give them nmethods.
> 
> I removed `void CodeCache::blobs_do(CodeBlobClosure* f)` since there's no more usage of that function. Is this OK?
> 
> I also opted to skipped calling the GC verification code from the iterator code:
> 
> Universe::heap()->verify_nmethod((nmethod*)cb);
> 
> IMHO, I think it is up to the GCs to decide if they want to perform extra nmethod verification. If someone wants to keep this verification in their favorite GC I can add calls to this function where we used to call CodeCache::blobs_do.
> 
> I've only done limited testing and will run extensive testing concurrent with the review.

Marked as reviewed by eosterlund (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18653#pullrequestreview-1987338271

From ysr at openjdk.org  Tue Apr  9 00:45:35 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 9 Apr 2024 00:45:35 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial [v4]
In-Reply-To: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
Message-ID: <Hi9G4QtqUwEO5dENpnGbiF2x_dx688BE8tatTXpyOi8=.25dac74c-c165-45eb-b75f-b18f74186aa3@github.com>

> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
> 
> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
> 
> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
> 
> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
> 
> **Testing**:
> - [x] GHA : there are crashes unrelated to this change that are being addressed in other PRs
> - [ ] Code pipeline testing : there is a crash, unrelated to this change, that needs investigation
> - [x] SPECjbb
> 
> **Performance**:
> - [x] SPECjbb : the variance in tests appears to fail significance under 2-tailed Mann-Whitney
> - [ ] SPECjbb (patched for wk ref issue) : TBD
> - [ ] Extremem : in progress

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits:

 - Merge branch 'master' into clear_census
 - Merge branch 'master' into clear_census
 - Merge branch 'master' into clear_census
 - Remove local_reset of age_census object inadvertently left behind in the
   GLOBAL gen concurrent mark, which was triggering the newly added
   reasonableness assert (yay!).
 - Avoid divide-by-zero.
 - Merge branch 'master' into clear_census
 - Fix word and byte size unit confusion in comparison/assert.
 - Loose verification of "reasonable completeness" of census.
 - Fix release build.
 - Support for verification of census' reasonableness, but no verification
   itself yet.
 - ... and 6 more: https://git.openjdk.org/shenandoah/compare/11295a6a...d5fb72cc

-------------

Changes: https://git.openjdk.org/shenandoah/pull/403/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=403&range=03
  Stats: 167 lines in 11 files changed: 122 ins; 27 del; 18 mod
  Patch: https://git.openjdk.org/shenandoah/pull/403.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/403/head:pull/403

PR: https://git.openjdk.org/shenandoah/pull/403

From ysr at openjdk.org  Tue Apr  9 01:45:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 9 Apr 2024 01:45:17 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v32]
In-Reply-To: <TkOEgN0er6u-mWRwXQCEgQStaHXB4TNKgbjrKwFPYmw=.fe570ad1-d01e-4b39-af4d-400bb350df13@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Mu9wTnVX6kIS89pKXTUOHQT2CB7P_m_kSWGQHOQU-ko=.77851b17-c53b-4b59-8a84-6d518a19d35b@github.com>
 <HgHiTh_t5gXIEQnpJ8EbuF3mVuuyGOOh0DFsikSmo9E=.906d3f45-9210-4c24-9817-720935389846@github.com>
 <-g5tscBZPsDgVrjfZnM-UBRy0_6qodnRN5_srPWunXk=.82eda809-141c-4bfb-82aa-e722eecd0469@github.com>
 <U6tUCW9TF58aCdQCJU1ZQ993BWlojiwj145QCaQALkY=.ed211af5-2493-4f9d-89ed-256a609d14d3@github.com>
 <DgXhJP37I84BIzuXBZ9cItwhOzClFBVphjCPAIn0EWI=.9c18075e-5be5-4ffe-bc05-b57f58175eb8@github.com>
 <TkOEgN0er6u-mWRwXQCEgQStaHXB4TNKgbjrKwFPYmw=.fe570ad1-d01e-4b39-af4d-400bb350df13@github.com>
Message-ID: <n7x7vwO_G0jEkW48kmLAwk3qQytGL5RwZ-s1XwSsjmo=.5b479891-fafe-4b0c-aacf-c7e06f75b96d@github.com>

On Mon, 8 Apr 2024 18:48:16 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Here's a rough outline of the differences between ShenandoahSimpleBitMap and src/hotspot/share/utilities/bitMap.*
>> 1. ShenandoahSimpleBitMap finds next bit in the forward and backward direction.  BitMap only searches in forward direction.  We use backward searches when we prefer to pack certain types of objects into the high end of the heap.
>> 2. ShenandoahSimpleBitMap also searches for clusters of bits, to facilitate efficient satisfaction of humongous allocation requests.  BitMap only searches for a single bit.  We could add this capability to BitMap.
>> 4. If we were to generalize BitMap to handle reverse searches, this may have far-reaching impact on the API.   The type of results and idx_t (typedef size_t).  When we do backward searches, ShenandoahSimpleBitMap treats -1 as the sentinel NOT_FOUND value.
>
> If consensus is we should not re-invent the wheel, I'll make an effort to make the additional changes required to use BitMap instead of ShenandoahSimpleBitMap.

Thank you Kelvin:

> Here's a rough outline of the differences between ShenandoahSimpleBitMap and src/hotspot/share/utilities/bitMap.*
> 
> 1. ShenandoahSimpleBitMap finds next bit in the forward and backward direction.  BitMap only searches in forward direction.  We use backward searches when we prefer to pack certain types of objects into the high end of the heap.

BitMap could be extended to optionally search backwards as well? (In fact as I realized after I wrote this, BitMap already has that capability, I think, or at least has an API that does so.)

> 2. ShenandoahSimpleBitMap also searches for clusters of bits, to facilitate efficient satisfaction of humongous allocation requests.  BitMap only searches for a single bit.  We could add this capability to BitMap.

I agree.

> 3. If we were to generalize BitMap to handle reverse searches, this may have far-reaching impact on the API.   The type of results and idx_t (typedef size_t).  When we do backward searches, ShenandoahSimpleBitMap treats -1 as the sentinel NOT_FOUND value.

I think that's easy to fix. Backwards searches from index x (exclusive), can use `find_last_set_bit(idx_t 0, idx_t x)`. If you get back `x` that indicates NOT_FOUND:


  // Return the index of the last set (or clear) bit in the range [beg, end),
  // or end if none found.
  // precondition: beg and end form a valid range for the bitmap.
  idx_t find_last_set_bit(idx_t beg, idx_t end) const;
  idx_t find_last_clear_bit(idx_t beg, idx_t end) const;


You can use a wrapper to return `-1` if you want for your specific use case.

Would that work?

PS: I have not looked at `cluster of bits`, but I am guessing it's not a bit but a specific pattern/sequence you are looking for. I'll take a look at the code of `ShenandoahSimpleBitMap` to understand.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1556726039

From stefank at openjdk.org  Tue Apr  9 07:28:40 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 9 Apr 2024 07:28:40 GMT
Subject: RFR: 8329629: GC interfaces should work directly against nmethod
 instead of CodeBlob [v2]
In-Reply-To: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
References: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
Message-ID: <P3FyeF5oVtIJkCWYpKhocnA5gzwnY2wBwChflw2RoeU=.e77e54de-edf6-4c07-b628-1dd3a0eaa4bb@github.com>

> The GCs scan and handles nmethods and ignores CodeBlobs of other kinds. The I propose that we stop sending in CodeBlobs to the GCs and make sure to only give them nmethods.
> 
> I removed `void CodeCache::blobs_do(CodeBlobClosure* f)` since there's no more usage of that function. Is this OK?
> 
> I also opted to skipped calling the GC verification code from the iterator code:
> 
> Universe::heap()->verify_nmethod((nmethod*)cb);
> 
> IMHO, I think it is up to the GCs to decide if they want to perform extra nmethod verification. If someone wants to keep this verification in their favorite GC I can add calls to this function where we used to call CodeCache::blobs_do.
> 
> I've only done limited testing and will run extensive testing concurrent with the review.

Stefan Karlsson has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:

 - Merge remote-tracking branch 'upstream/master' into 8329629_do_code_blob
 - 8329629: GC interfaces should work directly against nmethod instead of CodeBlob

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18653/files
  - new: https://git.openjdk.org/jdk/pull/18653/files/e10683e2..1c2bdaea

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18653&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18653&range=00-01

  Stats: 5545 lines in 115 files changed: 3743 ins; 1387 del; 415 mod
  Patch: https://git.openjdk.org/jdk/pull/18653.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18653/head:pull/18653

PR: https://git.openjdk.org/jdk/pull/18653

From stefank at openjdk.org  Tue Apr  9 07:28:40 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 9 Apr 2024 07:28:40 GMT
Subject: RFR: 8329629: GC interfaces should work directly against nmethod
 instead of CodeBlob
In-Reply-To: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
References: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
Message-ID: <YDqSWAJuSm7Gy3zkm_sKrsicCTHd8eoASeQZYtwNgYY=.42e735cf-81b3-4165-b064-7416daba1310@github.com>

On Fri, 5 Apr 2024 12:32:30 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> The GCs scan and handles nmethods and ignores CodeBlobs of other kinds. The I propose that we stop sending in CodeBlobs to the GCs and make sure to only give them nmethods.
> 
> I removed `void CodeCache::blobs_do(CodeBlobClosure* f)` since there's no more usage of that function. Is this OK?
> 
> I also opted to skipped calling the GC verification code from the iterator code:
> 
> Universe::heap()->verify_nmethod((nmethod*)cb);
> 
> IMHO, I think it is up to the GCs to decide if they want to perform extra nmethod verification. If someone wants to keep this verification in their favorite GC I can add calls to this function where we used to call CodeCache::blobs_do.
> 
> I've only done limited testing and will run extensive testing concurrent with the review.

Thanks for the reviews! I'll let GHA complete before integrating.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18653#issuecomment-2044313453

From stefank at openjdk.org  Tue Apr  9 12:31:21 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 9 Apr 2024 12:31:21 GMT
Subject: Integrated: 8329629: GC interfaces should work directly against
 nmethod instead of CodeBlob
In-Reply-To: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
References: <FrjeNeatB_30oeaLOmusRqYRCeNXB7oa--CpGdqr1Mc=.b216c2dc-ddac-4c7c-a9f8-b8af2529ff50@github.com>
Message-ID: <kFpO1Tq7tHMOBKYEjCwO6Yjbov4E7mWqChBkkico45w=.74d68d0c-1daf-4483-9290-6630560225c0@github.com>

On Fri, 5 Apr 2024 12:32:30 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> The GCs scan and handles nmethods and ignores CodeBlobs of other kinds. The I propose that we stop sending in CodeBlobs to the GCs and make sure to only give them nmethods.
> 
> I removed `void CodeCache::blobs_do(CodeBlobClosure* f)` since there's no more usage of that function. Is this OK?
> 
> I also opted to skipped calling the GC verification code from the iterator code:
> 
> Universe::heap()->verify_nmethod((nmethod*)cb);
> 
> IMHO, I think it is up to the GCs to decide if they want to perform extra nmethod verification. If someone wants to keep this verification in their favorite GC I can add calls to this function where we used to call CodeCache::blobs_do.
> 
> I've only done limited testing and will run extensive testing concurrent with the review.

This pull request has now been integrated.

Changeset: 87131fb2
Author:    Stefan Karlsson <stefank at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/87131fb2f77188a483fd0852da5f9228aafd5336
Stats:     850 lines in 74 files changed: 238 ins; 318 del; 294 mod

8329629: GC interfaces should work directly against nmethod instead of CodeBlob

Reviewed-by: ayang, eosterlund

-------------

PR: https://git.openjdk.org/jdk/pull/18653

From cslucas at openjdk.org  Tue Apr  9 19:10:30 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Tue, 9 Apr 2024 19:10:30 GMT
Subject: RFR: JDK-8241503: C2: Share MacroAssembler between mach nodes
 during code emission [v12]
In-Reply-To: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
Message-ID: <K_czfhnzzDNtM1cx9T0HbYCw2UVFQ9mgA4fKKWgUc3M=.c031beb4-c719-46c1-9043-9b2f00b6e355@github.com>

> # Description
> 
> Please review this PR with a patch to re-use the same C2_MacroAssembler object to emit all instructions in the same compilation unit.
> 
> Overall, the change is pretty simple. However, due to the renaming of the variable to access C2_MacroAssembler, from `_masm.` to `masm->`, and also some method prototype changes, the patch became quite large.
> 
> # Help Needed for Testing
> 
> I don't have access to all platforms necessary to test this. I hope some other folks can help with testing on `S390`, `RISC-V` and `PPC`.

Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:

  Fix ARM32 AD file

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/16484/files
  - new: https://git.openjdk.org/jdk/pull/16484/files/b4d73c98..693c7ef8

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=16484&range=11
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16484&range=10-11

  Stats: 21 lines in 1 file changed: 12 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/16484.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16484/head:pull/16484

PR: https://git.openjdk.org/jdk/pull/16484

From cslucas at openjdk.org  Tue Apr  9 19:10:33 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Tue, 9 Apr 2024 19:10:33 GMT
Subject: RFR: JDK-8241503: C2: Share MacroAssembler between mach nodes
 during code emission [v11]
In-Reply-To: <JOS3FkAoDAb2RHxJSGe4a14GlH60vDLOx3f-4Ugkq6E=.60d72071-60f6-47eb-b449-7f0c9266963a@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <dQZjFk-E_tYaTTg7CzDj_bFhPb4RHsI6EKlga4vsNbQ=.9339a5c7-72c5-425c-b90b-6261d53f5a58@github.com>
 <JOS3FkAoDAb2RHxJSGe4a14GlH60vDLOx3f-4Ugkq6E=.60d72071-60f6-47eb-b449-7f0c9266963a@github.com>
Message-ID: <ngaAkzLxfbOGSnNB5Q3_6PEW0tfoYUklDWcS3fKcI0A=.718bebdf-db02-41f3-b770-46f327e76fdf@github.com>

On Mon, 8 Apr 2024 06:09:14 GMT, Boris Ulasevich <bulasevich at openjdk.org> wrote:

>> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into reuse-macroasm
>>  - Fix AArch64 build & improve comment about InstructionMark
>>  - Catching up with changes in master
>>  - Catching up with origin/master
>>  - Catch up with origin/master
>>  - Merge with origin/master
>>  - Fix build, copyright dates, m4 files.
>>  - Fix merge
>>  - Catch up with master branch.
>>    
>>    Merge remote-tracking branch 'origin/master' into reuse-macroasm
>>  - Some inst_mark fixes; Catch up with master.
>>  - ... and 2 more: https://git.openjdk.org/jdk/compare/89e0889a...b4d73c98
>
> Do you need help understanding the problem? The crash occurred because you removed the line `fprintf(fp, "  cbuf.set_insts_mark();\n");` from the generator of AD nodes ::emit() methods. That is why emit_call_reloc finds cbuf.insts->_mark unitialized.
> 
> 
> // Call Runtime Instruction
> instruct CallRuntimeDirect(method meth) %{
>   match(CallRuntime);
>   effect(USE meth);
>   ins_cost(CALL_COST);
>   format %{ "CALL,runtime" %}
>   ins_encode( Java_To_Runtime( meth ),
>               call_epilog );
>   ins_pipe(simple_call);
> %}
> 
> -->
> 
> void CallRuntimeDirectNode::emit(CodeBuffer& cbuf, PhaseRegAlloc* ra_) const {
>   cbuf.set_insts_mark();
>   // Start at oper_input_base() and count operands
>   unsigned idx0 = 1;
>   unsigned idx1 = 1;    //
>   {
> #line 1217 "/home/boris/jdk-bulasevich/src/hotspot/cpu/arm/arm.ad"
>     // CALL directly to the runtime
>     emit_call_reloc(cbuf, as_MachCall(), opnd_array(1), runtime_call_Relocation::spec());
> #line 999999
>   }
>   {
> #line 1213 "/home/boris/jdk-bulasevich/src/hotspot/cpu/arm/arm.ad"
>     // nothing
> #line 999999
>   }
> }

@bulasevich - I just pushed a fix for ARM32. Can you please run your tests again? Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16484#issuecomment-2045889314

From rkennke at openjdk.org  Tue Apr  9 19:36:16 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 9 Apr 2024 19:36:16 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v32]
In-Reply-To: <n7x7vwO_G0jEkW48kmLAwk3qQytGL5RwZ-s1XwSsjmo=.5b479891-fafe-4b0c-aacf-c7e06f75b96d@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Mu9wTnVX6kIS89pKXTUOHQT2CB7P_m_kSWGQHOQU-ko=.77851b17-c53b-4b59-8a84-6d518a19d35b@github.com>
 <HgHiTh_t5gXIEQnpJ8EbuF3mVuuyGOOh0DFsikSmo9E=.906d3f45-9210-4c24-9817-720935389846@github.com>
 <-g5tscBZPsDgVrjfZnM-UBRy0_6qodnRN5_srPWunXk=.82eda809-141c-4bfb-82aa-e722eecd0469@github.com>
 <U6tUCW9TF58aCdQCJU1ZQ993BWlojiwj145QCaQALkY=.ed211af5-2493-4f9d-89ed-256a609d14d3@github.com>
 <DgXhJP37I84BIzuXBZ9cItwhOzClFBVphjCPAIn0EWI=.9c18075e-5be5-4ffe-bc05-b57f58175eb8@github.com>
 <TkOEgN0er6u-mWRwXQCEgQStaHXB4TNKgbjrKwFPYmw=.fe570ad1-d01e-4b39-af4d-400bb350df13@github.com>
 <n7x7vwO_G0jEkW48kmLAwk3qQytGL5RwZ-s1XwSsjmo=.5b479891-fafe-4b0c-aacf-c7e06f75b96d@github.com>
Message-ID: <oeAEnM5vQQtialWn3PfoyeCJmnt9fysm3oDgHUbWh-E=.f2db1d97-4226-4e43-b9ee-1390226691fa@github.com>

On Tue, 9 Apr 2024 01:41:49 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> If consensus is we should not re-invent the wheel, I'll make an effort to make the additional changes required to use BitMap instead of ShenandoahSimpleBitMap.
>
> Thank you Kelvin:
> 
>> Here's a rough outline of the differences between ShenandoahSimpleBitMap and src/hotspot/share/utilities/bitMap.*
>> 
>> 1. ShenandoahSimpleBitMap finds next bit in the forward and backward direction.  BitMap only searches in forward direction.  We use backward searches when we prefer to pack certain types of objects into the high end of the heap.
> 
> BitMap could be extended to optionally search backwards as well? (In fact as I realized after I wrote this, BitMap already has that capability, I think, or at least has an API that does so.)
> 
>> 2. ShenandoahSimpleBitMap also searches for clusters of bits, to facilitate efficient satisfaction of humongous allocation requests.  BitMap only searches for a single bit.  We could add this capability to BitMap.
> 
> I agree.
> 
>> 3. If we were to generalize BitMap to handle reverse searches, this may have far-reaching impact on the API.   The type of results and idx_t (typedef size_t).  When we do backward searches, ShenandoahSimpleBitMap treats -1 as the sentinel NOT_FOUND value.
> 
> I think that's easy to fix. Backwards searches from index x (exclusive), can use `find_last_set_bit(idx_t 0, idx_t x)`. If you get back `x` that indicates NOT_FOUND:
> 
> 
>   // Return the index of the last set (or clear) bit in the range [beg, end),
>   // or end if none found.
>   // precondition: beg and end form a valid range for the bitmap.
>   idx_t find_last_set_bit(idx_t beg, idx_t end) const;
>   idx_t find_last_clear_bit(idx_t beg, idx_t end) const;
> 
> 
> You can use a wrapper to return `-1` if you want for your specific use case.
> 
> Would that work?
> 
> PS: I have not looked at `cluster of bits`, but I am guessing it's not a bit but a specific pattern/sequence you are looking for. I'll take a look at the code of `ShenandoahSimpleBitMap` to understand.

I'd say, let's keep the duplicate ShSimpleBitMap for now, but make a note (bug-entry?) to attempt a merge later on. We can help that by trying to keep the style, types, algorithmic and API approaches similar.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1558185173

From rkennke at openjdk.org  Tue Apr  9 19:36:16 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 9 Apr 2024 19:36:16 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v39]
In-Reply-To: <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
Message-ID: <IimvSux5GEemCzBEY0lZHUCxFTFzhTM7mvCm7AD3nV4=.ed14fcb8-d5a3-4439-a2f8-8444cf7df123@github.com>

On Thu, 28 Mar 2024 16:06:58 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove debugging instrumentation

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 45:

> 43: // break abstraction rules, because efficient implementation requires assumptions about superclass internals that
> 44: // might be violatee through future software maintenance.
> 45: class ShenandoahSimpleBitMap {

I think this class should go into its own set of files. It would certainly help readability of ShFreeSet which is now somewhat dominated by ShSimpleBitMap.

src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 915:

> 913: public:
> 914:   ShenandoahPostCompactClosure() : _heap(ShenandoahHeap::heap()), _live(0) {
> 915:     _heap->free_set()->clear();

Why is that ok?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1558187740
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1558185718

From ysr at openjdk.org  Tue Apr  9 20:05:21 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 9 Apr 2024 20:05:21 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial [v4]
In-Reply-To: <Hi9G4QtqUwEO5dENpnGbiF2x_dx688BE8tatTXpyOi8=.25dac74c-c165-45eb-b75f-b18f74186aa3@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
 <Hi9G4QtqUwEO5dENpnGbiF2x_dx688BE8tatTXpyOi8=.25dac74c-c165-45eb-b75f-b18f74186aa3@github.com>
Message-ID: <rz_6C5yXvg_aD6u_I6cQfBCUYJHn3huDctO61KmY6H8=.f7947124-876b-425c-8e47-05e8aca14505@github.com>

On Tue, 9 Apr 2024 00:45:35 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
>> 
>> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
>> 
>> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
>> 
>> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
>> 
>> **Testing**:
>> - [x] GHA 
>> - [x] Code pipeline testing : rerunning because of an infra script glitch; tests were all green though.
>> - [x] SPECjbb
>> 
>> **Performance**: Tests are being rerun
>> - [x] SPECjbb : the variance in tests appears to fail significance under 2-tailed Mann-Whitney
>> - [ ] SPECjbb (patched for wk ref issue) : TBD
>> - [ ] Extremem : in progress
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits:
> 
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Remove local_reset of age_census object inadvertently left behind in the
>    GLOBAL gen concurrent mark, which was triggering the newly added
>    reasonableness assert (yay!).
>  - Avoid divide-by-zero.
>  - Merge branch 'master' into clear_census
>  - Fix word and byte size unit confusion in comparison/assert.
>  - Loose verification of "reasonable completeness" of census.
>  - Fix release build.
>  - Support for verification of census' reasonableness, but no verification
>    itself yet.
>  - ... and 6 more: https://git.openjdk.org/shenandoah/compare/11295a6a...d5fb72cc

After the fixing of some of the upstream issues that had snuck in, the testing is all green now. I'll rerun the performance measurements and update the description in the PR. If all goes well, we plan to intergrate it as soon as performance numbers are at hand.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/403#issuecomment-2045960020

From rkennke at openjdk.org  Wed Apr 10 11:54:18 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 10 Apr 2024 11:54:18 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v39]
In-Reply-To: <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
Message-ID: <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>

On Thu, 28 Mar 2024 16:06:58 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove debugging instrumentation

Only partial review, but addressing the comments likely changes a lot (types, names, etc) all over the place, so I leave it at that for now.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 42:

> 40:     case Mutator: return "Mutator";
> 41:     case Collector: return "Collector";
> 42:     default: return "Unrecognized";

I believe using an 'enum class' for ShenandoahFreeSetPartitionId means you don't need to have a default.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 49:

> 47:   assert((start_idx >= 0) && (start_idx < _num_bits), "precondition");
> 48:   size_t array_idx = start_idx >> LogBitsPerWord;
> 49:   uintx element_bits = _bitmap[array_idx];

Same here, try to stick with what is done in bitMap.hpp (typedef uintptr_t bm_word_t, you could just as well typedef uintx bm_word_t, if you prefer).

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 93:

> 91:   uintx complement = ~element_bits;
> 92:   uintx trailing_ones;
> 93:   if (complement) {

complement is not a bool type. If you meant to say 'if (complement != 0)' then say so, to make it easier for the reader. Same applies for a few other places.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 102:

> 100:   } else if (trailing_ones == bits_to_examine) {
> 101:      // Tail recursion
> 102:     return is_forward_consecutive_ones(start_idx + bits_to_examine, count - bits_to_examine);

How bad can this get? I haven't analyzed the algorithm, but if this is not bounded (e.g. we know that it can only go 1 level deep), then I'd prefer to have this as loop, to avoid stack overflow. Same applies for a few other places.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 186:

> 184: }
> 185: 
> 186: ssize_t ShenandoahSimpleBitMap::find_prev_consecutive_bits(

It looks weird to me to have an opening ( and then all the args on the next line. I think in HotSpot, it's usually done like:

ssize_t ShenandoahSimpleBitMap::find_prev_consecutive_bits(const size_t num_bits,
                                                           ssize_t last_idx,
                                                           const ssize_t boundary_idx) const {

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 165:

> 163: 
> 164: // Each ShenandoahHeapRegion is associated with a ShenandoahFreeSetPartitionId.
> 165: enum ShenandoahFreeSetPartitionId : uint8_t {

Make this an 'enum class' instead.
I see that you are using the enum as index, in this case you can declare the enum class like 'enum class ShenandoahFreeSetPartitionId : uint8_t'

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.inline.hpp line 31:

> 29: #include "gc/shenandoah/shenandoahFreeSet.hpp"
> 30: 
> 31: inline ssize_t ShenandoahSimpleBitMap::find_next_set_bit(ssize_t start_idx, ssize_t boundary_idx) const {

bitMap.hpp calls it get_next_one_offset(), maybe keep names, args (and impl?) the same as there, to make future merge easier? I guess that is true for all methods that also exist in bitMap.hpp. If you do, then say in comment that the whole block is taken from bitMap.* to make review and later merge easier.

-------------

Changes requested by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-1991417328
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559263831
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559270594
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559274704
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559280964
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559286212
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559262690
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559301482

From rkennke at openjdk.org  Wed Apr 10 11:54:18 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 10 Apr 2024 11:54:18 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v32]
In-Reply-To: <-g5tscBZPsDgVrjfZnM-UBRy0_6qodnRN5_srPWunXk=.82eda809-141c-4bfb-82aa-e722eecd0469@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <Mu9wTnVX6kIS89pKXTUOHQT2CB7P_m_kSWGQHOQU-ko=.77851b17-c53b-4b59-8a84-6d518a19d35b@github.com>
 <HgHiTh_t5gXIEQnpJ8EbuF3mVuuyGOOh0DFsikSmo9E=.906d3f45-9210-4c24-9817-720935389846@github.com>
 <-g5tscBZPsDgVrjfZnM-UBRy0_6qodnRN5_srPWunXk=.82eda809-141c-4bfb-82aa-e722eecd0469@github.com>
Message-ID: <HbluEhv-jTRXGT6Fa6u-4pkJUOVMMYCEEoX2Zedihug=.f485c544-8ad9-4bda-a8dc-6daa7252bbfd@github.com>

On Wed, 20 Mar 2024 19:51:48 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 46:
>> 
>>> 44: }
>>> 45: 
>>> 46: size_t ShenandoahSimpleBitMap::count_leading_ones(ssize_t start_idx) const {
>> 
>> Are you sure that you can't use the BitMap class in src/hotspot/share/utilities/bitMap.*? Or maybe only src/hotspot/share/utilities/count_leading_zeros.hpp or src/hotspot/share/utilities/count_trailing_zeros.hpp? That would emit CPU instructions that do the counting really fast, if possible. I.e. reverse all bits after loading, then find leading/trailing zeros, problem solved real fast. I would even argue to augment the BitMap class with count_* methods as needed, using the CPU-optimized routines.
>
> I will see if I can exploit the services in src/hotspot/share/utilities to improve performance.

Why use ssize_t for start_idx? or, at all? In bitMap.hpp, it declares a 'typedef size_t idx_t' and use that for bit and word indices. I'd try to stick to the same pattern as much as reasonably possible.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559267896

From rkennke at openjdk.org  Wed Apr 10 11:54:18 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 10 Apr 2024 11:54:18 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v14]
In-Reply-To: <XzlTwJFYWhjXumZ-mexXz39jhm5-hjqfBnmWayHTVZg=.e5d25e3d-54f0-4478-a1ed-afacaecf9af9@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <DtAJtZcZPNv2FMbF9-FeJIT6T8Oio1TW9yYnw3cceF0=.ed94d2d3-39c2-4f8d-9e00-5dbd81683c93@github.com>
 <PVrVW-I2bKkYzrdBafHAaWF8mw6WtANB3PoOZW5ekRw=.2ab54392-0f5b-4f6f-b5d7-7b7a9b6a0a97@github.com>
 <XzlTwJFYWhjXumZ-mexXz39jhm5-hjqfBnmWayHTVZg=.e5d25e3d-54f0-4478-a1ed-afacaecf9af9@github.com>
Message-ID: <LSfLue-6QPygFz_XhIZa_kihwsc98z6MOA7z1TXtsP8=.1fb122df-f06a-4dcb-8091-4610ab8a3081@github.com>

On Fri, 1 Mar 2024 21:22:54 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 68:
>> 
>>> 66:   // Count consecutive ones in forward order, starting from start_idx.  Requires that there is at least one zero
>>> 67:   // between start_idx and index value (_num_bits - 1), inclusive.
>>> 68:   size_t count_leading_ones(ssize_t start_idx) const;
>> 
>> Does the argument have to me `ssize_t`, not `size_t`? Is this because you want it to be signed?
>
> I'm going to add the following comment at the top of shenandoahFreeSet.hpp:
> 
> // The API and internal implementation of ShenandoahSimpleBitMap and ShenandoahRegionPartitions use ssize_t to                      
> // represent index, even though index is "inherently" unsigned.  There are two reasons for this choice:                             
> //  1. We use -1 as a sentinel value to represent empty partitions.  This same value may be used to represent                       
> //     failure to find a previous set bit or previous range of set bits.                                                            
> //  2. Certain loops are written most naturally if the iterator, which may hold the sentinel -1 value, can be                       
> //     declared as signed and the terminating condition can be < 0.

Please use types, method signatures and keep structure similar to bitMap.hpp, which use idx_t, which is a size_t (see e.g. BitMap::count_one_bits()).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1559294278

From bulasevich at openjdk.org  Wed Apr 10 21:24:46 2024
From: bulasevich at openjdk.org (Boris Ulasevich)
Date: Wed, 10 Apr 2024 21:24:46 GMT
Subject: RFR: JDK-8241503: C2: Share MacroAssembler between mach nodes
 during code emission [v11]
In-Reply-To: <JOS3FkAoDAb2RHxJSGe4a14GlH60vDLOx3f-4Ugkq6E=.60d72071-60f6-47eb-b449-7f0c9266963a@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <dQZjFk-E_tYaTTg7CzDj_bFhPb4RHsI6EKlga4vsNbQ=.9339a5c7-72c5-425c-b90b-6261d53f5a58@github.com>
 <JOS3FkAoDAb2RHxJSGe4a14GlH60vDLOx3f-4Ugkq6E=.60d72071-60f6-47eb-b449-7f0c9266963a@github.com>
Message-ID: <K7X8auWi1t0cntjoqs7mGqqlvkBMvttj5Th1p-pE-JM=.d68a50b7-b573-4f75-ad13-ca222aff3950@github.com>

On Mon, 8 Apr 2024 06:09:14 GMT, Boris Ulasevich <bulasevich at openjdk.org> wrote:

>> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into reuse-macroasm
>>  - Fix AArch64 build & improve comment about InstructionMark
>>  - Catching up with changes in master
>>  - Catching up with origin/master
>>  - Catch up with origin/master
>>  - Merge with origin/master
>>  - Fix build, copyright dates, m4 files.
>>  - Fix merge
>>  - Catch up with master branch.
>>    
>>    Merge remote-tracking branch 'origin/master' into reuse-macroasm
>>  - Some inst_mark fixes; Catch up with master.
>>  - ... and 2 more: https://git.openjdk.org/jdk/compare/89e0889a...b4d73c98
>
> Do you need help understanding the problem? The crash occurred because you removed the line `fprintf(fp, "  cbuf.set_insts_mark();\n");` from the generator of AD nodes ::emit() methods. That is why emit_call_reloc finds cbuf.insts->_mark unitialized.
> 
> 
> // Call Runtime Instruction
> instruct CallRuntimeDirect(method meth) %{
>   match(CallRuntime);
>   effect(USE meth);
>   ins_cost(CALL_COST);
>   format %{ "CALL,runtime" %}
>   ins_encode( Java_To_Runtime( meth ),
>               call_epilog );
>   ins_pipe(simple_call);
> %}
> 
> -->
> 
> void CallRuntimeDirectNode::emit(CodeBuffer& cbuf, PhaseRegAlloc* ra_) const {
>   cbuf.set_insts_mark();
>   // Start at oper_input_base() and count operands
>   unsigned idx0 = 1;
>   unsigned idx1 = 1;    //
>   {
> #line 1217 "/home/boris/jdk-bulasevich/src/hotspot/cpu/arm/arm.ad"
>     // CALL directly to the runtime
>     emit_call_reloc(cbuf, as_MachCall(), opnd_array(1), runtime_call_Relocation::spec());
> #line 999999
>   }
>   {
> #line 1213 "/home/boris/jdk-bulasevich/src/hotspot/cpu/arm/arm.ad"
>     // nothing
> #line 999999
>   }
> }

> @bulasevich - I just pushed a fix for ARM32. Can you please run your tests again? Thanks!

Good. Tests are OK now.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16484#issuecomment-2048456503

From bulasevich at openjdk.org  Wed Apr 10 21:24:47 2024
From: bulasevich at openjdk.org (Boris Ulasevich)
Date: Wed, 10 Apr 2024 21:24:47 GMT
Subject: RFR: JDK-8241503: C2: Share MacroAssembler between mach nodes
 during code emission [v12]
In-Reply-To: <K_czfhnzzDNtM1cx9T0HbYCw2UVFQ9mgA4fKKWgUc3M=.c031beb4-c719-46c1-9043-9b2f00b6e355@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <K_czfhnzzDNtM1cx9T0HbYCw2UVFQ9mgA4fKKWgUc3M=.c031beb4-c719-46c1-9043-9b2f00b6e355@github.com>
Message-ID: <vMlFUycsWIZKQODO5ZZgawl2ZpE1q-RtJgeehPgrsso=.cf33d1ce-3772-48a4-bc50-ab02d451a24f@github.com>

On Tue, 9 Apr 2024 19:10:30 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> # Description
>> 
>> Please review this PR with a patch to re-use the same C2_MacroAssembler object to emit all instructions in the same compilation unit.
>> 
>> Overall, the change is pretty simple. However, due to the renaming of the variable to access C2_MacroAssembler, from `_masm.` to `masm->`, and also some method prototype changes, the patch became quite large.
>> 
>> # Help Needed for Testing
>> 
>> I don't have access to all platforms necessary to test this. I hope some other folks can help with testing on `S390`, `RISC-V` and `PPC`.
>
> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix ARM32 AD file

src/hotspot/cpu/arm/arm.ad line 1877:

> 1875: %}
> 1876: 
> 1877: // Pointer Immediate

Why do you introduce immN operands for arm32?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16484#discussion_r1560068373

From cslucas at openjdk.org  Wed Apr 10 23:49:48 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Wed, 10 Apr 2024 23:49:48 GMT
Subject: RFR: 8241503: C2: Share MacroAssembler between mach nodes during
 code emission [v12]
In-Reply-To: <vMlFUycsWIZKQODO5ZZgawl2ZpE1q-RtJgeehPgrsso=.cf33d1ce-3772-48a4-bc50-ab02d451a24f@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <K_czfhnzzDNtM1cx9T0HbYCw2UVFQ9mgA4fKKWgUc3M=.c031beb4-c719-46c1-9043-9b2f00b6e355@github.com>
 <vMlFUycsWIZKQODO5ZZgawl2ZpE1q-RtJgeehPgrsso=.cf33d1ce-3772-48a4-bc50-ab02d451a24f@github.com>
Message-ID: <LnSU-Z-mOdKybwAhL_c1vpFjxz2Kq8lBeO2SkPMBalA=.e10c4c8c-459d-4aec-ae1c-cd0d9b240bcc@github.com>

On Wed, 10 Apr 2024 21:20:11 GMT, Boris Ulasevich <bulasevich at openjdk.org> wrote:

>> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix ARM32 AD file
>
> src/hotspot/cpu/arm/arm.ad line 1877:
> 
>> 1875: %}
>> 1876: 
>> 1877: // Pointer Immediate
> 
> Why do you introduce immN operands for arm32?

This was accidental. I'll remove it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16484#discussion_r1560174793

From cslucas at openjdk.org  Wed Apr 10 23:53:00 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Wed, 10 Apr 2024 23:53:00 GMT
Subject: RFR: 8241503: C2: Share MacroAssembler between mach nodes during
 code emission [v13]
In-Reply-To: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
Message-ID: <hbW78wfehwwUMAX_Bc03EJjClx5Dm2N7h-DIHgyGb2g=.1e0e5d12-79f1-4fb9-a36c-b1713e3972a2@github.com>

> # Description
> 
> Please review this PR with a patch to re-use the same C2_MacroAssembler object to emit all instructions in the same compilation unit.
> 
> Overall, the change is pretty simple. However, due to the renaming of the variable to access C2_MacroAssembler, from `_masm.` to `masm->`, and also some method prototype changes, the patch became quite large.
> 
> # Help Needed for Testing
> 
> I don't have access to all platforms necessary to test this. I hope some other folks can help with testing on `S390`, `RISC-V` and `PPC`.

Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:

  Remove unused operands in arm.ad

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/16484/files
  - new: https://git.openjdk.org/jdk/pull/16484/files/693c7ef8..44e63ee0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=16484&range=12
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16484&range=11-12

  Stats: 30 lines in 1 file changed: 0 ins; 30 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/16484.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16484/head:pull/16484

PR: https://git.openjdk.org/jdk/pull/16484

From bulasevich at openjdk.org  Thu Apr 11 05:08:48 2024
From: bulasevich at openjdk.org (Boris Ulasevich)
Date: Thu, 11 Apr 2024 05:08:48 GMT
Subject: RFR: 8241503: C2: Share MacroAssembler between mach nodes during
 code emission [v12]
In-Reply-To: <LnSU-Z-mOdKybwAhL_c1vpFjxz2Kq8lBeO2SkPMBalA=.e10c4c8c-459d-4aec-ae1c-cd0d9b240bcc@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <K_czfhnzzDNtM1cx9T0HbYCw2UVFQ9mgA4fKKWgUc3M=.c031beb4-c719-46c1-9043-9b2f00b6e355@github.com>
 <vMlFUycsWIZKQODO5ZZgawl2ZpE1q-RtJgeehPgrsso=.cf33d1ce-3772-48a4-bc50-ab02d451a24f@github.com>
 <LnSU-Z-mOdKybwAhL_c1vpFjxz2Kq8lBeO2SkPMBalA=.e10c4c8c-459d-4aec-ae1c-cd0d9b240bcc@github.com>
Message-ID: <M9aIfs7Qn7eRpUh1iRIO57FeJJ3mG6Qum1BG2H7QAoo=.9572d315-7c45-4d6b-8fc7-7156cc78ecc8@github.com>

On Wed, 10 Apr 2024 23:47:32 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> src/hotspot/cpu/arm/arm.ad line 1877:
>> 
>>> 1875: %}
>>> 1876: 
>>> 1877: // Pointer Immediate
>> 
>> Why do you introduce immN operands for arm32?
>
> This was accidental. I'll remove it.

Ok. The ARM32 changes look good to me now!

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16484#discussion_r1560439593

From pminborg at openjdk.org  Thu Apr 11 10:42:50 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Thu, 11 Apr 2024 10:42:50 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find
Message-ID: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>

While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.

This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

-------------

Commit messages:
 - Remove white space
 - Add SymbolLookup::findOrThrow

Changes: https://git.openjdk.org/jdk/pull/18474/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8314592
  Stats: 93 lines in 21 files changed: 30 ins; 0 del; 63 mod
  Patch: https://git.openjdk.org/jdk/pull/18474.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18474/head:pull/18474

PR: https://git.openjdk.org/jdk/pull/18474

From mcimadamore at openjdk.org  Thu Apr 11 11:43:42 2024
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 11 Apr 2024 11:43:42 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <O5hSvRzOvRt7EDz0BAgIw5bOYFiN10jBwad19H0uc9Q=.c6021c11-0647-4375-a4f9-00aae201a67f@github.com>

On Mon, 25 Mar 2024 14:56:23 GMT, Per Minborg <pminborg at openjdk.org> wrote:

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

src/java.base/share/classes/java/lang/foreign/SymbolLookup.java line 152:

> 150: 
> 151:     /**
> 152:      * {@return the address of the symbol with the provided {@code name} or throws an

I suggest using the same javadoc structure as the main `find` method. Note that there we have a summary, then a more detailed `@return` which talks about zero length memory segments. This should do the same.

src/java.base/share/classes/java/lang/foreign/SymbolLookup.java line 155:

> 153:      *          {@linkplain IllegalArgumentException} if no such address can be found}
> 154:      *<p>
> 155:      * This is a convenience method that provides better exception messages compared

I would reframe this as: "this is equivalent to the following code, but with better exception message" (for consistency with other API points in FFM where we show what a method boils down to)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1560889723
PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1560893974

From mcimadamore at openjdk.org  Thu Apr 11 11:46:40 2024
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 11 Apr 2024 11:46:40 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <NjGEgqr5SEJ-IadHMjjwV24YJLxIk8h529VASlqhqkE=.0b564bc8-a088-4820-8fa2-ac13bd654445@github.com>

On Mon, 25 Mar 2024 14:56:23 GMT, Per Minborg <pminborg at openjdk.org> wrote:

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

src/java.base/share/classes/java/lang/foreign/SymbolLookup.java line 177:

> 175:         }
> 176:         throw new IllegalArgumentException(
> 177:                 "Unable to to find a symbol with the name: " + name);

Another, more succinct, text could be "Symbol not found: <name>"

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1560897671

From kdnilsen at openjdk.org  Thu Apr 11 15:26:06 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 11 Apr 2024 15:26:06 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
Message-ID: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>

This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.

-------------

Commit messages:
 - Change behavior of max_old and min_old
 - Merge branch 'openjdk:master' into master
 - Make satb-mode Info logging less verbose
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Revert "Round LAB sizes down rather than up to force alignment"
 - Round LAB sizes down rather than up to force alignment
 - Revert "Remove dead code for inelastic plabs"
 - Remove dead code for inelastic plabs

Changes: https://git.openjdk.org/shenandoah/pull/421/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=421&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8330071
  Stats: 6 lines in 1 file changed: 4 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/421.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/421/head:pull/421

PR: https://git.openjdk.org/shenandoah/pull/421

From cslucas at openjdk.org  Thu Apr 11 15:36:48 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 11 Apr 2024 15:36:48 GMT
Subject: RFR: 8241503: C2: Share MacroAssembler between mach nodes during
 code emission [v13]
In-Reply-To: <hbW78wfehwwUMAX_Bc03EJjClx5Dm2N7h-DIHgyGb2g=.1e0e5d12-79f1-4fb9-a36c-b1713e3972a2@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
 <hbW78wfehwwUMAX_Bc03EJjClx5Dm2N7h-DIHgyGb2g=.1e0e5d12-79f1-4fb9-a36c-b1713e3972a2@github.com>
Message-ID: <bv9b2st58I28Kq_Gpej1X1OU60IHe0DUCQi3BdGR5iY=.aba7ac1a-0a1c-4054-91ab-e5e04c6b4279@github.com>

On Wed, 10 Apr 2024 23:53:00 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> # Description
>> 
>> Please review this PR with a patch to re-use the same C2_MacroAssembler object to emit all instructions in the same compilation unit.
>> 
>> Overall, the change is pretty simple. However, due to the renaming of the variable to access C2_MacroAssembler, from `_masm.` to `masm->`, and also some method prototype changes, the patch became quite large.
>> 
>> # Help Needed for Testing
>> 
>> I don't have access to all platforms necessary to test this. I hope some other folks can help with testing on `S390`, `RISC-V` and `PPC`.
>
> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove unused operands in arm.ad

Thank you all for reviewing!!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16484#issuecomment-2049976529

From cslucas at openjdk.org  Thu Apr 11 15:47:53 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 11 Apr 2024 15:47:53 GMT
Subject: Integrated: 8241503: C2: Share MacroAssembler between mach nodes
 during code emission
In-Reply-To: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
References: <ExCrwPns55nw2P62UI3vzSVTmFCh3dzJq7u3Kugp9EA=.c20cd86c-2c16-4869-ba7f-76f2a53821d0@github.com>
Message-ID: <QrUZ-AFWjokvRQsVCpfhG94B0u5n7jCH0Bt3Mi9e7bg=.03f93cbe-a8c5-4f2c-9118-0fe1de227861@github.com>

On Thu, 2 Nov 2023 22:17:43 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

> # Description
> 
> Please review this PR with a patch to re-use the same C2_MacroAssembler object to emit all instructions in the same compilation unit.
> 
> Overall, the change is pretty simple. However, due to the renaming of the variable to access C2_MacroAssembler, from `_masm.` to `masm->`, and also some method prototype changes, the patch became quite large.
> 
> # Help Needed for Testing
> 
> I don't have access to all platforms necessary to test this. I hope some other folks can help with testing on `S390`, `RISC-V` and `PPC`.

This pull request has now been integrated.

Changeset: 31ee5108
Author:    Cesar Soares Lucas <cslucas at openjdk.org>
Committer: Martin Doerr <mdoerr at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/31ee5108e059afae0a3809947adb7b91e19baec6
Stats:     2144 lines in 60 files changed: 118 ins; 431 del; 1595 mod

8241503: C2: Share MacroAssembler between mach nodes during code emission

Reviewed-by: kvn, mdoerr, amitkumar, lucy

-------------

PR: https://git.openjdk.org/jdk/pull/16484

From kdnilsen at openjdk.org  Thu Apr 11 15:58:02 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 11 Apr 2024 15:58:02 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
In-Reply-To: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
Message-ID: <ah0yw5_KuNpungZQtgWyyQ8GoGddvZYjX6t3AeBIsbI=.88e3af55-1c82-44e0-b711-1c5069503f48@github.com>

On Thu, 11 Apr 2024 15:20:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.

It would appear that our existing CI performance pipeline tests are not sufficiently sensitive to failure to expand OLD when necessary to support mixed evacuations.  Here's a test that does show good sensitivity.

On existing master, the results are:

Customer Preparation Processing[100000]: P50(295) P95(448) P99(768) P99.9(58408) P99.99(351320) P99.999(414070) P100(424719)

[1145.223s][info   ][gc,stats       ] 12939 Completed GCs
[1145.223s][info   ][gc,stats       ] 12894 Successful Concurrent GCs (99.65%)
[1145.223s][info   ][gc,stats       ]    12 Completed Old GCs (0.09%)
[1145.223s][info   ][gc,stats       ]      96 mixed
[1145.223s][info   ][gc,stats       ]    30 Degenerated GCs (0.23%)
[1145.223s][info   ][gc,stats       ]     4 Full GCs (0.03%)

With this PR, I get:

Customer Preparation Processing[100000]: P50(293) P95(414) P99(484) P99.9(738) P99.99(4977) P99.999(103219) P100(113555)

[1145.299s][info   ][gc,stats    ]   844 Completed GCs
[1145.299s][info   ][gc,stats    ]   828 Successful Concurrent GCs (98.10%)
[1145.299s][info   ][gc,stats    ]    14 Completed Old GCs (1.66%)
[1145.299s][info   ][gc,stats    ]     238 mixed
[1145.299s][info   ][gc,stats    ]     2 Degenerated GCs (0.24%)
[1145.299s][info   ][gc,stats    ]     0 Full GCs (0.00%)

The test is Extremem based:

~/github/shenandoah.allow-old-to-expand-again/build/linux-x86_64-server-release/jdk/bin/java \
  -XX:+AlwaysPreTouch -XX:+DisableExplicitGC -Xms3g -Xmx3g \
  -XX:+UseShenandoahGC \
  -XX:ShenandoahGCMode=generational -XX:+UnlockExperimentalVMOptions \
  -XX:-ShenandoahPacing -XX:+UsePerfData \
  -XX:ShenandoahGuaranteedYoungGCInterval=12000 \
  -Xlog:"gc*=info,ergo" \
  -XX:+UnlockDiagnosticVMOptions \
  -XX:-UseCompressedOops -XX:-UseCompressedClassPointers \
  -XX:+UnlockExperimentalVMOptions \
  -jar ~/github/heapothesys/Extremem/target/extremem-1.0-SNAPSHOT.jar \
  -dInitializationDelay=60s -dDictionarySize=400000 -dNumCustomers=350000 \
  -dNumProducts=12000 -dCustomerThreads=200 -dCustomerPeriod=2s -dCustomerThinkTime=1s \
  -dKeywordSearchCount=4 -dServerThreads=5 -dServerPeriod=1s -dProductNameLength=10 \
  -dBrowsingHistoryQueueCount=5 \
  -dSalesTransactionQueueCount=5 \
  -dProductDescriptionLength=64 -dProductReplacementPeriod=5s -dProductReplacementCount=3500 \
  -dCustomerReplacementPeriod=5s -dCustomerReplacementCount=4000 -dBrowsingExpiration=1m \
  -dPhasedUpdates=true \
  -dPhasedUpdateInterval=30s \
  -dSimulationDuration=18m -dResponseTimeMeasurements=100000

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/421#issuecomment-2050017928

From azafari at openjdk.org  Thu Apr 11 15:59:50 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 15:59:50 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to family
 of os::reserve/commit/uncommit memory API
Message-ID: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>

`MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.

Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

-------------

Commit messages:
 - 8330076: [NMT] add/make a mandatory MEMFLAGS argument to family of os::reserve/commit/uncommit memory API
 - some missed changes
 - reserve memory functions are also updated to have mandatory MEMFLAGS.
 - uncommit has also mandatory MEMFLAGS arg
 - virtual memory commit has mandatory MEMFLAGS arg.

Changes: https://git.openjdk.org/jdk/pull/18745/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8330076
  Stats: 299 lines in 59 files changed: 13 ins; 39 del; 247 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Thu Apr 11 16:10:10 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 16:10:10 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  fixed missing change.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/c45aa21c..1f7d6fbb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From stuefe at openjdk.org  Thu Apr 11 16:10:11 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Thu, 11 Apr 2024 16:10:11 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <It0zIs5B7tugN_xQGff4Ud7zO0cTbCiBQ2NAjEck64s=.35d093c1-c235-44c9-9442-394759635252@github.com>

On Thu, 11 Apr 2024 15:54:38 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

The general thrust of this is okay, and I think it makes sense. It also combines nicely with the VMATree work Johan does.

One question though, why do you want to abolish default values for MEMFLAGS? Allowing default values would reduce this patch by quite a bit, and make backporting less painful.

Another idea: To alleviate the need to pass MEMFLAGS all the time, could we have something like a "active MEMFLAGS" state per Thread, and set that stack-based with a XXMark object? That way, one could say at the entrance of Metaspace, for instance, "whatever is allocated under the scope of this function, please mark with mtMetaspace".

Just an idea. Otherwise, I think this is good, and thanks for doing this onerous work.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2050037989

From kdnilsen at openjdk.org  Thu Apr 11 16:12:42 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 11 Apr 2024 16:12:42 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v2]
In-Reply-To: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
Message-ID: <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>

> This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Spelling errors in comment

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/421/files
  - new: https://git.openjdk.org/shenandoah/pull/421/files/d8813000..f23113f5

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=421&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=421&range=00-01

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/421.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/421/head:pull/421

PR: https://git.openjdk.org/shenandoah/pull/421

From wkemper at openjdk.org  Thu Apr 11 16:12:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 11 Apr 2024 16:12:43 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v2]
In-Reply-To: <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
 <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
Message-ID: <Loosu0qGyHFPl-al7KGcYq4J5vOj2Cn70DPo_5BmujY=.2ea4873e-4a2c-49b8-9f49-93fb4fc02be2@github.com>

On Thu, 11 Apr 2024 16:09:14 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Spelling errors in comment

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/421#pullrequestreview-1994630958

From azafari at openjdk.org  Thu Apr 11 16:21:52 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 16:21:52 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v3]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <npwvm2ufmP_51jvaKYJpajtELeAeuLiEfaIjm0cMTgA=.0d1e064f-8cd7-4d26-b866-2910b3d4fcec@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  fixed shenandoah missed changes.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/1f7d6fbb..b009556e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Thu Apr 11 16:24:41 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 16:24:41 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API
In-Reply-To: <It0zIs5B7tugN_xQGff4Ud7zO0cTbCiBQ2NAjEck64s=.35d093c1-c235-44c9-9442-394759635252@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <It0zIs5B7tugN_xQGff4Ud7zO0cTbCiBQ2NAjEck64s=.35d093c1-c235-44c9-9442-394759635252@github.com>
Message-ID: <czKeAMR5G2shmCPI4EAQnAtOnlNpw6974_iNR4o6feE=.679289c4-04c1-4161-b948-74698a7e2fe4@github.com>

On Thu, 11 Apr 2024 16:06:35 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> One question though, why do you want to abolish default values for MEMFLAGS? Allowing default values would reduce this patch by quite a bit, and make backporting less painful.

To be sure that all the calls without MEMFLAGS are changed in PR, Ii made it mandatory to let compiler find all the instances.  There are some cases hidden or not found easily if I let the arg be optional.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2050063555

From azafari at openjdk.org  Thu Apr 11 16:30:40 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 16:30:40 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API
In-Reply-To: <It0zIs5B7tugN_xQGff4Ud7zO0cTbCiBQ2NAjEck64s=.35d093c1-c235-44c9-9442-394759635252@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <It0zIs5B7tugN_xQGff4Ud7zO0cTbCiBQ2NAjEck64s=.35d093c1-c235-44c9-9442-394759635252@github.com>
Message-ID: <AOE_tOA2nq2mYmnCsPAso6ctj2Onkp0AQbyEdHFa4Uk=.306398b3-ac3f-4671-b955-f37dd1de6840@github.com>

On Thu, 11 Apr 2024 16:06:35 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> Another idea: To alleviate the need to pass MEMFLAGS all the time, could we have something like a "active MEMFLAGS" state per Thread, and set that stack-based with a XXMark object? That way, one could say at the entrance of Metaspace, for instance, "whatever is allocated under the scope of this function, please mark with mtMetaspace".

Not sure if I understood your idea, the question is if a thread always uses only ONE type of memory and not mix of them? For example, CDS uses both mtClass and mtClassShared. If a Thread has an active MEMFLAG, it has to switch this flag between A and B whenever it uses type A or B.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2050072695

From kdnilsen at openjdk.org  Thu Apr 11 16:35:05 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 11 Apr 2024 16:35:05 GMT
Subject: RFR: 8329789: GenShen: Over assertive assert when scanning
 remembered set
In-Reply-To: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
References: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
Message-ID: <WmWOD1A-yNOP1cPEWtiux199iCUfVFFbcP7gq8QmaBQ=.d62f5d88-8056-4caa-9b7a-5a73670cb32a@github.com>

On Fri, 5 Apr 2024 17:45:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When scanning the remembered set for update-refs, it is possible for the scanner to encounter a card that was dirtied by a mutator during the scan. This is by design.

src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.inline.hpp line 776:

> 774: 
> 775:       // If we are using the write table (during update refs, e.g.), a mutator may dirty
> 776:       // a card at any time. This is fine for the algorithm below because it is only

Didn't finish your thought in this comment.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/418#discussion_r1561305798

From stefank at openjdk.org  Thu Apr 11 16:37:43 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 11 Apr 2024 16:37:43 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
Message-ID: <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>

On Thu, 11 Apr 2024 16:10:10 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fixed missing change.

I think it is good that we are fixing this. I've on the verge of doing it myself a couple of times. I've left an initial set of comments below. It's not an exhaustive list, but I think it is a good set to start with.

src/hotspot/os/bsd/gc/x/xPhysicalMemoryBacking_bsd.cpp line 81:

> 79: 
> 80:   // Reserve address space for backing memory
> 81:   _base = (uintptr_t)os::reserve_memory(max_capacity, false, mtJavaHeap);

I think this should be using !MemExec and not a raw 'false' argument.

With that said, I really think it would be best if we actually split os::reserve_memory into two distinct functions:
1) os::reserve_memory(max_capacity, mtJavaHeap) // Non-executable memory
2) os::reserve_executable_memory(max_capacity, mtCode) // Executable

Maybe we could think about that in separate RFE.

src/hotspot/os/bsd/gc/z/zPhysicalMemoryBacking_bsd.cpp line 82:

> 80: 
> 81:   // Reserve address space for backing memory
> 82:   _base = (uintptr_t)os::reserve_memory(max_capacity, false, mtJavaHeap);

Use !MemExec - this goes for all other places in the patch that fills in the `exec` parameter.

src/hotspot/os/linux/os_linux.cpp line 4684:

> 4682:   char* hint = (char*)(os::Linux::initial_thread_stack_bottom() -
> 4683:                        (StackOverflow::stack_guard_zone_size() + page_size));
> 4684:   char* codebuf = os::attempt_reserve_memory_at(hint, page_size, false, mtInternal);

Should these be `mtInternal` or is there a `mtStack` that is more suitable?

src/hotspot/os/windows/os_windows.cpp line 3137:

> 3135:   // If reservation failed, return null
> 3136:   if (p_buf == nullptr) return nullptr;
> 3137:   MemTracker::record_virtual_memory_reserve((address)p_buf, size_of_reserve, CALLER_PC, mtNone);

Why is this (and the other places in this file) using `mtNone`? Shouldn't it at least be using `mtInternal`?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1414:

> 1412:   assert(SafepointSynchronize::is_at_safepoint(), "safe iteration is only available during safepoints");
> 1413: 
> 1414:   if (!_aux_bitmap_region_special && !os::commit_memory((char*)_aux_bitmap_region.start(), _aux_bitmap_region.byte_size(), false, mtJavaHeap)) {

Why is this `mtJavaHeap` and not `mtGC`?

src/hotspot/share/gc/z/zMarkStackAllocator.cpp line 107:

> 105: 
> 106:     const uintptr_t shrink_start = _end - shrink_size;
> 107:     os::uncommit_memory((char*)shrink_start, shrink_size, mtGC, false /* executable */);

`uncommit_memory` places the order of executable and flags differently to what we have for `commmit_memory_or_exit`. We might want to consider doing something about the order here.

src/hotspot/share/memory/metaspace.cpp line 592:

> 590:     // Fallback: reserve anywhere
> 591:     log_debug(metaspace, map)("Trying anywhere...");
> 592:     result = os::reserve_memory_aligned(size, Metaspace::reserve_alignment(), false, mtMetaspace);

It's unclear to me if some of these `mtMetaspace` should be `mtClass`. This comment applies to other places where we're setting up memory for the compressed class space.

src/hotspot/share/memory/virtualspace.cpp line 71:

> 69: ReservedSpace::ReservedSpace(size_t size,
> 70:                              size_t alignment,
> 71:                              size_t page_size, MEMFLAGS flag,

I think this function was written to have one argument per line. You should probably keep the style.

I'm also unsure why this param is put as the next to last param instead of the last, as we do in many other places.

src/hotspot/share/memory/virtualspace.cpp line 366:

> 364:   ReservedSpace space;
> 365:   space.initialize_members(base, size, alignment, page_size, special, executable);
> 366:   space.set_nmt_flag(flag);

Why is this calling a set_nmt_flag instead of making initialize_member take a flag?

src/hotspot/share/memory/virtualspace.cpp line 693:

> 691:   _special                = false;
> 692:   _executable             = false;
> 693:   _nmt_flag                = mtNone;

Weird indentation.

src/hotspot/share/memory/virtualspace.hpp line 45:

> 43:   bool   _special;
> 44:   int    _fd_for_heap;
> 45:   MEMFLAGS _nmt_flag;

Indentation is now off.

src/hotspot/share/memory/virtualspace.hpp line 72:

> 70: 
> 71:   inline MEMFLAGS nmt_flag() { return _nmt_flag; }
> 72:   inline void set_nmt_flag(MEMFLAGS flag) { _nmt_flag = flag; }

No need for the inline specifier here.

-------------

Changes requested by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-1994644693
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561286933
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561287120
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561291740
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561294869
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561297501
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561300014
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561301420
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561305069
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561305892
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561306136
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561306502
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561307268

From azafari at openjdk.org  Thu Apr 11 16:43:42 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 16:43:42 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
Message-ID: <u047VVg6mHMnnVJDKvoKSX1-Jqf4fKh711H2f7mhM8U=.8a64ce76-886d-41bc-9e12-7eb47ee44882@github.com>

On Thu, 11 Apr 2024 16:19:08 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed missing change.
>
> src/hotspot/os/linux/os_linux.cpp line 4684:
> 
>> 4682:   char* hint = (char*)(os::Linux::initial_thread_stack_bottom() -
>> 4683:                        (StackOverflow::stack_guard_zone_size() + page_size));
>> 4684:   char* codebuf = os::attempt_reserve_memory_at(hint, page_size, false, mtInternal);
> 
> Should these be `mtInternal` or is there a `mtStack` that is more suitable?

In line 4699, a few lines later, the original developer used `mtInternal`. I copied it here too.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561315756

From wkemper at openjdk.org  Thu Apr 11 17:11:19 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 11 Apr 2024 17:11:19 GMT
Subject: RFR: 8329789: GenShen: Over assertive assert when scanning
 remembered set [v2]
In-Reply-To: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
References: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
Message-ID: <M_3rUe0ecovEEKHyD9HdllgCSTYS-hIeehA8douI_mY=.bf5873bc-d1e9-4cee-b82d-a51016a4900c@github.com>

> When scanning the remembered set for update-refs, it is possible for the scanner to encounter a card that was dirtied by a mutator during the scan. This is by design.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Finish comment

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/418/files
  - new: https://git.openjdk.org/shenandoah/pull/418/files/7bd0251e..c4e0be2b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=418&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=418&range=00-01

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/418.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/418/head:pull/418

PR: https://git.openjdk.org/shenandoah/pull/418

From wkemper at openjdk.org  Thu Apr 11 17:11:19 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 11 Apr 2024 17:11:19 GMT
Subject: RFR: 8329789: GenShen: Over assertive assert when scanning
 remembered set [v2]
In-Reply-To: <WmWOD1A-yNOP1cPEWtiux199iCUfVFFbcP7gq8QmaBQ=.d62f5d88-8056-4caa-9b7a-5a73670cb32a@github.com>
References: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
 <WmWOD1A-yNOP1cPEWtiux199iCUfVFFbcP7gq8QmaBQ=.d62f5d88-8056-4caa-9b7a-5a73670cb32a@github.com>
Message-ID: <T34FSncYEMRce6ZMNyRnasQyhU2zSX9M86Vm3PrhMLQ=.99ed6254-8e25-43e8-a2cc-9b84e77d13de@github.com>

On Thu, 11 Apr 2024 16:31:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Finish comment
>
> src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.inline.hpp line 776:
> 
>> 774: 
>> 775:       // If we are using the write table (during update refs, e.g.), a mutator may dirty
>> 776:       // a card at any time. This is fine for the algorithm below because it is only
> 
> Didn't finish your thought in this comment.

Hmm, strange. Fixed it.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/418#discussion_r1561345129

From azafari at openjdk.org  Thu Apr 11 18:08:44 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 18:08:44 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
Message-ID: <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>

On Thu, 11 Apr 2024 16:15:04 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed missing change.
>
> src/hotspot/os/bsd/gc/x/xPhysicalMemoryBacking_bsd.cpp line 81:
> 
>> 79: 
>> 80:   // Reserve address space for backing memory
>> 81:   _base = (uintptr_t)os::reserve_memory(max_capacity, false, mtJavaHeap);
> 
> I think this should be using !MemExec and not a raw 'false' argument.
> 
> With that said, I really think it would be best if we actually split os::reserve_memory into two distinct functions:
> 1) os::reserve_memory(max_capacity, mtJavaHeap) // Non-executable memory
> 2) os::reserve_executable_memory(max_capacity, mtCode) // Executable
> 
> Maybe we could think about that in separate RFE.

`false/true` constants are not used in executable args.
separate reserve_memory functions can be left for another RFE.

> src/hotspot/os/bsd/gc/z/zPhysicalMemoryBacking_bsd.cpp line 82:
> 
>> 80: 
>> 81:   // Reserve address space for backing memory
>> 82:   _base = (uintptr_t)os::reserve_memory(max_capacity, false, mtJavaHeap);
> 
> Use !MemExec - this goes for all other places in the patch that fills in the `exec` parameter.

Fixed.

> src/hotspot/os/windows/os_windows.cpp line 3137:
> 
>> 3135:   // If reservation failed, return null
>> 3136:   if (p_buf == nullptr) return nullptr;
>> 3137:   MemTracker::record_virtual_memory_reserve((address)p_buf, size_of_reserve, CALLER_PC, mtNone);
> 
> Why is this (and the other places in this file) using `mtNone`? Shouldn't it at least be using `mtInternal`?

Fixed.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1414:
> 
>> 1412:   assert(SafepointSynchronize::is_at_safepoint(), "safe iteration is only available during safepoints");
>> 1413: 
>> 1414:   if (!_aux_bitmap_region_special && !os::commit_memory((char*)_aux_bitmap_region.start(), _aux_bitmap_region.byte_size(), false, mtJavaHeap)) {
> 
> Why is this `mtJavaHeap` and not `mtGC`?

Fixed.

> src/hotspot/share/gc/z/zMarkStackAllocator.cpp line 107:
> 
>> 105: 
>> 106:     const uintptr_t shrink_start = _end - shrink_size;
>> 107:     os::uncommit_memory((char*)shrink_start, shrink_size, mtGC, false /* executable */);
> 
> `uncommit_memory` places the order of executable and flags differently to what we have for `commmit_memory_or_exit`. We might want to consider doing something about the order here.

Fixed.

> src/hotspot/share/memory/virtualspace.cpp line 71:
> 
>> 69: ReservedSpace::ReservedSpace(size_t size,
>> 70:                              size_t alignment,
>> 71:                              size_t page_size, MEMFLAGS flag,
> 
> I think this function was written to have one argument per line. You should probably keep the style.
> 
> I'm also unsure why this param is put as the next to last param instead of the last, as we do in many other places.

Put in separate line.
the last param is optional, and flag is to be mandatory.

> src/hotspot/share/memory/virtualspace.cpp line 693:
> 
>> 691:   _special                = false;
>> 692:   _executable             = false;
>> 693:   _nmt_flag                = mtNone;
> 
> Weird indentation.

Fixed.

> src/hotspot/share/memory/virtualspace.hpp line 45:
> 
>> 43:   bool   _special;
>> 44:   int    _fd_for_heap;
>> 45:   MEMFLAGS _nmt_flag;
> 
> Indentation is now off.

Fixed.

> src/hotspot/share/memory/virtualspace.hpp line 72:
> 
>> 70: 
>> 71:   inline MEMFLAGS nmt_flag() { return _nmt_flag; }
>> 72:   inline void set_nmt_flag(MEMFLAGS flag) { _nmt_flag = flag; }
> 
> No need for the inline specifier here.

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561408632
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561409363
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561410373
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561412485
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561413236
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561418594
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561421020
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561423257
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561424255

From jsjolen at openjdk.org  Thu Apr 11 18:08:44 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Thu, 11 Apr 2024 18:08:44 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
 <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>
Message-ID: <KS6Skd4acbP0NpDLjxR2P1KTxMxztAxpGQbhJZT2X_A=.44b57499-a203-49fe-a31a-19ab2ab8135c@github.com>

On Thu, 11 Apr 2024 17:58:37 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/os/bsd/gc/x/xPhysicalMemoryBacking_bsd.cpp line 81:
>> 
>>> 79: 
>>> 80:   // Reserve address space for backing memory
>>> 81:   _base = (uintptr_t)os::reserve_memory(max_capacity, false, mtJavaHeap);
>> 
>> I think this should be using !MemExec and not a raw 'false' argument.
>> 
>> With that said, I really think it would be best if we actually split os::reserve_memory into two distinct functions:
>> 1) os::reserve_memory(max_capacity, mtJavaHeap) // Non-executable memory
>> 2) os::reserve_executable_memory(max_capacity, mtCode) // Executable
>> 
>> Maybe we could think about that in separate RFE.
>
> `false/true` constants are not used in executable args.
> separate reserve_memory functions can be left for another RFE.

The executable argument really is only false in the original, can we keep this from doing any functional changes here and keep that to separate PR:s?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561415984

From jsjolen at openjdk.org  Thu Apr 11 18:18:45 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Thu, 11 Apr 2024 18:18:45 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v3]
In-Reply-To: <npwvm2ufmP_51jvaKYJpajtELeAeuLiEfaIjm0cMTgA=.0d1e064f-8cd7-4d26-b866-2910b3d4fcec@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <npwvm2ufmP_51jvaKYJpajtELeAeuLiEfaIjm0cMTgA=.0d1e064f-8cd7-4d26-b866-2910b3d4fcec@github.com>
Message-ID: <uy3fNX3oKYtPUN4xYz-aW-22sRGQ-gTQllhEAa2tvkA=.90f06eac-4978-4b9e-a68e-b81aec82dd51@github.com>

On Thu, 11 Apr 2024 16:21:52 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fixed shenandoah missed changes.

Hi Afshin,

Thank you for this!

I found a couple of things.

@tstuefe , would you mind having a look at the Metaspace changes?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2267:

> 2265:   char* start = (char*) _bitmap_region.start() + off;
> 2266: 
> 2267:   if (!os::commit_memory(start, len, false)) {

I think this should probably be `mtGC`.  @shipilev, you don't happen to know whether this should be accounted under the Java heap? Thank you.

src/hotspot/share/nmt/virtualMemoryTracker.cpp line 460:

> 458:   assert(_reserved_regions != nullptr, "Sanity check");
> 459: 
> 460:   ReservedMemoryRegion  rgn(addr, size, NativeCallStack::empty_stack(), flag);

Instead, change the constructor so that it takes a flag?

```c++
  ReservedMemoryRegion(address base, size_t size, MEMFLAGS flag) :
    VirtualMemoryRegion(base, size), _stack(NativeCallStack::empty_stack()), _flag(flag) { }


Or does that break somewhere else?

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-1994902004
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561430517
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561443098

From shade at openjdk.org  Thu Apr 11 18:25:42 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 11 Apr 2024 18:25:42 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v3]
In-Reply-To: <uy3fNX3oKYtPUN4xYz-aW-22sRGQ-gTQllhEAa2tvkA=.90f06eac-4978-4b9e-a68e-b81aec82dd51@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <npwvm2ufmP_51jvaKYJpajtELeAeuLiEfaIjm0cMTgA=.0d1e064f-8cd7-4d26-b866-2910b3d4fcec@github.com>
 <uy3fNX3oKYtPUN4xYz-aW-22sRGQ-gTQllhEAa2tvkA=.90f06eac-4978-4b9e-a68e-b81aec82dd51@github.com>
Message-ID: <Z1PQa0kngvNqgmw9Z7ayhxOR7Ivxv4eMepW1PU6oAGY=.6960f32c-2fa6-4737-ace2-5493d8ea727b@github.com>

On Thu, 11 Apr 2024 18:09:08 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed shenandoah missed changes.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2267:
> 
>> 2265:   char* start = (char*) _bitmap_region.start() + off;
>> 2266: 
>> 2267:   if (!os::commit_memory(start, len, false)) {
> 
> I think this should probably be `mtGC`.  @shipilev, you don't happen to know whether this should be accounted under the Java heap? Thank you.

This is bitmap slice, so it is not Java heap, it is `mtGC`. In the sister method, `uncommit_bitmap_slice`, we do the right thing: `mtGC`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561459688

From ysr at openjdk.org  Thu Apr 11 19:58:53 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 11 Apr 2024 19:58:53 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v2]
In-Reply-To: <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
 <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
Message-ID: <twbyUdzayvnJk3dLOAqcaf0Wgj8R-dycCaOOyHYDnsI=.2687ba03-be80-4c54-b55d-668d464579e4@github.com>

On Thu, 11 Apr 2024 16:12:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Spelling errors in comment

What would go wrong if, as indicated in your comments, you were to return `heap - min_young` for `max(old)`, and `heap - max_young` for `min(old)` ?

If nothing would go wrong, why not do that? If something would go wrong if one did that, may be that needs to be documented along with the comment you added.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/421#pullrequestreview-1995160319

From kdnilsen at openjdk.org  Thu Apr 11 20:03:01 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 11 Apr 2024 20:03:01 GMT
Subject: RFR: 8329699: GenShen: Move promotion logic out of shHeap and
 shHeapRegion [v2]
In-Reply-To: <9GiLUn4uHDiTw4ZeLPh8R5TtV5pWVywDzlVaQ7XdKq8=.e90a858a-28f2-4075-a659-a5b219d785d6@github.com>
References: <G9ZJV5hPgMHKOgnZjpXYbkMfdeSGA3nabJOCQh_4-Fo=.62da6e6e-16bd-44b2-b4fb-f10a5a4eb676@github.com>
 <9GiLUn4uHDiTw4ZeLPh8R5TtV5pWVywDzlVaQ7XdKq8=.e90a858a-28f2-4075-a659-a5b219d785d6@github.com>
Message-ID: <I1uQJv9FKg_dFhPjdNms6ZIh0TmhZ6FsbfyfWGDaCyA=.a84ebb42-bb58-4eb7-b9ac-53beedc894db@github.com>

On Thu, 4 Apr 2024 18:40:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> The code to handle promotions during evacuation has been moved into a new class in its own files: `shGenerationalEvacuationTask`
>> 
>> The methods for managing `plabs` have been moved into `shGenerationalHeap`. `plabs` are now also only created when running the generational mode.
>> 
>> 
>> -  inline HeapWord* allocate_from_plab(Thread* thread, size_t size, bool is_promotion);
>> -  HeapWord* allocate_from_plab_slow(Thread* thread, size_t size, bool is_promotion);
>> -  HeapWord* allocate_new_plab(size_t min_size, size_t word_size, size_t* actual_size);
>> -  void retire_plab(PLAB* plab);
>> -  void retire_plab(PLAB* plab, Thread* thread);
>> 
>> 
>> The code to handle promotion in place has been moved from `shHeapRegion` into `shGenerationalEvacuationTask`:
>> 
>> -  void promote_humongous();
>> -  void promote_in_place();
>> 
>> 
>> Unused methods related to global coalesce and fill have been deleted from `shHeapRegion`:
>> 
>> -  void global_oop_iterate_and_fill_dead(OopIterateClosure* cl);
>> -  void global_oop_iterate_objects_and_fill_dead(OopIterateClosure* cl);
>> -  void oop_iterate_humongous(OopIterateClosure* cl);
>> -  void oop_iterate_humongous(OopIterateClosure* cl, HeapWord* start, size_t words);
>> 
>> 
>> Additionally, these methods in `shHeap` which just delegated to old or young generation instances have more or less been inlined at the usage site (many `includes` have been fixed):
>> 
>> -  ShenandoahOldHeuristics* old_heuristics();
>> -  ShenandoahYoungHeuristics* young_heuristics();
>> -
>> -  bool doing_mixed_evacuations();
>> -  bool is_old_bitmap_stable() const;
>> -  bool is_gc_generation_young() const;
>> 
>> 
>> The boolean `is_promotion` that was passed through all of the allocation calls has been moved into `shAllocRequest` to reduce the upstream delta for `shHeap`.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix mac build

Very nice cleanup.  Thanks for your work on this.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/415#pullrequestreview-1995168043

From ysr at openjdk.org  Thu Apr 11 20:10:54 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 11 Apr 2024 20:10:54 GMT
Subject: RFR: 8329789: GenShen: Over assertive assert when scanning
 remembered set [v2]
In-Reply-To: <M_3rUe0ecovEEKHyD9HdllgCSTYS-hIeehA8douI_mY=.bf5873bc-d1e9-4cee-b82d-a51016a4900c@github.com>
References: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
 <M_3rUe0ecovEEKHyD9HdllgCSTYS-hIeehA8douI_mY=.bf5873bc-d1e9-4cee-b82d-a51016a4900c@github.com>
Message-ID: <YdCSESNJmCwUBYcLwRFsm-IVpDJA9pakJw1lVuyuvvo=.9a67a22c-b885-4c62-a1c2-69e078991554@github.com>

On Thu, 11 Apr 2024 17:11:19 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> When scanning the remembered set for update-refs, it is possible for the scanner to encounter a card that was dirtied by a mutator during the scan. This is by design.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Finish comment

LGTM!

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/418#pullrequestreview-1995179626

From ysr at openjdk.org  Thu Apr 11 20:10:54 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 11 Apr 2024 20:10:54 GMT
Subject: RFR: 8329789: GenShen: Over assertive assert when scanning
 remembered set [v2]
In-Reply-To: <T34FSncYEMRce6ZMNyRnasQyhU2zSX9M86Vm3PrhMLQ=.99ed6254-8e25-43e8-a2cc-9b84e77d13de@github.com>
References: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
 <WmWOD1A-yNOP1cPEWtiux199iCUfVFFbcP7gq8QmaBQ=.d62f5d88-8056-4caa-9b7a-5a73670cb32a@github.com>
 <T34FSncYEMRce6ZMNyRnasQyhU2zSX9M86Vm3PrhMLQ=.99ed6254-8e25-43e8-a2cc-9b84e77d13de@github.com>
Message-ID: <pCwHsRstqpa54s1rDiv06trHWem98BhzyRX2d_SPo3o=.b0b36af4-beda-49d1-a769-794ca9ee990e@github.com>

On Thu, 11 Apr 2024 17:08:16 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.inline.hpp line 776:
>> 
>>> 774: 
>>> 775:       // If we are using the write table (during update refs, e.g.), a mutator may dirty
>>> 776:       // a card at any time. This is fine for the algorithm below because it is only
>> 
>> Didn't finish your thought in this comment.
>
> Hmm, strange. Fixed it.

The weakening of the assert and the explanatory comment look good to me; thanks!

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/418#discussion_r1561586603

From azafari at openjdk.org  Thu Apr 11 20:34:42 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 20:34:42 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v3]
In-Reply-To: <Z1PQa0kngvNqgmw9Z7ayhxOR7Ivxv4eMepW1PU6oAGY=.6960f32c-2fa6-4737-ace2-5493d8ea727b@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <npwvm2ufmP_51jvaKYJpajtELeAeuLiEfaIjm0cMTgA=.0d1e064f-8cd7-4d26-b866-2910b3d4fcec@github.com>
 <uy3fNX3oKYtPUN4xYz-aW-22sRGQ-gTQllhEAa2tvkA=.90f06eac-4978-4b9e-a68e-b81aec82dd51@github.com>
 <Z1PQa0kngvNqgmw9Z7ayhxOR7Ivxv4eMepW1PU6oAGY=.6960f32c-2fa6-4737-ace2-5493d8ea727b@github.com>
Message-ID: <c_UsF6ViNOc1RjtmSBD5DPtRXvyub4vUaPGEkTtPmWg=.45b8bbf4-07e1-4fbb-b80d-79cbec481f11@github.com>

On Thu, 11 Apr 2024 18:21:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2267:
>> 
>>> 2265:   char* start = (char*) _bitmap_region.start() + off;
>>> 2266: 
>>> 2267:   if (!os::commit_memory(start, len, false)) {
>> 
>> I think this should probably be `mtGC`.  @shipilev, you don't happen to know whether this should be accounted under the Java heap? Thank you.
>
> This is bitmap slice, so it is not Java heap, it is `mtGC`. In the sister method, `uncommit_bitmap_slice`, we do the right thing: `mtGC`.

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561641931

From azafari at openjdk.org  Thu Apr 11 20:34:44 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 20:34:44 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
Message-ID: <r5lvmZLs8692S6WA76-tMhRa5lDjOhDCc86IHLV8BEs=.d6d3d03c-18a8-485d-a9ab-4757d7adb4b7@github.com>

On Thu, 11 Apr 2024 16:27:41 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed missing change.
>
> src/hotspot/share/memory/metaspace.cpp line 592:
> 
>> 590:     // Fallback: reserve anywhere
>> 591:     log_debug(metaspace, map)("Trying anywhere...");
>> 592:     result = os::reserve_memory_aligned(size, Metaspace::reserve_alignment(), false, mtMetaspace);
> 
> It's unclear to me if some of these `mtMetaspace` should be `mtClass`. This comment applies to other places where we're setting up memory for the compressed class space.

Anywhere compressed class is used, the flag is set to `mtClass`.

> src/hotspot/share/memory/virtualspace.cpp line 366:
> 
>> 364:   ReservedSpace space;
>> 365:   space.initialize_members(base, size, alignment, page_size, special, executable);
>> 366:   space.set_nmt_flag(flag);
> 
> Why is this calling a set_nmt_flag instead of making initialize_member take a flag?

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561644315
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561645352

From azafari at openjdk.org  Thu Apr 11 20:40:44 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 20:40:44 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v3]
In-Reply-To: <uy3fNX3oKYtPUN4xYz-aW-22sRGQ-gTQllhEAa2tvkA=.90f06eac-4978-4b9e-a68e-b81aec82dd51@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <npwvm2ufmP_51jvaKYJpajtELeAeuLiEfaIjm0cMTgA=.0d1e064f-8cd7-4d26-b866-2910b3d4fcec@github.com>
 <uy3fNX3oKYtPUN4xYz-aW-22sRGQ-gTQllhEAa2tvkA=.90f06eac-4978-4b9e-a68e-b81aec82dd51@github.com>
Message-ID: <5xywkAFrVzkc3ZPUaxxUf4vn4uEpqvppSDHPdjnnzbY=.cb922cee-aef8-49b5-9490-1315da1299c0@github.com>

On Thu, 11 Apr 2024 18:13:55 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   fixed shenandoah missed changes.
>
> src/hotspot/share/nmt/virtualMemoryTracker.cpp line 460:
> 
>> 458:   assert(_reserved_regions != nullptr, "Sanity check");
>> 459: 
>> 460:   ReservedMemoryRegion  rgn(addr, size, NativeCallStack::empty_stack(), flag);
> 
> Instead, change the constructor so that it takes a flag?
> 
> ```c++
>   ReservedMemoryRegion(address base, size_t size, MEMFLAGS flag) :
>     VirtualMemoryRegion(base, size), _stack(NativeCallStack::empty_stack()), _flag(flag) { }
> 
> 
> Or does that break somewhere else?

Fixed. No problem.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1561653418

From kdnilsen at openjdk.org  Thu Apr 11 20:42:53 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 11 Apr 2024 20:42:53 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v2]
In-Reply-To: <twbyUdzayvnJk3dLOAqcaf0Wgj8R-dycCaOOyHYDnsI=.2687ba03-be80-4c54-b55d-668d464579e4@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
 <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
 <twbyUdzayvnJk3dLOAqcaf0Wgj8R-dycCaOOyHYDnsI=.2687ba03-be80-4c54-b55d-668d464579e4@github.com>
Message-ID: <pa2SWJVQhsnCEKthlHYWZy2H4ec94X1qB5HpGUbyE68=.73463036-0e61-4ac5-8ed8-6abd0e84bae0@github.com>

On Thu, 11 Apr 2024 19:56:13 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Spelling errors in comment
>
> What would go wrong if, as indicated in your comments, you were to return `heap - min_young` for `max(old)`, and `heap - max_young` for `min(old)` ?
> 
> If nothing would go wrong, why not do that? If something would go wrong if one did that, may be that needs to be documented along with the comment you added.

@ysramakrishna : Nothing would go wrong.  It's just more computation for no increased value.  Before transferring regions from generation src to generation dest, we confirm that the proposed transfer does not violate minimum size of src generation or maximum size of dest generation.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/421#issuecomment-2050499435

From azafari at openjdk.org  Thu Apr 11 21:25:55 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 21:25:55 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  review comments applied.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/b009556e..9d66735f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=02-03

  Stats: 122 lines in 35 files changed: 5 ins; 1 del; 116 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Thu Apr 11 21:25:55 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 11 Apr 2024 21:25:55 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
Message-ID: <tBONrLD1pVUd4DnUjazQ2qNIm2D1plafYtvO1wgwI5Q=.840c93e9-b638-4097-a321-ad58ffffec78@github.com>

On Thu, 11 Apr 2024 21:23:08 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   review comments applied.

All comments applied. Ready for another round of reviews.

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-1995509280

From ysr at openjdk.org  Thu Apr 11 21:48:10 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 11 Apr 2024 21:48:10 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v2]
In-Reply-To: <twbyUdzayvnJk3dLOAqcaf0Wgj8R-dycCaOOyHYDnsI=.2687ba03-be80-4c54-b55d-668d464579e4@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
 <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
 <twbyUdzayvnJk3dLOAqcaf0Wgj8R-dycCaOOyHYDnsI=.2687ba03-be80-4c54-b55d-668d464579e4@github.com>
Message-ID: <AoAWt0eOeU99WTVBBOjbrZts9FHi_hu7xcUtpnA2h8M=.5a36d443-10f2-4d50-87f6-bd24f2dafdf1@github.com>

On Thu, 11 Apr 2024 19:56:13 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Spelling errors in comment
>
> What would go wrong if, as indicated in your comments, you were to return `heap - min_young` for `max(old)`, and `heap - max_young` for `min(old)` ?
> 
> If nothing would go wrong, why not do that? If something would go wrong if one did that, may be that needs to be documented along with the comment you added.

> @ysramakrishna : Nothing would go wrong. It's just more computation for no increased value. Before transferring regions from generation src to generation dest, we confirm that the proposed transfer does not violate minimum size of src generation or maximum size of dest generation.

I'd err in that case on the side of the trivial extra work in the method to keep its semantics clear and complete, rather than expect its clients to carry that knowledge, best to avoid surprises and keep the code maintainable. I doubt that the little extra compute is worth the potential future trouble.

I realize this view is somewhat subjective, so I'll let you decide which way you go on this, even though I personally vote for clarity and cleanliness in code rather than cleverness and frugality that might lead to less maintainability in the future. 

Reviewed!

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/421#issuecomment-2050606302

From ysr at openjdk.org  Thu Apr 11 21:48:10 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 11 Apr 2024 21:48:10 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v2]
In-Reply-To: <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
 <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
Message-ID: <5qr7iy6HiPqxYtQN-OR3TRciYfWzXIFNyKlTsmsamDU=.0403689b-c107-45d2-921f-081b7b6f8b2a@github.com>

On Thu, 11 Apr 2024 16:12:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Spelling errors in comment

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/421#pullrequestreview-1995537779

From wkemper at openjdk.org  Thu Apr 11 22:24:55 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 11 Apr 2024 22:24:55 GMT
Subject: Integrated: 8329789: GenShen: Over assertive assert when scanning
 remembered set
In-Reply-To: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
References: <4DwJmELWpKR6v56sgl_lSEF0Apv8_hBIAv8HAxn7p9c=.607271a2-b634-44f9-bef9-aa1cccf88911@github.com>
Message-ID: <QIz8irkqtbBiTIpXY-FsErLazEvMjYNyVwTkN8EVMTo=.e5642e3c-1ce8-444c-9267-9cb089f5ce23@github.com>

On Fri, 5 Apr 2024 17:45:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When scanning the remembered set for update-refs, it is possible for the scanner to encounter a card that was dirtied by a mutator during the scan. This is by design.

This pull request has now been integrated.

Changeset: 69281629
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/69281629a4a537b988f5dfd3a93733b3ddccd109
Stats:     5 lines in 1 file changed: 4 ins; 0 del; 1 mod

8329789: GenShen: Over assertive assert when scanning remembered set

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/418

From wkemper at openjdk.org  Thu Apr 11 22:51:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 11 Apr 2024 22:51:28 GMT
Subject: RFR: 8329789: GenShen: Over assertive assert when scanning remembered
 set
Message-ID: <DX93ICC_EgOyNTIhEme2nBZeD19BMcksujPX1BOeimg=.06665a81-efc9-4f46-bf6d-055e44e5c8b4@github.com>

Clean backport, assert only and assert only applies to non-product code.

-------------

Commit messages:
 - Backport 69281629a4a537b988f5dfd3a93733b3ddccd109

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/34/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=34&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329789
  Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/34.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/34/head:pull/34

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/34

From stefank at openjdk.org  Fri Apr 12 07:04:43 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 12 Apr 2024 07:04:43 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <KS6Skd4acbP0NpDLjxR2P1KTxMxztAxpGQbhJZT2X_A=.44b57499-a203-49fe-a31a-19ab2ab8135c@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
 <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>
 <KS6Skd4acbP0NpDLjxR2P1KTxMxztAxpGQbhJZT2X_A=.44b57499-a203-49fe-a31a-19ab2ab8135c@github.com>
Message-ID: <YVOF16KJ6kmUN4qSxeD64_fMH4pinOU3lRAhC31Im-k=.308dadd7-19ed-42e0-9946-f6820d6e4abd@github.com>

On Thu, 11 Apr 2024 18:02:05 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> `false/true` constants are not used in executable args.
>> separate reserve_memory functions can be left for another RFE.
>
> The executable argument really is only false in the original, can we keep this from doing any functional changes here and keep that to separate PR:s?

I'm not sure I understand. Are you proposing something else than what I proposed that we could do in a separate RFE?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562106153

From stefank at openjdk.org  Fri Apr 12 07:04:44 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 12 Apr 2024 07:04:44 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <u047VVg6mHMnnVJDKvoKSX1-Jqf4fKh711H2f7mhM8U=.8a64ce76-886d-41bc-9e12-7eb47ee44882@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
 <u047VVg6mHMnnVJDKvoKSX1-Jqf4fKh711H2f7mhM8U=.8a64ce76-886d-41bc-9e12-7eb47ee44882@github.com>
Message-ID: <Hyd4GUAZbPkI4OURRLZ4IYU-qy3C0IftDQtc1GFRoM4=.ab9992f5-b6a3-480c-a067-21708b0dbd00@github.com>

On Thu, 11 Apr 2024 16:40:59 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/os/linux/os_linux.cpp line 4684:
>> 
>>> 4682:   char* hint = (char*)(os::Linux::initial_thread_stack_bottom() -
>>> 4683:                        (StackOverflow::stack_guard_zone_size() + page_size));
>>> 4684:   char* codebuf = os::attempt_reserve_memory_at(hint, page_size, false, mtInternal);
>> 
>> Should these be `mtInternal` or is there a `mtStack` that is more suitable?
>
> In line 4699, a few lines later, the original developer used `mtInternal`. I copied it here too.

OK. Then this is fine for now.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562106768

From stuefe at openjdk.org  Fri Apr 12 07:42:42 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 12 Apr 2024 07:42:42 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API
In-Reply-To: <AOE_tOA2nq2mYmnCsPAso6ctj2Onkp0AQbyEdHFa4Uk=.306398b3-ac3f-4671-b955-f37dd1de6840@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <It0zIs5B7tugN_xQGff4Ud7zO0cTbCiBQ2NAjEck64s=.35d093c1-c235-44c9-9442-394759635252@github.com>
 <AOE_tOA2nq2mYmnCsPAso6ctj2Onkp0AQbyEdHFa4Uk=.306398b3-ac3f-4671-b955-f37dd1de6840@github.com>
Message-ID: <2cBgaKxlWyMBVMct-_dRFueGfbEVsSNnrYfNAt8w82E=.f39d1048-574a-473d-8979-b15494364491@github.com>

On Thu, 11 Apr 2024 16:27:51 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

> > Another idea: To alleviate the need to pass MEMFLAGS all the time, could we have something like a "active MEMFLAGS" state per Thread, and set that stack-based with a XXMark object? That way, one could say at the entrance of Metaspace, for instance, "whatever is allocated under the scope of this function, please mark with mtMetaspace".
> 
> Not sure if I understood your idea, the question is if a thread always uses only ONE type of memory and not mix of them? For example, CDS uses both mtClass and mtClassShared. If a Thread has an active MEMFLAGS, it has to switch this flag between A and B whenever it uses type A or B.

No, the idea was to do it stack-based with a chained mark object. But never mind, we can do this in a follow up PR. Maybe it has no merit.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2051192960

From stefank at openjdk.org  Fri Apr 12 07:52:53 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 12 Apr 2024 07:52:53 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
 <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>
Message-ID: <XYyYksgW6JBXiCCS0UxCIdptnH1vJLdROnmrqIfKF7s=.a35852d4-49ad-4ffc-8e9d-ba49cc6fd2f9@github.com>

On Thu, 11 Apr 2024 18:04:48 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/virtualspace.cpp line 693:
>> 
>>> 691:   _special                = false;
>>> 692:   _executable             = false;
>>> 693:   _nmt_flag                = mtNone;
>> 
>> Weird indentation.
>
> Fixed.

Still looks weird when to me.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562153193

From stefank at openjdk.org  Fri Apr 12 07:52:52 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 12 Apr 2024 07:52:52 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
Message-ID: <lhFIh2qUbeY3kzGklUGuLCMZCC9lMzqMevukRHLzdik=.8c3147d8-62bd-4e7e-9630-d5c7103d0418@github.com>

On Thu, 11 Apr 2024 21:25:55 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   review comments applied.

A few more comments.

src/hotspot/os/windows/os_windows.cpp line 3137:

> 3135:   // If reservation failed, return null
> 3136:   if (p_buf == nullptr) return nullptr;
> 3137:   MemTracker::record_virtual_memory_reserve((address)p_buf, size_of_reserve, CALLER_PC, mtInternal);

I think that allocate_pages_individually should take a MEMFLAGS argument instead of using mtInternal here.

src/hotspot/os/windows/os_windows.cpp line 3198:

> 3196:         // the release.
> 3197:         MemTracker::record_virtual_memory_reserve((address)p_buf,
> 3198:                                                   bytes_to_release, CALLER_PC, mtNone);

I don't think we should ever use `mtNone` in code outside of the NMT code. If you follow my suggestion above that allocate_pages_individually should take a MEMFLAG arg, then it could be used here.

src/hotspot/os/windows/os_windows.cpp line 3218:

> 3216:     MemTracker::record_virtual_memory_reserve_and_commit((address)p_buf, bytes, CALLER_PC);
> 3217:   } else {
> 3218:     MemTracker::record_virtual_memory_reserve((address)p_buf, bytes, CALLER_PC, mtNone);

Use the correct MEMFLAG here instead of mtNone.

src/hotspot/os/windows/os_windows.cpp line 3771:

> 3769:   if (!is_committed) {
> 3770:     commit_memory_or_exit(addr, bytes, prot == MEM_PROT_RWX,
> 3771:                           "cannot commit protection page", mtNone);

This should probably be something else than mtNone.

src/hotspot/share/jfr/recorder/storage/jfrVirtualMemory.cpp line 107:

> 105:   _rs = ReservedSpace(reservation_size_request_bytes,
> 106:                       os::vm_allocation_granularity(),
> 107:                       os::vm_page_size(), mtTracing);

The mtTracing should probably be on a separate line, so that it follows the style of the surrounding code.

src/hotspot/share/memory/virtualspace.cpp line 45:

> 43: // Dummy constructor
> 44: ReservedSpace::ReservedSpace() : _base(nullptr), _size(0), _noaccess_prefix(0),
> 45:     _alignment(0), _special(false), _fd_for_heap(-1), _nmt_flag(mtNone), _executable(false) {

In almost all code we pass in the executable before the flag, but in ReservedSpace the flag is located before the executable. I think it would be nice to flip the order in this class. I understand that _executable is in the private section, while the other members are protected, but I don't think that it needs to be that way. The _executable could probably just be moved together with the rest of the members.

OTOH, I think the entire class needs some cleanups. Let's leave this for a separate RFE.

src/hotspot/share/memory/virtualspace.cpp line 615:

> 613: 
> 614: ReservedHeapSpace::ReservedHeapSpace(size_t size, size_t alignment, size_t page_size, const char* heap_allocation_directory) : ReservedSpace() {
> 615:   set_nmt_flag(mtJavaHeap);

It seems odd that we only initialize the _nmt_flag when `size == 0`. Could this be done after that check? If not, why not?

There's also a call to record_virtual_memory_type further down in the code. Why is that needed? Why isn't it enough to pass in the correct type to the os::reserve_memory call in the initialize function?

src/hotspot/share/memory/virtualspace.cpp line 672:

> 670:                                      size_t rs_align,
> 671:                                      size_t rs_page_size) : ReservedSpace() {
> 672:   set_nmt_flag(mtCode);

Why isn't this a part of the initialize call? This looks like a bug to me. `initialize` will call clear_members, which will undo this setting.

src/hotspot/share/memory/virtualspace.cpp line 708:

> 706:   assert(max_commit_granularity > 0, "Granularity must be non-zero.");
> 707: 
> 708:   _nmt_flag = rs.nmt_flag();

The code seems to be written with blank lines to separate various members that belong together. Please add a blank line after this line.

src/hotspot/share/memory/virtualspace.hpp line 72:

> 70: 
> 71:   MEMFLAGS nmt_flag() { return _nmt_flag; }
> 72:   void set_nmt_flag(MEMFLAGS flag) { _nmt_flag = flag; }

I have a feeling that set_nmt_flag should not exist and be replaced by updated initialize functions.

src/hotspot/share/memory/virtualspace.hpp line 199:

> 197:   size_t _upper_alignment;
> 198: 
> 199:   MEMFLAGS _nmt_flag;

The VirtualSpace::initialize functions used to initialize these members in the order that they are specified here. That is now messed up by adding the _nmt_flag at the end here, but in the beginning in the initialize function. I would propose that you move it to after _executable, both here and in the initialize function.

src/hotspot/share/nmt/virtualMemoryTracker.hpp line 307:

> 305: 
> 306:   ReservedMemoryRegion(address base, size_t size, MEMFLAGS flag) :
> 307:     VirtualMemoryRegion(base, size), _stack(NativeCallStack::empty_stack()), _flag(flag) { }

The function above uses mtNone. I find that a bit dubious, but I understand that it is done to be able to write code like this:

    ReservedMemoryRegion* rmr = VirtualMemoryTracker::_reserved_regions->find(ReservedMemoryRegion(addr, size));


Unfortunately, it opens up the door for people to accidentally use that version instead of this new version that you have written. Could we get rid of the version using mtNone somehow?

The same question goes for the version above that, which has a "MEMFLAGS flag = mtNone". (GH doesn't allow me to comment on lines that you haven't changed)

src/hotspot/share/runtime/os.hpp line 511:

> 509:   // and is added to be used for implementation of -XX:AllocateHeapAt
> 510:   static char* map_memory_to_file(size_t size, int fd, MEMFLAGS flag = mtNone);
> 511:   static char* map_memory_to_file_aligned(size_t size, size_t alignment, int fd, MEMFLAGS flag);

There are still a few mtNone usages in this file.

test/hotspot/gtest/gc/g1/test_freeRegionList.cpp line 53:

> 51:   size_t bot_size = G1BlockOffsetTable::compute_size(heap.word_size());
> 52:   HeapWord* bot_data = NEW_C_HEAP_ARRAY(HeapWord, bot_size, mtGC);
> 53:   ReservedSpace bot_rs(G1BlockOffsetTable::compute_size(heap.word_size()), mtGC);

mtGC => mtTest?

test/hotspot/gtest/gc/z/test_zForwarding.cpp line 103:

> 101:     _reserved = reserved;
> 102: 
> 103:     os::commit_memory((char*)_reserved, ZGranuleSize, !ExecMem /* executable */, mtGC);

mtGC => mtTest?

test/hotspot/gtest/gc/z/test_zForwarding.cpp line 114:

> 112:     ZGeneration::_young = _old_young;
> 113:     if (_reserved != nullptr) {
> 114:       os::uncommit_memory((char*)_reserved, ZGranuleSize, !ExecMem, mtGC);

mtGC => mtTest?

test/hotspot/gtest/memory/test_virtualspace.cpp line 223:

> 221:         return ReservedSpace(reserve_size_aligned,
> 222:                              os::vm_allocation_granularity(),
> 223:                              os::vm_page_size(), mtTest);

newline before mtTest.

-------------

Changes requested by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-1996032947
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562112282
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562114435
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562114730
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562116176
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562126580
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562134024
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562150698
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562152813
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562154150
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562155538
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562158292
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562163590
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562165152
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562165753
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562166089
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562166154
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562166709

From stuefe at openjdk.org  Fri Apr 12 08:03:46 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 12 Apr 2024 08:03:46 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
Message-ID: <4eN_yJUIi_0MTBROX0yxeIZIYo4W3KNlBGGOSA3glI4=.8e6ec837-1cb3-414f-959c-86fb3e3c9907@github.com>

On Thu, 11 Apr 2024 21:25:55 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   review comments applied.

Mostly good. Small nits inside.

src/hotspot/share/memory/metaspace/testHelpers.cpp line 81:

> 79:   if (reserve_limit > 0) {
> 80:     // have reserve limit -> non-expandable context
> 81:     _rs = ReservedSpace(reserve_limit * BytesPerWord, Metaspace::reserve_alignment(), os::vm_page_size(), mtTest);

mtMetaspace

src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 112:

> 110: 
> 111:   // Commit...
> 112:   if (os::commit_memory((char*)p, word_size * BytesPerWord, !ExecMem, _rs.nmt_flag()) == false) {

just use mtMetaspace here, its easier

src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 191:

> 189: 
> 190:   // Uncommit...
> 191:   if (os::uncommit_memory((char*)p, word_size * BytesPerWord, !ExecMem, _rs.nmt_flag()) == false) {

mtMetaspace

src/hotspot/share/runtime/os.hpp line 521:

> 519:                            bool allow_exec = false, MEMFLAGS flags = mtNone);
> 520:   static bool   unmap_memory(char *addr, size_t bytes);
> 521:   static void   free_memory(char *addr, size_t bytes, size_t alignment_hint, MEMFLAGS flag);

While looking at this, I noticed a couple of odd things about this function. I think it should be revised and I opened https://bugs.openjdk.org/browse/JDK-8330144. The result of that revision will be that we don't need MEMFLAGS, nor do would we need the alignment hint. 

But leave the MEMFLAGS in for now. If I happen to push that change first, you can adapt the change, if you push first I'll manage.

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-1996103064
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562159668
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562159155
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562159372
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562178816

From stefank at openjdk.org  Fri Apr 12 08:18:43 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 12 Apr 2024 08:18:43 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
 <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>
Message-ID: <OnazIuzb6ee0NaLnoCnDczhj8sU3iSwcfgwxK1dPNyY=.626e490a-3f74-4ff1-8482-bb8a6094c819@github.com>

On Thu, 11 Apr 2024 18:05:57 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/virtualspace.hpp line 45:
>> 
>>> 43:   bool   _special;
>>> 44:   int    _fd_for_heap;
>>> 45:   MEMFLAGS _nmt_flag;
>> 
>> Indentation is now off.
>
> Fixed.

It still looks wrong.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1562197284

From wkemper at openjdk.org  Fri Apr 12 14:17:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 12 Apr 2024 14:17:37 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <JN__f1fGhwA-dVcQn1-EqJaV_ZdfSVgHr9fs8Huj1B0=.11034e3f-c0df-443f-8dc7-990d822b231b@github.com>

Merges tag jdk-23+18

-------------

Commit messages:
 - 8329254: optimize integral reverse operations on x86 GFNI target.
 - 8329656: assertion failed in MAP_ARCHIVE_MMAP_FAILURE path: Invalid immediate -5 0
 - 8329491: GetThreadListStackTraces function should use JvmtiHandshake
 - 8329432: PopFrame and ForceEarlyReturn functions should use JvmtiHandshake
 - 8330033: com/sun/net/httpserver/bugs/B6431193.java fails in AssertionError after JDK-8326568
 - 8329961: Buffer overflow in os::Linux::kernel_version
 - 8327137: Add test for ConcurrentModificationException in BasicDirectoryModel
 - 8309751: Duplicate constant pool entries added during default method processing
 - 8326568: jdk/test/com/sun/net/httpserver/bugs/B6431193.java should use try-with-resource and try-finally
 - 8330002: Remove redundant public keyword in BarrierSet
 - ... and 166 more: https://git.openjdk.org/shenandoah/compare/d580bcf9...b04b3047

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=422&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/422/files
  Stats: 32165 lines in 779 files changed: 14386 ins; 12566 del; 5213 mod
  Patch: https://git.openjdk.org/shenandoah/pull/422.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/422/head:pull/422

PR: https://git.openjdk.org/shenandoah/pull/422

From snazarki at openjdk.org  Fri Apr 12 14:44:52 2024
From: snazarki at openjdk.org (Sergey Nazarkin)
Date: Fri, 12 Apr 2024 14:44:52 GMT
Subject: RFR: 8330171: Lazy W^X swtich implementation
Message-ID: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>

An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 

Additional testing:
- [x] MacOS AArch64 server fastdebug *gtets*
- [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
- [ ] Benchmarking

@apangin and @parttimenerd could you please check the patch on your scenarios??

-------------

Commit messages:
 - Fix non-macos builds
 - Revert "8304725: AsyncGetCallTrace can cause SIGBUS on M1"
 - Remove ThreadWXEnable guard
 - Lazy W^X state switch

Changes: https://git.openjdk.org/jdk/pull/18762/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18762&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8330171
  Stats: 325 lines in 46 files changed: 17 ins; 305 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/18762.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18762/head:pull/18762

PR: https://git.openjdk.org/jdk/pull/18762

From vkempik at openjdk.org  Fri Apr 12 14:53:44 2024
From: vkempik at openjdk.org (Vladimir Kempik)
Date: Fri, 12 Apr 2024 14:53:44 GMT
Subject: RFR: 8330171: Lazy W^X swtich implementation
In-Reply-To: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
Message-ID: <eDj_EGy4fShd2Q4spfYunpQjWHCwYBZUEEiVk7mo9QE=.4a721375-2f5f-4450-b943-f7061208d7a9@github.com>

On Fri, 12 Apr 2024 14:40:05 GMT, Sergey Nazarkin <snazarki at openjdk.org> wrote:

> An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 
> 
> Additional testing:
> - [x] MacOS AArch64 server fastdebug *gtets*
> - [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
> - [ ] Benchmarking
> 
> @apangin and @parttimenerd could you please check the patch on your scenarios??

Hello Sergey.
W^X mode was initially forced by Apple to prevent writing to executable memory, as a security feature.
This change just eliminates this security feature at all, doesn't it ?
Basically: "want to write to Executable memory ? ok, here you go"

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2051910824

From aph at openjdk.org  Fri Apr 12 15:28:43 2024
From: aph at openjdk.org (Andrew Haley)
Date: Fri, 12 Apr 2024 15:28:43 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <eDj_EGy4fShd2Q4spfYunpQjWHCwYBZUEEiVk7mo9QE=.4a721375-2f5f-4450-b943-f7061208d7a9@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
 <eDj_EGy4fShd2Q4spfYunpQjWHCwYBZUEEiVk7mo9QE=.4a721375-2f5f-4450-b943-f7061208d7a9@github.com>
Message-ID: <88Jsd_RmZ8QTcODe6MsTx2j54J8Dk6dJX-ZUpVIdxVs=.abd71be6-dba9-4851-9f93-009858d0c175@github.com>

On Fri, 12 Apr 2024 14:50:46 GMT, Vladimir Kempik <vkempik at openjdk.org> wrote:

> Hello Sergey. W^X mode was initially forced by Apple to prevent writing to executable memory, as a security feature. This change just eliminates this security feature at all, doesn't it ? Basically: "want to write to Executable memory ? ok, here you go"

Yes @VladimirKempik, you are right. No, we should not do this.

Instead, when we enter the VM we could track the current state of W^X and whenever we enter a block that needs to write into code space we would set W if needed. When we leave the VM or when we call back into Java we would set X, if needed. The cost of doing this would be small, but we'd have to find all the blocks that need to write into code space. This might be more effort than we want to make, though.

So where would be need to make the transitions to W? At a guess, whenever we start assembling something, and in all of the methods in nativeInst_aarch64.?pp, and in class Patcher. And to X, in the call stub and a few other places.

That would minimize the transitions exactly to the set of places we actually need.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2051977752

From vkempik at openjdk.org  Fri Apr 12 15:32:41 2024
From: vkempik at openjdk.org (Vladimir Kempik)
Date: Fri, 12 Apr 2024 15:32:41 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
Message-ID: <WSMsn6iefWdLHx7hsN6_50_Plkrjyi59W7BRVsblyDU=.9540d841-6feb-41c5-afc7-ef48527cce46@github.com>

On Fri, 12 Apr 2024 14:40:05 GMT, Sergey Nazarkin <snazarki at openjdk.org> wrote:

> An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 
> 
> Additional testing:
> - [x] MacOS AArch64 server fastdebug *gtets*
> - [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
> - [ ] Benchmarking
> 
> @apangin and @parttimenerd could you please check the patch on your scenarios??

This can be left as an addition to existing mechanism. Disabled by default and can be enabled with a special (DEVELOP) option. So this can't be enabled on production builds, but can be useful to debug w^x issues on debug builds

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2051984322

From kdnilsen at openjdk.org  Fri Apr 12 16:04:17 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 12 Apr 2024 16:04:17 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v3]
In-Reply-To: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
Message-ID: <eZrW3nJo_cgbZ6vISTt97AEYPEsR0BEOAblt-5k7RUo=.907a73cd-6f3b-42b3-beb1-defe76b7db81@github.com>

> This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Respond to reviewer feedback
  
  Return practical limits for OLD generation max and min sizes

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/421/files
  - new: https://git.openjdk.org/shenandoah/pull/421/files/f23113f5..7570b2d4

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=421&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=421&range=01-02

  Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/shenandoah/pull/421.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/421/head:pull/421

PR: https://git.openjdk.org/shenandoah/pull/421

From snazarki at openjdk.org  Fri Apr 12 17:18:41 2024
From: snazarki at openjdk.org (Sergey Nazarkin)
Date: Fri, 12 Apr 2024 17:18:41 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <88Jsd_RmZ8QTcODe6MsTx2j54J8Dk6dJX-ZUpVIdxVs=.abd71be6-dba9-4851-9f93-009858d0c175@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
 <eDj_EGy4fShd2Q4spfYunpQjWHCwYBZUEEiVk7mo9QE=.4a721375-2f5f-4450-b943-f7061208d7a9@github.com>
 <88Jsd_RmZ8QTcODe6MsTx2j54J8Dk6dJX-ZUpVIdxVs=.abd71be6-dba9-4851-9f93-009858d0c175@github.com>
Message-ID: <MHy1h0_wjYYtDV61FSSch_11MEEqc3Z-FmktjJP76gc=.dcb04a48-08e2-40a4-a662-33a9d151006d@github.com>

On Fri, 12 Apr 2024 15:26:03 GMT, Andrew Haley <aph at openjdk.org> wrote:

>> Hello Sergey.
>> W^X mode was initially forced by Apple to prevent writing to executable memory, as a security feature.
>> This change just eliminates this security feature at all, doesn't it ?
>> Basically: "want to write to Executable memory ? ok, here you go"
>
>> Hello Sergey. W^X mode was initially forced by Apple to prevent writing to executable memory, as a security feature. This change just eliminates this security feature at all, doesn't it ? Basically: "want to write to Executable memory ? ok, here you go"
> 
> Yes @VladimirKempik, you are right. No, we should not do this.
> 
> Instead, when we enter the VM we could track the current state of W^X and whenever we enter a block that needs to write into code space we would set W if needed. When we leave the VM or when we call back into Java we would set X, if needed. The cost of doing this would be small, but we'd have to find all the blocks that need to write into code space. This might be more effort than we want to make, though.
> 
> So where would be need to make the transitions to W? At a guess, whenever we start assembling something, and in all of the methods in nativeInst_aarch64.?pp, and in class Patcher. And to X, in the call stub and a few other places.
> 
> That would minimize the transitions exactly to the set of places we actually need.

Thanks @theRealAph, @VladimirKempik
> Instead, when we enter the VM we could track the current state of W^X and whenever we enter a block that needs to write into code space we would set W if needed. When we leave the VM or when we call back into Java we would set X, if needed. The cost of doing this would be small, but we'd have to find all the blocks that need to write into code space. This might be more effort than we want to make, though.

?It is the way in which it is implemented in the current code.  Unfortunately, it is hardly maintainable solution that suffers from issues like [1-5].   

As I understand it, your concern is that the implementation doesn't prevent rogue from writing to the code cache with some some unsafe api? 

1. https://bugs.openjdk.org/browse/JDK-8302736
2. https://bugs.openjdk.org/browse/JDK-8327990
3. https://bugs.openjdk.org/browse/JDK-8327036
4. https://bugs.openjdk.org/browse/JDK-8304725
5. https://bugs.openjdk.org/browse/JDK-8307549

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2052164890

From kdnilsen at openjdk.org  Fri Apr 12 18:05:09 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 12 Apr 2024 18:05:09 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v2]
In-Reply-To: <AoAWt0eOeU99WTVBBOjbrZts9FHi_hu7xcUtpnA2h8M=.5a36d443-10f2-4d50-87f6-bd24f2dafdf1@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
 <VONCsebiQBQqPHlzDxTulXzP1ehSkKZF-gvmmnTbs2U=.b5c69d3d-3929-4344-baf7-866fe8a80e6b@github.com>
 <twbyUdzayvnJk3dLOAqcaf0Wgj8R-dycCaOOyHYDnsI=.2687ba03-be80-4c54-b55d-668d464579e4@github.com>
 <AoAWt0eOeU99WTVBBOjbrZts9FHi_hu7xcUtpnA2h8M=.5a36d443-10f2-4d50-87f6-bd24f2dafdf1@github.com>
Message-ID: <aTXXkwm3M840RdD2zOi6uzvxm9V6DDyT2-ALjHS3mCQ=.ed5f1f34-414e-4165-acdd-bd34b3fa43c4@github.com>

On Thu, 11 Apr 2024 21:45:03 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> What would go wrong if, as indicated in your comments, you were to return `heap - min_young` for `max(old)`, and `heap - max_young` for `min(old)` ?
>> 
>> If nothing would go wrong, why not do that? If something would go wrong if one did that, may be that needs to be documented along with the comment you added.
>
>> @ysramakrishna : Nothing would go wrong. It's just more computation for no increased value. Before transferring regions from generation src to generation dest, we confirm that the proposed transfer does not violate minimum size of src generation or maximum size of dest generation.
> 
> I'd err in that case on the side of the trivial extra work in the method to keep its semantics clear and complete, rather than expect its clients to carry that knowledge, best to avoid surprises and keep the code maintainable. I doubt that the little extra compute is worth the potential future trouble.
> 
> I realize this view is somewhat subjective, so I'll let you decide which way you go on this, even though I personally vote for clarity and cleanliness in code rather than cleverness and frugality that might lead to less maintainability in the future. 
> 
> Reviewed!

Thanks @ysramakrishna for your suggestion.  I will make this change.  Am retesting before integration.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/421#issuecomment-2052227205

From azafari at openjdk.org  Fri Apr 12 19:10:09 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 12 Apr 2024 19:10:09 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v5]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <asRwE8-BGk8hnVyAK7lHj0f0Esb5kq_Nx9fu0txB2Pc=.144e81bd-2d5d-4d57-9b85-1a281b6ca163@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  mtCode and mtMetaspace were missed from System Dump map

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/9d66735f..a3940639

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=03-04

  Stats: 11 lines in 3 files changed: 2 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From wkemper at openjdk.org  Fri Apr 12 20:38:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 12 Apr 2024 20:38:11 GMT
Subject: Integrated: 8329789: GenShen: Over assertive assert when scanning
 remembered set
In-Reply-To: <DX93ICC_EgOyNTIhEme2nBZeD19BMcksujPX1BOeimg=.06665a81-efc9-4f46-bf6d-055e44e5c8b4@github.com>
References: <DX93ICC_EgOyNTIhEme2nBZeD19BMcksujPX1BOeimg=.06665a81-efc9-4f46-bf6d-055e44e5c8b4@github.com>
Message-ID: <tljj9L6WgO9VTcwGSnJywmVfvOo5lnWXtny9QHmRzOY=.b582783d-1e52-4e43-8fbd-78e9b47fc18c@github.com>

On Thu, 11 Apr 2024 22:45:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport, assert only and assert only applies to non-product code.

This pull request has now been integrated.

Changeset: b33cb0a7
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/b33cb0a7fa6f1edebcb5e83295f94e2118b668a9
Stats:     5 lines in 1 file changed: 4 ins; 0 del; 1 mod

8329789: GenShen: Over assertive assert when scanning remembered set

Backport-of: 69281629a4a537b988f5dfd3a93733b3ddccd109

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/34

From wkemper at openjdk.org  Fri Apr 12 20:39:09 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 12 Apr 2024 20:39:09 GMT
Subject: Integrated: 8329699: GenShen: Move promotion logic out of shHeap and
 shHeapRegion
In-Reply-To: <G9ZJV5hPgMHKOgnZjpXYbkMfdeSGA3nabJOCQh_4-Fo=.62da6e6e-16bd-44b2-b4fb-f10a5a4eb676@github.com>
References: <G9ZJV5hPgMHKOgnZjpXYbkMfdeSGA3nabJOCQh_4-Fo=.62da6e6e-16bd-44b2-b4fb-f10a5a4eb676@github.com>
Message-ID: <w26W_AcQ6L3TbvLKqepbXqQGEBgCDSrGlX-5MA7BlB4=.54c6e133-5792-4c24-a91c-b3e98deef859@github.com>

On Thu, 4 Apr 2024 18:15:38 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The code to handle promotions during evacuation has been moved into a new class in its own files: `shGenerationalEvacuationTask`
> 
> The methods for managing `plabs` have been moved into `shGenerationalHeap`. `plabs` are now also only created when running the generational mode.
> 
> 
> -  inline HeapWord* allocate_from_plab(Thread* thread, size_t size, bool is_promotion);
> -  HeapWord* allocate_from_plab_slow(Thread* thread, size_t size, bool is_promotion);
> -  HeapWord* allocate_new_plab(size_t min_size, size_t word_size, size_t* actual_size);
> -  void retire_plab(PLAB* plab);
> -  void retire_plab(PLAB* plab, Thread* thread);
> 
> 
> The code to handle promotion in place has been moved from `shHeapRegion` into `shGenerationalEvacuationTask`:
> 
> -  void promote_humongous();
> -  void promote_in_place();
> 
> 
> Unused methods related to global coalesce and fill have been deleted from `shHeapRegion`:
> 
> -  void global_oop_iterate_and_fill_dead(OopIterateClosure* cl);
> -  void global_oop_iterate_objects_and_fill_dead(OopIterateClosure* cl);
> -  void oop_iterate_humongous(OopIterateClosure* cl);
> -  void oop_iterate_humongous(OopIterateClosure* cl, HeapWord* start, size_t words);
> 
> 
> Additionally, these methods in `shHeap` which just delegated to old or young generation instances have more or less been inlined at the usage site (many `includes` have been fixed):
> 
> -  ShenandoahOldHeuristics* old_heuristics();
> -  ShenandoahYoungHeuristics* young_heuristics();
> -
> -  bool doing_mixed_evacuations();
> -  bool is_old_bitmap_stable() const;
> -  bool is_gc_generation_young() const;
> 
> 
> The boolean `is_promotion` that was passed through all of the allocation calls has been moved into `shAllocRequest` to reduce the upstream delta for `shHeap`.

This pull request has now been integrated.

Changeset: 9d869ca1
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/9d869ca1b4d5b210a93dbb4d209189d913c92446
Stats:     1876 lines in 36 files changed: 968 ins; 791 del; 117 mod

8329699: GenShen: Move promotion logic out of shHeap and shHeapRegion

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/415

From kdnilsen at openjdk.org  Fri Apr 12 23:00:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 12 Apr 2024 23:00:12 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
 <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
Message-ID: <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>

On Fri, 12 Apr 2024 22:57:23 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Pinned regions that are not humongous may be added to the set of _candidates_ for mixed collection. They still may not be added to the collection set for a mixed collection. Regions that are not in this collection of _candidates_ will not be coalesced and filled.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>  - Use is_regular_pinned to restore candidate selection logic to its previous form
>  - Introduce 'regular pinned' state
>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>  - Allow pinned/regular regions into mixed collection candidates
>  - Little clean up

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 327:

> 325:     live_data += live_bytes;
> 326: 
> 327:     if (region->is_trash()) {

In revised code structure, we handle trash regular regions with same code that handles trash humongous regions.  I like this improvement.  This is a note to self.  Please let me know if I am misunderstaning.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 331:

> 329: 
> 330:     // Only place regular regions into the candidate set
> 331:     if (region->is_regular()) {

I understand that the rationale for the PR is to make sure we do not overlook pinned regions.  What I'm not seeing is how the original code is excluding the pinned regions from being inserted into the "old collection set".  To help my review, can you point me to that part of the original code?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 332:

> 330:       immediate_garbage += garbage;
> 331:     } else if (region->is_humongous_start()) {
> 332:       // This will handle humongous start regions whether they are also pinned, or not.

Since we've already handled region->is_trash() above, do we need this code?  If we want to preserve the assert(!is_pinned), we can insert that into the is_trash() code above.  Right?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 347:

> 345:       }
> 346:     } else if (region->is_regular() || region->is_pinned()) {
> 347:       if (!region->has_live()) {

Here, I'm not sure why we ask "|| region->is_pinned()".  We don't want humongous regions that are pinned.  And we do want regular regions whether or not they are pinned.  Also, I think it better to handle the case of !region->has_live() at line 327 above.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 443:

> 441:            (total_uncollected_old_regions < 15 * span_of_uncollected_regions / 16)) {
> 442:       ShenandoahHeapRegion* r = candidates[_last_old_collection_candidate]._region;
> 443:       assert((r->is_regular() || r->is_pinned()) && !r->is_humongous(), "Region " SIZE_FORMAT " has wrong state for collection: %s",

I think assert should be r->is_regular(). (testing for r->is_pinned() and !r->is_humongous() seems redundant and unnecessary).  Or explain the rationale for assert as written with comment.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 469:

> 467:   log_info(gc)("Old-Gen Immediate Garbage: " PROPERFMT " over " SIZE_FORMAT " regions",
> 468:               PROPERFMTARGS(immediate_garbage), immediate_regions);
> 469:   log_info(gc)("Old regions selected for defragmentation: " SIZE_FORMAT, defrag_count);

Might be good to add a comment here that defrag_count is a subset of old_candidates.  defrag_count represents regions that were placed into the old collection set in order to defragment the memory that we try to "reserve" for humongous allocations.  old_candidates includes defrag_count, plus it also includes old-gen regions that are to be collected because they contain an abundance of garbage.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1554451689
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1554459766
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1554452329
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1554457264
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1554450956
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1554458847

From wkemper at openjdk.org  Fri Apr 12 23:00:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 12 Apr 2024 23:00:11 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
Message-ID: <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>

> Pinned regions that are not humongous may be added to the set of _candidates_ for mixed collection. They still may not be added to the collection set for a mixed collection. Regions that are not in this collection of _candidates_ will not be coalesced and filled.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:

 - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
 - Use is_regular_pinned to restore candidate selection logic to its previous form
 - Introduce 'regular pinned' state
 - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
 - Allow pinned/regular regions into mixed collection candidates
 - Little clean up

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/420/files
  - new: https://git.openjdk.org/shenandoah/pull/420/files/b2d5e8ce..b0e3b2ab

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=420&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=420&range=00-01

  Stats: 1918 lines in 37 files changed: 988 ins; 803 del; 127 mod
  Patch: https://git.openjdk.org/shenandoah/pull/420.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/420/head:pull/420

PR: https://git.openjdk.org/shenandoah/pull/420

From kdnilsen at openjdk.org  Fri Apr 12 23:00:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 12 Apr 2024 23:00:12 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
 <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
 <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
Message-ID: <188vF6jEHcvbTRFERHezjlCjAf_7AjEQNj4GjKSoarM=.b6eb6776-ef98-4b8b-820e-4e661b9a0561@github.com>

On Sat, 6 Apr 2024 00:31:10 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>>  - Use is_regular_pinned to restore candidate selection logic to its previous form
>>  - Introduce 'regular pinned' state
>>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>>  - Allow pinned/regular regions into mixed collection candidates
>>  - Little clean up
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 327:
> 
>> 325:     live_data += live_bytes;
>> 326: 
>> 327:     if (region->is_trash()) {
> 
> In revised code structure, we handle trash regular regions with same code that handles trash humongous regions.  I like this improvement.  This is a note to self.  Please let me know if I am misunderstaning.

Need to double-check myself here.  is_trash() may not be exactly the same as !region->has_live().  If these are distinct, maybe we want to check for either condition right here.

> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 331:
> 
>> 329: 
>> 330:     // Only place regular regions into the candidate set
>> 331:     if (region->is_regular()) {
> 
> I understand that the rationale for the PR is to make sure we do not overlook pinned regions.  What I'm not seeing is how the original code is excluding the pinned regions from being inserted into the "old collection set".  To help my review, can you point me to that part of the original code?

No need for this.  I now understand that is_regular() does not include is_pinned_regular().

> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 347:
> 
>> 345:       }
>> 346:     } else if (region->is_regular() || region->is_pinned()) {
>> 347:       if (!region->has_live()) {
> 
> Here, I'm not sure why we ask "|| region->is_pinned()".  We don't want humongous regions that are pinned.  And we do want regular regions whether or not they are pinned.  Also, I think it better to handle the case of !region->has_live() at line 327 above.

I get it now.  But would prefer that we rename these region-state accessors as proposed above.  I've discovered after more thorough testing of the customer workload that we will occasionally also encounter old pinned_cset!  That would represent an unintentional match to the guard for this conditional code.  See draft https://github.com/openjdk/shenandoah/pull/414 for how I've handled this on my development and test branch.

I'd like for your PR to go in first, and then I'll merge any additional (refactored) changes from PR414.  I expect I'll be dividing PR414 into fix-coalesce-and-fill and improvements to ShenandoahVerify and maybe something else...

> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 443:
> 
>> 441:            (total_uncollected_old_regions < 15 * span_of_uncollected_regions / 16)) {
>> 442:       ShenandoahHeapRegion* r = candidates[_last_old_collection_candidate]._region;
>> 443:       assert((r->is_regular() || r->is_pinned()) && !r->is_humongous(), "Region " SIZE_FORMAT " has wrong state for collection: %s",
> 
> I think assert should be r->is_regular(). (testing for r->is_pinned() and !r->is_humongous() seems redundant and unnecessary).  Or explain the rationale for assert as written with comment.

After better understanding of the region state attributes, I now understand how this code works.  Refecting on how this bug came to exist, I think part of the "root cause" is inconsistency in how these attributes are named and accessed.  Conceptually, pinned and "region type" are orthogonal. We can have pinned humongous and pinned regular and pinned cset.  The naming of RegionState enum constants is inconsistent:

1. We have pinned_cset, pinned_humongous_start, but not pinned_regular.  Instead, pinned means pinned_regular.  Would it not be better to replace pinned with pinned_regular?
2. Also, we have is_humongous_start() which is defined as (state == humongous_start || state == pinned_humongous_start)
3. We have is_cset(), defined as (state == cset || state == pinned_cset)
4. But we have is_regular() which does not include state == pinned_regular.  Wouldn't it be better to define is_regular() as (state == regular || state == pinned_regular).  Making this change would require scrutiny of every existing use of is_regular() and revise as appropriate

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1554454748
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1559634302
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1559632710
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1559614470

From kdnilsen at openjdk.org  Fri Apr 12 23:00:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 12 Apr 2024 23:00:12 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <188vF6jEHcvbTRFERHezjlCjAf_7AjEQNj4GjKSoarM=.b6eb6776-ef98-4b8b-820e-4e661b9a0561@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
 <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
 <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
 <188vF6jEHcvbTRFERHezjlCjAf_7AjEQNj4GjKSoarM=.b6eb6776-ef98-4b8b-820e-4e661b9a0561@github.com>
Message-ID: <nPF8j5BB47lvqDp8s4GDiQnVkPjbdy6xkCSUMO38HDM=.fbd326d0-e5ad-4a7b-ae03-ac968c344253@github.com>

On Sat, 6 Apr 2024 00:39:27 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 327:
>> 
>>> 325:     live_data += live_bytes;
>>> 326: 
>>> 327:     if (region->is_trash()) {
>> 
>> In revised code structure, we handle trash regular regions with same code that handles trash humongous regions.  I like this improvement.  This is a note to self.  Please let me know if I am misunderstaning.
>
> Need to double-check myself here.  is_trash() may not be exactly the same as !region->has_live().  If these are distinct, maybe we want to check for either condition right here.

I see that we need to explicitly make_trash() after we discover that !region->has_live().  So these are different conditions.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1559616445

From wkemper at openjdk.org  Fri Apr 12 23:00:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 12 Apr 2024 23:00:12 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <188vF6jEHcvbTRFERHezjlCjAf_7AjEQNj4GjKSoarM=.b6eb6776-ef98-4b8b-820e-4e661b9a0561@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
 <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
 <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
 <188vF6jEHcvbTRFERHezjlCjAf_7AjEQNj4GjKSoarM=.b6eb6776-ef98-4b8b-820e-4e661b9a0561@github.com>
Message-ID: <0zCNqE-ay0iwmftIrW-t4d-qZA51q7uIiGdq_tzsqe8=.451ab8c6-eeea-4352-a31c-6be4015dd1ae@github.com>

On Wed, 10 Apr 2024 15:12:05 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 347:
>> 
>>> 345:       }
>>> 346:     } else if (region->is_regular() || region->is_pinned()) {
>>> 347:       if (!region->has_live()) {
>> 
>> Here, I'm not sure why we ask "|| region->is_pinned()".  We don't want humongous regions that are pinned.  And we do want regular regions whether or not they are pinned.  Also, I think it better to handle the case of !region->has_live() at line 327 above.
>
> I get it now.  But would prefer that we rename these region-state accessors as proposed above.  I've discovered after more thorough testing of the customer workload that we will occasionally also encounter old pinned_cset!  That would represent an unintentional match to the guard for this conditional code.  See draft https://github.com/openjdk/shenandoah/pull/414 for how I've handled this on my development and test branch.
> 
> I'd like for your PR to go in first, and then I'll merge any additional (refactored) changes from PR414.  I expect I'll be dividing PR414 into fix-coalesce-and-fill and improvements to ShenandoahVerify and maybe something else...

I agree, it's confusing to have `is_pinned` mask the underlying state of the region.

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 443:
>> 
>>> 441:            (total_uncollected_old_regions < 15 * span_of_uncollected_regions / 16)) {
>>> 442:       ShenandoahHeapRegion* r = candidates[_last_old_collection_candidate]._region;
>>> 443:       assert((r->is_regular() || r->is_pinned()) && !r->is_humongous(), "Region " SIZE_FORMAT " has wrong state for collection: %s",
>> 
>> I think assert should be r->is_regular(). (testing for r->is_pinned() and !r->is_humongous() seems redundant and unnecessary).  Or explain the rationale for assert as written with comment.
>
> After better understanding of the region state attributes, I now understand how this code works.  Refecting on how this bug came to exist, I think part of the "root cause" is inconsistency in how these attributes are named and accessed.  Conceptually, pinned and "region type" are orthogonal. We can have pinned humongous and pinned regular and pinned cset.  The naming of RegionState enum constants is inconsistent:
> 
> 1. We have pinned_cset, pinned_humongous_start, but not pinned_regular.  Instead, pinned means pinned_regular.  Would it not be better to replace pinned with pinned_regular?
> 2. Also, we have is_humongous_start() which is defined as (state == humongous_start || state == pinned_humongous_start)
> 3. We have is_cset(), defined as (state == cset || state == pinned_cset)
> 4. But we have is_regular() which does not include state == pinned_regular.  Wouldn't it be better to define is_regular() as (state == regular || state == pinned_regular).  Making this change would require scrutiny of every existing use of is_regular() and revise as appropriate

I'm adding a `is_regular_pinned` method to `shHeapRegion`.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1563239639
PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1563237822

From wkemper at openjdk.org  Fri Apr 12 23:00:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 12 Apr 2024 23:00:12 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
 <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
 <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
Message-ID: <3YvUliHqqkDsAYj2ABbdSLGtBHtvGC_lMpCk7MpI0pU=.78207800-b0a1-4c98-b689-37b5f3468eb9@github.com>

On Sat, 6 Apr 2024 00:43:05 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>>  - Use is_regular_pinned to restore candidate selection logic to its previous form
>>  - Introduce 'regular pinned' state
>>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>>  - Allow pinned/regular regions into mixed collection candidates
>>  - Little clean up
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 469:
> 
>> 467:   log_info(gc)("Old-Gen Immediate Garbage: " PROPERFMT " over " SIZE_FORMAT " regions",
>> 468:               PROPERFMTARGS(immediate_garbage), immediate_regions);
>> 469:   log_info(gc)("Old regions selected for defragmentation: " SIZE_FORMAT, defrag_count);
> 
> Might be good to add a comment here that defrag_count is a subset of old_candidates.  defrag_count represents regions that were placed into the old collection set in order to defragment the memory that we try to "reserve" for humongous allocations.  old_candidates includes defrag_count, plus it also includes old-gen regions that are to be collected because they contain an abundance of garbage.

Added a comment.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1563314072

From ysr at openjdk.org  Sat Apr 13 00:02:53 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 13 Apr 2024 00:02:53 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial [v4]
In-Reply-To: <Hi9G4QtqUwEO5dENpnGbiF2x_dx688BE8tatTXpyOi8=.25dac74c-c165-45eb-b75f-b18f74186aa3@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
 <Hi9G4QtqUwEO5dENpnGbiF2x_dx688BE8tatTXpyOi8=.25dac74c-c165-45eb-b75f-b18f74186aa3@github.com>
Message-ID: <TChTvyFovQk0v1Bf1A9rxEGgip2IGszr-0DTMAmQRoQ=.f8fedfa2-6f67-4963-b42d-0cbd70cca363@github.com>

On Tue, 9 Apr 2024 00:45:35 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
>> 
>> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
>> 
>> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
>> 
>> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
>> 
>> **Testing**:
>> - [x] GHA 
>> - [x] Code pipeline testing : one intermittent stress dacapo failure, ascribed to an existing bug in coalesce-and-fill
>> - [x] SPECjbb
>> 
>> **Performance**: 
>> - [x] SPECjbb : the variance in tests fails any significant change under 2-tailed Mann-Whitney
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits:
> 
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Remove local_reset of age_census object inadvertently left behind in the
>    GLOBAL gen concurrent mark, which was triggering the newly added
>    reasonableness assert (yay!).
>  - Avoid divide-by-zero.
>  - Merge branch 'master' into clear_census
>  - Fix word and byte size unit confusion in comparison/assert.
>  - Loose verification of "reasonable completeness" of census.
>  - Fix release build.
>  - Support for verification of census' reasonableness, but no verification
>    itself yet.
>  - ... and 6 more: https://git.openjdk.org/shenandoah/compare/11295a6a...d5fb72cc

No performance differences were noted; however, the previous census was partial and therefore potentially incorrect. A correct census should allow for future performance tuning.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/403#issuecomment-2052708265

From ysr at openjdk.org  Sat Apr 13 00:24:08 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 13 Apr 2024 00:24:08 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial [v5]
In-Reply-To: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
Message-ID: <JWyhtWUiQjIKems-D3kykmTPtU19QVm72Nx9_DZ4kHY=.42ae3338-965c-430f-bd4c-dcfc6bff6a7c@github.com>

> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
> 
> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
> 
> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
> 
> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
> 
> **Testing**:
> - [x] GHA 
> - [x] Code pipeline testing : one intermittent stress dacapo failure, ascribed to an existing bug in coalesce-and-fill
> - [x] SPECjbb
> 
> **Performance**: 
> - [x] SPECjbb : the variance in tests fails any significant change under 2-tailed Mann-Whitney

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits:

 - Merge branch 'master' into clear_census
 - Merge branch 'master' into clear_census
 - Merge branch 'master' into clear_census
 - Merge branch 'master' into clear_census
 - Remove local_reset of age_census object inadvertently left behind in the
   GLOBAL gen concurrent mark, which was triggering the newly added
   reasonableness assert (yay!).
 - Avoid divide-by-zero.
 - Merge branch 'master' into clear_census
 - Fix word and byte size unit confusion in comparison/assert.
 - Loose verification of "reasonable completeness" of census.
 - Fix release build.
 - ... and 7 more: https://git.openjdk.org/shenandoah/compare/9d869ca1...787e37a8

-------------

Changes: https://git.openjdk.org/shenandoah/pull/403/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=403&range=04
  Stats: 166 lines in 11 files changed: 121 ins; 27 del; 18 mod
  Patch: https://git.openjdk.org/shenandoah/pull/403.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/403/head:pull/403

PR: https://git.openjdk.org/shenandoah/pull/403

From ysr at openjdk.org  Sat Apr 13 01:35:03 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 13 Apr 2024 01:35:03 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v6]
In-Reply-To: <EceXsAE6gdlyE7-w00aQxM7rmZ0Fkt9PBOD2QIXdNw0=.7a66381f-3f1b-42a9-b5ec-d5d819bedb76@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <EceXsAE6gdlyE7-w00aQxM7rmZ0Fkt9PBOD2QIXdNw0=.7a66381f-3f1b-42a9-b5ec-d5d819bedb76@github.com>
Message-ID: <eNjnhqWn06XrIJqi6JH5x-V_DXR8jE4CaHwmJgNDdfo=.bdb3fc6d-9e13-4469-ad7e-d8402e014c7f@github.com>

On Thu, 4 Apr 2024 15:32:05 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
>> 
>> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
>> 
>> 
>>    Control: shenandoah-x86-template
>> Experiment: enable-old-growth-triggers-gh-x86
>> 
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>>                                Genshen/specjbb2015 |                               context_switch_count
>>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>>                             Shenandoah/specjbb2015 |                                      trigger_learn
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
>> 
>> Genshen
>> -------------------------------------------------------------------------------------------------------
>> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>>   Control:      2.500   (+/-  0.68  )         30
>>   Test:        19.625   (+/-  4.79  )         10
>> 
>> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>>   Control:      2.625   (+/-  0.92  )         30
>>   Test:        17.375   (+/-  3.89  )         10
>> 
>> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>>   Control:      9.833   (+/-  3.48  )         30
>>   Test:        32.000   (+/-  2.58  )       ...
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Move trigger code to ShenandoahOldHeuristics from ShenandoahOldGeneration

A few comments, but may be not worth addressing all of them at this time. The division of code & state between the heuristics object on the one hand, and the heap/generation/space objects on the other is still a bit unclear, and we might want to take a fresh look afterwards and clean it up to make things more self-contained, if possible. SpaceInfo goes part of the way in abstracting these APIs and embedding a hook into the heuristics object, but specific heuristics objects also seem to want to look at more specific information from the generations/spaces in places, which makes this a bit difficult (or at least somewhat awkward in places).

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 548:

> 546: }
> 547: 
> 548: void ShenandoahOldHeuristics::trigger_collection_if_fragmented(ShenandoahGenerationalHeap* gen_heap, ShenandoahOldGeneration* old_gen,

Should `ShenandoahOldHeurtitics` just keep pointers to `gen_heap` and `old_gen`, and slim down these APIs so these don't need to be passed around? (Similarly `ShenandoahYoungHeuristics`?)

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 564:

> 562:     // humongous allocation was not required.
> 563: 
> 564:     size_t old_available = old_gen->available();

Can you use `_space_info->available()` and avoid passing `old_gen` to this method?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 566:

> 564:     size_t old_available = old_gen->available();
> 565:     size_t region_size_bytes = ShenandoahHeapRegion::region_size_bytes();
> 566:     size_t old_unaffiliated_available = old_gen->free_unaffiliated_regions() * region_size_bytes;

Alas, `ShenandoahSpaceInfo` doesn't have the notion of `free_unaffliated_regions()`, although perhaps it could be extended to include it? I am not sure if all the implementors of the `ShenandoahSpaceInfo` interface would have a notion of "free" and "affiliated" regions, though (in particular, the non-generational heap's space info).

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 579:

> 577:       double density_threshold = (eighths - 2) / 8.0;
> 578:       if ((old_region_span >= span_threshold) && (old_density < density_threshold)) {
> 579:         gen_heap->old_heuristics()->trigger_old_is_fragmented(old_density, first_old_region, last_old_region);

Just `trigger_old_is_fragmented()` because `gen_heap->old_heuristics()` is `this` ?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 588:

> 586: 
> 587: void ShenandoahOldHeuristics::trigger_collection_if_overgrown(ShenandoahGenerationalHeap* gen_heap, ShenandoahOldGeneration* old_gen) {
> 588:   size_t old_used = old_gen->used() + old_gen->get_humongous_waste();

There's a `used_including_humongous_waste()` method in `ShenandoahGeneration` but unfortunately not yet in `ShenandoahSpaceInfo`.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 593:

> 591:   assert(old_used <= gen_heap->capacity(),
> 592:          "Old used (" SIZE_FORMAT ", " SIZE_FORMAT") must not be more than heap capacity (" SIZE_FORMAT ")",
> 593:          old_gen->used(), old_gen->get_humongous_waste(), gen_heap->capacity());

Does this assert belong here, rather than in the `used()` method?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 595:

> 593:          old_gen->used(), old_gen->get_humongous_waste(), gen_heap->capacity());
> 594:   if (old_used > trigger_threshold) {
> 595:     gen_heap->old_heuristics()->trigger_old_has_grown();

Just `trigger_old_has_grown()` because `gen_heap->old_heuristics()` is the same as`this` ?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/409#pullrequestreview-1998774551
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1563432976
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1563440112
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1563452131
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1563430879
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1563504301
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1563510168
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1563500293

From kdnilsen at openjdk.org  Sat Apr 13 01:35:29 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 13 Apr 2024 01:35:29 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v40]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <PfJkYaz8jUcQ5KZjhkj0TDDqPgrDJ84pEJ9PbyxmPWU=.324d4738-564d-4682-a79b-6349a2c334d0@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Make ShenadoahFreeSetPartitionId an enum class
 - Separate ShenandoahSimpleBitMap into distinct source files

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/84c5875d..b64739ad

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=39
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=38-39

  Stats: 985 lines in 6 files changed: 389 ins; 330 del; 266 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From ysr at openjdk.org  Sat Apr 13 01:35:56 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 13 Apr 2024 01:35:56 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
 [v3]
In-Reply-To: <eZrW3nJo_cgbZ6vISTt97AEYPEsR0BEOAblt-5k7RUo=.907a73cd-6f3b-42b3-beb1-defe76b7db81@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
 <eZrW3nJo_cgbZ6vISTt97AEYPEsR0BEOAblt-5k7RUo=.907a73cd-6f3b-42b3-beb1-defe76b7db81@github.com>
Message-ID: <0ETm3aexp6Z8J7lGr3i4ULiQSQFsfMH5rvTqQ9SNoYw=.00707aa2-c195-4e53-b738-3b37b2417f25@github.com>

On Fri, 12 Apr 2024 16:04:17 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Respond to reviewer feedback
>   
>   Return practical limits for OLD generation max and min sizes

Thanks... LGTM! ?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/421#pullrequestreview-1998912270

From ysr at openjdk.org  Sat Apr 13 02:35:57 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 13 Apr 2024 02:35:57 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial [v5]
In-Reply-To: <JWyhtWUiQjIKems-D3kykmTPtU19QVm72Nx9_DZ4kHY=.42ae3338-965c-430f-bd4c-dcfc6bff6a7c@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
 <JWyhtWUiQjIKems-D3kykmTPtU19QVm72Nx9_DZ4kHY=.42ae3338-965c-430f-bd4c-dcfc6bff6a7c@github.com>
Message-ID: <9KJYekFlzBInuEOXlw5hWz6bJ3kPsQPEaFlQLynBjd0=.5c77d5e4-7df1-4297-a14d-28f598a18827@github.com>

On Sat, 13 Apr 2024 00:24:08 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
>> 
>> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
>> 
>> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
>> 
>> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
>> 
>> **Testing**:
>> - [x] GHA 
>> - [x] Code pipeline testing : one intermittent stress dacapo failure, ascribed to an existing bug in coalesce-and-fill
>> - [x] SPECjbb
>> 
>> **Performance**: 
>> - [x] SPECjbb : the variance in tests fails any significant change under 2-tailed Mann-Whitney
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits:
> 
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Remove local_reset of age_census object inadvertently left behind in the
>    GLOBAL gen concurrent mark, which was triggering the newly added
>    reasonableness assert (yay!).
>  - Avoid divide-by-zero.
>  - Merge branch 'master' into clear_census
>  - Fix word and byte size unit confusion in comparison/assert.
>  - Loose verification of "reasonable completeness" of census.
>  - Fix release build.
>  - ... and 7 more: https://git.openjdk.org/shenandoah/compare/9d869ca1...787e37a8

Note: The linux-64/test (tier1) runtime failure is just an infra failure to upload test results. The tests all passed as verified from the test info:

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR   
   jtreg:test/hotspot/jtreg:tier1_runtime              642   642     0     0   
==============================
TEST SUCCESS


So the red x against the linux-64/tier1_runtime can be ignored.

Once the rest of the ongoing GHA tests complete, I'll consider integrating this change.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/403#issuecomment-2053034076

From stuefe at openjdk.org  Sat Apr 13 05:41:42 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Sat, 13 Apr 2024 05:41:42 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v5]
In-Reply-To: <asRwE8-BGk8hnVyAK7lHj0f0Esb5kq_Nx9fu0txB2Pc=.144e81bd-2d5d-4d57-9b85-1a281b6ca163@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <asRwE8-BGk8hnVyAK7lHj0f0Esb5kq_Nx9fu0txB2Pc=.144e81bd-2d5d-4d57-9b85-1a281b6ca163@github.com>
Message-ID: <SfhnbpVFZHB4JMJoyP_mpPKqBLq1uEYjKIeno5lgj-k=.8ca29f8e-c944-4970-b157-1d67ed2c9f87@github.com>

On Fri, 12 Apr 2024 19:10:09 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   mtCode and mtMetaspace were missed from System Dump map

Just a thought: one (manual) test I would do would be that several JVMs run with the same conditions (I would do at least one with -Xmx=Xms and AlwaysPreTouch) accumulate the same NMT numbers, current, and peak. Just to make sure we use the same flags before and after.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2053511560

From kdnilsen at openjdk.org  Sat Apr 13 15:19:50 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 13 Apr 2024 15:19:50 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v39]
In-Reply-To: <IimvSux5GEemCzBEY0lZHUCxFTFzhTM7mvCm7AD3nV4=.ed14fcb8-d5a3-4439-a2f8-8444cf7df123@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <IimvSux5GEemCzBEY0lZHUCxFTFzhTM7mvCm7AD3nV4=.ed14fcb8-d5a3-4439-a2f8-8444cf7df123@github.com>
Message-ID: <99oUiKynBFbBtK69VEkcI1rATPTdtKM8HIZE4WKE6gs=.59ed38a5-4781-41c8-ad36-6d3645ecf892@github.com>

On Tue, 9 Apr 2024 19:33:52 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove debugging instrumentation
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 45:
> 
>> 43: // break abstraction rules, because efficient implementation requires assumptions about superclass internals that
>> 44: // might be violatee through future software maintenance.
>> 45: class ShenandoahSimpleBitMap {
> 
> I think this class should go into its own set of files. It would certainly help readability of ShFreeSet which is now somewhat dominated by ShSimpleBitMap.

Thanks for suggestion.  Done.

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 165:
> 
>> 163: 
>> 164: // Each ShenandoahHeapRegion is associated with a ShenandoahFreeSetPartitionId.
>> 165: enum ShenandoahFreeSetPartitionId : uint8_t {
> 
> Make this an 'enum class' instead.
> I see that you are using the enum as index, in this case you can declare the enum class like 'enum class ShenandoahFreeSetPartitionId : uint8_t'

Thanks for this suggestion.  Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1564090012
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1564090128

From ysr at openjdk.org  Sat Apr 13 15:22:01 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 13 Apr 2024 15:22:01 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial [v5]
In-Reply-To: <JWyhtWUiQjIKems-D3kykmTPtU19QVm72Nx9_DZ4kHY=.42ae3338-965c-430f-bd4c-dcfc6bff6a7c@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
 <JWyhtWUiQjIKems-D3kykmTPtU19QVm72Nx9_DZ4kHY=.42ae3338-965c-430f-bd4c-dcfc6bff6a7c@github.com>
Message-ID: <URJ_ruE0_YrSDSHbuzuGq5HV8mrlncTPlkaW13XdtuU=.c5cf52cc-aeca-40b8-9024-02b00579277a@github.com>

On Sat, 13 Apr 2024 00:24:08 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
>> 
>> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
>> 
>> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
>> 
>> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
>> 
>> **Testing**:
>> - [x] GHA 
>> - [x] Code pipeline testing : one intermittent stress dacapo failure, ascribed to an existing bug in coalesce-and-fill
>> - [x] SPECjbb
>> 
>> **Performance**: 
>> - [x] SPECjbb : the variance in tests fails any significant change under 2-tailed Mann-Whitney
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits:
> 
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Merge branch 'master' into clear_census
>  - Remove local_reset of age_census object inadvertently left behind in the
>    GLOBAL gen concurrent mark, which was triggering the newly added
>    reasonableness assert (yay!).
>  - Avoid divide-by-zero.
>  - Merge branch 'master' into clear_census
>  - Fix word and byte size unit confusion in comparison/assert.
>  - Loose verification of "reasonable completeness" of census.
>  - Fix release build.
>  - ... and 7 more: https://git.openjdk.org/shenandoah/compare/9d869ca1...787e37a8

The tests were all successful after latest merge as well (see note above). There were no changes after the most recent review, which is current.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/403#issuecomment-2053675115

From ysr at openjdk.org  Sat Apr 13 15:22:01 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 13 Apr 2024 15:22:01 GMT
Subject: Integrated: 8327388: GenShen: census during marking is partial
In-Reply-To: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
References: <g_Iob3H5hUHpr6iN1nzSSSM_u7_8z9hO6Al52hRtyUc=.ee22a343-8e84-454f-8da0-541cd3146026@github.com>
Message-ID: <4V0WHYFz7RiSPoyURwnNzyJfT3VCyEbEXmdtZ9whlzU=.e0470978-e5ff-4d85-8f4f-0ad2fb4d4a19@github.com>

On Tue, 5 Mar 2024 17:43:20 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> There was a bug in the placement of the call to clear stale census data before starting a fresh one for a new marking cycle that marks through the younger generation. This bug resulted in the use of a co-terminal suffix of the census collection, losing all data from the earlier iterations of an iterative collection process that may run up to 5 times.
> 
> We stumbled upon the defect when working on a refactoring task involving separation of generational extensions of Shenandoah from its non-generational version. The (performance) defect has existed since day zero of the adaptive tenuring code in GenShen.
> 
> Along with fixing the defect, an assertion has been added to check the "reasonable completeness" of the census, which came in useful to detect a reset call inadvertently left behind in one place.
> 
> Some ShenandoahAgeCensus APIs have been narrowed and cleaned up a bit, and documentation clarified a bit more.
> 
> **Testing**:
> - [x] GHA 
> - [x] Code pipeline testing : one intermittent stress dacapo failure, ascribed to an existing bug in coalesce-and-fill
> - [x] SPECjbb
> 
> **Performance**: 
> - [x] SPECjbb : the variance in tests fails any significant change under 2-tailed Mann-Whitney

This pull request has now been integrated.

Changeset: 10febd95
Author:    Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/10febd95d093165ca2e91c12e385e3754dc47ea0
Stats:     166 lines in 11 files changed: 121 ins; 27 del; 18 mod

8327388: GenShen: census during marking is partial

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/403

From stuefe at openjdk.org  Sat Apr 13 18:18:41 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Sat, 13 Apr 2024 18:18:41 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
Message-ID: <PQCCRRVbiMnuCRhJp6mNeIyPU1zFSP0UXpWQ5tq4Fuc=.14c30570-f138-4615-bae6-e6e861c6be4b@github.com>

On Fri, 12 Apr 2024 14:40:05 GMT, Sergey Nazarkin <snazarki at openjdk.org> wrote:

> An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 
> 
> Additional testing:
> - [x] MacOS AArch64 server fastdebug *gtets*
> - [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
> - [ ] Benchmarking
> 
> @apangin and @parttimenerd could you please check the patch on your scenarios??

I have one question, and I'm sorry if it has been answered before. How expensive is changing the mode? Is it just a status variable in user-space pthread lib? Or does it need a system call? 

In other words, how fine granular can we get without incurring too high a cost?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2053721713

From aph at openjdk.org  Sat Apr 13 19:09:46 2024
From: aph at openjdk.org (Andrew Haley)
Date: Sat, 13 Apr 2024 19:09:46 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <PQCCRRVbiMnuCRhJp6mNeIyPU1zFSP0UXpWQ5tq4Fuc=.14c30570-f138-4615-bae6-e6e861c6be4b@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
 <PQCCRRVbiMnuCRhJp6mNeIyPU1zFSP0UXpWQ5tq4Fuc=.14c30570-f138-4615-bae6-e6e861c6be4b@github.com>
Message-ID: <64tQxEFk-VGE424wdgtQgzEQnj9R5POxrLCKyGEpEGw=.7f40b04f-fdb8-4485-b530-1841384dbc8a@github.com>

On Sat, 13 Apr 2024 18:16:21 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> I have one question, and I'm sorry if it has been answered before. How expensive is changing the mode? Is it just a status variable in user-space pthread lib? Or does it need a system call?
> 
> In other words, how fine granular can we get without incurring too high a cost?

It's expensive. We've seen it cause significant slowdowns in Java->VM transitions.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2053734174

From burban at openjdk.org  Mon Apr 15 08:35:42 2024
From: burban at openjdk.org (Bernhard Urban-Forster)
Date: Mon, 15 Apr 2024 08:35:42 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
Message-ID: <RccCLpcBzmE5HG415eFBvMaHEcPsq92INC7JBl80N8k=.1af1c97b-2404-4db9-b594-9e9fec38a57c@github.com>

On Fri, 12 Apr 2024 14:40:05 GMT, Sergey Nazarkin <snazarki at openjdk.org> wrote:

> An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 
> 
> Additional testing:
> - [x] MacOS AArch64 server fastdebug *gtets*
> - [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
> - [ ] Benchmarking
> 
> @apangin and @parttimenerd could you please check the patch on your scenarios??

I agree that this PR effectively removes the whole idea behind JIT_MAP and thus is a bad idea security wise.  Still it has some value.

@snazarkin do you have numbers on how many transitions are done with your PR vs. the current state when running the same program?  That would give us a lower bound on the amount of transitions needed and define a goal for future improvements in that area.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2056182560

From pminborg at openjdk.org  Mon Apr 15 08:37:55 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Mon, 15 Apr 2024 08:37:55 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v2]
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <PnQQ1SSVPLCXgkF0ibFC6zS_4whAKNo5w3vf0CxsZ7g=.14452427-ceff-49c6-9327-7687e7594515@github.com>

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

Per Minborg has updated the pull request incrementally with two additional commits since the last revision:

 - Fix typo
 - Update after PR comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18474/files
  - new: https://git.openjdk.org/jdk/pull/18474/files/159b9c44..7de90243

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=00-01

  Stats: 13 lines in 1 file changed: 3 ins; 2 del; 8 mod
  Patch: https://git.openjdk.org/jdk/pull/18474.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18474/head:pull/18474

PR: https://git.openjdk.org/jdk/pull/18474

From mcimadamore at openjdk.org  Mon Apr 15 09:44:42 2024
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 15 Apr 2024 09:44:42 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v2]
In-Reply-To: <PnQQ1SSVPLCXgkF0ibFC6zS_4whAKNo5w3vf0CxsZ7g=.14452427-ceff-49c6-9327-7687e7594515@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <PnQQ1SSVPLCXgkF0ibFC6zS_4whAKNo5w3vf0CxsZ7g=.14452427-ceff-49c6-9327-7687e7594515@github.com>
Message-ID: <en2p_TxXXhpIpL1UoVnKaahC_ciqD6FxkrAXvYJn2Z4=.f7daf05a-ce73-4106-b22b-d15992e4101a@github.com>

On Mon, 15 Apr 2024 08:37:55 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Fix typo
>  - Update after PR comments

src/java.base/share/classes/java/lang/foreign/SymbolLookup.java line 154:

> 152:      * Returns the address of the symbol with the given name or throws an Exception.
> 153:      *<p>
> 154:      * This is equivalent to the following code, but is more resource efficient:

Suggestion:

     * This is equivalent to the following code, but is more efficient:

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1565485693

From pminborg at openjdk.org  Mon Apr 15 13:48:09 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Mon, 15 Apr 2024 13:48:09 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v3]
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <ohwlY2f69wykOw4yOEJ02Gr0oQjD_jAi6JRZsavE-O4=.e10e68ca-f02a-4f57-96c6-55f918a5b199@github.com>

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

Per Minborg has updated the pull request incrementally with one additional commit since the last revision:

  Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
  
  Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18474/files
  - new: https://git.openjdk.org/jdk/pull/18474/files/7de90243..e173ff09

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/18474.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18474/head:pull/18474

PR: https://git.openjdk.org/jdk/pull/18474

From kdnilsen at openjdk.org  Mon Apr 15 13:50:30 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 13:50:30 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v7]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <DRnsMhTu00OkOZkhAg1Z8RyLzHVufjV5g7LPJqq91ko=.cdc432da-7e14-4822-89eb-21dfa7c25f06@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits:

 - Merge branch 'master' of https://git.openjdk.org/shenandoah into enable-old-growth-triggers
 - Move trigger code to ShenandoahOldHeuristics from ShenandoahOldGeneration
 - Fix another typo
 - Merge branch 'master' of https://git.openjdk.org/shenandoah into enable-old-growth-triggers
 - Fix typo introduced during refactoring
 - Respond to reviewer feedback
 - Add a jtreg test for old growth trigger
 - Re-enable old-growth trigger
 - Make satb-mode Info logging less verbose
 - Merge branch 'openjdk:master' into master
 - ... and 5 more: https://git.openjdk.org/shenandoah/compare/10febd95...ef967a55

-------------

Changes: https://git.openjdk.org/shenandoah/pull/409/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=06
  Stats: 225 lines in 6 files changed: 173 ins; 49 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From mcimadamore at openjdk.org  Mon Apr 15 13:54:44 2024
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 15 Apr 2024 13:54:44 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v3]
In-Reply-To: <ohwlY2f69wykOw4yOEJ02Gr0oQjD_jAi6JRZsavE-O4=.e10e68ca-f02a-4f57-96c6-55f918a5b199@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <ohwlY2f69wykOw4yOEJ02Gr0oQjD_jAi6JRZsavE-O4=.e10e68ca-f02a-4f57-96c6-55f918a5b199@github.com>
Message-ID: <zYxe2VpE1LLGI_dpgWKKvk7TZ9fwuLQhIHgIm9Mq85g=.6c40cf8e-9053-4a98-bc0c-ba9442669006@github.com>

On Mon, 15 Apr 2024 13:48:09 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
>   
>   Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>

src/java.base/share/classes/java/lang/foreign/SymbolLookup.java line 152:

> 150: 
> 151:     /**
> 152:      * Returns the address of the symbol with the given name or throws an Exception.

Suggestion:

     * Returns the address of the symbol with the given name or throws an exception.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1565837858

From pminborg at openjdk.org  Mon Apr 15 14:02:56 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Mon, 15 Apr 2024 14:02:56 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v4]
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <5xcpEj85UX0BJUK8vYDDXymSvB9MrVjXOCCmUlyVE9k=.86e0b7c5-edf9-47a9-83eb-9dac4570935c@github.com>

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

Per Minborg has updated the pull request incrementally with one additional commit since the last revision:

  Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
  
  Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18474/files
  - new: https://git.openjdk.org/jdk/pull/18474/files/e173ff09..fa86d837

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=02-03

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/18474.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18474/head:pull/18474

PR: https://git.openjdk.org/jdk/pull/18474

From mcimadamore at openjdk.org  Mon Apr 15 14:48:01 2024
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 15 Apr 2024 14:48:01 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v4]
In-Reply-To: <5xcpEj85UX0BJUK8vYDDXymSvB9MrVjXOCCmUlyVE9k=.86e0b7c5-edf9-47a9-83eb-9dac4570935c@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <5xcpEj85UX0BJUK8vYDDXymSvB9MrVjXOCCmUlyVE9k=.86e0b7c5-edf9-47a9-83eb-9dac4570935c@github.com>
Message-ID: <yaXPZJz7zNPigYlPVl6JraAAIWWmGd8KE_9umn-m71U=.3e0367fd-f66d-4689-a593-3778a7a6730c@github.com>

On Mon, 15 Apr 2024 14:02:56 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
>   
>   Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>

Marked as reviewed by mcimadamore (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18474#pullrequestreview-2001319293

From azafari at openjdk.org  Mon Apr 15 15:35:56 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 15:35:56 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v6]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <vduE9ti23X23bgiLvePCtb_A3b2M5hY23u-FjNGAPsA=.d7cd58e6-3915-4b64-945b-aa86722eba47@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  mtNone is not used anymore as default for optional args.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/a3940639..930d2748

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=04-05

  Stats: 109 lines in 19 files changed: 23 ins; 9 del; 77 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Mon Apr 15 15:41:06 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 15:41:06 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v2]
In-Reply-To: <OnazIuzb6ee0NaLnoCnDczhj8sU3iSwcfgwxK1dPNyY=.626e490a-3f74-4ff1-8482-bb8a6094c819@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <3id02uEHGujrMSIg0lKlstLRL0x2yTsn7lPWrmEwGBU=.747fa9c2-e782-462f-95ba-bc567944a502@github.com>
 <MJ0ntztJgKvmb7eL1-Hlu_MnQZkzNPwIeUnquORXmrw=.42254d3f-afb5-4ddf-bc14-0f8150f780b8@github.com>
 <LszyVGEC4vZO25L_HI7lyhR-XXJJSEGrkM1mjqCppJA=.8598c81b-02e8-41af-9318-161cb0788f30@github.com>
 <OnazIuzb6ee0NaLnoCnDczhj8sU3iSwcfgwxK1dPNyY=.626e490a-3f74-4ff1-8482-bb8a6094c819@github.com>
Message-ID: <G9hWKXh0j-HBRqEti4_QwjvmjEhTotTtxNBbCB52vfI=.82371235-ca90-4e5d-bde2-e8f12697f80e@github.com>

On Fri, 12 Apr 2024 08:15:56 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Fixed.
>
> It still looks wrong.

Should be fixed now.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1565994207

From azafari at openjdk.org  Mon Apr 15 15:41:06 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 15:41:06 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <lhFIh2qUbeY3kzGklUGuLCMZCC9lMzqMevukRHLzdik=.8c3147d8-62bd-4e7e-9630-d5c7103d0418@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
 <lhFIh2qUbeY3kzGklUGuLCMZCC9lMzqMevukRHLzdik=.8c3147d8-62bd-4e7e-9630-d5c7103d0418@github.com>
Message-ID: <wqTyxKvkXPWRHXtRxq0BfR9H33NhbK6somlPzp1zYHQ=.3dd6bc06-a854-4679-9c4a-177661494d58@github.com>

On Fri, 12 Apr 2024 07:45:43 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   review comments applied.
>
> src/hotspot/share/nmt/virtualMemoryTracker.hpp line 307:
> 
>> 305: 
>> 306:   ReservedMemoryRegion(address base, size_t size, MEMFLAGS flag) :
>> 307:     VirtualMemoryRegion(base, size), _stack(NativeCallStack::empty_stack()), _flag(flag) { }
> 
> The function above uses mtNone. I find that a bit dubious, but I understand that it is done to be able to write code like this:
> 
>     ReservedMemoryRegion* rmr = VirtualMemoryTracker::_reserved_regions->find(ReservedMemoryRegion(addr, size));
> 
> 
> Unfortunately, it opens up the door for people to accidentally use that version instead of this new version that you have written. Could we get rid of the version using mtNone somehow?
> 
> The same question goes for the version above that, which has a "MEMFLAGS flag = mtNone". (GH doesn't allow me to comment on lines that you haven't changed)

`mtNone` as default value is no longer valid.

> src/hotspot/share/runtime/os.hpp line 511:
> 
>> 509:   // and is added to be used for implementation of -XX:AllocateHeapAt
>> 510:   static char* map_memory_to_file(size_t size, int fd, MEMFLAGS flag = mtNone);
>> 511:   static char* map_memory_to_file_aligned(size_t size, size_t alignment, int fd, MEMFLAGS flag);
> 
> There are still a few mtNone usages in this file.

`mtNone` as default value for optional arguments is removed from all function definitions.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1565995694
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1565997114

From azafari at openjdk.org  Mon Apr 15 15:41:07 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 15:41:07 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <4eN_yJUIi_0MTBROX0yxeIZIYo4W3KNlBGGOSA3glI4=.8e6ec837-1cb3-414f-959c-86fb3e3c9907@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
 <4eN_yJUIi_0MTBROX0yxeIZIYo4W3KNlBGGOSA3glI4=.8e6ec837-1cb3-414f-959c-86fb3e3c9907@github.com>
Message-ID: <8FDhtk1BZwnz60dyaHDmEzcgQUjX2bCGf8nn7nA5WrY=.eaccab67-6ba1-499d-946d-cb07383cf16b@github.com>

On Fri, 12 Apr 2024 07:59:06 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   review comments applied.
>
> src/hotspot/share/runtime/os.hpp line 521:
> 
>> 519:                            bool allow_exec = false, MEMFLAGS flags = mtNone);
>> 520:   static bool   unmap_memory(char *addr, size_t bytes);
>> 521:   static void   free_memory(char *addr, size_t bytes, size_t alignment_hint, MEMFLAGS flag);
> 
> While looking at this, I noticed a couple of odd things about this function. I think it should be revised and I opened https://bugs.openjdk.org/browse/JDK-8330144. The result of that revision will be that we don't need MEMFLAGS, nor do would we need the alignment hint. 
> 
> But leave the MEMFLAGS in for now. If I happen to push that change first, you can adapt the change, if you push first I'll manage.

OK.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1565997790

From azafari at openjdk.org  Mon Apr 15 15:50:07 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 15:50:07 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <lhFIh2qUbeY3kzGklUGuLCMZCC9lMzqMevukRHLzdik=.8c3147d8-62bd-4e7e-9630-d5c7103d0418@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
 <lhFIh2qUbeY3kzGklUGuLCMZCC9lMzqMevukRHLzdik=.8c3147d8-62bd-4e7e-9630-d5c7103d0418@github.com>
Message-ID: <-5QgNzuuNNMIYopm2_6TQa09bBKILK8RInI-HnMYdJY=.282cd643-3e61-416f-bd16-2db958e5f69c@github.com>

On Fri, 12 Apr 2024 07:33:51 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   review comments applied.
>
> src/hotspot/share/memory/virtualspace.cpp line 615:
> 
>> 613: 
>> 614: ReservedHeapSpace::ReservedHeapSpace(size_t size, size_t alignment, size_t page_size, const char* heap_allocation_directory) : ReservedSpace() {
>> 615:   set_nmt_flag(mtJavaHeap);
> 
> It seems odd that we only initialize the _nmt_flag when `size == 0`. Could this be done after that check? If not, why not?
> 
> There's also a call to record_virtual_memory_type further down in the code. Why is that needed? Why isn't it enough to pass in the correct type to the os::reserve_memory call in the initialize function?

Corrected.

> src/hotspot/share/memory/virtualspace.cpp line 672:
> 
>> 670:                                      size_t rs_align,
>> 671:                                      size_t rs_page_size) : ReservedSpace() {
>> 672:   set_nmt_flag(mtCode);
> 
> Why isn't this a part of the initialize call? This looks like a bug to me. `initialize` will call clear_members, which will undo this setting.

`set_nmt_flag()`, `initialize` and `initialize_members` are chagned accorrding to the related comments.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566006111
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566009377

From jvernee at openjdk.org  Mon Apr 15 16:10:04 2024
From: jvernee at openjdk.org (Jorn Vernee)
Date: Mon, 15 Apr 2024 16:10:04 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v4]
In-Reply-To: <5xcpEj85UX0BJUK8vYDDXymSvB9MrVjXOCCmUlyVE9k=.86e0b7c5-edf9-47a9-83eb-9dac4570935c@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <5xcpEj85UX0BJUK8vYDDXymSvB9MrVjXOCCmUlyVE9k=.86e0b7c5-edf9-47a9-83eb-9dac4570935c@github.com>
Message-ID: <pZ7FQ20nyd2zTFvoqQxM9u4IvW6tzkk7AuGFacJfHv4=.4812ae1c-db82-4be6-923e-2fe40931d2c8@github.com>

On Mon, 15 Apr 2024 14:02:56 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
>   
>   Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>

Marked as reviewed by jvernee (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18474#pullrequestreview-2001521250

From azafari at openjdk.org  Mon Apr 15 16:11:13 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 16:11:13 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  alignment in coding style changed.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/930d2748..abcfcccd

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=06
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=05-06

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Mon Apr 15 16:11:15 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 16:11:15 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <lhFIh2qUbeY3kzGklUGuLCMZCC9lMzqMevukRHLzdik=.8c3147d8-62bd-4e7e-9630-d5c7103d0418@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
 <lhFIh2qUbeY3kzGklUGuLCMZCC9lMzqMevukRHLzdik=.8c3147d8-62bd-4e7e-9630-d5c7103d0418@github.com>
Message-ID: <B5FD1uvKRJ-OYZSmxyXUBmvp1XF4UyzFKvyFpAO-j0w=.71e06611-77d1-46fb-be4b-5f6a9e5bd5a1@github.com>

On Fri, 12 Apr 2024 07:06:50 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   review comments applied.
>
> src/hotspot/os/windows/os_windows.cpp line 3137:
> 
>> 3135:   // If reservation failed, return null
>> 3136:   if (p_buf == nullptr) return nullptr;
>> 3137:   MemTracker::record_virtual_memory_reserve((address)p_buf, size_of_reserve, CALLER_PC, mtInternal);
> 
> I think that allocate_pages_individually should take a MEMFLAGS argument instead of using mtInternal here.

It takes now. All corresponding functions in call hierarchy are also changed.

> src/hotspot/os/windows/os_windows.cpp line 3198:
> 
>> 3196:         // the release.
>> 3197:         MemTracker::record_virtual_memory_reserve((address)p_buf,
>> 3198:                                                   bytes_to_release, CALLER_PC, mtNone);
> 
> I don't think we should ever use `mtNone` in code outside of the NMT code. If you follow my suggestion above that allocate_pages_individually should take a MEMFLAG arg, then it could be used here.

Corrected.

> src/hotspot/os/windows/os_windows.cpp line 3218:
> 
>> 3216:     MemTracker::record_virtual_memory_reserve_and_commit((address)p_buf, bytes, CALLER_PC);
>> 3217:   } else {
>> 3218:     MemTracker::record_virtual_memory_reserve((address)p_buf, bytes, CALLER_PC, mtNone);
> 
> Use the correct MEMFLAG here instead of mtNone.

Done.

> src/hotspot/os/windows/os_windows.cpp line 3771:
> 
>> 3769:   if (!is_committed) {
>> 3770:     commit_memory_or_exit(addr, bytes, prot == MEM_PROT_RWX,
>> 3771:                           "cannot commit protection page", mtNone);
> 
> This should probably be something else than mtNone.

Changed to `mtInternal`. When `protect_memory` is called with uncommitted area (`is_committed == false`), the flag is mtInternal.

> src/hotspot/share/jfr/recorder/storage/jfrVirtualMemory.cpp line 107:
> 
>> 105:   _rs = ReservedSpace(reservation_size_request_bytes,
>> 106:                       os::vm_allocation_granularity(),
>> 107:                       os::vm_page_size(), mtTracing);
> 
> The mtTracing should probably be on a separate line, so that it follows the style of the surrounding code.

Done.

> src/hotspot/share/memory/virtualspace.cpp line 45:
> 
>> 43: // Dummy constructor
>> 44: ReservedSpace::ReservedSpace() : _base(nullptr), _size(0), _noaccess_prefix(0),
>> 45:     _alignment(0), _special(false), _fd_for_heap(-1), _nmt_flag(mtNone), _executable(false) {
> 
> In almost all code we pass in the executable before the flag, but in ReservedSpace the flag is located before the executable. I think it would be nice to flip the order in this class. I understand that _executable is in the private section, while the other members are protected, but I don't think that it needs to be that way. The _executable could probably just be moved together with the rest of the members.
> 
> OTOH, I think the entire class needs some cleanups. Let's leave this for a separate RFE.

Changed.

> src/hotspot/share/memory/virtualspace.cpp line 708:
> 
>> 706:   assert(max_commit_granularity > 0, "Granularity must be non-zero.");
>> 707: 
>> 708:   _nmt_flag = rs.nmt_flag();
> 
> The code seems to be written with blank lines to separate various members that belong together. Please add a blank line after this line.

Fixed and moved.

> src/hotspot/share/memory/virtualspace.hpp line 199:
> 
>> 197:   size_t _upper_alignment;
>> 198: 
>> 199:   MEMFLAGS _nmt_flag;
> 
> The VirtualSpace::initialize functions used to initialize these members in the order that they are specified here. That is now messed up by adding the _nmt_flag at the end here, but in the beginning in the initialize function. I would propose that you move it to after _executable, both here and in the initialize function.

Fixed.

> test/hotspot/gtest/gc/g1/test_freeRegionList.cpp line 53:
> 
>> 51:   size_t bot_size = G1BlockOffsetTable::compute_size(heap.word_size());
>> 52:   HeapWord* bot_data = NEW_C_HEAP_ARRAY(HeapWord, bot_size, mtGC);
>> 53:   ReservedSpace bot_rs(G1BlockOffsetTable::compute_size(heap.word_size()), mtGC);
> 
> mtGC => mtTest?

Done.

> test/hotspot/gtest/gc/z/test_zForwarding.cpp line 103:
> 
>> 101:     _reserved = reserved;
>> 102: 
>> 103:     os::commit_memory((char*)_reserved, ZGranuleSize, !ExecMem /* executable */, mtGC);
> 
> mtGC => mtTest?

Done.

> test/hotspot/gtest/gc/z/test_zForwarding.cpp line 114:
> 
>> 112:     ZGeneration::_young = _old_young;
>> 113:     if (_reserved != nullptr) {
>> 114:       os::uncommit_memory((char*)_reserved, ZGranuleSize, !ExecMem, mtGC);
> 
> mtGC => mtTest?

Done.

> test/hotspot/gtest/memory/test_virtualspace.cpp line 223:
> 
>> 221:         return ReservedSpace(reserve_size_aligned,
>> 222:                              os::vm_allocation_granularity(),
>> 223:                              os::vm_page_size(), mtTest);
> 
> newline before mtTest.

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566039719
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566038228
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566037774
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566037525
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566033618
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566026381
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566027896
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566028800
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566029979
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566032851
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566032647
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566030330

From azafari at openjdk.org  Mon Apr 15 16:11:15 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 16:11:15 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <4eN_yJUIi_0MTBROX0yxeIZIYo4W3KNlBGGOSA3glI4=.8e6ec837-1cb3-414f-959c-86fb3e3c9907@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
 <4eN_yJUIi_0MTBROX0yxeIZIYo4W3KNlBGGOSA3glI4=.8e6ec837-1cb3-414f-959c-86fb3e3c9907@github.com>
Message-ID: <nwx8_k7fUWqYX7WR2uEhtGohrA4_-UPWDw0bJU2hnHs=.929ca2e7-217c-4c86-947d-3ba4564da2e0@github.com>

On Fri, 12 Apr 2024 07:42:11 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   review comments applied.
>
> src/hotspot/share/memory/metaspace/testHelpers.cpp line 81:
> 
>> 79:   if (reserve_limit > 0) {
>> 80:     // have reserve limit -> non-expandable context
>> 81:     _rs = ReservedSpace(reserve_limit * BytesPerWord, Metaspace::reserve_alignment(), os::vm_page_size(), mtTest);
> 
> mtMetaspace

Done

> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 112:
> 
>> 110: 
>> 111:   // Commit...
>> 112:   if (os::commit_memory((char*)p, word_size * BytesPerWord, !ExecMem, _rs.nmt_flag()) == false) {
> 
> just use mtMetaspace here, its easier

Done.

> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 191:
> 
>> 189: 
>> 190:   // Uncommit...
>> 191:   if (os::uncommit_memory((char*)p, word_size * BytesPerWord, !ExecMem, _rs.nmt_flag()) == false) {
> 
> mtMetaspace

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566031812
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566032265
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1566032016

From azafari at openjdk.org  Mon Apr 15 16:16:44 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Mon, 15 Apr 2024 16:16:44 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v5]
In-Reply-To: <SfhnbpVFZHB4JMJoyP_mpPKqBLq1uEYjKIeno5lgj-k=.8ca29f8e-c944-4970-b157-1d67ed2c9f87@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <asRwE8-BGk8hnVyAK7lHj0f0Esb5kq_Nx9fu0txB2Pc=.144e81bd-2d5d-4d57-9b85-1a281b6ca163@github.com>
 <SfhnbpVFZHB4JMJoyP_mpPKqBLq1uEYjKIeno5lgj-k=.8ca29f8e-c944-4970-b157-1d67ed2c9f87@github.com>
Message-ID: <1zx5BUSqZfy81_KftcRahy0vtrBYXBDvZPxOApOqWcs=.39fd3462-221f-4a07-a043-6e2bc4dc918e@github.com>

On Sat, 13 Apr 2024 05:38:11 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> Just a thought: one (manual) test I would do would be that several JVMs run with the same conditions (I would do at least one with -Xmx=Xms and AlwaysPreTouch) accumulate the same NMT numbers, current, and peak. Just to make sure we use the same flags before and after.

I understand your idea as: run JVM with options to show virtual memory report, one for master branch (before this PR) and one using the PR's branch. Then it is expected that the reports show the same numbers. Right? 
Nice idea. Will do it. Thanks.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2057232308

From wkemper at openjdk.org  Mon Apr 15 16:18:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 15 Apr 2024 16:18:00 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
 <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
 <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
Message-ID: <Ry3NBuTT8BKw5YI2zzGXRcgtC10q208rOPln8B10g58=.41e2891f-e714-422d-b50a-f1aec054404c@github.com>

On Sat, 6 Apr 2024 00:33:38 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>>  - Use is_regular_pinned to restore candidate selection logic to its previous form
>>  - Introduce 'regular pinned' state
>>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>>  - Allow pinned/regular regions into mixed collection candidates
>>  - Little clean up
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 332:
> 
>> 330:       immediate_garbage += garbage;
>> 331:     } else if (region->is_humongous_start()) {
>> 332:       // This will handle humongous start regions whether they are also pinned, or not.
> 
> Since we've already handled region->is_trash() above, do we need this code?  If we want to preserve the assert(!is_pinned), we can insert that into the is_trash() code above.  Right?

I didn't change the implementation of `is_humongous_start`, so it still includes pinned humongous objects.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1566048445

From kdnilsen at openjdk.org  Mon Apr 15 16:28:09 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 16:28:09 GMT
Subject: Integrated: 8330071: GenShen: Allow old to expand again at end of each
 GC
In-Reply-To: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
References: <mMr3NtNFi08E2AWeP9x_XmXqER3011yltxeYCNfToHE=.0c97e0d3-61c0-43ad-a6c5-fa603d2078b3@github.com>
Message-ID: <LQmJf53elocDPCHP2XPgjnilsG1fmWx5W7DA9JPUTXg=.5c4f3ae0-c035-4d5b-9ff2-3b79ae0595f6@github.com>

On Thu, 11 Apr 2024 15:20:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This corrects errors in the implementations of max_size_for(OLD) and min_size_for(OLD).  These errors were preventing expansion of OLD in preparation for subsequent mixed-evacuation cycles.

This pull request has now been integrated.

Changeset: 0578af80
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/0578af8015bc0dfcd4a2b726b1a8b52f68712cf2
Stats:     5 lines in 1 file changed: 4 ins; 0 del; 1 mod

8330071: GenShen: Allow old to expand again at end of each GC

Reviewed-by: wkemper, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/421

From snazarki at openjdk.org  Mon Apr 15 16:34:01 2024
From: snazarki at openjdk.org (Sergey Nazarkin)
Date: Mon, 15 Apr 2024 16:34:01 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <RccCLpcBzmE5HG415eFBvMaHEcPsq92INC7JBl80N8k=.1af1c97b-2404-4db9-b594-9e9fec38a57c@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
 <RccCLpcBzmE5HG415eFBvMaHEcPsq92INC7JBl80N8k=.1af1c97b-2404-4db9-b594-9e9fec38a57c@github.com>
Message-ID: <C40feDZ9aXj9-By9rGsyZiL8JfzTsZfDLOgZ9Lr5IOU=.b73e712d-1ea9-4143-b87d-1dc7e47b4828@github.com>

On Mon, 15 Apr 2024 08:33:25 GMT, Bernhard Urban-Forster <burban at openjdk.org> wrote:

> do you have numbers on how many transitions are done with your PR vs. the current state when running the same program?

With just simple **java -version** it is ~180 vs ~9500 (new vs old),  for **java -help** ~1120 vs ~86300. For the applications the ration is about the same.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2057280998

From kdnilsen at openjdk.org  Mon Apr 15 16:34:05 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 16:34:05 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
 <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
Message-ID: <LrlJBrMk1TqGUPVGXqnGTjsN9cI971AtJGl2GAAs6Hw=.e4a6ddd8-1e6b-47c5-a88a-58472e7decc2@github.com>

On Fri, 12 Apr 2024 23:00:11 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Pinned regions that are not humongous may be added to the set of _candidates_ for mixed collection. They still may not be added to the collection set for a mixed collection. Regions that are not in this collection of _candidates_ will not be coalesced and filled.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>  - Use is_regular_pinned to restore candidate selection logic to its previous form
>  - Introduce 'regular pinned' state
>  - Merge remote-tracking branch 'shenandoah/master' into coalesce-old-pinned-regions
>  - Allow pinned/regular regions into mixed collection candidates
>  - Little clean up

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/420#pullrequestreview-2001595615

From kdnilsen at openjdk.org  Mon Apr 15 16:34:05 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 16:34:05 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable [v2]
In-Reply-To: <Ry3NBuTT8BKw5YI2zzGXRcgtC10q208rOPln8B10g58=.41e2891f-e714-422d-b50a-f1aec054404c@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
 <jfYPiLBxKDhQhMo5p8XPreVe2HFhT0zfuq9MjJMxbVs=.1f366d94-9bb6-4c3a-a4f9-fcf4b61a5b7c@github.com>
 <rWIa8WIWQrWY3JraEMBxIe5tkq3YcVHOBxH954GZegk=.89438c48-5850-4dd0-b835-cd4539b9231d@github.com>
 <Ry3NBuTT8BKw5YI2zzGXRcgtC10q208rOPln8B10g58=.41e2891f-e714-422d-b50a-f1aec054404c@github.com>
Message-ID: <sMrCtiVJ10kUhy-tOkCRPToCW4RiAaZpf-F8kcOGtqY=.f91a5d0e-054c-43b1-a6c4-a8f1261ba1dc@github.com>

On Mon, 15 Apr 2024 16:15:45 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 332:
>> 
>>> 330:       immediate_garbage += garbage;
>>> 331:     } else if (region->is_humongous_start()) {
>>> 332:       // This will handle humongous start regions whether they are also pinned, or not.
>> 
>> Since we've already handled region->is_trash() above, do we need this code?  If we want to preserve the assert(!is_pinned), we can insert that into the is_trash() code above.  Right?
>
> I didn't change the implementation of `is_humongous_start`, so it still includes pinned humongous objects.

My question was misguided.  Ignore it.  (I was confused when I was first reading this PR.)

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/420#discussion_r1566090417

From kdnilsen at openjdk.org  Mon Apr 15 16:42:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 16:42:12 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v8]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <0acv4pylPZFurPz-zDgZydsnQ3Md5nvyrvjU4QBycN8=.a1766d1b-5d88-4447-9b8e-c52d1b40e1e6@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Refactoring for reviewer feedback

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/409/files
  - new: https://git.openjdk.org/shenandoah/pull/409/files/ef967a55..e4e79b07

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=07
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=06-07

  Stats: 54 lines in 4 files changed: 24 ins; 14 del; 16 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From wkemper at openjdk.org  Mon Apr 15 16:46:05 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 15 Apr 2024 16:46:05 GMT
Subject: Integrated: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable
In-Reply-To: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
References: <fKkRyBx68KxSIFARozABon_LgxKW4txxSK8T4is7DV4=.b1d0d120-7e35-4e18-a3d4-45c5be3751eb@github.com>
Message-ID: <3-J_x_KERnMeQ_jLysj7Pw8eqt2WYei1_6gmsga30nI=.e16bfac2-a1a4-48af-a318-f0b40abd4646@github.com>

On Fri, 5 Apr 2024 22:06:14 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Pinned regions that are not humongous may be added to the set of _candidates_ for mixed collection. They still may not be added to the collection set for a mixed collection. Regions that are not in this collection of _candidates_ will not be coalesced and filled.

This pull request has now been integrated.

Changeset: 64b44a8c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/64b44a8c0e3a296d23905ac9e2233fc237057bab
Stats:     40 lines in 3 files changed: 17 ins; 8 del; 15 mod

8329790: GenShen: Old regions that are pinned during final mark are not made parseable

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/420

From kdnilsen at openjdk.org  Mon Apr 15 17:31:17 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 17:31:17 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v9]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <UJpxPc_hTk_tZpg9v43LrZZRXyt8DPxGStteZyQytHQ=.21a132aa-bd98-4d5c-bc37-08016c1f5a1d@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 22 commits:

 - Increase workload to force OLD collections
 - Merge remote-tracking branch 'origin/master' into enable-old-growth-triggers
 - Merge branch 'openjdk:master' into master
 - Revert "Change behavior of max_old and min_old"
   
   This reverts commit d88130000c764dacf08ec27723132dd2a3d968de.
 - Change behavior of max_old and min_old
 - Merge branch 'openjdk:master' into master
 - Refactoring for reviewer feedback
 - Merge branch 'master' of https://git.openjdk.org/shenandoah into enable-old-growth-triggers
 - Move trigger code to ShenandoahOldHeuristics from ShenandoahOldGeneration
 - Fix another typo
 - ... and 12 more: https://git.openjdk.org/shenandoah/compare/64b44a8c...901dbe0d

-------------

Changes: https://git.openjdk.org/shenandoah/pull/409/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=08
  Stats: 254 lines in 6 files changed: 191 ins; 57 del; 6 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From kdnilsen at openjdk.org  Mon Apr 15 17:31:18 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 17:31:18 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v6]
In-Reply-To: <eNjnhqWn06XrIJqi6JH5x-V_DXR8jE4CaHwmJgNDdfo=.bdb3fc6d-9e13-4469-ad7e-d8402e014c7f@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <EceXsAE6gdlyE7-w00aQxM7rmZ0Fkt9PBOD2QIXdNw0=.7a66381f-3f1b-42a9-b5ec-d5d819bedb76@github.com>
 <eNjnhqWn06XrIJqi6JH5x-V_DXR8jE4CaHwmJgNDdfo=.bdb3fc6d-9e13-4469-ad7e-d8402e014c7f@github.com>
Message-ID: <q00A3IfGchKsSVo3CIAdCepC3gb4BJH25gFdDkTouXw=.3e798210-9084-4ee8-b0ac-10fef492edd3@github.com>

On Sat, 13 Apr 2024 00:39:44 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Move trigger code to ShenandoahOldHeuristics from ShenandoahOldGeneration
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 548:
> 
>> 546: }
>> 547: 
>> 548: void ShenandoahOldHeuristics::trigger_collection_if_fragmented(ShenandoahGenerationalHeap* gen_heap, ShenandoahOldGeneration* old_gen,
> 
> Should `ShenandoahOldHeurtitics` just keep pointers to `gen_heap` and `old_gen`, and slim down these APIs so these don't need to be passed around? (Similarly `ShenandoahYoungHeuristics`?)

Agreed.  Implemented.

> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 579:
> 
>> 577:       double density_threshold = (eighths - 2) / 8.0;
>> 578:       if ((old_region_span >= span_threshold) && (old_density < density_threshold)) {
>> 579:         gen_heap->old_heuristics()->trigger_old_is_fragmented(old_density, first_old_region, last_old_region);
> 
> Just `trigger_old_is_fragmented()` because `gen_heap->old_heuristics()` is `this` ?

Thanks. Corrected.

> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 595:
> 
>> 593:          old_gen->used(), old_gen->get_humongous_waste(), gen_heap->capacity());
>> 594:   if (old_used > trigger_threshold) {
>> 595:     gen_heap->old_heuristics()->trigger_old_has_grown();
> 
> Just `trigger_old_has_grown()` because `gen_heap->old_heuristics()` is the same as`this` ?

Thanks.  Corrected.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1566200544
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1566203591
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1566204093

From kdnilsen at openjdk.org  Mon Apr 15 17:31:18 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 17:31:18 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v5]
In-Reply-To: <bIDuBxwNPnOKhQqaed3myym2agkFZnB5Mu--StVRg8w=.6039a8cd-2135-4119-97de-3bfd84a34534@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <BFuizLduIbx5NeR5F_0MVUqDCbZJsCbGGrzVESBsS8w=.5848ac32-fcb0-4a0e-94ce-b76f34fa2629@github.com>
 <_Vk9JRksaktAcBJNgK1sLPR0Q0-7_cuSwbI9UBDFQNA=.ee4a4f87-bf86-4f5a-8273-8d95d552c5e0@github.com>
 <bIDuBxwNPnOKhQqaed3myym2agkFZnB5Mu--StVRg8w=.6039a8cd-2135-4119-97de-3bfd84a34534@github.com>
Message-ID: <Jp669_b6uCfGd7A8L3V2CkJvT5pTRX8kg1_iNJ8uJRM=.4c4e27b4-83ff-43de-a8fd-1c86e8eee7c6@github.com>

On Mon, 1 Apr 2024 15:48:28 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3202:
>> 
>>> 3200: 
>>> 3201:   if (mode()->is_generational()) {
>>> 3202:     ShenandoahGenerationalHeap* gen_heap = (ShenandoahGenerationalHeap*) this;
>> 
>> May be `ShenandoahGenerationalHeap::heap()` like was done further above? (subjective nit can be ignored; see another suggestion further below that affects this as well.)
>
> The object on which the method is called could keep a reference to the generational heap to avoid the need to pass it as an argument in the call. (In particular, do the heuristics objets -- if these methods were to move there -- keep a reference to the heap and to the appropriate generations?)

Thanks.  Making these suggested changes.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1566198819

From kdnilsen at openjdk.org  Mon Apr 15 17:31:19 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 17:31:19 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v5]
In-Reply-To: <_Vk9JRksaktAcBJNgK1sLPR0Q0-7_cuSwbI9UBDFQNA=.ee4a4f87-bf86-4f5a-8273-8d95d552c5e0@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <BFuizLduIbx5NeR5F_0MVUqDCbZJsCbGGrzVESBsS8w=.5848ac32-fcb0-4a0e-94ce-b76f34fa2629@github.com>
 <_Vk9JRksaktAcBJNgK1sLPR0Q0-7_cuSwbI9UBDFQNA=.ee4a4f87-bf86-4f5a-8273-8d95d552c5e0@github.com>
Message-ID: <4pRIYdVzaT1yWJjI2X9spkfhCS5wYU-bJM072L6w4So=.05759f8e-3abc-4850-af8f-9e2e90f8d47a@github.com>

On Mon, 1 Apr 2024 15:39:06 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix another typo
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3202:
> 
>> 3200: 
>> 3201:   if (mode()->is_generational()) {
>> 3202:     ShenandoahGenerationalHeap* gen_heap = (ShenandoahGenerationalHeap*) this;
> 
> One other question: would something like this not be under the purview of the appropriate heuristics object associated with the heap? May be ShenandoahOldHeuristics? Just wondering about the correct abstraction boundaries here.
> 
> How does the regular (non-generational collector) do this following the rebuild?

Thanks.  Moving these trigger functions into OldHeuristics

> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 510:
> 
>> 508: }
>> 509: 
>> 510: void ShenandoahOldGeneration::trigger_collection_if_fragmented(ShenandoahGenerationalHeap* gen_heap, size_t first_old_region,
> 
> As pointed out further above, would this more naturally belong here or in ShenandoahOldHeuristics? There are a bunch of triggers & checks there as well. Would appear to make sense to keep all of this in one place rather than scattering them between generation and heuristic for clear abstraction boundaries and maintainability of code.

Agreed.  Thanks.  Moved.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1566199418
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1566199674

From kdnilsen at openjdk.org  Mon Apr 15 17:56:05 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 17:56:05 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v6]
In-Reply-To: <eNjnhqWn06XrIJqi6JH5x-V_DXR8jE4CaHwmJgNDdfo=.bdb3fc6d-9e13-4469-ad7e-d8402e014c7f@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <EceXsAE6gdlyE7-w00aQxM7rmZ0Fkt9PBOD2QIXdNw0=.7a66381f-3f1b-42a9-b5ec-d5d819bedb76@github.com>
 <eNjnhqWn06XrIJqi6JH5x-V_DXR8jE4CaHwmJgNDdfo=.bdb3fc6d-9e13-4469-ad7e-d8402e014c7f@github.com>
Message-ID: <8q7B8NwkAM9-RsVbgxXNgvS_V6II7BABbKUUpddwwhw=.8be8acb4-a544-412f-8a3c-a5bea6515735@github.com>

On Sat, 13 Apr 2024 01:14:23 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Move trigger code to ShenandoahOldHeuristics from ShenandoahOldGeneration
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 588:
> 
>> 586: 
>> 587: void ShenandoahOldHeuristics::trigger_collection_if_overgrown(ShenandoahGenerationalHeap* gen_heap, ShenandoahOldGeneration* old_gen) {
>> 588:   size_t old_used = old_gen->used() + old_gen->get_humongous_waste();
> 
> There's a `used_including_humongous_waste()` method in `ShenandoahGeneration` but unfortunately not yet in `ShenandoahSpaceInfo`.

I'm inclined to keep code as is for this integration.  We can think about refactoring and using ShenandoahSpaceInfo services in a future refinement.

> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 593:
> 
>> 591:   assert(old_used <= gen_heap->capacity(),
>> 592:          "Old used (" SIZE_FORMAT ", " SIZE_FORMAT") must not be more than heap capacity (" SIZE_FORMAT ")",
>> 593:          old_gen->used(), old_gen->get_humongous_waste(), gen_heap->capacity());
> 
> Does this assert belong here, rather than in the `used()` method?

I think I'll keep it here for two reasons:
1. We know the context here, and we know that this invariant should be true here.  Sometimes, during certain transitions that occur during Full GC for example, the invariant might be briefly violated, which could cause spurious false flags if I put the assert into used() method.
2. I'm actually summing two values to computed the value of old_used that feeds into this assert.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1566229498
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1566228371

From kdnilsen at openjdk.org  Mon Apr 15 18:02:17 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 15 Apr 2024 18:02:17 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v10]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <VylWleQgYliPHaaF7uKCiBE9YUpFe7QnSd0U3r2c1M0=.5c87bc14-5f25-4aef-856a-0c3dbb0ac423@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix zero build

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/409/files
  - new: https://git.openjdk.org/shenandoah/pull/409/files/901dbe0d..b767ab9f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=09
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=08-09

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From wkemper at openjdk.org  Mon Apr 15 20:41:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 15 Apr 2024 20:41:34 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <JN__f1fGhwA-dVcQn1-EqJaV_ZdfSVgHr9fs8Huj1B0=.11034e3f-c0df-443f-8dc7-990d822b231b@github.com>
References: <JN__f1fGhwA-dVcQn1-EqJaV_ZdfSVgHr9fs8Huj1B0=.11034e3f-c0df-443f-8dc7-990d822b231b@github.com>
Message-ID: <35C4X9mT946C6SDvAx6dYDFpGrfXnEtRcwsAIkBQW_c=.db757c4d-2462-4ed1-90a5-a42f414a9580@github.com>

> Merges tag jdk-23+18

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 177 commits:

 - Merge remote-tracking branch 'shenandoah/master' into merge-jdk-23+18
 - 8329254: optimize integral reverse operations on x86 GFNI target.
   
   Reviewed-by: sviswanathan
 - 8329656: assertion failed in MAP_ARCHIVE_MMAP_FAILURE path: Invalid immediate -5 0
   
   Reviewed-by: ccheung, iklam
 - 8329491: GetThreadListStackTraces function should use JvmtiHandshake
   
   Reviewed-by: pchilanomate, lmesnik
 - 8329432: PopFrame and ForceEarlyReturn functions should use JvmtiHandshake
   
   Reviewed-by: pchilanomate, lmesnik
 - 8330033: com/sun/net/httpserver/bugs/B6431193.java fails in AssertionError after JDK-8326568
   
   Reviewed-by: jpai, dfuchs
 - 8329961: Buffer overflow in os::Linux::kernel_version
   
   Reviewed-by: rehn, stuefe
 - 8327137: Add test for ConcurrentModificationException in BasicDirectoryModel
   
   Reviewed-by: serb, tr
 - 8309751: Duplicate constant pool entries added during default method processing
   
   Co-authored-by: Ashutosh Mehra <asmehra at openjdk.org>
   Reviewed-by: matsaave, dholmes
 - 8326568: jdk/test/com/sun/net/httpserver/bugs/B6431193.java should use try-with-resource and try-finally
   
   Reviewed-by: dfuchs, jpai
 - ... and 167 more: https://git.openjdk.org/shenandoah/compare/0578af80...3f6b43c1

-------------

Changes: https://git.openjdk.org/shenandoah/pull/422/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=422&range=01
  Stats: 32169 lines in 779 files changed: 14385 ins; 12567 del; 5217 mod
  Patch: https://git.openjdk.org/shenandoah/pull/422.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/422/head:pull/422

PR: https://git.openjdk.org/shenandoah/pull/422

From kdnilsen at openjdk.org  Tue Apr 16 00:19:21 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 16 Apr 2024 00:19:21 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v41]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <5hQgvnpnla8R3VSK-_PlHydOWOGIOQGjLaEnv_6ibPQ=.a9e70378-4938-4df1-a01f-05455f621398@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Cosmetic reformat after edits that changed source line length

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/b64739ad..0cdd7745

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=40
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=39-40

  Stats: 108 lines in 2 files changed: 50 ins; 0 del; 58 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 16 00:19:22 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 16 Apr 2024 00:19:22 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v39]
In-Reply-To: <99oUiKynBFbBtK69VEkcI1rATPTdtKM8HIZE4WKE6gs=.59ed38a5-4781-41c8-ad36-6d3645ecf892@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <IimvSux5GEemCzBEY0lZHUCxFTFzhTM7mvCm7AD3nV4=.ed14fcb8-d5a3-4439-a2f8-8444cf7df123@github.com>
 <99oUiKynBFbBtK69VEkcI1rATPTdtKM8HIZE4WKE6gs=.59ed38a5-4781-41c8-ad36-6d3645ecf892@github.com>
Message-ID: <XxPudSWp0o5Wv2LaXyLbCP5DxzmAy0XlGYXSxZkzRko=.621f541a-00ce-40ff-9e0e-929e9d0939c5@github.com>

On Sat, 13 Apr 2024 15:17:21 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 165:
>> 
>>> 163: 
>>> 164: // Each ShenandoahHeapRegion is associated with a ShenandoahFreeSetPartitionId.
>>> 165: enum ShenandoahFreeSetPartitionId : uint8_t {
>> 
>> Make this an 'enum class' instead.
>> I see that you are using the enum as index, in this case you can declare the enum class like 'enum class ShenandoahFreeSetPartitionId : uint8_t'
>
> Thanks for this suggestion.  Done.

Thanks.  This is done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1566566426

From kdnilsen at openjdk.org  Tue Apr 16 00:26:52 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 16 Apr 2024 00:26:52 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v39]
In-Reply-To: <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
Message-ID: <2t0urUU2aUpG3wa9oNtxeaz35ijBJ03tcQ31oWNY1Bc=.6fcc2de4-4af2-4b36-91fc-f7cbea0e914d@github.com>

On Wed, 10 Apr 2024 11:16:11 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove debugging instrumentation
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 42:
> 
>> 40:     case Mutator: return "Mutator";
>> 41:     case Collector: return "Collector";
>> 42:     default: return "Unrecognized";
> 
> I believe using an 'enum class' for ShenandoahFreeSetPartitionId means you don't need to have a default.

Compiler still wants the default clause.  Otherwise, "control reaches end of non-void function".  You'd think the compiler could be less strict, but will leave as is, unless we can figure out a way to get around this.

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 49:
> 
>> 47:   assert((start_idx >= 0) && (start_idx < _num_bits), "precondition");
>> 48:   size_t array_idx = start_idx >> LogBitsPerWord;
>> 49:   uintx element_bits = _bitmap[array_idx];
> 
> Same here, try to stick with what is done in bitMap.hpp (typedef uintptr_t bm_word_t, you could just as well typedef uintx bm_word_t, if you prefer).

This is one of the "philosophical" and "tactical" differences between ShenandoahSimpleBitMap and BitMap.  ShenandoahSimpleBitMap allows searches to go in the backward direction, and uses Sentinel value -1 to indicate NotFound, and to represent "NoLimitOnBackwardSearch".  BitMap only searches in the forward direction.  Thus, it doesn't need signed values.

I could introduce a typedef to make future translation simpler if you think that would be useful.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1566568060
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1566570315

From kdnilsen at openjdk.org  Tue Apr 16 00:34:06 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 16 Apr 2024 00:34:06 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v39]
In-Reply-To: <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
Message-ID: <ncq8H2kyodnMfmpC91RojDtenVbG7iA4IHJCJStXIJY=.24deb52a-5b90-4423-94bb-d77a132e9dbd@github.com>

On Wed, 10 Apr 2024 11:26:03 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove debugging instrumentation
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 93:
> 
>> 91:   uintx complement = ~element_bits;
>> 92:   uintx trailing_ones;
>> 93:   if (complement) {
> 
> complement is not a bool type. If you meant to say 'if (complement != 0)' then say so, to make it easier for the reader. Same applies for a few other places.

Thanks.  Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1566573253

From kdnilsen at openjdk.org  Tue Apr 16 00:41:06 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 16 Apr 2024 00:41:06 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v42]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <W8_u7PZSo3-yunuP-Fef05kyWMflxbwsDrY-gq605cU=.f0d6346d-6265-41b4-a70d-4f2fd84bb65f@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Replace tail recursion with iteration
 - Do not treat integer values as booleans
   
   Instead test for equal to zero or not equal to zero.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/0cdd7745..43e851d3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=41
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=40-41

  Stats: 53 lines in 1 file changed: 8 ins; 0 del; 45 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 16 00:41:06 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 16 Apr 2024 00:41:06 GMT
Subject: RFR: 8324649: Shenandoah: refactor implementation of free set
 [v39]
In-Reply-To: <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
Message-ID: <V8CYyYs-jqWP7ZgILsdTWrFmlSvhqE1T9-LJan9absM=.69f79661-32d0-4d99-921a-6c1f4c5a01f0@github.com>

On Wed, 10 Apr 2024 11:31:43 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove debugging instrumentation
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 102:
> 
>> 100:   } else if (trailing_ones == bits_to_examine) {
>> 101:      // Tail recursion
>> 102:     return is_forward_consecutive_ones(start_idx + bits_to_examine, count - bits_to_examine);
> 
> How bad can this get? I haven't analyzed the algorithm, but if this is not bounded (e.g. we know that it can only go 1 level deep), then I'd prefer to have this as loop, to avoid stack overflow. Same applies for a few other places.

I've replace tail recursion with iteration.  Thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1566576781

From kdnilsen at openjdk.org  Tue Apr 16 00:48:20 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 16 Apr 2024 00:48:20 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v43]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <6f4x8SxTaBzvXBV4RPLiaUe17t6Qo81iaW7jTI0tN1o=.5b8b31bb-2e82-4535-918b-35028deb770e@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix indentation of function declaration

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/43e851d3..4b1d6a32

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=42
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=41-42

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 16 00:48:20 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 16 Apr 2024 00:48:20 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v39]
In-Reply-To: <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <pfNxx05qDlbrYP69FrimR1VT5inoqy0_2fJPXOmtItw=.098581b1-7afc-461c-b79f-f8c52cf94ae8@github.com>
Message-ID: <A3p2vKCzQNutz3fJiiPV5KGfeB6dyR6mf7RgExc5AOY=.0d76e7e8-6b3a-478c-abcb-2049f2976fba@github.com>

On Wed, 10 Apr 2024 11:36:45 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove debugging instrumentation
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 186:
> 
>> 184: }
>> 185: 
>> 186: ssize_t ShenandoahSimpleBitMap::find_prev_consecutive_bits(
> 
> It looks weird to me to have an opening ( and then all the args on the next line. I think in HotSpot, it's usually done like:
> 
> ssize_t ShenandoahSimpleBitMap::find_prev_consecutive_bits(const size_t num_bits,
>                                                            ssize_t last_idx,
>                                                            const ssize_t boundary_idx) const {

Fixed

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.inline.hpp line 31:
> 
>> 29: #include "gc/shenandoah/shenandoahFreeSet.hpp"
>> 30: 
>> 31: inline ssize_t ShenandoahSimpleBitMap::find_next_set_bit(ssize_t start_idx, ssize_t boundary_idx) const {
> 
> bitMap.hpp calls it get_next_one_offset(), maybe keep names, args (and impl?) the same as there, to make future merge easier? I guess that is true for all methods that also exist in bitMap.hpp. If you do, then say in comment that the whole block is taken from bitMap.* to make review and later merge easier.

I'll work on this tomorrow.  Thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1566579114
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1566579300

From azafari at openjdk.org  Tue Apr 16 11:32:01 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 16 Apr 2024 11:32:01 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
Message-ID: <EhdLz2OMqwsQkrR0C9BL0Nz3EInXuFHBJWB_1Z5AiV4=.efdbe0bf-e60b-4bbc-90d8-3652f48b6204@github.com>

On Mon, 15 Apr 2024 16:11:13 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   alignment in coding style changed.

Tested `runtime/NMT/VirtualAlloc*.java` tests for master and PR branches. All `mmap` values are the same.
Ready for next round of review.
Ping @tstuefe, @stefank and @jdksjolen.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2058865629

From jvernee at openjdk.org  Tue Apr 16 14:14:03 2024
From: jvernee at openjdk.org (Jorn Vernee)
Date: Tue, 16 Apr 2024 14:14:03 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v4]
In-Reply-To: <5xcpEj85UX0BJUK8vYDDXymSvB9MrVjXOCCmUlyVE9k=.86e0b7c5-edf9-47a9-83eb-9dac4570935c@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <5xcpEj85UX0BJUK8vYDDXymSvB9MrVjXOCCmUlyVE9k=.86e0b7c5-edf9-47a9-83eb-9dac4570935c@github.com>
Message-ID: <OvTATV03nGgwnaa2FAkfK1fbWIbbq-XvpqvzDsgH7rU=.477dd0bd-ab01-41d3-9b0c-57cd82e9ab1c@github.com>

On Mon, 15 Apr 2024 14:02:56 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
>   
>   Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>

src/java.base/share/classes/java/lang/foreign/package-info.java line 114:

> 112:  *
> 113:  * Here, we obtain a {@linkplain java.lang.foreign.Linker#nativeLinker() native linker}
> 114:  * and we use it to {@linkplain java.lang.foreign.SymbolLookup#findOrThrow(java.lang.String)}

Looks like the plain text was dropped here:
Suggestion:

 * and we use it to {@linkplain java.lang.foreign.SymbolLookup#findOrThrow(java.lang.String) look up}

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1567435862

From wkemper at openjdk.org  Tue Apr 16 20:41:24 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 16 Apr 2024 20:41:24 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <GtaI6c6G2DzJRpcqA8gmYG0bYFdGN_nzvzkExOOt8RM=.600bda9a-bc33-4c30-9d0b-de4179aee2dc@github.com>
References: <SVEFHy342xY7V0EteTxnF5m-D_tOu83VpQnG9FgPW_4=.7e8c45ed-bf24-4e17-ab06-291385ac301a@github.com>
 <GtaI6c6G2DzJRpcqA8gmYG0bYFdGN_nzvzkExOOt8RM=.600bda9a-bc33-4c30-9d0b-de4179aee2dc@github.com>
Message-ID: <lwip_FcRT20wGeXzzCr9eWU1zShrPBUvaj5f1XdyvEk=.fad158ff-6789-4646-9103-fc4ecb737ac0@github.com>

On Fri, 5 Apr 2024 21:47:08 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Resolved build issue with changes to `ageTable` API: https://github.com/openjdk/shenandoah/pull/416/files#diff-bccc00e8e28e415d1eaeac169ebb325ff962d7adc171e821e537c2c7f88f848d
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 77 commits:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into merge-jdk-23-17
>  - Merge tag 'jdk-23+17' into merge-jdk-23-17
>    
>    Added tag jdk-23+17 for changeset 8efd7aa6
>  - 8328786: [AIX] move some important warnings/errors from trcVerbose to UL
>    
>    Reviewed-by: lucy, stuefe
>  - 8327110: Refactor create_bool_from_template_assertion_predicate() to separate class and fix identical cloning cases used for Loop Unswitching and Split If
>    
>    Reviewed-by: epeter, kvn
>  - 8328702: C2: Crash during parsing because sub type check is not folded
>    
>    Reviewed-by: roland, kvn
>  - 8326962: C2 SuperWord: cache VPointer
>    
>    Reviewed-by: chagedorn, kvn
>  - 8328938: C2 SuperWord: disable vectorization for large stride and scale
>    
>    Reviewed-by: chagedorn, kvn
>  - 8329494: Serial: Merge GenMarkSweep into MarkSweep
>    
>    Reviewed-by: ihse, ayang, tschatzl
>  - 8329470: Remove obsolete CDS SharedStrings tests
>    
>    Reviewed-by: ccheung
>  - 8329564: [JVMCI] TranslatedException::debugPrintStackTrace does not work in the libjvmci compiler.
>    
>    Reviewed-by: dnsimon
>  - ... and 67 more: https://git.openjdk.org/shenandoah/compare/11295a6a...254699d2

Superseded by https://github.com/openjdk/shenandoah/pull/422

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/416#issuecomment-2059885222

From wkemper at openjdk.org  Tue Apr 16 20:41:24 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 16 Apr 2024 20:41:24 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <SVEFHy342xY7V0EteTxnF5m-D_tOu83VpQnG9FgPW_4=.7e8c45ed-bf24-4e17-ab06-291385ac301a@github.com>
References: <SVEFHy342xY7V0EteTxnF5m-D_tOu83VpQnG9FgPW_4=.7e8c45ed-bf24-4e17-ab06-291385ac301a@github.com>
Message-ID: <wllMVYCmT850zS6DetSCCm33xbaxHSmeX886G4MZsQQ=.f45d11ee-95ca-4c50-b038-5d89b9e441fc@github.com>

On Thu, 4 Apr 2024 19:22:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Resolved build issue with changes to `ageTable` API: https://github.com/openjdk/shenandoah/pull/416/files#diff-bccc00e8e28e415d1eaeac169ebb325ff962d7adc171e821e537c2c7f88f848d

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/416

From wkemper at openjdk.org  Tue Apr 16 20:43:04 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 16 Apr 2024 20:43:04 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <JN__f1fGhwA-dVcQn1-EqJaV_ZdfSVgHr9fs8Huj1B0=.11034e3f-c0df-443f-8dc7-990d822b231b@github.com>
References: <JN__f1fGhwA-dVcQn1-EqJaV_ZdfSVgHr9fs8Huj1B0=.11034e3f-c0df-443f-8dc7-990d822b231b@github.com>
Message-ID: <C7TsET3Qppz3_28UL3FXkDZ1JbibgIIzJZBkDNYECBw=.b620dad2-6b92-4da4-882d-8f5afe779206@github.com>

On Fri, 12 Apr 2024 14:10:04 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+18

This pull request has now been integrated.

Changeset: ae15774c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/ae15774cb6f4d112a3924a473bf47fa7a5c4cc0d
Stats:     32169 lines in 779 files changed: 14385 ins; 12567 del; 5217 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/422

From pminborg at openjdk.org  Wed Apr 17 11:24:28 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Wed, 17 Apr 2024 11:24:28 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v5]
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <nG3gW3IfgyIcZuOq6igDvfWcW2T3D4QCl3EcCYnDMzs=.dccd37db-c186-49e3-9245-50a90c17db64@github.com>

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

Per Minborg has updated the pull request incrementally with one additional commit since the last revision:

  Update src/java.base/share/classes/java/lang/foreign/package-info.java
  
  Co-authored-by: Jorn Vernee <JornVernee at users.noreply.github.com>

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18474/files
  - new: https://git.openjdk.org/jdk/pull/18474/files/fa86d837..2ebac9fc

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=03-04

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/18474.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18474/head:pull/18474

PR: https://git.openjdk.org/jdk/pull/18474

From stefank at openjdk.org  Wed Apr 17 13:08:06 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 17 Apr 2024 13:08:06 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
Message-ID: <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>

On Mon, 15 Apr 2024 16:11:13 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   alignment in coding style changed.

Here's a new set of comments.

src/hotspot/os/windows/os_windows.cpp line 5110:

> 5108: 
> 5109:     // Record virtual memory allocation
> 5110:     MemTracker::record_virtual_memory_reserve_and_commit((address)addr, bytes, CALLER_PC, flag);

Should this really be called here? The posix version don't call this, so I don't understand why it is called here in the Windows code.

src/hotspot/share/cds/filemap.cpp line 1697:

> 1695: static char* map_memory(int fd, const char* file_name, size_t file_offset,
> 1696:                         char *addr, size_t bytes, bool read_only,
> 1697:                         bool allow_exec, MEMFLAGS flags) {

It is odd that `map_memory` and `os::map_memory` has different parameter order. I understand that this is done because of default values, but I'd like to suggest that you get rid of these default values and fix the order.

(Side-note: Wouldn't it be better to rename this `map_memory` to something that clearly shows the difference between this function and `os::map_memory`)

src/hotspot/share/classfile/compactHashtable.cpp line 243:

> 241:     quit("Unable to open hashtable dump file", filename);
> 242:   }
> 243:   _base = os::map_memory(_fd, filename, 0, nullptr, _size, mtInternal, true, false);

Isn't this CDS code. Should ths be mtClassShared or something else that indicates that this is CDS code?

src/hotspot/share/nmt/virtualMemoryTracker.cpp line 460:

> 458:   assert(_reserved_regions != nullptr, "Sanity check");
> 459: 
> 460:   ReservedMemoryRegion  rgn(addr, size, flag);

I'm not sure about this. `rgn` is just used to find the memory region we want to uncommit. The flag isn't used in the search, and passing it forces the callers to also pass in the flag.

I understand that this happens after the request to remove the mtNone default value. Is there a way that allows us to skip using mtNone, but still don't have to unnecessarily provide a flag?

Maybe we could create a helper function `ReservedMemoryRegion rgn = ReservedMemoryRegion::create_find_key(addr, size)`, which sets up a ReserveMemoryRegion with mtNone?

src/hotspot/share/runtime/os.cpp line 1817:

> 1815: 
> 1816: char* os::reserve_memory(size_t bytes, bool executable, MEMFLAGS flags) {
> 1817:   char* result = pd_reserve_memory(bytes, executable, flags);

Doesn't it look weird that we pass in flags here and then still call MemTracker::record_ below? I think this is an artifact from mixing if we put the NMT calls in shared or in platform dependent code. I understand that you need this for this patch, but I also think there needs to be some RFE to figure out if this can be reworked.

src/hotspot/share/runtime/os.cpp line 2187:

> 2185:                      MEMFLAGS flags,
> 2186:                      bool read_only,
> 2187:                      bool allow_exec) {

The function was written with multiple parameters per line here, and then you changed it so that some of the params where placed on individual lines. This should likely be reverted.

src/hotspot/share/runtime/os.hpp line 233:

> 231:                            char *addr, size_t bytes,
> 232:                            MEMFLAGS flag,
> 233:                            bool read_only = false,

Mixes param layout style. (Plus earlier comment that the default values should probably be removed so that MEMFLAGS can be put last).

src/hotspot/share/runtime/os.hpp line 471:

> 469:   // vm_exit_out_of_memory() with the specified mesg.
> 470:   static void   commit_memory_or_exit(char* addr, size_t bytes,
> 471:                                       bool executable, const char* mesg, MEMFLAGS flag);

I think that we should change the parameter order here, so that it is like `commit_memory` and then the extra mesg param goes with the `_or_exit` part (if that makes sense).
Suggestion:

                                      bool executable, MEMFLAGS flag, const char* mesg);

src/hotspot/share/runtime/os.hpp line 522:

> 520:                            MEMFLAGS flag,
> 521:                            bool read_only = false,
> 522:                            bool allow_exec = false);

params layout style.

-------------

Changes requested by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2005841897
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568730671
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568735525
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568722870
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568775270
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568802391
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568808811
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568810492
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568726202
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1568812091

From kdnilsen at openjdk.org  Wed Apr 17 16:13:06 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 17 Apr 2024 16:13:06 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v39]
In-Reply-To: <99oUiKynBFbBtK69VEkcI1rATPTdtKM8HIZE4WKE6gs=.59ed38a5-4781-41c8-ad36-6d3645ecf892@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <IimvSux5GEemCzBEY0lZHUCxFTFzhTM7mvCm7AD3nV4=.ed14fcb8-d5a3-4439-a2f8-8444cf7df123@github.com>
 <99oUiKynBFbBtK69VEkcI1rATPTdtKM8HIZE4WKE6gs=.59ed38a5-4781-41c8-ad36-6d3645ecf892@github.com>
Message-ID: <op4VU5Iy-iTUPgwt-ycDUDieZ3rjozVXPB_TabMLv9A=.0baed719-7161-41c9-b354-188922936abc@github.com>

On Sat, 13 Apr 2024 15:17:07 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 45:
>> 
>>> 43: // break abstraction rules, because efficient implementation requires assumptions about superclass internals that
>>> 44: // might be violatee through future software maintenance.
>>> 45: class ShenandoahSimpleBitMap {
>> 
>> I think this class should go into its own set of files. It would certainly help readability of ShFreeSet which is now somewhat dominated by ShSimpleBitMap.
>
> Thanks for suggestion.  Done.

Thanks.  I've separated this code into distinct source files.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1569105090

From kdnilsen at openjdk.org  Wed Apr 17 16:13:07 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 17 Apr 2024 16:13:07 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v43]
In-Reply-To: <IimvSux5GEemCzBEY0lZHUCxFTFzhTM7mvCm7AD3nV4=.ed14fcb8-d5a3-4439-a2f8-8444cf7df123@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <IimvSux5GEemCzBEY0lZHUCxFTFzhTM7mvCm7AD3nV4=.ed14fcb8-d5a3-4439-a2f8-8444cf7df123@github.com>
Message-ID: <yyffC-N_oOimWAEYVnCi-HMaFq9bDW5TG9WHfAIVODY=.d49d00d7-39e2-48ab-a5e8-5361ea15e598@github.com>

On Tue, 9 Apr 2024 19:32:08 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix indentation of function declaration
>
> src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 915:
> 
>> 913: public:
>> 914:   ShenandoahPostCompactClosure() : _heap(ShenandoahHeap::heap()), _live(0) {
>> 915:     _heap->free_set()->clear();
> 
> Why is that ok?

There is no need to clear() because prepare_to_rebuild() invokes clear_internal() before it establishes new partition assignments.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1569103078

From kdnilsen at openjdk.org  Wed Apr 17 16:35:36 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 17 Apr 2024 16:35:36 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v11]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <NYkfEAg9c4wNd6GFeRYTA2Ux68BtW8Kjos3R3fSJVXQ=.6113a611-4bc9-4ec4-a2cd-51bcb9176d2c@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Correct typo in symbolic constant
  
  FRACTIONAL_DENOMINATOR should be a power of 2.  Small round-off errors
  were observed because there was an error in one digit of this contant.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/409/files
  - new: https://git.openjdk.org/shenandoah/pull/409/files/b767ab9f..8e5164c6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=10
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=09-10

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From wkemper at openjdk.org  Wed Apr 17 17:54:07 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 17 Apr 2024 17:54:07 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v43]
In-Reply-To: <6f4x8SxTaBzvXBV4RPLiaUe17t6Qo81iaW7jTI0tN1o=.5b8b31bb-2e82-4535-918b-35028deb770e@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <6f4x8SxTaBzvXBV4RPLiaUe17t6Qo81iaW7jTI0tN1o=.5b8b31bb-2e82-4535-918b-35028deb770e@github.com>
Message-ID: <CUIlLHnq7wScnm4qvo8CcxOqWYO_5HB4RGlDdMm_LBs=.77541b1a-80dd-42b2-8f72-fb8cdb6429ae@github.com>

On Tue, 16 Apr 2024 00:48:20 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix indentation of function declaration

Marked as reviewed by wkemper (Committer).

Zero build and others look a bit broken.

-------------

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-2006729729
PR Comment: https://git.openjdk.org/jdk/pull/17561#issuecomment-2061884429

From kdnilsen at openjdk.org  Wed Apr 17 18:22:31 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 17 Apr 2024 18:22:31 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v12]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <IBuERxV2R1GkpdPp54ueySDHqs6bB05C9DYpbX5NL10=.73a9d985-4d3a-4f3b-afab-cb19ed4be421@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Minor refinements to old-growth heuristics
  
  1. If old-marking finds old-gen is entirely empty, reset the heuristics
     to initial state, so we can restart the search for appropriate
     old-gen size
  2. When testing whether old-gen growth "still" exceeds the trigger
     threshold following mixed evacuations, include both old usage and
     old humongous waste, as that is the quantity that was used for
     initial trigger.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/409/files
  - new: https://git.openjdk.org/shenandoah/pull/409/files/8e5164c6..0dc8abaa

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=11
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=10-11

  Stats: 12 lines in 2 files changed: 7 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From kdnilsen at openjdk.org  Wed Apr 17 18:31:35 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 17 Apr 2024 18:31:35 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v44]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <Vnj2TKN2Dz_W586OQlULBUd47074Koo2CPe12fbE5zk=.1ac26339-a3eb-41ad-8e43-54ab5e1dc041@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Respond to reviewer suggestions

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/4b1d6a32..8c2d8015

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=43
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=42-43

  Stats: 276 lines in 5 files changed: 17 ins; 1 del; 258 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Wed Apr 17 23:57:44 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 17 Apr 2024 23:57:44 GMT
Subject: RFR: 8330071: GenShen: Allow old to expand again at end of each GC
Message-ID: <4JOkyMaUzYnaXHdYzbrvb7chUbK1LIYSErXNBhGe8fk=.3641aaed-a040-4fdd-bddb-db275fb124e8@github.com>

This pull request contains a backport of commit 0578af80 from the openjdk/shenandoah repository.

The commit being backported was authored by Kelvin Nilsen on 15 Apr 2024 and was reviewed by William Kemper and Y. Srinivas Ramakrishna.

-------------

Commit messages:
 - Backport 0578af8015bc0dfcd4a2b726b1a8b52f68712cf2

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/35/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=35&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8330071
  Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/35.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/35/head:pull/35

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/35

From azafari at openjdk.org  Thu Apr 18 08:42:10 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 08:42:10 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
Message-ID: <CIhg1QFIpSZudH254R9aMQJhSf1we6xmqkR4uEFJftc=.079e39bd-aef4-42ed-8f66-d686c741beef@github.com>

On Wed, 17 Apr 2024 12:02:50 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   alignment in coding style changed.
>
> src/hotspot/os/windows/os_windows.cpp line 5110:
> 
>> 5108: 
>> 5109:     // Record virtual memory allocation
>> 5110:     MemTracker::record_virtual_memory_reserve_and_commit((address)addr, bytes, CALLER_PC, flag);
> 
> Should this really be called here? The posix version don't call this, so I don't understand why it is called here in the Windows code.

It uses the requested address rather than the result of allocation that seems a bug.
Removed.

> src/hotspot/share/cds/filemap.cpp line 1697:
> 
>> 1695: static char* map_memory(int fd, const char* file_name, size_t file_offset,
>> 1696:                         char *addr, size_t bytes, bool read_only,
>> 1697:                         bool allow_exec, MEMFLAGS flags) {
> 
> It is odd that `map_memory` and `os::map_memory` has different parameter order. I understand that this is done because of default values, but I'd like to suggest that you get rid of these default values and fix the order.
> 
> (Side-note: Wouldn't it be better to rename this `map_memory` to something that clearly shows the difference between this function and `os::map_memory`)

`map_memory` is renamed to `map_and_pretouch_memory`.
argument orders match with `os::map_memory`.

> src/hotspot/share/classfile/compactHashtable.cpp line 243:
> 
>> 241:     quit("Unable to open hashtable dump file", filename);
>> 242:   }
>> 243:   _base = os::map_memory(_fd, filename, 0, nullptr, _size, mtInternal, true, false);
> 
> Isn't this CDS code. Should ths be mtClassShared or something else that indicates that this is CDS code?

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570270206
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570272820
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570273489

From azafari at openjdk.org  Thu Apr 18 08:59:59 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 08:59:59 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
Message-ID: <mA7NxX2MDxeixrhXBJhL3A7WObyGWLSvEtQ6tXSY4b8=.10fa6929-1e50-489c-90a3-d4a3674efd6c@github.com>

On Wed, 17 Apr 2024 12:57:43 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   alignment in coding style changed.
>
> src/hotspot/share/runtime/os.cpp line 1817:
> 
>> 1815: 
>> 1816: char* os::reserve_memory(size_t bytes, bool executable, MEMFLAGS flags) {
>> 1817:   char* result = pd_reserve_memory(bytes, executable, flags);
> 
> Doesn't it look weird that we pass in flags here and then still call MemTracker::record_ below? I think this is an artifact from mixing if we put the NMT calls in shared or in platform dependent code. I understand that you need this for this patch, but I also think there needs to be some RFE to figure out if this can be reworked.

The flag is needed on Windows for this call hierarchy:

reserve_memory
  pd_reserve_memory
    pd_attempt_reserve_memory_at
      allocate_pages_individually(..., MEMFLAGS flag)

Other platforms ignore the flag.
Agreed on a new RFE for handling this.

> src/hotspot/share/runtime/os.cpp line 2187:
> 
>> 2185:                      MEMFLAGS flags,
>> 2186:                      bool read_only,
>> 2187:                      bool allow_exec) {
> 
> The function was written with multiple parameters per line here, and then you changed it so that some of the params where placed on individual lines. This should likely be reverted.

Fixed.

> src/hotspot/share/runtime/os.hpp line 233:
> 
>> 231:                            char *addr, size_t bytes,
>> 232:                            MEMFLAGS flag,
>> 233:                            bool read_only = false,
> 
> Mixes param layout style. (Plus earlier comment that the default values should probably be removed so that MEMFLAGS can be put last).

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570302144
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570305926
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570306322

From azafari at openjdk.org  Thu Apr 18 09:03:00 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 09:03:00 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
Message-ID: <sla9cBFGB_qAwtZqks0nXL4vVdD5dPQElJc4ZO1LBhI=.8707dff3-07bc-422a-bb0e-2c9ccfb9c9dd@github.com>

On Wed, 17 Apr 2024 11:59:01 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   alignment in coding style changed.
>
> src/hotspot/share/runtime/os.hpp line 471:
> 
>> 469:   // vm_exit_out_of_memory() with the specified mesg.
>> 470:   static void   commit_memory_or_exit(char* addr, size_t bytes,
>> 471:                                       bool executable, const char* mesg, MEMFLAGS flag);
> 
> I think that we should change the parameter order here, so that it is like `commit_memory` and then the extra mesg param goes with the `_or_exit` part (if that makes sense).
> Suggestion:
> 
>                                       bool executable, MEMFLAGS flag, const char* mesg);

Changed the order.
Note that, now it is only the `commit_memory_or_exit` in `os.hpp` that the MEMFLAGS is not the last param.

> src/hotspot/share/runtime/os.hpp line 522:
> 
>> 520:                            MEMFLAGS flag,
>> 521:                            bool read_only = false,
>> 522:                            bool allow_exec = false);
> 
> params layout style.

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570311465
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570311935

From azafari at openjdk.org  Thu Apr 18 09:22:34 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 09:22:34 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v8]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <u_KDlu1560TjnYhOHrHBezQD1VrqUAfvMgkoBJqdoT0=.04b9164c-6157-4838-898d-c09f6e8e1190@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  order of params are consistent now. style is corrected.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/abcfcccd..229bc890

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=06-07

  Stats: 49 lines in 14 files changed: 2 ins; 10 del; 37 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Thu Apr 18 09:22:34 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 09:22:34 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v8]
In-Reply-To: <u_KDlu1560TjnYhOHrHBezQD1VrqUAfvMgkoBJqdoT0=.04b9164c-6157-4838-898d-c09f6e8e1190@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <u_KDlu1560TjnYhOHrHBezQD1VrqUAfvMgkoBJqdoT0=.04b9164c-6157-4838-898d-c09f6e8e1190@github.com>
Message-ID: <mFpPf2gt0jnsUVxBWLW8Mp_Lik2Iq6IMTfPByqtIhBo=.bc64d534-b695-4c34-af71-ae408aac616a@github.com>

On Thu, 18 Apr 2024 09:19:08 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   order of params are consistent now. style is corrected.

Comments applied.

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2008324970

From azafari at openjdk.org  Thu Apr 18 09:34:40 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 09:34:40 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v9]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <2q_nxOZzJ8y0s4otI6yWw90csbVGgSNaWyswAjlxly0=.644c8479-0061-4e0d-982b-e97fe7b802a2@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  fixed a missed file from param reordering.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/229bc890..769166f8

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=08
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=07-08

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From stefank at openjdk.org  Thu Apr 18 09:41:03 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 18 Apr 2024 09:41:03 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v8]
In-Reply-To: <u_KDlu1560TjnYhOHrHBezQD1VrqUAfvMgkoBJqdoT0=.04b9164c-6157-4838-898d-c09f6e8e1190@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <u_KDlu1560TjnYhOHrHBezQD1VrqUAfvMgkoBJqdoT0=.04b9164c-6157-4838-898d-c09f6e8e1190@github.com>
Message-ID: <ymOEd0w-x-QmKvzn5GBjo-TDQ6J9HVEvWXzioaa8LHE=.5046d4f6-886f-412d-bb0c-2e0f48f0b528@github.com>

On Thu, 18 Apr 2024 09:22:34 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   order of params are consistent now. style is corrected.

More comments.

src/hotspot/os/windows/os_windows.cpp line 5068:

> 5066: char* os::pd_map_memory(int fd, const char* file_name, size_t file_offset,
> 5067:                         char *addr, size_t bytes,
> 5068:                         bool read_only,

This should be reverted.

src/hotspot/share/cds/filemap.cpp line 1701:

> 1699:                              AlwaysPreTouch ? false : read_only,
> 1700:                              allow_exec,
> 1701:                              flags);

Revert this change

src/hotspot/share/cds/filemap.cpp line 1729:

> 1727:                               false /* !read_only */,
> 1728:                               r->allow_exec(),
> 1729:                               mtClassShared);

This mixes styles between multiple arguments per line vs one argument per line.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 217:

> 215:   if (!_heap_region_special) {
> 216:     os::commit_memory_or_exit(sh_rs.base(), _initial_size, heap_alignment, !ExecMem,
> 217:                               "Cannot commit heap memory", mtGC);

The argument order needs to be changed here and below.

src/hotspot/share/runtime/os.cpp line 2185:

> 2183: char* os::map_memory(int fd, const char* file_name, size_t file_offset,
> 2184:                      char *addr, size_t bytes,
> 2185:                      bool read_only, bool allow_exec, MEMFLAGS flags) {

You should probably move back read_only to the line below.

src/hotspot/share/runtime/os.hpp line 520:

> 518:                            bool read_only,
> 519:                            bool allow_exec,
> 520:                            MEMFLAGS flag);

Style inconsistency

-------------

Changes requested by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2008344764
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570357660
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570358795
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570360021
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570361901
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570371965
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570373261

From stefank at openjdk.org  Thu Apr 18 09:41:04 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 18 Apr 2024 09:41:04 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <CIhg1QFIpSZudH254R9aMQJhSf1we6xmqkR4uEFJftc=.079e39bd-aef4-42ed-8f66-d686c741beef@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
 <CIhg1QFIpSZudH254R9aMQJhSf1we6xmqkR4uEFJftc=.079e39bd-aef4-42ed-8f66-d686c741beef@github.com>
Message-ID: <NgUvH7BTWcBKpOLtPL0OGh-CjPOVv1esFEDNDuG5MD0=.5b28fc9e-c547-4def-81b7-c046fe9189c8@github.com>

On Thu, 18 Apr 2024 08:39:25 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/cds/filemap.cpp line 1697:
>> 
>>> 1695: static char* map_memory(int fd, const char* file_name, size_t file_offset,
>>> 1696:                         char *addr, size_t bytes, bool read_only,
>>> 1697:                         bool allow_exec, MEMFLAGS flags) {
>> 
>> It is odd that `map_memory` and `os::map_memory` has different parameter order. I understand that this is done because of default values, but I'd like to suggest that you get rid of these default values and fix the order.
>> 
>> (Side-note: Wouldn't it be better to rename this `map_memory` to something that clearly shows the difference between this function and `os::map_memory`)
>
> `map_memory` is renamed to `map_and_pretouch_memory`.
> argument orders match with `os::map_memory`.

Thanks. The parames need to be adjusted after the rename.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570358350

From stefank at openjdk.org  Thu Apr 18 09:41:05 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 18 Apr 2024 09:41:05 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v4]
In-Reply-To: <B5FD1uvKRJ-OYZSmxyXUBmvp1XF4UyzFKvyFpAO-j0w=.71e06611-77d1-46fb-be4b-5f6a9e5bd5a1@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <QBIq1__mpiCibBldTDqBJXRSrpb-Vk_fLRwQZ88JdvE=.1fbe51ac-352e-4a05-a336-03a77ccb227b@github.com>
 <lhFIh2qUbeY3kzGklUGuLCMZCC9lMzqMevukRHLzdik=.8c3147d8-62bd-4e7e-9630-d5c7103d0418@github.com>
 <B5FD1uvKRJ-OYZSmxyXUBmvp1XF4UyzFKvyFpAO-j0w=.71e06611-77d1-46fb-be4b-5f6a9e5bd5a1@github.com>
Message-ID: <TgR4c8YdmKEVPrubTNsEYEvBhFb12wfqNmrUAFZtuDo=.d7f9071f-5155-40f4-871d-143ff287d968@github.com>

On Mon, 15 Apr 2024 16:02:01 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/virtualspace.hpp line 199:
>> 
>>> 197:   size_t _upper_alignment;
>>> 198: 
>>> 199:   MEMFLAGS _nmt_flag;
>> 
>> The VirtualSpace::initialize functions used to initialize these members in the order that they are specified here. That is now messed up by adding the _nmt_flag at the end here, but in the beginning in the initialize function. I would propose that you move it to after _executable, both here and in the initialize function.
>
> Fixed.

I'd like to see some consistency between ReservedHeap and VritualSpace. Could you change the variables layout and intialization to be:

_special
_executable
_nmt_flags

in both classes?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570369399

From pminborg at openjdk.org  Thu Apr 18 11:32:13 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Thu, 18 Apr 2024 11:32:13 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v6]
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <2FFaFRgutA2t8ufmdNJUHF1N02EEQeIyt_4wj8qCYn4=.079b00e1-e4c5-4f78-9b91-0633426191a7@github.com>

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

Per Minborg has updated the pull request incrementally with one additional commit since the last revision:

  Change exception type

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18474/files
  - new: https://git.openjdk.org/jdk/pull/18474/files/2ebac9fc..e2f6c4c9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=04-05

  Stats: 4 lines in 1 file changed: 1 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/18474.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18474/head:pull/18474

PR: https://git.openjdk.org/jdk/pull/18474

From jvernee at openjdk.org  Thu Apr 18 12:21:58 2024
From: jvernee at openjdk.org (Jorn Vernee)
Date: Thu, 18 Apr 2024 12:21:58 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v6]
In-Reply-To: <2FFaFRgutA2t8ufmdNJUHF1N02EEQeIyt_4wj8qCYn4=.079b00e1-e4c5-4f78-9b91-0633426191a7@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <2FFaFRgutA2t8ufmdNJUHF1N02EEQeIyt_4wj8qCYn4=.079b00e1-e4c5-4f78-9b91-0633426191a7@github.com>
Message-ID: <ftS-TAS4lTtUvayLC83NpNpioHhv2W3ohZMkPKyAydY=.8d2decb3-19ed-4795-89b5-c1a44183a947@github.com>

On Thu, 18 Apr 2024 11:32:13 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Change exception type

Marked as reviewed by jvernee (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18474#pullrequestreview-2008729407

From azafari at openjdk.org  Thu Apr 18 13:55:06 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 13:55:06 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v8]
In-Reply-To: <ymOEd0w-x-QmKvzn5GBjo-TDQ6J9HVEvWXzioaa8LHE=.5046d4f6-886f-412d-bb0c-2e0f48f0b528@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <u_KDlu1560TjnYhOHrHBezQD1VrqUAfvMgkoBJqdoT0=.04b9164c-6157-4838-898d-c09f6e8e1190@github.com>
 <ymOEd0w-x-QmKvzn5GBjo-TDQ6J9HVEvWXzioaa8LHE=.5046d4f6-886f-412d-bb0c-2e0f48f0b528@github.com>
Message-ID: <WyQhcNW6WVt0hLjI7pQTHJqxbGNwgH421Py3-Kge5Gs=.667a92a1-a167-4e84-b1be-2a03c3bc9b9f@github.com>

On Thu, 18 Apr 2024 09:36:59 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   order of params are consistent now. style is corrected.
>
> src/hotspot/share/runtime/os.cpp line 2185:
> 
>> 2183: char* os::map_memory(int fd, const char* file_name, size_t file_offset,
>> 2184:                      char *addr, size_t bytes,
>> 2185:                      bool read_only, bool allow_exec, MEMFLAGS flags) {
> 
> You should probably move back read_only to the line below.

move back means line above?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570806464

From wkemper at openjdk.org  Thu Apr 18 14:21:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 18 Apr 2024 14:21:40 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <gGX8XGhIrkE1lWhag_jqV3Fl7hF6upA9EFuZ4IOA7bk=.35c9b576-cb7a-409b-9bcf-dbda25c22259@github.com>

Merges tag jdk-21.0.3-ga

-------------

Commit messages:
 - 8329838: [21u] Remove designator DEFAULT_PROMOTED_VERSION_PRE=ea for release 21.0.3
 - 8319851: Improve exception logging
 - 8322122: Enhance generation of addresses
 - 8318340: Improve RSA key implementations
 - 8315708: Enhance HTTP/2 client usage

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/36/files
  Stats: 173 lines in 19 files changed: 48 ins; 51 del; 74 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/36.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/36/head:pull/36

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/36

From azafari at openjdk.org  Thu Apr 18 14:27:37 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 14:27:37 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v10]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <Vdgb1WZjBNWUZHb-79AYDS86k4NJJyMNH7jOoatg7qY=.8feb0bd1-bf2c-48f1-98c9-9042cc19ba94@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  more improvements in style/alignments/adjustments.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/769166f8..897b4b30

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=09
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=08-09

  Stats: 26 lines in 6 files changed: 3 ins; 9 del; 14 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Thu Apr 18 14:27:38 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Thu, 18 Apr 2024 14:27:38 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v8]
In-Reply-To: <ymOEd0w-x-QmKvzn5GBjo-TDQ6J9HVEvWXzioaa8LHE=.5046d4f6-886f-412d-bb0c-2e0f48f0b528@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <u_KDlu1560TjnYhOHrHBezQD1VrqUAfvMgkoBJqdoT0=.04b9164c-6157-4838-898d-c09f6e8e1190@github.com>
 <ymOEd0w-x-QmKvzn5GBjo-TDQ6J9HVEvWXzioaa8LHE=.5046d4f6-886f-412d-bb0c-2e0f48f0b528@github.com>
Message-ID: <mLB7ma5-KM_1OoUozVzlAE-UPV7Kq5d1b_z2KLo7Aag=.bb3f43fe-9423-484e-bafa-e754881db3f1@github.com>

On Thu, 18 Apr 2024 09:28:05 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   order of params are consistent now. style is corrected.
>
> src/hotspot/os/windows/os_windows.cpp line 5068:
> 
>> 5066: char* os::pd_map_memory(int fd, const char* file_name, size_t file_offset,
>> 5067:                         char *addr, size_t bytes,
>> 5068:                         bool read_only,
> 
> This should be reverted.

Done.

> src/hotspot/share/cds/filemap.cpp line 1701:
> 
>> 1699:                              AlwaysPreTouch ? false : read_only,
>> 1700:                              allow_exec,
>> 1701:                              flags);
> 
> Revert this change

Done.

> src/hotspot/share/cds/filemap.cpp line 1729:
> 
>> 1727:                               false /* !read_only */,
>> 1728:                               r->allow_exec(),
>> 1729:                               mtClassShared);
> 
> This mixes styles between multiple arguments per line vs one argument per line.

related params (file, memory, info) are in the same line.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 217:
> 
>> 215:   if (!_heap_region_special) {
>> 216:     os::commit_memory_or_exit(sh_rs.base(), _initial_size, heap_alignment, !ExecMem,
>> 217:                               "Cannot commit heap memory", mtGC);
> 
> The argument order needs to be changed here and below.

Done.

> src/hotspot/share/runtime/os.hpp line 520:
> 
>> 518:                            bool read_only,
>> 519:                            bool allow_exec,
>> 520:                            MEMFLAGS flag);
> 
> Style inconsistency

related params (file, memory, flags) are in the same line.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570872004
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570871096
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570870825
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570870421
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1570868039

From mcimadamore at openjdk.org  Thu Apr 18 17:35:04 2024
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 18 Apr 2024 17:35:04 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v6]
In-Reply-To: <2FFaFRgutA2t8ufmdNJUHF1N02EEQeIyt_4wj8qCYn4=.079b00e1-e4c5-4f78-9b91-0633426191a7@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <2FFaFRgutA2t8ufmdNJUHF1N02EEQeIyt_4wj8qCYn4=.079b00e1-e4c5-4f78-9b91-0633426191a7@github.com>
Message-ID: <UDD_yQgZ0kLqCklBpSu6Q-uIXxfH7Du9VrFCOWtsbVU=.76d48499-8973-433e-8684-1dcec989d154@github.com>

On Thu, 18 Apr 2024 11:32:13 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Change exception type

We need a test for the new method, e.g. to check that the right exception is thrown, and the message is right. The fact that no test needed to be updated when you changed the exception type is a smell.

-------------

Changes requested by mcimadamore (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18474#pullrequestreview-2009500390

From prr at openjdk.org  Thu Apr 18 18:02:57 2024
From: prr at openjdk.org (Phil Race)
Date: Thu, 18 Apr 2024 18:02:57 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v6]
In-Reply-To: <2FFaFRgutA2t8ufmdNJUHF1N02EEQeIyt_4wj8qCYn4=.079b00e1-e4c5-4f78-9b91-0633426191a7@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <2FFaFRgutA2t8ufmdNJUHF1N02EEQeIyt_4wj8qCYn4=.079b00e1-e4c5-4f78-9b91-0633426191a7@github.com>
Message-ID: <TwcBMY5juNZRirnFrvCpsIhPdKl28QKyc_RV2eHuZZo=.25b492f9-ee3d-49ca-bbfe-d85d299c106e@github.com>

On Thu, 18 Apr 2024 11:32:13 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Change exception type

I'm OK with the minimal changes in the desktop code.

-------------

Marked as reviewed by prr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18474#pullrequestreview-2009552015

From wkemper at openjdk.org  Fri Apr 19 00:04:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 19 Apr 2024 00:04:34 GMT
Subject: RFR: 8330414: GenShen: Class unloading requires old regions be made
 parseable
Message-ID: <ezemrXOHMIFYee3Ex7in1IBnjxQINhWAg3eeFq8OQPs=.08f3516e-f057-4ec7-a8a6-cebe0e6b847a@github.com>

Even when the generational mode is using the mark bitmap to scan the remembered set's card tables, the card table offsets may refer to unmarked objects. If the classes referred to by these unmarked objects have been reloaded in metaspace, the class pointer may no longer be valid or may refer to a class with a different size than expected by the parser. This may cause the remembered set scan to miss oops that should be included in the root set.

GenShen will attempt to make these old regions parseable after mixed evacuations have completed, but only if no GCs have been requested. When GenShen is under memory pressure and running back-to-back GCs, it may not make these old regions parseable before the remembered set is scanned again. This change adds an uninterruptible phase to make old regions parseable when a global cycle has unloaded classes.

-------------

Commit messages:
 - Replace coalesce and fill heap region closure with regular worker
 - Fix phase timing and reports for coalesce and fill
 - Move old gc state transition logic and fix it
 - Do not coalesce and fill again after mixed collections that follow a global collection
 - Add comment
 - Merge remote-tracking branch 'shenandoah/master' into coalesce-after-class-unloading
 - Degen cycles should also check if old regions are parseable
 - WIP: Fill old regions at end of global cycle
 - Merge remote-tracking branch 'shenandoah/master' into coalesce-after-class-unloading
 - Assert that old regions are parseable before scanning
 - ... and 1 more: https://git.openjdk.org/shenandoah/compare/ae15774c...3fa9bef4

Changes: https://git.openjdk.org/shenandoah/pull/424/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=424&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8330414
  Stats: 296 lines in 19 files changed: 205 ins; 57 del; 34 mod
  Patch: https://git.openjdk.org/shenandoah/pull/424.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/424/head:pull/424

PR: https://git.openjdk.org/shenandoah/pull/424

From stefank at openjdk.org  Fri Apr 19 06:45:04 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 19 Apr 2024 06:45:04 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v10]
In-Reply-To: <Vdgb1WZjBNWUZHb-79AYDS86k4NJJyMNH7jOoatg7qY=.8feb0bd1-bf2c-48f1-98c9-9042cc19ba94@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <Vdgb1WZjBNWUZHb-79AYDS86k4NJJyMNH7jOoatg7qY=.8feb0bd1-bf2c-48f1-98c9-9042cc19ba94@github.com>
Message-ID: <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>

On Thu, 18 Apr 2024 14:27:37 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   more improvements in style/alignments/adjustments.

Changes requested by stefank (Reviewer).

src/hotspot/os/windows/os_windows.cpp line 3401:

> 3399: // Reserve memory at an arbitrary address, only if that area is
> 3400: // available (and not reserved for something else).
> 3401: char* os::pd_attempt_reserve_memory_at(char* addr, size_t bytes, bool exec, MEMFLAGS nmt_flag) {

Most of the time the MEMFLAGS parameters are called flag but in some places they are called nmt_flag. Could they always be called flag? This might require some minor renames in some files, but I think that would be OK.

src/hotspot/share/cds/filemap.cpp line 1726:

> 1724:   char *base = os::map_memory(_fd, _full_path, r->file_offset(),                      //file info
> 1725:                               addr, size,                                             //memory info
> 1726:                               false /* !read_only */, r->allow_exec(), mtClassShared); // flags

Your new comments are not aligned properly.

However, I still don't think you should do this structural change. It makes the code inconsistent with the code below. If you do want to do it, I think you should change the rest of the code as well. But I don't think you should do that in this RFE, so I'd prefer if you just change this to:
Suggestion:

  char *base = os::map_memory(_fd, _full_path, r->file_offset(),
                              addr, size, false /* !read_only */,
                              r->allow_exec(), mtClassShared);

src/hotspot/share/gc/shared/cardTable.cpp line 87:

> 85:   ReservedSpace heap_rs(_byte_map_size, rs_align, _page_size, mtGC);
> 86: 
> 87:     os::trace_page_sizes("Card Table", num_bytes, num_bytes,

The indention is now messed up here.

src/hotspot/share/gc/shared/cardTable.cpp line 176:

> 174:                                 old_committed.word_size() - new_committed.word_size());
> 175:     bool res = os::uncommit_memory((char*)delta.start(),
> 176:                                    delta.byte_size(), !ExecMem, mtGCCardSet);

Suggestion:

                                   delta.byte_size(),
                                   !ExecMem,
                                   mtGCCardSet);

src/hotspot/share/memory/virtualspace.hpp line 180:

> 178:   bool   _executable;
> 179: 
> 180:   MEMFLAGS _nmt_flag;

Don't place the variable against the comment below.
Suggestion:

  MEMFLAGS _nmt_flag;

src/hotspot/share/memory/virtualspace.hpp line 199:

> 197:   size_t _upper_alignment;
> 198: 
> 199: 

Stray blankline

src/hotspot/share/nmt/virtualMemoryTracker.hpp line 306:

> 304:     VirtualMemoryRegion(base, size), _stack(NativeCallStack::empty_stack()), _flag(mtNone) { }
> 305: 
> 306:   ReservedMemoryRegion(address base, size_t size, MEMFLAGS flag) :

(Commenting here because GitHub doesn't allow me to add comments to unchanged lines). I'd like to see a follow-up RFE that makes

  ReservedMemoryRegion(address base, size_t size) :
    VirtualMemoryRegion(base, size), _stack(NativeCallStack::empty_stack()), _flag(mtNone) { }

a private constructor, so that it is only used for our find operations and never accidentally used for other code.

src/hotspot/share/runtime/os.cpp line 2185:

> 2183: char* os::map_memory(int fd, const char* file_name, size_t file_offset,  // file info
> 2184:                      char *addr, size_t bytes,                           // memory info
> 2185:                      bool read_only, bool allow_exec, MEMFLAGS flags) {  // flags

I find it odd to make this documentation and separation only for this function. Could you remove that for this PR?
Suggestion:

char* os::map_memory(int fd, const char* file_name, size_t file_offset,
                     char *addr, size_t bytes, bool read_only
                     bool allow_exec, MEMFLAGS flags) {

src/hotspot/share/runtime/os.hpp line 232:

> 230:   static char*  pd_map_memory(int fd, const char* file_name, size_t file_offset,
> 231:                               char *addr, size_t bytes,
> 232:                               bool read_only = false, bool allow_exec = false);

I'd like to see this reverted.

src/hotspot/share/runtime/os.hpp line 518:

> 516:   static char*  map_memory(int fd, const char* file_name, size_t file_offset, // file info
> 517:                            char *addr, size_t bytes,                          // memory info
> 518:                            bool read_only, bool allow_exec, MEMFLAGS flag);   // flags

Suggestion:

  static char*  map_memory(int fd, const char* file_name, size_t file_offset,
                           char *addr, size_t bytes, bool read_only,
                           bool allow_exec, MEMFLAGS flag);

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2010616842
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571879226
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571880120
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571885564
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571886492
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571892287
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571892588
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571896273
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571898696
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571899968
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1571901493

From rrich at openjdk.org  Fri Apr 19 07:46:58 2024
From: rrich at openjdk.org (Richard Reingruber)
Date: Fri, 19 Apr 2024 07:46:58 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
Message-ID: <DGjOTJjWNyRdiUDzupUmGBzS6N7anW3ECzYAtjRUZOQ=.310e6e2a-a387-407b-a293-a14381c04980@github.com>

On Fri, 12 Apr 2024 14:40:05 GMT, Sergey Nazarkin <snazarki at openjdk.org> wrote:

> An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 
> 
> Additional testing:
> - [x] MacOS AArch64 server fastdebug *gtets*
> - [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
> - [ ] Benchmarking
> 
> @apangin and @parttimenerd could you please check the patch on your scenarios??

What about granting `WXWrite` only if the current thread is in `_thread_in_vm`?
That would be more restrictive and roughly equivalent how it currently works. Likely there are some places then that should be granted `WXWrite` eagerly because they need `WXWrite` without `_thread_in_vm`. E.g. the JIT compiler threads should have `WXWrite` and never `WXExec` (I assume) which should be checked in the signal handler.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2065983074

From azafari at openjdk.org  Fri Apr 19 08:18:00 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 19 Apr 2024 08:18:00 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v10]
In-Reply-To: <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <Vdgb1WZjBNWUZHb-79AYDS86k4NJJyMNH7jOoatg7qY=.8feb0bd1-bf2c-48f1-98c9-9042cc19ba94@github.com>
 <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>
Message-ID: <RLV6IbmciS7PF9NjinEBLGb32VcAZsBmWYle1HxdSho=.ca50fc30-d2fb-4357-8c04-7242310bfeb1@github.com>

On Fri, 19 Apr 2024 06:40:35 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   more improvements in style/alignments/adjustments.
>
> src/hotspot/share/runtime/os.hpp line 232:
> 
>> 230:   static char*  pd_map_memory(int fd, const char* file_name, size_t file_offset,
>> 231:                               char *addr, size_t bytes,
>> 232:                               bool read_only = false, bool allow_exec = false);
> 
> I'd like to see this reverted.

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572006493

From azafari at openjdk.org  Fri Apr 19 08:33:16 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 19 Apr 2024 08:33:16 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v11]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <W3jdmVLQZnjFnJ38BUpuFlNHYr-M8tMZIkct_bomZ08=.e2975864-0d65-4fdb-84a5-055323ba8ebf@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  more on alignments.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/897b4b30..33f2cf69

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=10
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=09-10

  Stats: 19 lines in 6 files changed: 3 ins; 1 del; 15 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Fri Apr 19 08:33:17 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 19 Apr 2024 08:33:17 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v10]
In-Reply-To: <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <Vdgb1WZjBNWUZHb-79AYDS86k4NJJyMNH7jOoatg7qY=.8feb0bd1-bf2c-48f1-98c9-9042cc19ba94@github.com>
 <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>
Message-ID: <ZALQGfPc2KFXuaSoIYmjJbr5s_lrdSpCnAbRO-Y6vvo=.ebdc7313-bd1f-4040-aab2-dfe6b4935522@github.com>

On Fri, 19 Apr 2024 06:23:27 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   more improvements in style/alignments/adjustments.
>
> src/hotspot/os/windows/os_windows.cpp line 3401:
> 
>> 3399: // Reserve memory at an arbitrary address, only if that area is
>> 3400: // available (and not reserved for something else).
>> 3401: char* os::pd_attempt_reserve_memory_at(char* addr, size_t bytes, bool exec, MEMFLAGS nmt_flag) {
> 
> Most of the time the MEMFLAGS parameters are called flag but in some places they are called nmt_flag. Could they always be called flag? This might require some minor renames in some files, but I think that would be OK.

`reserve_large_pages_individually` has a local `flags` variable and
`allocate_pages_individually` has a `flags` param in their definitions. So, the `nmt_flag` is used to avoid confusing different flags while reading the code.
Other cases where there would be no ambiguity, the `nmt_flag` is renamed to `flag`.

> src/hotspot/share/cds/filemap.cpp line 1726:
> 
>> 1724:   char *base = os::map_memory(_fd, _full_path, r->file_offset(),                      //file info
>> 1725:                               addr, size,                                             //memory info
>> 1726:                               false /* !read_only */, r->allow_exec(), mtClassShared); // flags
> 
> Your new comments are not aligned properly.
> 
> However, I still don't think you should do this structural change. It makes the code inconsistent with the code below. If you do want to do it, I think you should change the rest of the code as well. But I don't think you should do that in this RFE, so I'd prefer if you just change this to:
> Suggestion:
> 
>   char *base = os::map_memory(_fd, _full_path, r->file_offset(),
>                               addr, size, false /* !read_only */,
>                               r->allow_exec(), mtClassShared);

Done.

> src/hotspot/share/gc/shared/cardTable.cpp line 87:
> 
>> 85:   ReservedSpace heap_rs(_byte_map_size, rs_align, _page_size, mtGC);
>> 86: 
>> 87:     os::trace_page_sizes("Card Table", num_bytes, num_bytes,
> 
> The indention is now messed up here.

Fixed.

> src/hotspot/share/gc/shared/cardTable.cpp line 176:
> 
>> 174:                                 old_committed.word_size() - new_committed.word_size());
>> 175:     bool res = os::uncommit_memory((char*)delta.start(),
>> 176:                                    delta.byte_size(), !ExecMem, mtGCCardSet);
> 
> Suggestion:
> 
>                                    delta.byte_size(),
>                                    !ExecMem,
>                                    mtGCCardSet);

Done.

> src/hotspot/share/memory/virtualspace.hpp line 180:
> 
>> 178:   bool   _executable;
>> 179: 
>> 180:   MEMFLAGS _nmt_flag;
> 
> Don't place the variable against the comment below.
> Suggestion:
> 
>   MEMFLAGS _nmt_flag;

Done.

> src/hotspot/share/memory/virtualspace.hpp line 199:
> 
>> 197:   size_t _upper_alignment;
>> 198: 
>> 199: 
> 
> Stray blankline

Fixed.

> src/hotspot/share/runtime/os.cpp line 2185:
> 
>> 2183: char* os::map_memory(int fd, const char* file_name, size_t file_offset,  // file info
>> 2184:                      char *addr, size_t bytes,                           // memory info
>> 2185:                      bool read_only, bool allow_exec, MEMFLAGS flags) {  // flags
> 
> I find it odd to make this documentation and separation only for this function. Could you remove that for this PR?
> Suggestion:
> 
> char* os::map_memory(int fd, const char* file_name, size_t file_offset,
>                      char *addr, size_t bytes, bool read_only
>                      bool allow_exec, MEMFLAGS flags) {

Done.

> src/hotspot/share/runtime/os.hpp line 518:
> 
>> 516:   static char*  map_memory(int fd, const char* file_name, size_t file_offset, // file info
>> 517:                            char *addr, size_t bytes,                          // memory info
>> 518:                            bool read_only, bool allow_exec, MEMFLAGS flag);   // flags
> 
> Suggestion:
> 
>   static char*  map_memory(int fd, const char* file_name, size_t file_offset,
>                            char *addr, size_t bytes, bool read_only,
>                            bool allow_exec, MEMFLAGS flag);

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572020983
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572021204
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572021411
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572021706
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572021948
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572022104
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572024667
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572024863

From azafari at openjdk.org  Fri Apr 19 08:47:00 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 19 Apr 2024 08:47:00 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v10]
In-Reply-To: <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <Vdgb1WZjBNWUZHb-79AYDS86k4NJJyMNH7jOoatg7qY=.8feb0bd1-bf2c-48f1-98c9-9042cc19ba94@github.com>
 <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>
Message-ID: <CcfkfA08ZQsrErRP9imi5DzwTqJYYIegOHy03WBFVcE=.4216c749-3fdb-44bf-9d4f-343fa2dd9d1e@github.com>

On Fri, 19 Apr 2024 06:37:41 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   more improvements in style/alignments/adjustments.
>
> src/hotspot/share/nmt/virtualMemoryTracker.hpp line 306:
> 
>> 304:     VirtualMemoryRegion(base, size), _stack(NativeCallStack::empty_stack()), _flag(mtNone) { }
>> 305: 
>> 306:   ReservedMemoryRegion(address base, size_t size, MEMFLAGS flag) :
> 
> (Commenting here because GitHub doesn't allow me to add comments to unchanged lines). I'd like to see a follow-up RFE that makes
> 
>   ReservedMemoryRegion(address base, size_t size) :
>     VirtualMemoryRegion(base, size), _stack(NativeCallStack::empty_stack()), _flag(mtNone) { }
> 
> a private constructor, so that it is only used for our find operations and never accidentally used for other code.

https://bugs.openjdk.org/browse/JDK-8330627 is created for this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572041557

From stefank at openjdk.org  Fri Apr 19 08:51:00 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 19 Apr 2024 08:51:00 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v10]
In-Reply-To: <ZALQGfPc2KFXuaSoIYmjJbr5s_lrdSpCnAbRO-Y6vvo=.ebdc7313-bd1f-4040-aab2-dfe6b4935522@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <Vdgb1WZjBNWUZHb-79AYDS86k4NJJyMNH7jOoatg7qY=.8feb0bd1-bf2c-48f1-98c9-9042cc19ba94@github.com>
 <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>
 <ZALQGfPc2KFXuaSoIYmjJbr5s_lrdSpCnAbRO-Y6vvo=.ebdc7313-bd1f-4040-aab2-dfe6b4935522@github.com>
Message-ID: <gxsyLdoJ3Mz0obll3sSf9fK2-Xxrdw7AzosFYuumFlA=.17379fe1-5cb3-4e3a-a54f-7334265b4339@github.com>

On Fri, 19 Apr 2024 08:27:09 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/os/windows/os_windows.cpp line 3401:
>> 
>>> 3399: // Reserve memory at an arbitrary address, only if that area is
>>> 3400: // available (and not reserved for something else).
>>> 3401: char* os::pd_attempt_reserve_memory_at(char* addr, size_t bytes, bool exec, MEMFLAGS nmt_flag) {
>> 
>> Most of the time the MEMFLAGS parameters are called flag but in some places they are called nmt_flag. Could they always be called flag? This might require some minor renames in some files, but I think that would be OK.
>
> `reserve_large_pages_individually` has a local `flags` variable and
> `allocate_pages_individually` has a `flags` param in their definitions. So, the `nmt_flag` is used to avoid confusing different flags while reading the code.
> Other cases where there would be no ambiguity, the `nmt_flag` is renamed to `flag`.

Could you instead rename the local 'flags' variable instead?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572047252

From azafari at openjdk.org  Fri Apr 19 09:09:29 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 19 Apr 2024 09:09:29 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v12]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <J9Obsgl6HgXQ5_WOH0kzGj1YJg5DL02_hXJR14y51mc=.f9c01938-2512-4a37-b983-5568349abdf8@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  local/param `flags` renamed to `alloc_type` to let have `MEMFLAGS flag` param.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/33f2cf69..2989e3a3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=11
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=10-11

  Stats: 12 lines in 1 file changed: 0 ins; 0 del; 12 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Fri Apr 19 09:09:29 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 19 Apr 2024 09:09:29 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v10]
In-Reply-To: <gxsyLdoJ3Mz0obll3sSf9fK2-Xxrdw7AzosFYuumFlA=.17379fe1-5cb3-4e3a-a54f-7334265b4339@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <Vdgb1WZjBNWUZHb-79AYDS86k4NJJyMNH7jOoatg7qY=.8feb0bd1-bf2c-48f1-98c9-9042cc19ba94@github.com>
 <e2hQtoG8UWb2pj4Q_mYf6z8nX1SJ1ry-JMwYmmngtrI=.2e653603-77c3-40c8-8303-f170ce7039b4@github.com>
 <ZALQGfPc2KFXuaSoIYmjJbr5s_lrdSpCnAbRO-Y6vvo=.ebdc7313-bd1f-4040-aab2-dfe6b4935522@github.com>
 <gxsyLdoJ3Mz0obll3sSf9fK2-Xxrdw7AzosFYuumFlA=.17379fe1-5cb3-4e3a-a54f-7334265b4339@github.com>
Message-ID: <Pfo9o4YJR3iQoDOqWgOOeAK87OYHNnphkGBb71Xxh-Q=.5a7fb8e1-6f2a-4be5-b87c-fe108df58611@github.com>

On Fri, 19 Apr 2024 08:48:36 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> `reserve_large_pages_individually` has a local `flags` variable and
>> `allocate_pages_individually` has a `flags` param in their definitions. So, the `nmt_flag` is used to avoid confusing different flags while reading the code.
>> Other cases where there would be no ambiguity, the `nmt_flag` is renamed to `flag`.
>
> Could you instead rename the local 'flags' variable instead?

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572069037

From stefank at openjdk.org  Fri Apr 19 09:36:02 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 19 Apr 2024 09:36:02 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v12]
In-Reply-To: <J9Obsgl6HgXQ5_WOH0kzGj1YJg5DL02_hXJR14y51mc=.f9c01938-2512-4a37-b983-5568349abdf8@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <J9Obsgl6HgXQ5_WOH0kzGj1YJg5DL02_hXJR14y51mc=.f9c01938-2512-4a37-b983-5568349abdf8@github.com>
Message-ID: <cXTT4BOiwmgEVGihlR3zHDPaJ8qhyqUJMlKP95_Niig=.38d72eff-ec99-4e46-a3d2-e062f6631485@github.com>

On Fri, 19 Apr 2024 09:09:29 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   local/param `flags` renamed to `alloc_type` to let have `MEMFLAGS flag` param.

There's one tiny nit left, from my POV, otherwise this looks good to me.

src/hotspot/share/memory/virtualspace.cpp line 709:

> 707:   assert(max_commit_granularity > 0, "Granularity must be non-zero.");
> 708: 
> 709: 

This blankline should be reverted.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2010969083
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572099614

From azafari at openjdk.org  Fri Apr 19 09:49:33 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 19 Apr 2024 09:49:33 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  removed extra blank line.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/2989e3a3..fa350261

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=12
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=11-12

  Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Fri Apr 19 09:49:34 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Fri, 19 Apr 2024 09:49:34 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v12]
In-Reply-To: <cXTT4BOiwmgEVGihlR3zHDPaJ8qhyqUJMlKP95_Niig=.38d72eff-ec99-4e46-a3d2-e062f6631485@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <J9Obsgl6HgXQ5_WOH0kzGj1YJg5DL02_hXJR14y51mc=.f9c01938-2512-4a37-b983-5568349abdf8@github.com>
 <cXTT4BOiwmgEVGihlR3zHDPaJ8qhyqUJMlKP95_Niig=.38d72eff-ec99-4e46-a3d2-e062f6631485@github.com>
Message-ID: <5yX9I1JoQY9gmxbIvTDPxuxQSu37KHG0LzlL7cq-3iQ=.38c06bf3-699b-466c-b934-aefedb37b17b@github.com>

On Fri, 19 Apr 2024 09:31:12 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   local/param `flags` renamed to `alloc_type` to let have `MEMFLAGS flag` param.
>
> src/hotspot/share/memory/virtualspace.cpp line 709:
> 
>> 707:   assert(max_commit_granularity > 0, "Granularity must be non-zero.");
>> 708: 
>> 709: 
> 
> This blankline should be reverted.

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1572118687

From stefank at openjdk.org  Fri Apr 19 11:57:02 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 19 Apr 2024 11:57:02 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
Message-ID: <CLX0jodEPmd882I8Y2RipNsGc5Ssnp5T3I1UZRrlnZM=.5ee6c8c9-f990-4058-b242-3da9bfb861f6@github.com>

On Fri, 19 Apr 2024 09:49:33 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removed extra blank line.

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2011231210

From wkemper at openjdk.org  Fri Apr 19 14:16:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 19 Apr 2024 14:16:31 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <yU5WbOCoZM-iA7KYFKfydf2n0xUqDfSGkJES7sOJxik=.283adb72-e3da-4253-b5d6-fdc490c22e3c@github.com>

Merges tag jdk-23+19

-------------

Commit messages:
 - 8330467: NoClassDefFoundError when lambda is in a hidden class
 - 8311098: Change comment in verificationType.hpp to refer to _sym
 - 8317376: Minor improvements to the 'this' escape analyzer
 - 8319516: AIX System::loadLibrary needs support to load a shared library from an archive object
 - 8325469: Freeze/Thaw code can crash in the presence of OSR frames
 - 8325494: C2: Broken graph after not skipping CastII node anymore for Assertion Predicates after JDK-8309902
 - 8329595: spurious  variable "might not have been initialized" on static final field
 - 8329948: Remove string template feature
 - 8330053: JFR: Use LocalDateTime instead ZonedDateTime
 - 8324683: Unify AttachListener code for Posix platforms
 - ... and 85 more: https://git.openjdk.org/shenandoah/compare/b04b3047...706b421c

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/425/files
  Stats: 26348 lines in 528 files changed: 11135 ins; 12090 del; 3123 mod
  Patch: https://git.openjdk.org/shenandoah/pull/425.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/425/head:pull/425

PR: https://git.openjdk.org/shenandoah/pull/425

From pminborg at openjdk.org  Mon Apr 22 08:46:53 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Mon, 22 Apr 2024 08:46:53 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v7]
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <lj7O4Bv1QYMArWAyHkuzZ8zbXQ7mj2WF7b_fDEt8hpk=.e22d8bb2-5dc3-4f8a-b4e1-261aab5c8b00@github.com>

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

Per Minborg has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 12 additional commits since the last revision:

 - Simplify tests
 - Add a test for null arg
 - Add a test for findOrThrow()
 - Merge branch 'master' into symbol-lookup-findorthrow
 - Change exception type
 - Update src/java.base/share/classes/java/lang/foreign/package-info.java
   
   Co-authored-by: Jorn Vernee <JornVernee at users.noreply.github.com>
 - Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
   
   Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>
 - Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
   
   Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>
 - Fix typo
 - Update after PR comments
 - ... and 2 more: https://git.openjdk.org/jdk/compare/76cd531f...0e06e0d6

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18474/files
  - new: https://git.openjdk.org/jdk/pull/18474/files/e2f6c4c9..0e06e0d6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=06
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=05-06

  Stats: 91042 lines in 1455 files changed: 42444 ins; 38886 del; 9712 mod
  Patch: https://git.openjdk.org/jdk/pull/18474.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18474/head:pull/18474

PR: https://git.openjdk.org/jdk/pull/18474

From jsjolen at openjdk.org  Mon Apr 22 12:55:36 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Mon, 22 Apr 2024 12:55:36 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
Message-ID: <toINWUmtB3CwD-3l5KQCd4DdcVnoc8TKO6GjuXp5xOI=.2ca93cad-754c-4e4f-84ae-ee0c5decc00b@github.com>

On Fri, 19 Apr 2024 09:49:33 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removed extra blank line.

A couple of questions, looks good over all.

src/hotspot/os/windows/os_windows.cpp line 5108:

> 5106: 
> 5107:     base = (char*) virtualAlloc(addr, bytes, MEM_COMMIT | MEM_RESERVE,
> 5108:                                 PAGE_READWRITE);

Why is this removed?

src/hotspot/share/memory/virtualspace.cpp line 45:

> 43: // Dummy constructor
> 44: ReservedSpace::ReservedSpace() : _base(nullptr), _size(0), _noaccess_prefix(0),
> 45:     _alignment(0), _fd_for_heap(-1), _special(false), _executable(false), _nmt_flag(mtNone) {

Isn't just `_flag` or `_memflag` sufficient as a name for `ReservedSpace`? We don' use `nmt_flag` anywhere else in the codebase.

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2014652979
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1574692818
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1574703805

From mcimadamore at openjdk.org  Mon Apr 22 13:49:37 2024
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 22 Apr 2024 13:49:37 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v7]
In-Reply-To: <lj7O4Bv1QYMArWAyHkuzZ8zbXQ7mj2WF7b_fDEt8hpk=.e22d8bb2-5dc3-4f8a-b4e1-261aab5c8b00@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <lj7O4Bv1QYMArWAyHkuzZ8zbXQ7mj2WF7b_fDEt8hpk=.e22d8bb2-5dc3-4f8a-b4e1-261aab5c8b00@github.com>
Message-ID: <YqMHwwkz2Lezr2Mvaav4QLyOAaO1OrjMCq8R3nhBpdA=.6ad77975-989f-42eb-9644-f3b576b68f00@github.com>

On Mon, 22 Apr 2024 08:46:53 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
>> 
>> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.
>
> Per Minborg has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 12 additional commits since the last revision:
> 
>  - Simplify tests
>  - Add a test for null arg
>  - Add a test for findOrThrow()
>  - Merge branch 'master' into symbol-lookup-findorthrow
>  - Change exception type
>  - Update src/java.base/share/classes/java/lang/foreign/package-info.java
>    
>    Co-authored-by: Jorn Vernee <JornVernee at users.noreply.github.com>
>  - Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
>    
>    Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>
>  - Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
>    
>    Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>
>  - Fix typo
>  - Update after PR comments
>  - ... and 2 more: https://git.openjdk.org/jdk/compare/9a68d47d...0e06e0d6

test/jdk/java/foreign/loaderLookup/TestSymbolLookupFindOrThrow.java line 41:

> 39: 
> 40:     static {
> 41:         System.loadLibrary("Foo");

Where is this lib defined?

test/jdk/java/foreign/loaderLookup/TestSymbolLookupFindOrThrow.java line 58:

> 56:     @Test
> 57:     void findOrThrowNullArg() {
> 58:         assertThrows(NullPointerException.class, () ->

I believe this should already be checked by the TestNulls test - which is a test that automatically checks that _all_ API methods handle nulls accordingly. Please check that, and maybe remove this?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1574788219
PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1574786946

From gli at openjdk.org  Mon Apr 22 16:30:39 2024
From: gli at openjdk.org (Guoxiong Li)
Date: Mon, 22 Apr 2024 16:30:39 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace
Message-ID: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>

Hi all,

This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.

The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.

Best Regards,
-- Guoxiong

-------------

Commit messages:
 - JDK-8330155

Changes: https://git.openjdk.org/jdk/pull/18894/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18894&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8330155
  Stats: 162 lines in 21 files changed: 10 ins; 127 del; 25 mod
  Patch: https://git.openjdk.org/jdk/pull/18894.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18894/head:pull/18894

PR: https://git.openjdk.org/jdk/pull/18894

From ayang at openjdk.org  Mon Apr 22 16:58:28 2024
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Mon, 22 Apr 2024 16:58:28 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace
In-Reply-To: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
Message-ID: <S22i2cp2Wrkqe-iRhje20i7D_IfrYY1yi1w-JbuTuqc=.81161eaa-fae3-4a71-b12b-0297cbb154e5@github.com>

On Mon, 22 Apr 2024 16:24:06 GMT, Guoxiong Li <gli at openjdk.org> wrote:

> Hi all,
> 
> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
> 
> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
> 
> Best Regards,
> -- Guoxiong

Marked as reviewed by ayang (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18894#pullrequestreview-2015334437

From kdnilsen at openjdk.org  Mon Apr 22 19:34:43 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 22 Apr 2024 19:34:43 GMT
Subject: Integrated: 8330071: GenShen: Allow old to expand again at end of each
 GC
In-Reply-To: <4JOkyMaUzYnaXHdYzbrvb7chUbK1LIYSErXNBhGe8fk=.3641aaed-a040-4fdd-bddb-db275fb124e8@github.com>
References: <4JOkyMaUzYnaXHdYzbrvb7chUbK1LIYSErXNBhGe8fk=.3641aaed-a040-4fdd-bddb-db275fb124e8@github.com>
Message-ID: <jVg48r9AIv1MZKHk0nQcRjcMdXt9F-fLIQTC_cvYWPk=.54eeeb39-c495-455e-b91e-c7efc7dc2688@github.com>

On Wed, 17 Apr 2024 23:51:32 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This pull request contains a backport of commit 0578af80 from the openjdk/shenandoah repository.
> 
> The commit being backported was authored by Kelvin Nilsen on 15 Apr 2024 and was reviewed by William Kemper and Y. Srinivas Ramakrishna.

This pull request has now been integrated.

Changeset: 34ad402b
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/34ad402b78d96d583ea25e2726e473d462f85352
Stats:     5 lines in 1 file changed: 4 ins; 0 del; 1 mod

8330071: GenShen: Allow old to expand again at end of each GC

Backport-of: 0578af8015bc0dfcd4a2b726b1a8b52f68712cf2

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/35

From cjplummer at openjdk.org  Mon Apr 22 19:56:27 2024
From: cjplummer at openjdk.org (Chris Plummer)
Date: Mon, 22 Apr 2024 19:56:27 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace
In-Reply-To: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
Message-ID: <NxGpRiVsg8Iz8t8urdjfOsDDnnr9crwRmGmX9EmzL4A=.4037484e-305f-4e68-aa65-aed80763b556@github.com>

On Mon, 22 Apr 2024 16:24:06 GMT, Guoxiong Li <gli at openjdk.org> wrote:

> Hi all,
> 
> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
> 
> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
> 
> Best Regards,
> -- Guoxiong

SA changes look good.

-------------

Marked as reviewed by cjplummer (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18894#pullrequestreview-2015689218

From kdnilsen at openjdk.org  Mon Apr 22 23:38:04 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 22 Apr 2024 23:38:04 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix typo

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/8c2d8015..24ccf880

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=44
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=43-44

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From dholmes at openjdk.org  Tue Apr 23 06:34:35 2024
From: dholmes at openjdk.org (David Holmes)
Date: Tue, 23 Apr 2024 06:34:35 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
Message-ID: <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>

On Fri, 19 Apr 2024 09:49:33 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removed extra blank line.

This is a big change, but the pattern of the changes is quite easy to follow.

I do have a couple of queries below.

Thanks

src/hotspot/share/cds/metaspaceShared.cpp line 1332:

> 1330:     // NMT: fix up the space tags
> 1331:     MemTracker::record_virtual_memory_type(archive_space_rs.base(), mtClassShared);
> 1332:     MemTracker::record_virtual_memory_type(class_space_rs.base(), mtClass);

I assumed these (and others) were removed because the `MemTracker` updates had been pushed down into `ReserveSpace` itself, but I can't find them there - what am I missing?

src/hotspot/share/gc/parallel/mutableSpace.cpp line 63:

> 61:       if (clear_space) {
> 62:         // Prefer page reallocation to migration.
> 63:         os::free_memory((char*)start, size, page_size, mtGC);

Not at all obvious where the corresponding allocation sets the type as mtGC. ??

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2016320972
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575693287
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575697729

From dholmes at openjdk.org  Tue Apr 23 06:34:37 2024
From: dholmes at openjdk.org (David Holmes)
Date: Tue, 23 Apr 2024 06:34:37 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
Message-ID: <BnFd0eq_V8otgX948ZUtFenGjsICkQNyMQcscwNjmeo=.4ffbfdeb-c7c9-4d45-b91d-51dca7b0ef39@github.com>

On Wed, 17 Apr 2024 12:38:23 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   alignment in coding style changed.
>
> src/hotspot/share/nmt/virtualMemoryTracker.cpp line 460:
> 
>> 458:   assert(_reserved_regions != nullptr, "Sanity check");
>> 459: 
>> 460:   ReservedMemoryRegion  rgn(addr, size, flag);
> 
> I'm not sure about this. `rgn` is just used to find the memory region we want to uncommit. The flag isn't used in the search, and passing it forces the callers to also pass in the flag.
> 
> I understand that this happens after the request to remove the mtNone default value. Is there a way that allows us to skip using mtNone, but still don't have to unnecessarily provide a flag?
> 
> Maybe we could create a helper function `ReservedMemoryRegion rgn = ReservedMemoryRegion::create_find_key(addr, size)`, which sets up a ReserveMemoryRegion with mtNone?

Was this comment from Stefan addressed?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575707097

From stuefe at openjdk.org  Tue Apr 23 06:49:35 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 23 Apr 2024 06:49:35 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
Message-ID: <vCCH5hH8ZAa_JMNZz91l92Ne0HACbasmKrc-i5JPSlw=.8ce56342-1d84-4cf3-8d25-e16ab0751a0f@github.com>

On Tue, 23 Apr 2024 06:18:14 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removed extra blank line.
>
> src/hotspot/share/gc/parallel/mutableSpace.cpp line 63:
> 
>> 61:       if (clear_space) {
>> 62:         // Prefer page reallocation to migration.
>> 63:         os::free_memory((char*)start, size, page_size, mtGC);
> 
> Not at all obvious where the corresponding allocation sets the type as mtGC. ??

We don't, and I am not sure this is right. AFAICS this API is used on java heap, in ParallelGC. So, that should be mtJavaHeap, I think.

Note that I would like and plan to simplify this API, if possible remove both the page size and the NMT flags. See https://bugs.openjdk.org/browse/JDK-8330144. (the tricky part is to make sure the proposed Linux alternative works with large pages and on old enough kernels)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575726805

From stuefe at openjdk.org  Tue Apr 23 07:21:48 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 23 Apr 2024 07:21:48 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
Message-ID: <kQ-4TZZW7fQOB7fMPXrlpblrlg5pjuNveFlwrh6VSUI=.6b2ad00e-5be5-489c-a8d4-e2b6e7957ac2@github.com>

On Fri, 19 Apr 2024 09:49:33 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removed extra blank line.

src/hotspot/share/memory/metaspace/testHelpers.cpp line 81:

> 79:   if (reserve_limit > 0) {
> 80:     // have reserve limit -> non-expandable context
> 81:     _rs = ReservedSpace(reserve_limit * BytesPerWord, Metaspace::reserve_alignment(), os::vm_page_size(), mtMetaspace);

I would make this mtTest. This should not increase the metaspace counters in NMT

src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 112:

> 110: 
> 111:   // Commit...
> 112:   if (os::commit_memory((char*)p, word_size * BytesPerWord, !ExecMem, mtMetaspace) == false) {

Not sure if I suggested something different in my first review, but thinking this over, this is wrong. Please don't hardwire mtMetaspace; take the flag from the ReservedSpace member of VirtualSpaceNode.

The reason is that metaspace can be used for at least two different flags, and may later be expanded for more.

src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 191:

> 189: 
> 190:   // Uncommit...
> 191:   if (os::uncommit_memory((char*)p, word_size * BytesPerWord, !ExecMem, mtMetaspace) == false) {

Same here.

src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 262:

> 260:     vm_exit_out_of_memory(word_size * BytesPerWord, OOM_MMAP_ERROR, "Failed to reserve memory for metaspace");
> 261:   }
> 262:   MemTracker::record_virtual_memory_type(rs.base(), mtMetaspace);

Looking at this, I don't particularly like it, but it is pre-existing. The fact that we hard-wire mtMetaspace works now relies on the fact that mtClass and mtMetaspace (as of now, the only two flags that are being used) are using different allocation paths. Long term we should change this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575752104
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575747557
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575747677
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575763725

From stuefe at openjdk.org  Tue Apr 23 07:21:49 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 23 Apr 2024 07:21:49 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <toINWUmtB3CwD-3l5KQCd4DdcVnoc8TKO6GjuXp5xOI=.2ca93cad-754c-4e4f-84ae-ee0c5decc00b@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <toINWUmtB3CwD-3l5KQCd4DdcVnoc8TKO6GjuXp5xOI=.2ca93cad-754c-4e4f-84ae-ee0c5decc00b@github.com>
Message-ID: <OsMCU4PNOiMpfgZLg5eZ4s6SeMbKj66FNsqlvIGIWyI=.7078a200-f861-4d8d-a2ed-696b6ff2c5f7@github.com>

On Mon, 22 Apr 2024 12:51:33 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removed extra blank line.
>
> src/hotspot/share/memory/virtualspace.cpp line 45:
> 
>> 43: // Dummy constructor
>> 44: ReservedSpace::ReservedSpace() : _base(nullptr), _size(0), _noaccess_prefix(0),
>> 45:     _alignment(0), _fd_for_heap(-1), _special(false), _executable(false), _nmt_flag(mtNone) {
> 
> Isn't just `_flag` or `_memflag` sufficient as a name for `ReservedSpace`? We don' use `nmt_flag` anywhere else in the codebase.

Yes, I would keep consistency with existing code, and maybe later rename all in one followup change

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575764827

From stefank at openjdk.org  Tue Apr 23 07:49:39 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 23 Apr 2024 07:49:39 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
Message-ID: <nr0NzhCGV_KeStoKcCnxhNv2dUFT2dDMdQIylrJl4c0=.54ae8536-d841-4b37-be96-b3c0d31980b2@github.com>

On Fri, 19 Apr 2024 09:49:33 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removed extra blank line.

Changes requested by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2016503659

From stefank at openjdk.org  Tue Apr 23 07:49:40 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 23 Apr 2024 07:49:40 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <BnFd0eq_V8otgX948ZUtFenGjsICkQNyMQcscwNjmeo=.4ffbfdeb-c7c9-4d45-b91d-51dca7b0ef39@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
 <BnFd0eq_V8otgX948ZUtFenGjsICkQNyMQcscwNjmeo=.4ffbfdeb-c7c9-4d45-b91d-51dca7b0ef39@github.com>
Message-ID: <DSUJ0PA6Eepc4Fxv793OggqBo82b0ywKFZuldVF60vI=.d76ad4e7-f26b-402a-bcb6-ed25f4818597@github.com>

On Tue, 23 Apr 2024 06:28:31 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> src/hotspot/share/nmt/virtualMemoryTracker.cpp line 460:
>> 
>>> 458:   assert(_reserved_regions != nullptr, "Sanity check");
>>> 459: 
>>> 460:   ReservedMemoryRegion  rgn(addr, size, flag);
>> 
>> I'm not sure about this. `rgn` is just used to find the memory region we want to uncommit. The flag isn't used in the search, and passing it forces the callers to also pass in the flag.
>> 
>> I understand that this happens after the request to remove the mtNone default value. Is there a way that allows us to skip using mtNone, but still don't have to unnecessarily provide a flag?
>> 
>> Maybe we could create a helper function `ReservedMemoryRegion rgn = ReservedMemoryRegion::create_find_key(addr, size)`, which sets up a ReserveMemoryRegion with mtNone?
>
> Was this comment from Stefan addressed?

David is right, this comment wasn't addressed. The code here went back and forth and we settled on hiding `ReservedMemoryRegion(address base, size_t size)` in a separate RFE. This means we probably should revert the usage of `flag` here and all the places that passes down `flag` just to reach this function.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575801191

From azafari at openjdk.org  Tue Apr 23 08:43:34 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 23 Apr 2024 08:43:34 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <toINWUmtB3CwD-3l5KQCd4DdcVnoc8TKO6GjuXp5xOI=.2ca93cad-754c-4e4f-84ae-ee0c5decc00b@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <toINWUmtB3CwD-3l5KQCd4DdcVnoc8TKO6GjuXp5xOI=.2ca93cad-754c-4e4f-84ae-ee0c5decc00b@github.com>
Message-ID: <7r1JsiGxPIa7-h0rpD2CbwK_qeqsWwRNEChBXVctsBw=.7ae4b1e5-4b97-4a04-af69-960ec2c47b6b@github.com>

On Mon, 22 Apr 2024 12:43:43 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removed extra blank line.
>
> src/hotspot/os/windows/os_windows.cpp line 5108:
> 
>> 5106: 
>> 5107:     base = (char*) virtualAlloc(addr, bytes, MEM_COMMIT | MEM_RESERVE,
>> 5108:                                 PAGE_READWRITE);
> 
> Why is this removed?

We found this call duplicated, since the `MemTracker::record_..._and_commit` is called inside the `os::map_memory` after `pd_map_memory` is called.
Here, the requested address is used, but there the result address is used.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575876476

From azafari at openjdk.org  Tue Apr 23 08:43:35 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 23 Apr 2024 08:43:35 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <OsMCU4PNOiMpfgZLg5eZ4s6SeMbKj66FNsqlvIGIWyI=.7078a200-f861-4d8d-a2ed-696b6ff2c5f7@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <toINWUmtB3CwD-3l5KQCd4DdcVnoc8TKO6GjuXp5xOI=.2ca93cad-754c-4e4f-84ae-ee0c5decc00b@github.com>
 <OsMCU4PNOiMpfgZLg5eZ4s6SeMbKj66FNsqlvIGIWyI=.7078a200-f861-4d8d-a2ed-696b6ff2c5f7@github.com>
Message-ID: <-lsJRmLenHFNlKUscKa9ho4ROYjZVW5MrKlTlov5h5k=.9e9972ed-299e-49b5-9bea-46e602fa6672@github.com>

On Tue, 23 Apr 2024 07:16:48 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> src/hotspot/share/memory/virtualspace.cpp line 45:
>> 
>>> 43: // Dummy constructor
>>> 44: ReservedSpace::ReservedSpace() : _base(nullptr), _size(0), _noaccess_prefix(0),
>>> 45:     _alignment(0), _fd_for_heap(-1), _special(false), _executable(false), _nmt_flag(mtNone) {
>> 
>> Isn't just `_flag` or `_memflag` sufficient as a name for `ReservedSpace`? We don' use `nmt_flag` anywhere else in the codebase.
>
> Yes, I would keep consistency with existing code, and maybe later rename all in one followup change

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575876690

From azafari at openjdk.org  Tue Apr 23 08:49:37 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 23 Apr 2024 08:49:37 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
Message-ID: <y9yJBJ3fAaei5cp4B7TXlrL4IgIUSz555arv5AAcG4o=.f163bf07-60a0-4aab-8fc7-096a0a087fcd@github.com>

On Tue, 23 Apr 2024 06:12:59 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removed extra blank line.
>
> src/hotspot/share/cds/metaspaceShared.cpp line 1332:
> 
>> 1330:     // NMT: fix up the space tags
>> 1331:     MemTracker::record_virtual_memory_type(archive_space_rs.base(), mtClassShared);
>> 1332:     MemTracker::record_virtual_memory_type(class_space_rs.base(), mtClass);
> 
> I assumed these (and others) were removed because the `MemTracker` updates had been pushed down into `ReserveSpace` itself, but I can't find them there - what am I missing?

`archive_space_rs` and `class_space_rs` pass the MEMFLAGS to the `ReservedSpace` ctors a few lines above at 1272, 1319 and 1321.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575884704

From azafari at openjdk.org  Tue Apr 23 08:59:34 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 23 Apr 2024 08:59:34 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <vCCH5hH8ZAa_JMNZz91l92Ne0HACbasmKrc-i5JPSlw=.8ce56342-1d84-4cf3-8d25-e16ab0751a0f@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
 <vCCH5hH8ZAa_JMNZz91l92Ne0HACbasmKrc-i5JPSlw=.8ce56342-1d84-4cf3-8d25-e16ab0751a0f@github.com>
Message-ID: <dfbZr5a5GCD7i9r2O9ftTanBpa7NtnYITiyouv-eqa4=.87f3e90e-f8f0-4b0d-bc40-ea0a9b9a7862@github.com>

On Tue, 23 Apr 2024 06:47:00 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> src/hotspot/share/gc/parallel/mutableSpace.cpp line 63:
>> 
>>> 61:       if (clear_space) {
>>> 62:         // Prefer page reallocation to migration.
>>> 63:         os::free_memory((char*)start, size, page_size, mtGC);
>> 
>> Not at all obvious where the corresponding allocation sets the type as mtGC. ??
>
> We don't, and I am not sure this is right. AFAICS this API is used on java heap, in ParallelGC. So, that should be mtJavaHeap, I think.
> 
> Note that I would like and plan to simplify this API, if possible remove both the page size and the NMT flags. See https://bugs.openjdk.org/browse/JDK-8330144. (the tricky part is to make sure the proposed Linux alternative works with large pages and on old enough kernels)

`os::free_memory` on Linux, re-commits the region to discard the existing committed memory. So MEMFLAGS is needed here to pass down to the `os::commit_memory` there.
Since it is actually an uncommit, can we use `mtNone` instead in the `pd_free_memory`? Or define a new `mtDontCare`, for example?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575894505

From azafari at openjdk.org  Tue Apr 23 08:59:37 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 23 Apr 2024 08:59:37 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <kQ-4TZZW7fQOB7fMPXrlpblrlg5pjuNveFlwrh6VSUI=.6b2ad00e-5be5-489c-a8d4-e2b6e7957ac2@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <kQ-4TZZW7fQOB7fMPXrlpblrlg5pjuNveFlwrh6VSUI=.6b2ad00e-5be5-489c-a8d4-e2b6e7957ac2@github.com>
Message-ID: <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>

On Tue, 23 Apr 2024 07:05:24 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removed extra blank line.
>
> src/hotspot/share/memory/metaspace/testHelpers.cpp line 81:
> 
>> 79:   if (reserve_limit > 0) {
>> 80:     // have reserve limit -> non-expandable context
>> 81:     _rs = ReservedSpace(reserve_limit * BytesPerWord, Metaspace::reserve_alignment(), os::vm_page_size(), mtMetaspace);
> 
> I would make this mtTest. This should not increase the metaspace counters in NMT

Done.

> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 112:
> 
>> 110: 
>> 111:   // Commit...
>> 112:   if (os::commit_memory((char*)p, word_size * BytesPerWord, !ExecMem, mtMetaspace) == false) {
> 
> Not sure if I suggested something different in my first review, but thinking this over, this is wrong. Please don't hardwire mtMetaspace; take the flag from the ReservedSpace member of VirtualSpaceNode.
> 
> The reason is that metaspace can be used for at least two different flags, and may later be expanded for more.

Done.

> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 191:
> 
>> 189: 
>> 190:   // Uncommit...
>> 191:   if (os::uncommit_memory((char*)p, word_size * BytesPerWord, !ExecMem, mtMetaspace) == false) {
> 
> Same here.

Done.

> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 262:
> 
>> 260:     vm_exit_out_of_memory(word_size * BytesPerWord, OOM_MMAP_ERROR, "Failed to reserve memory for metaspace");
>> 261:   }
>> 262:   MemTracker::record_virtual_memory_type(rs.base(), mtMetaspace);
> 
> Looking at this, I don't particularly like it, but it is pre-existing. The fact that we hard-wire mtMetaspace works now relies on the fact that mtClass and mtMetaspace (as of now, the only two flags that are being used) are using different allocation paths. Long term we should change this.

Should I create a RFE for it?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575896068
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575895625
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575895817
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575899343

From azafari at openjdk.org  Tue Apr 23 09:05:35 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 23 Apr 2024 09:05:35 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <DSUJ0PA6Eepc4Fxv793OggqBo82b0ywKFZuldVF60vI=.d76ad4e7-f26b-402a-bcb6-ed25f4818597@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
 <BnFd0eq_V8otgX948ZUtFenGjsICkQNyMQcscwNjmeo=.4ffbfdeb-c7c9-4d45-b91d-51dca7b0ef39@github.com>
 <DSUJ0PA6Eepc4Fxv793OggqBo82b0ywKFZuldVF60vI=.d76ad4e7-f26b-402a-bcb6-ed25f4818597@github.com>
Message-ID: <ytrs69dbj12MYkRFY9uciEzuqWCwMjq5xNBjkIPHQdg=.aeddde70-442c-4259-9065-3ef273b7573d@github.com>

On Tue, 23 Apr 2024 07:46:16 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Was this comment from Stefan addressed?
>
> David is right, this comment wasn't addressed. The code here went back and forth and we settled on hiding `ReservedMemoryRegion(address base, size_t size)` in a separate RFE. This means we probably should revert the usage of `flag` here and all the places that passes down `flag` just to reach this function.

We discussed that having flag here, we can use it for checking if the requested flag matches the actual memory flag or not. This check is missed now. 
What to do? reverting all the calls up to `os::uncommit_memory()`? and reverting the `ExecMem` param as optional?

Or adding check of flags?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1575907604

From tschatzl at openjdk.org  Tue Apr 23 09:44:29 2024
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Tue, 23 Apr 2024 09:44:29 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace
In-Reply-To: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
Message-ID: <1ZzvLCjrIROXngFcoJgvFqRUKp7ZIqi2r-WmaMSFt_A=.ba452467-a067-4177-9acf-4bee62658f26@github.com>

On Mon, 22 Apr 2024 16:24:06 GMT, Guoxiong Li <gli at openjdk.org> wrote:

> Hi all,
> 
> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
> 
> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
> 
> Best Regards,
> -- Guoxiong

Marked as reviewed by tschatzl (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18894#pullrequestreview-2016759032

From snazarki at openjdk.org  Tue Apr 23 10:51:28 2024
From: snazarki at openjdk.org (Sergey Nazarkin)
Date: Tue, 23 Apr 2024 10:51:28 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <DGjOTJjWNyRdiUDzupUmGBzS6N7anW3ECzYAtjRUZOQ=.310e6e2a-a387-407b-a293-a14381c04980@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
 <DGjOTJjWNyRdiUDzupUmGBzS6N7anW3ECzYAtjRUZOQ=.310e6e2a-a387-407b-a293-a14381c04980@github.com>
Message-ID: <VViOgQRmw8OsRtS1ik_5cTXBWImuvU8IxST5BlLcrNc=.6624ce84-7e3b-46cf-ba39-f71ad3e221a0@github.com>

On Fri, 19 Apr 2024 07:44:17 GMT, Richard Reingruber <rrich at openjdk.org> wrote:

>> An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 
>> 
>> Additional testing:
>> - [x] MacOS AArch64 server fastdebug *gtets*
>> - [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
>> - [ ] Benchmarking
>> 
>> @apangin and @parttimenerd could you please check the patch on your scenarios??
>
> What about granting `WXWrite` only if the current thread is in `_thread_in_vm`?
> That would be more restrictive and roughly equivalent how it currently works. Likely there are some places then that should be granted `WXWrite` eagerly because they need `WXWrite` without `_thread_in_vm`. E.g. the JIT compiler threads should have `WXWrite` and never `WXExec` (I assume) which should be checked in the signal handler.

The patch doesn't protect against native agents, as this is obviously impossible. The current code doesn't do that either.  For the bytecode, it doesn't prevent the attacker from abusing unsafe api to modify code cache. However unsafe functions are already considered "safe" and we proactively enable WXWrite as well as move thread to `_thread_in_vm` state (@reinrich). JITed code can't write to the cache either with or without the patch.  

I totally get the sense of loss of security. But is this really the case?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2071988941

From azafari at openjdk.org  Tue Apr 23 12:23:35 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 23 Apr 2024 12:23:35 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <ytrs69dbj12MYkRFY9uciEzuqWCwMjq5xNBjkIPHQdg=.aeddde70-442c-4259-9065-3ef273b7573d@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
 <BnFd0eq_V8otgX948ZUtFenGjsICkQNyMQcscwNjmeo=.4ffbfdeb-c7c9-4d45-b91d-51dca7b0ef39@github.com>
 <DSUJ0PA6Eepc4Fxv793OggqBo82b0ywKFZuldVF60vI=.d76ad4e7-f26b-402a-bcb6-ed25f4818597@github.com>
 <ytrs69dbj12MYkRFY9uciEzuqWCwMjq5xNBjkIPHQdg=.aeddde70-442c-4259-9065-3ef273b7573d@github.com>
Message-ID: <H4cv4a_FjsNT5qRiwO1znRdxKjI0ADNe23ujyjpXF-s=.e262e1b3-3afd-4109-a85c-76a900338105@github.com>

On Tue, 23 Apr 2024 09:02:49 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> David is right, this comment wasn't addressed. The code here went back and forth and we settled on hiding `ReservedMemoryRegion(address base, size_t size)` in a separate RFE. This means we probably should revert the usage of `flag` here and all the places that passes down `flag` just to reach this function.
>
> We discussed that having flag here, we can use it for checking if the requested flag matches the actual memory flag or not. This check is missed now. 
> What to do? reverting all the calls up to `os::uncommit_memory()`? and reverting the `ExecMem` param as optional?
> 
> Or adding check of flags?

New info:
I added the check and it always failed, meaning that no uncommit flag matches with commit flag.
I will remove the mandatory flag from uncommit, but it hides the issue of non-matching commit-uncommit flags.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1576156475

From jsjolen at openjdk.org  Tue Apr 23 13:11:43 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Tue, 23 Apr 2024 13:11:43 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <H4cv4a_FjsNT5qRiwO1znRdxKjI0ADNe23ujyjpXF-s=.e262e1b3-3afd-4109-a85c-76a900338105@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
 <BnFd0eq_V8otgX948ZUtFenGjsICkQNyMQcscwNjmeo=.4ffbfdeb-c7c9-4d45-b91d-51dca7b0ef39@github.com>
 <DSUJ0PA6Eepc4Fxv793OggqBo82b0ywKFZuldVF60vI=.d76ad4e7-f26b-402a-bcb6-ed25f4818597@github.com>
 <ytrs69dbj12MYkRFY9uciEzuqWCwMjq5xNBjkIPHQdg=.aeddde70-442c-4259-9065-3ef273b7573d@github.com>
 <H4cv4a_FjsNT5qRiwO1znRdxKjI0ADNe23ujyjpXF-s=.e262e1b3-3afd-4109-a85c-76a900338105@github.com>
Message-ID: <EplJ402MgliSr9Sb2B7BMaSmoWhTWT4AB4s21c_8ses=.1e04608d-b109-4cf8-8e96-cc87a87ff8b9@github.com>

On Tue, 23 Apr 2024 12:20:29 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> We discussed that having flag here, we can use it for checking if the requested flag matches the actual memory flag or not. This check is missed now. 
>> What to do? reverting all the calls up to `os::uncommit_memory()`? and reverting the `ExecMem` param as optional?
>> 
>> Or adding check of flags?
>
> New info:
> I added the check and it always failed, meaning that no uncommit flag matches with commit flag.
> I will remove the mandatory flag from uncommit, but it hides the issue of non-matching commit-uncommit flags.

Keeping the flag argument for `os::uncommit_memory` is important as it is equivalent to `reserve`:ing the memory. This makes future work easier, as we don't have to look at the region to figure out what flag it needs to be reserved as.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1576228846

From aph at openjdk.org  Tue Apr 23 15:13:29 2024
From: aph at openjdk.org (Andrew Haley)
Date: Tue, 23 Apr 2024 15:13:29 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <DGjOTJjWNyRdiUDzupUmGBzS6N7anW3ECzYAtjRUZOQ=.310e6e2a-a387-407b-a293-a14381c04980@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
 <DGjOTJjWNyRdiUDzupUmGBzS6N7anW3ECzYAtjRUZOQ=.310e6e2a-a387-407b-a293-a14381c04980@github.com>
Message-ID: <Ypy3borKtPa_pBCqZKQnF6ULfIuw93R1YDBQhaDUl4o=.aa7eef47-d3d2-48ef-a448-75d400ffbb28@github.com>

On Fri, 19 Apr 2024 07:44:17 GMT, Richard Reingruber <rrich at openjdk.org> wrote:

>> An alternative for preemptively switching the W^X thread mode on macOS with an AArch64 CPU. This implementation triggers the switch in response to the SIGBUS signal if the *si_addr* belongs to the CodeCache area. With this approach, it is now feasible to eliminate all WX guards and avoid potentially costly operations. However, no significant improvement or degradation in performance has been observed.  Additionally, considering the issue with AsyncGetCallTrace, the patched JVM has been successfully operated with [asgct_bottom](https://github.com/parttimenerd/asgct_bottom) and [async-profiler](https://github.com/async-profiler/async-profiler). 
>> 
>> Additional testing:
>> - [x] MacOS AArch64 server fastdebug *gtets*
>> - [ ] MacOS AArch64 server fastdebug *jtreg:hotspot:tier4*
>> - [ ] Benchmarking
>> 
>> @apangin and @parttimenerd could you please check the patch on your scenarios??
>
> What about granting `WXWrite` only if the current thread is in `_thread_in_vm`?
> That would be more restrictive and roughly equivalent how it currently works. Likely there are some places then that should be granted `WXWrite` eagerly because they need `WXWrite` without `_thread_in_vm`. E.g. the JIT compiler threads should have `WXWrite` and never `WXExec` (I assume) which should be checked in the signal handler.

> The patch doesn't protect against native agents, as this is obviously impossible. The current code doesn't do that either. For the bytecode, it doesn't prevent the attacker from abusing unsafe api to modify code cache. However unsafe functions are already considered "safe" and we proactively enable WXWrite as well as move thread to `_thread_in_vm` state (@reinrich). JITed code can't write to the cache either with or without the patch.
> 
> I totally get the sense of loss of security. But is this really the case?

I think it is. W^X is intended (amongst other things) to protect against the use of gadgets, from buffer overflow exploits in non-java code to ROP programming. At present, in order to generate code and execute it, you first have to be able to make the JIT code writable, then write the code, then make it executable. then jump to the code. And the exploit writer might have to do some or all of this by finding gadgets. If we were to merge this patch then all the attacker would have to do is write code to memory and find a way to jump to it, and the automatic switch-on-segfault in this patch would do the all the work the attacker needs.

It makes far more sense to tag those places that actually need to change W^X access, and only switch there.

You could argue that any switching of W^X on a write to code space, then switching it back on jumping (or returning) to Java code, even what we already do, is effectively the same thing. Kinda, but it's not on just any attempt to write to code space or any attempt to jump into code, it's at the places we choose, and we can be careful to limit those places.

But surely the JDK is not the most vulnerable part of the stack anyway? I'd agree with that, of course, but I don't think that's sufficient reason to decide to bypass an OS security mechanism.

We are trying to reduce the size of the attack surface.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2072639243

From gli at openjdk.org  Tue Apr 23 16:04:37 2024
From: gli at openjdk.org (Guoxiong Li)
Date: Tue, 23 Apr 2024 16:04:37 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace [v2]
In-Reply-To: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
Message-ID: <IKpXYKw7Ei_x1eEhiUVJEcX9cWfrzFZUtfrs2mhBxTQ=.602f3041-d61c-486d-9999-3e9312639ae5@github.com>

> Hi all,
> 
> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
> 
> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
> 
> Best Regards,
> -- Guoxiong

Guoxiong Li has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:

 - Remove file.
 - Merge branch 'master' into REMOVE_TENURED_SPACE
   
   # Conflicts:
   #	src/hotspot/share/gc/serial/defNewGeneration.inline.hpp
 - JDK-8330155

-------------

Changes: https://git.openjdk.org/jdk/pull/18894/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18894&range=01
  Stats: 161 lines in 20 files changed: 10 ins; 127 del; 24 mod
  Patch: https://git.openjdk.org/jdk/pull/18894.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18894/head:pull/18894

PR: https://git.openjdk.org/jdk/pull/18894

From stuefe at openjdk.org  Tue Apr 23 16:49:34 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 23 Apr 2024 16:49:34 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <kQ-4TZZW7fQOB7fMPXrlpblrlg5pjuNveFlwrh6VSUI=.6b2ad00e-5be5-489c-a8d4-e2b6e7957ac2@github.com>
 <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>
Message-ID: <SIgKSkzqoi8VMHd990_UrxX9sZgcyXgV9jVR_TxK7kI=.b1566f14-de2b-4ef2-9ed4-7058decf1010@github.com>

On Tue, 23 Apr 2024 08:56:55 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 262:
>> 
>>> 260:     vm_exit_out_of_memory(word_size * BytesPerWord, OOM_MMAP_ERROR, "Failed to reserve memory for metaspace");
>>> 261:   }
>>> 262:   MemTracker::record_virtual_memory_type(rs.base(), mtMetaspace);
>> 
>> Looking at this, I don't particularly like it, but it is pre-existing. The fact that we hard-wire mtMetaspace works now relies on the fact that mtClass and mtMetaspace (as of now, the only two flags that are being used) are using different allocation paths. Long term we should change this.
>
> Should I create a RFE for it?

Sure, go ahead. You can assign this to me, if you want.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1576572430

From gli at openjdk.org  Tue Apr 23 17:22:41 2024
From: gli at openjdk.org (Guoxiong Li)
Date: Tue, 23 Apr 2024 17:22:41 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace [v3]
In-Reply-To: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
Message-ID: <eCWiw2FKdSBPI5TBJbZwTgg2qSlR3UnnYwo-BHgrSvg=.4a3cd2f5-f3a7-485e-8954-f69b14da602c@github.com>

> Hi all,
> 
> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
> 
> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
> 
> Best Regards,
> -- Guoxiong

Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision:

  Fix included header file error after merging master.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18894/files
  - new: https://git.openjdk.org/jdk/pull/18894/files/0796e0b4..5478742c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18894&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18894&range=01-02

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/18894.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18894/head:pull/18894

PR: https://git.openjdk.org/jdk/pull/18894

From gli at openjdk.org  Tue Apr 23 17:33:31 2024
From: gli at openjdk.org (Guoxiong Li)
Date: Tue, 23 Apr 2024 17:33:31 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace [v3]
In-Reply-To: <eCWiw2FKdSBPI5TBJbZwTgg2qSlR3UnnYwo-BHgrSvg=.4a3cd2f5-f3a7-485e-8954-f69b14da602c@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
 <eCWiw2FKdSBPI5TBJbZwTgg2qSlR3UnnYwo-BHgrSvg=.4a3cd2f5-f3a7-485e-8954-f69b14da602c@github.com>
Message-ID: <13XcdYfz-Qdc4pOKtZK0IIYJM7xrCFuiM4LSpgQ2JhE=.26e5fa50-0d25-4e3e-b603-4299cf837614@github.com>

On Tue, 23 Apr 2024 17:22:41 GMT, Guoxiong Li <gli at openjdk.org> wrote:

>> Hi all,
>> 
>> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
>> 
>> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
>> 
>> Best Regards,
>> -- Guoxiong
>
> Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix included header file error after merging master.

I merged the master branch in order to solve the file conflict and added the missed header file after merging. Please take a look at the newest code.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18894#issuecomment-2072993947

From rkennke at openjdk.org  Tue Apr 23 17:33:43 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 23 Apr 2024 17:33:43 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
Message-ID: <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>

On Mon, 22 Apr 2024 23:38:04 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several objectives:
>> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
>> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
>> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
>> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
>> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
>> 
>> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
>> 
>> Comparing 105235.0 metrics from control, 220638.0 from experiment.
>> Compare: 0.589s
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                                  Shenandoah/jython |                                          cwr_total
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
>> 
>> Shenandoah
>> -------------------------------------------------------------------------------------------------------
>> +5.64% jython/cwr_total p=0.00037
>>   Control:      1.928ms (+/-272.40us)        170
>>   Test:         2.037ms (+/-322.73us)        344
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix typo

Ok, I've reviewed all but the shenandoahSimpleBitMap.* stuff, which I want to look at in a separate round.
This is overall much better to read, thanks for the updates! I have a few comments.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 130:

> 128: }
> 129: 
> 130: inline idx_t ShenandoahRegionPartitions:: leftmost(ShenandoahFreeSetPartitionId which_partition) const {

There is an excess whitespace after ::

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 142:

> 140: 
> 141: inline idx_t ShenandoahRegionPartitions::rightmost(ShenandoahFreeSetPartitionId which_partition) const {
> 142:   assert (int(which_partition) < NumPartitions, "selected free partition must be valid");

Is the cast needed because NumPartitions is int? You could also make NumPartitions be uintx_8 or make the enum class a intx_8 instead, and avoid some casts? Also seems more natural to have NumPartitions be the same type as the enum.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 164:

> 162:                                                      idx_t mutator_leftmost_empty, idx_t mutator_rightmost_empty,
> 163:                                                      size_t mutator_region_count, size_t mutator_used) {
> 164:   _region_counts[int(ShenandoahFreeSetPartitionId::Mutator)] = mutator_region_count;

uhh all those casts are kinda ugly. I wasn't aware that this is necessary when I suggested enum classes. But, afaict, this is the least intrusive way to use the enum as indices, short of going back to C-style-enums. Too bad (and sorry).

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 227:

> 225: }
> 226: 
> 227: inline void ShenandoahRegionPartitions::shrink_interval_if_boundary_modified(ShenandoahFreeSetPartitionId partition, idx_t idx) {

This almost looks like a variant of shrink_interval_if_range_modifies_either_boundary(...), maybe it could just call shrink_interval_if_range_modifies_either_boundary(partition, idx, idx)?

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 365:

> 363:   assert (idx < _max, "index is sane: " SIZE_FORMAT " < " SIZE_FORMAT, idx, _max);
> 364:   ShenandoahFreeSetPartitionId result = ShenandoahFreeSetPartitionId::NotFree;
> 365:   for (uint partition_id = 0; partition_id < NumPartitions; partition_id++) {

Looks to me like you could just call the below membership() method (and make membership() available outside of ASSERT), and then pass the result to partition_name()?

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 401:

> 399: }
> 400: 
> 401: inline idx_t ShenandoahRegionPartitions::find_index_of_next_available_region(

If I read this correctly, calling code assumes that the result is >= start_index, right? Otherwise loops might not terminate. Maybe assert that this holds?

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 413:

> 411: }
> 412: 
> 413: inline idx_t ShenandoahRegionPartitions::find_index_of_previous_available_region(

Same as above but other way around?

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 906:

> 904:     }
> 905: 
> 906:     r->set_update_watermark(r->bottom());

Why is that here? It means we would update-refs in this humongous object, but why is it needed in a new object?

-------------

PR Review: https://git.openjdk.org/jdk/pull/17561#pullrequestreview-2017386664
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576332123
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576481126
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576510630
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576519664
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576589712
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576605590
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576606244
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576612773

From rkennke at openjdk.org  Tue Apr 23 17:33:44 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 23 Apr 2024 17:33:44 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
 <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
Message-ID: <fPhnZ450ekb1VmnlqGfqD-fSsbnYMaqMx9WLLu2cqkc=.c532ea4d-aa69-48b1-85b7-1b69b3f24a80@github.com>

On Tue, 23 Apr 2024 15:59:16 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix typo
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 164:
> 
>> 162:                                                      idx_t mutator_leftmost_empty, idx_t mutator_rightmost_empty,
>> 163:                                                      size_t mutator_region_count, size_t mutator_used) {
>> 164:   _region_counts[int(ShenandoahFreeSetPartitionId::Mutator)] = mutator_region_count;
> 
> uhh all those casts are kinda ugly. I wasn't aware that this is necessary when I suggested enum classes. But, afaict, this is the least intrusive way to use the enum as indices, short of going back to C-style-enums. Too bad (and sorry).

maybe it'd indeed be cleaner to just use C-style-enums. I leave that up to you.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576511605

From rkennke at openjdk.org  Tue Apr 23 17:33:45 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 23 Apr 2024 17:33:45 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <yyffC-N_oOimWAEYVnCi-HMaFq9bDW5TG9WHfAIVODY=.d49d00d7-39e2-48ab-a5e8-5361ea15e598@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <aRJBboj41vv0D7F4jIrN2kzBZoXYEbIMrMfAIO3IMjg=.9e7917b1-7912-4409-bf51-bb073f133374@github.com>
 <IimvSux5GEemCzBEY0lZHUCxFTFzhTM7mvCm7AD3nV4=.ed14fcb8-d5a3-4439-a2f8-8444cf7df123@github.com>
 <yyffC-N_oOimWAEYVnCi-HMaFq9bDW5TG9WHfAIVODY=.d49d00d7-39e2-48ab-a5e8-5361ea15e598@github.com>
Message-ID: <p7lWkvvZcG0Zk0JxC1TL9Wej0nwjZWCJ3wliVtIeIbk=.787e54a3-29bb-4309-8640-efc712db98fe@github.com>

On Wed, 17 Apr 2024 16:08:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 915:
>> 
>>> 913: public:
>>> 914:   ShenandoahPostCompactClosure() : _heap(ShenandoahHeap::heap()), _live(0) {
>>> 915:     _heap->free_set()->clear();
>> 
>> Why is that ok?
>
> There is no need to clear() because prepare_to_rebuild() invokes clear_internal() before it establishes new partition assignments.

Ok.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1576629210

From wkemper at openjdk.org  Tue Apr 23 20:22:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 23 Apr 2024 20:22:14 GMT
Subject: RFR: 8330414: GenShen: Class unloading requires old regions be
 made parseable [v2]
In-Reply-To: <ezemrXOHMIFYee3Ex7in1IBnjxQINhWAg3eeFq8OQPs=.08f3516e-f057-4ec7-a8a6-cebe0e6b847a@github.com>
References: <ezemrXOHMIFYee3Ex7in1IBnjxQINhWAg3eeFq8OQPs=.08f3516e-f057-4ec7-a8a6-cebe0e6b847a@github.com>
Message-ID: <ALCSyGbWofhz_ghvrjIkDL-eKeMhI63VpQRPxWf_Ids=.80f4b925-9825-494a-97c5-cc5ec97e0fad@github.com>

> Even when the generational mode is using the mark bitmap to scan the remembered set's card tables, the card table offsets may refer to unmarked objects. If the classes referred to by these unmarked objects have been reloaded in metaspace, the class pointer may no longer be valid or may refer to a class with a different size than expected by the parser. This may cause the remembered set scan to miss oops that should be included in the root set.
> 
> GenShen will attempt to make these old regions parseable after mixed evacuations have completed, but only if no GCs have been requested. When GenShen is under memory pressure and running back-to-back GCs, it may not make these old regions parseable before the remembered set is scanned again. This change adds an uninterruptible phase to make old regions parseable when a global cycle has unloaded classes.

William Kemper has updated the pull request incrementally with three additional commits since the last revision:

 - Full GC should indicate that it has made old regions parseable
 - Fix typo and warning
 - Update state diagram ascii art

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/424/files
  - new: https://git.openjdk.org/shenandoah/pull/424/files/3fa9bef4..5e1350ff

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=424&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=424&range=00-01

  Stats: 43 lines in 2 files changed: 17 ins; 1 del; 25 mod
  Patch: https://git.openjdk.org/shenandoah/pull/424.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/424/head:pull/424

PR: https://git.openjdk.org/shenandoah/pull/424

From kdnilsen at openjdk.org  Tue Apr 23 20:22:14 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 23 Apr 2024 20:22:14 GMT
Subject: RFR: 8330414: GenShen: Class unloading requires old regions be
 made parseable [v2]
In-Reply-To: <ALCSyGbWofhz_ghvrjIkDL-eKeMhI63VpQRPxWf_Ids=.80f4b925-9825-494a-97c5-cc5ec97e0fad@github.com>
References: <ezemrXOHMIFYee3Ex7in1IBnjxQINhWAg3eeFq8OQPs=.08f3516e-f057-4ec7-a8a6-cebe0e6b847a@github.com>
 <ALCSyGbWofhz_ghvrjIkDL-eKeMhI63VpQRPxWf_Ids=.80f4b925-9825-494a-97c5-cc5ec97e0fad@github.com>
Message-ID: <Jp9xGz2jXVIdwrHk5Z3fASHCwSzmkzfG4kfZJIs9lYo=.be53a65c-843f-43ff-9e1c-2a64704373cd@github.com>

On Tue, 23 Apr 2024 20:18:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Even when the generational mode is using the mark bitmap to scan the remembered set's card tables, the card table offsets may refer to unmarked objects. If the classes referred to by these unmarked objects have been reloaded in metaspace, the class pointer may no longer be valid or may refer to a class with a different size than expected by the parser. This may cause the remembered set scan to miss oops that should be included in the root set.
>> 
>> GenShen will attempt to make these old regions parseable after mixed evacuations have completed, but only if no GCs have been requested. When GenShen is under memory pressure and running back-to-back GCs, it may not make these old regions parseable before the remembered set is scanned again. This change adds an uninterruptible phase to make old regions parseable when a global cycle has unloaded classes.
>
> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Full GC should indicate that it has made old regions parseable
>  - Fix typo and warning
>  - Update state diagram ascii art

Thanks for working through this fix, and the code cleanup also.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/424#pullrequestreview-2018148833

From wkemper at openjdk.org  Tue Apr 23 20:22:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 23 Apr 2024 20:22:14 GMT
Subject: Integrated: 8330414: GenShen: Class unloading requires old regions be
 made parseable
In-Reply-To: <ezemrXOHMIFYee3Ex7in1IBnjxQINhWAg3eeFq8OQPs=.08f3516e-f057-4ec7-a8a6-cebe0e6b847a@github.com>
References: <ezemrXOHMIFYee3Ex7in1IBnjxQINhWAg3eeFq8OQPs=.08f3516e-f057-4ec7-a8a6-cebe0e6b847a@github.com>
Message-ID: <yk1RhuSt67ZVR_fiV9Svz1Ig2uJyiTFLF_2Je_Bqqu8=.6bd41312-a7f7-475a-a5d5-6590c7d46cee@github.com>

On Fri, 19 Apr 2024 00:00:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Even when the generational mode is using the mark bitmap to scan the remembered set's card tables, the card table offsets may refer to unmarked objects. If the classes referred to by these unmarked objects have been reloaded in metaspace, the class pointer may no longer be valid or may refer to a class with a different size than expected by the parser. This may cause the remembered set scan to miss oops that should be included in the root set.
> 
> GenShen will attempt to make these old regions parseable after mixed evacuations have completed, but only if no GCs have been requested. When GenShen is under memory pressure and running back-to-back GCs, it may not make these old regions parseable before the remembered set is scanned again. This change adds an uninterruptible phase to make old regions parseable when a global cycle has unloaded classes.

This pull request has now been integrated.

Changeset: 6d095910
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/6d09591099fa78a98a1f963987637138089f702e
Stats:     339 lines in 19 files changed: 222 ins; 58 del; 59 mod

8330414: GenShen: Class unloading requires old regions be made parseable

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/424

From wkemper at openjdk.org  Tue Apr 23 20:48:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 23 Apr 2024 20:48:06 GMT
Subject: RFR: 8329350: GenShen: Do not reset mark bitmaps on a safepoint
Message-ID: <NjVawqMIwlGi36T_im23EXAPk4YM0EeyYF0JRHBWbjY=.181a4fb3-02ee-48a7-9aea-b7ed45e63368@github.com>

Clean backport

-------------

Commit messages:
 - Backport f43034d8d1dc571390f357dc3617f376cafa085f

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/37/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=37&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329350
  Stats: 173 lines in 12 files changed: 101 ins; 57 del; 15 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/37.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/37/head:pull/37

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/37

From kdnilsen at openjdk.org  Tue Apr 23 20:51:07 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 23 Apr 2024 20:51:07 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v13]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <9QjfP11ecbH0nCaFW8LdaJWp-yudfIxdiN6E8YGJHb0=.a99b1b79-98ca-4d8e-9b91-56c22f2228dc@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 27 commits:

 - Merge remote-tracking branch 'origin/master' into enable-old-growth-triggers
 - Merge branch 'openjdk:master' into master
 - Minor refinements to old-growth heuristics
   
   1. If old-marking finds old-gen is entirely empty, reset the heuristics
      to initial state, so we can restart the search for appropriate
      old-gen size
   2. When testing whether old-gen growth "still" exceeds the trigger
      threshold following mixed evacuations, include both old usage and
      old humongous waste, as that is the quantity that was used for
      initial trigger.
 - Correct typo in symbolic constant
   
   FRACTIONAL_DENOMINATOR should be a power of 2.  Small round-off errors
   were observed because there was an error in one digit of this contant.
 - Fix zero build
 - Increase workload to force OLD collections
 - Merge remote-tracking branch 'origin/master' into enable-old-growth-triggers
 - Merge branch 'openjdk:master' into master
 - Revert "Change behavior of max_old and min_old"
   
   This reverts commit d88130000c764dacf08ec27723132dd2a3d968de.
 - Change behavior of max_old and min_old
 - ... and 17 more: https://git.openjdk.org/shenandoah/compare/6d095910...7e708569

-------------

Changes: https://git.openjdk.org/shenandoah/pull/409/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=12
  Stats: 268 lines in 6 files changed: 199 ins; 57 del; 12 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From kdnilsen at openjdk.org  Tue Apr 23 21:15:07 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 23 Apr 2024 21:15:07 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v14]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <8sf2btk2yl1awurhyjbetvDdPUBmtckXH-rnBHsrQZI=.c5973e64-c16c-4b13-8890-fda553830c7e@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix typo

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/409/files
  - new: https://git.openjdk.org/shenandoah/pull/409/files/7e708569..c2adbbec

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=13
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=12-13

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From wkemper at openjdk.org  Wed Apr 24 00:53:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 24 Apr 2024 00:53:54 GMT
Subject: Integrated: 8329350: GenShen: Do not reset mark bitmaps on a safepoint
In-Reply-To: <NjVawqMIwlGi36T_im23EXAPk4YM0EeyYF0JRHBWbjY=.181a4fb3-02ee-48a7-9aea-b7ed45e63368@github.com>
References: <NjVawqMIwlGi36T_im23EXAPk4YM0EeyYF0JRHBWbjY=.181a4fb3-02ee-48a7-9aea-b7ed45e63368@github.com>
Message-ID: <MLlKsnK4OvxOy-aq4ufbzzdfygd-hlWOz-MVQvSWb5k=.6eec0714-bcba-4d51-88bf-386965156245@github.com>

On Tue, 23 Apr 2024 20:41:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport

This pull request has now been integrated.

Changeset: a3722176
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/a3722176528fbacdc74a271081061cd3a3bca7aa
Stats:     173 lines in 12 files changed: 101 ins; 57 del; 15 mod

8329350: GenShen: Do not reset mark bitmaps on a safepoint

Backport-of: f43034d8d1dc571390f357dc3617f376cafa085f

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/37

From stefank at openjdk.org  Wed Apr 24 05:39:35 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 24 Apr 2024 05:39:35 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
Message-ID: <BL1hJirW3lSoQU7KL3zck04tAA3yA1JZLfB9MLvTmqk=.d430e1fe-4629-4cba-85f3-cfcd763098a1@github.com>

On Fri, 19 Apr 2024 09:49:33 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removed extra blank line.

I'm approving this. I think it will be OK to handle the last few comments as separate RFEs.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2018871645

From stefank at openjdk.org  Wed Apr 24 05:39:35 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 24 Apr 2024 05:39:35 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <EplJ402MgliSr9Sb2B7BMaSmoWhTWT4AB4s21c_8ses=.1e04608d-b109-4cf8-8e96-cc87a87ff8b9@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
 <BnFd0eq_V8otgX948ZUtFenGjsICkQNyMQcscwNjmeo=.4ffbfdeb-c7c9-4d45-b91d-51dca7b0ef39@github.com>
 <DSUJ0PA6Eepc4Fxv793OggqBo82b0ywKFZuldVF60vI=.d76ad4e7-f26b-402a-bcb6-ed25f4818597@github.com>
 <ytrs69dbj12MYkRFY9uciEzuqWCwMjq5xNBjkIPHQdg=.aeddde70-442c-4259-9065-3ef273b7573d@github.com>
 <H4cv4a_FjsNT5qRiwO1znRdxKjI0ADNe23ujyjpXF-s=.e262e1b3-3afd-4109-a85c-76a900338105@github.com>
 <EplJ402MgliSr9Sb2B7BMaSmoWhTWT4AB4s21c_8ses=.1e04608d-b109-4cf8-8e96-cc87a87ff8b9@github.com>
Message-ID: <hDWZjPxI1UkNOAEvdwWp_CdRaKOGWUxqJuevi-jooAI=.613531ee-8a80-4bc3-85df-1d3b34b64e68@github.com>

On Tue, 23 Apr 2024 13:08:27 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> New info:
>> I added the check and it always failed, meaning that no uncommit flag matches with commit flag.
>> I will remove the mandatory flag from uncommit, but it hides the issue of non-matching commit-uncommit flags.
>
> Keeping the flag argument for `os::uncommit_memory` is important as it is equivalent to `reserve`:ing the memory. This makes future work easier, as we don't have to look at the region to figure out what flag it needs to be reserved as.

Is there a problem with looking at the region to figure out the flag of the currently committed memory region? On the surface that seems like a reasonable thing to do.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1577284106

From smonteith at openjdk.org  Wed Apr 24 07:20:29 2024
From: smonteith at openjdk.org (Stuart Monteith)
Date: Wed, 24 Apr 2024 07:20:29 GMT
Subject: RFR: 8330171: Lazy W^X switch implementation
In-Reply-To: <Ypy3borKtPa_pBCqZKQnF6ULfIuw93R1YDBQhaDUl4o=.aa7eef47-d3d2-48ef-a448-75d400ffbb28@github.com>
References: <9eymaXovxUNFdkAkzojFQP5trwl_yyY0jE2GzcMEjR4=.02ee2ef9-c476-4c7c-9e4a-e021425c38bc@github.com>
 <DGjOTJjWNyRdiUDzupUmGBzS6N7anW3ECzYAtjRUZOQ=.310e6e2a-a387-407b-a293-a14381c04980@github.com>
 <Ypy3borKtPa_pBCqZKQnF6ULfIuw93R1YDBQhaDUl4o=.aa7eef47-d3d2-48ef-a448-75d400ffbb28@github.com>
Message-ID: <_o-DnTvlCXTP8lho_6sJOEwgOBgn5lzYEJno-uCVRqQ=.ffb3f85e-89dd-44b4-b650-6a4ba79ba20d@github.com>

On Tue, 23 Apr 2024 15:11:10 GMT, Andrew Haley <aph at openjdk.org> wrote:

>> What about granting `WXWrite` only if the current thread is in `_thread_in_vm`?
>> That would be more restrictive and roughly equivalent how it currently works. Likely there are some places then that should be granted `WXWrite` eagerly because they need `WXWrite` without `_thread_in_vm`. E.g. the JIT compiler threads should have `WXWrite` and never `WXExec` (I assume) which should be checked in the signal handler.
>
>> The patch doesn't protect against native agents, as this is obviously impossible. The current code doesn't do that either. For the bytecode, it doesn't prevent the attacker from abusing unsafe api to modify code cache. However unsafe functions are already considered "safe" and we proactively enable WXWrite as well as move thread to `_thread_in_vm` state (@reinrich). JITed code can't write to the cache either with or without the patch.
>> 
>> I totally get the sense of loss of security. But is this really the case?
> 
> I think it is. W^X is intended (amongst other things) to protect against the use of gadgets, from buffer overflow exploits in non-java code to ROP programming. At present, in order to generate code and execute it, you first have to be able to make the JIT code writable, then write the code, then make it executable. then jump to the code. And the exploit writer might have to do some or all of this by finding gadgets. If we were to merge this patch then all the attacker would have to do is write code to memory and find a way to jump to it, and the automatic switch-on-segfault in this patch would do the all the work the attacker needs.
> 
> It makes far more sense to tag those places that actually need to change W^X access, and only switch there.
> 
> You could argue that any switching of W^X on a write to code space, then switching it back on jumping (or returning) to Java code, even what we already do, is effectively the same thing. Kinda, but it's not on just any attempt to write to code space or any attempt to jump into code, it's at the places we choose, and we can be careful to limit those places.
> 
> But surely the JDK is not the most vulnerable part of the stack anyway? I'd agree with that, of course, but I don't think that's sufficient reason to decide to bypass an OS security mechanism.
> 
> We are trying to reduce the size of the attack surface.

To add a little to @theRealAph 's point, we should avoid painting ourselves into a corner. I don't know how the platform is going to evolve, but I'd be nervous about fighting against the intentions of the protections.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18762#issuecomment-2074244082

From ayang at openjdk.org  Wed Apr 24 07:29:29 2024
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Wed, 24 Apr 2024 07:29:29 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace [v3]
In-Reply-To: <eCWiw2FKdSBPI5TBJbZwTgg2qSlR3UnnYwo-BHgrSvg=.4a3cd2f5-f3a7-485e-8954-f69b14da602c@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
 <eCWiw2FKdSBPI5TBJbZwTgg2qSlR3UnnYwo-BHgrSvg=.4a3cd2f5-f3a7-485e-8954-f69b14da602c@github.com>
Message-ID: <ctIIBxh4YJAGPlbMjUoRFub0kab_rSi4r0e_FM7ZlgI=.1099938c-d8d5-47ef-a847-5ecafe93ba9c@github.com>

On Tue, 23 Apr 2024 17:22:41 GMT, Guoxiong Li <gli at openjdk.org> wrote:

>> Hi all,
>> 
>> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
>> 
>> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
>> 
>> Best Regards,
>> -- Guoxiong
>
> Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix included header file error after merging master.

Marked as reviewed by ayang (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/18894#pullrequestreview-2019079434

From jsjolen at openjdk.org  Wed Apr 24 09:13:37 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Wed, 24 Apr 2024 09:13:37 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
Message-ID: <-qHwqRoCF7ddx1FHf5SVmh4fPB6Yx5ZNIufv8a1kREs=.549e1fed-3dad-4094-a3b4-006a10becd7e@github.com>

On Fri, 19 Apr 2024 09:49:33 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removed extra blank line.

I am also fine with approving this, but I'd like to wait for Thomas's approval before this gets integrated since we're touching so much of metaspace.

-------------

Marked as reviewed by jsjolen (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2019329390

From jsjolen at openjdk.org  Wed Apr 24 09:13:38 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Wed, 24 Apr 2024 09:13:38 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <hDWZjPxI1UkNOAEvdwWp_CdRaKOGWUxqJuevi-jooAI=.613531ee-8a80-4bc3-85df-1d3b34b64e68@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
 <BnFd0eq_V8otgX948ZUtFenGjsICkQNyMQcscwNjmeo=.4ffbfdeb-c7c9-4d45-b91d-51dca7b0ef39@github.com>
 <DSUJ0PA6Eepc4Fxv793OggqBo82b0ywKFZuldVF60vI=.d76ad4e7-f26b-402a-bcb6-ed25f4818597@github.com>
 <ytrs69dbj12MYkRFY9uciEzuqWCwMjq5xNBjkIPHQdg=.aeddde70-442c-4259-9065-3ef273b7573d@github.com>
 <H4cv4a_FjsNT5qRiwO1znRdxKjI0ADNe23ujyjpXF-s=.e262e1b3-3afd-4109-a85c-76a900338105@github.com>
 <EplJ402MgliSr9Sb2B7BMaSmoWhTWT4AB4s21c_8ses=.1e04608d-b109-4cf8-8e96-cc87a87ff8b9@github.com>
 <hDWZjPxI1UkNOAEvdwWp_CdRaKOGWUxqJuevi-jooAI=.613531ee-8a80-4bc3-85df-1d3b34b64e68@github.com>
Message-ID: <dcrMcey7ugAH8TOsOATg0Yvik_Cgt1bnvujPHC9Amf0=.7fad7512-7529-45f4-8691-f6b8f1261455@github.com>

On Wed, 24 Apr 2024 05:35:37 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Keeping the flag argument for `os::uncommit_memory` is important as it is equivalent to `reserve`:ing the memory. This makes future work easier, as we don't have to look at the region to figure out what flag it needs to be reserved as.
>
> Is there a problem with looking at the region to figure out the flag of the currently committed memory region? On the surface that seems like a reasonable thing to do.

It's not a problem per se, it's just nice not to have to :). Not a blocker, to be clear.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1577558517

From azafari at openjdk.org  Wed Apr 24 10:00:37 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 24 Apr 2024 10:00:37 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <SIgKSkzqoi8VMHd990_UrxX9sZgcyXgV9jVR_TxK7kI=.b1566f14-de2b-4ef2-9ed4-7058decf1010@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <kQ-4TZZW7fQOB7fMPXrlpblrlg5pjuNveFlwrh6VSUI=.6b2ad00e-5be5-489c-a8d4-e2b6e7957ac2@github.com>
 <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>
 <SIgKSkzqoi8VMHd990_UrxX9sZgcyXgV9jVR_TxK7kI=.b1566f14-de2b-4ef2-9ed4-7058decf1010@github.com>
Message-ID: <sXv94w48H2DXEm8CO1AogalJMSJ1RY_pImpBu_sfWwY=.b77bc83e-0b0e-4c5f-a5a2-a8d4226092ae@github.com>

On Tue, 23 Apr 2024 16:46:39 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Should I create a RFE for it?
>
> Sure, go ahead. You can assign this to me, if you want.

This is created for this issue: 
https://bugs.openjdk.org/browse/JDK-8331039

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1577625047

From dholmes at openjdk.org  Wed Apr 24 10:25:41 2024
From: dholmes at openjdk.org (David Holmes)
Date: Wed, 24 Apr 2024 10:25:41 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <y9yJBJ3fAaei5cp4B7TXlrL4IgIUSz555arv5AAcG4o=.f163bf07-60a0-4aab-8fc7-096a0a087fcd@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
 <y9yJBJ3fAaei5cp4B7TXlrL4IgIUSz555arv5AAcG4o=.f163bf07-60a0-4aab-8fc7-096a0a087fcd@github.com>
Message-ID: <FLfrcbRYLMOTdUCPzO758P3AORXu_f-RAwPsB3by6iY=.3e050f0f-17d0-4665-9993-f6c781caa33c@github.com>

On Tue, 23 Apr 2024 08:46:42 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/cds/metaspaceShared.cpp line 1332:
>> 
>>> 1330:     // NMT: fix up the space tags
>>> 1331:     MemTracker::record_virtual_memory_type(archive_space_rs.base(), mtClassShared);
>>> 1332:     MemTracker::record_virtual_memory_type(class_space_rs.base(), mtClass);
>> 
>> I assumed these (and others) were removed because the `MemTracker` updates had been pushed down into `ReserveSpace` itself, but I can't find them there - what am I missing?
>
> `archive_space_rs` and `class_space_rs` pass the MEMFLAGS to the `ReservedSpace` ctors a few lines above at 1272, 1319 and 1321.

Yes I see the flags being passed to `ReservedSpace` but I don't see where `ReservedSpace` then calls `record_virtual_memory_type`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1577656637

From azafari at openjdk.org  Wed Apr 24 11:38:37 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 24 Apr 2024 11:38:37 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <FLfrcbRYLMOTdUCPzO758P3AORXu_f-RAwPsB3by6iY=.3e050f0f-17d0-4665-9993-f6c781caa33c@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
 <y9yJBJ3fAaei5cp4B7TXlrL4IgIUSz555arv5AAcG4o=.f163bf07-60a0-4aab-8fc7-096a0a087fcd@github.com>
 <FLfrcbRYLMOTdUCPzO758P3AORXu_f-RAwPsB3by6iY=.3e050f0f-17d0-4665-9993-f6c781caa33c@github.com>
Message-ID: <LFPJWdVOk1uexpfNzvmm1_AZkfjAkHypp3D0qLA30bk=.ab49d7c5-0bbd-44e7-99d3-a24b622ad2f0@github.com>

On Wed, 24 Apr 2024 10:23:21 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> `archive_space_rs` and `class_space_rs` pass the MEMFLAGS to the `ReservedSpace` ctors a few lines above at 1272, 1319 and 1321.
>
> Yes I see the flags being passed to `ReservedSpace` but I don't see where `ReservedSpace` then calls `record_virtual_memory_type`.

The `flag` is passed down in the call chain of `ReservedSpace ctor` -> `initialize()` -> `reserve()` -> `reserve_memory()` -> `os::xxx_reserve_memory_yyy()` where is passed to `MemTracker`. There will be no need to specifically call `MemTracker::record_virtual_memory_type(..., flag)` since the flag is already sent to `MemTracker` when  reserving the region.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1577737436

From gli at openjdk.org  Wed Apr 24 11:44:35 2024
From: gli at openjdk.org (Guoxiong Li)
Date: Wed, 24 Apr 2024 11:44:35 GMT
Subject: RFR: 8330155: Serial: Remove TenuredSpace [v3]
In-Reply-To: <eCWiw2FKdSBPI5TBJbZwTgg2qSlR3UnnYwo-BHgrSvg=.4a3cd2f5-f3a7-485e-8954-f69b14da602c@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
 <eCWiw2FKdSBPI5TBJbZwTgg2qSlR3UnnYwo-BHgrSvg=.4a3cd2f5-f3a7-485e-8954-f69b14da602c@github.com>
Message-ID: <PyOUrqqqsUpvf3MCv-Ysfv99rXWRC37u9xt3BrGwGHI=.1f3242db-40c7-499b-8513-6f8ba8bd63c0@github.com>

On Tue, 23 Apr 2024 17:22:41 GMT, Guoxiong Li <gli at openjdk.org> wrote:

>> Hi all,
>> 
>> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
>> 
>> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
>> 
>> Best Regards,
>> -- Guoxiong
>
> Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix included header file error after merging master.

Thanks for the reviews. Integrating.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18894#issuecomment-2074744514

From gli at openjdk.org  Wed Apr 24 11:44:36 2024
From: gli at openjdk.org (Guoxiong Li)
Date: Wed, 24 Apr 2024 11:44:36 GMT
Subject: Integrated: 8330155: Serial: Remove TenuredSpace
In-Reply-To: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
References: <akIgBAk_kI6GQ8orQgn1h1KDPcs1H2AZlq9EHgVLYWI=.e6363e84-1349-464a-98c9-232ba97c2758@github.com>
Message-ID: <2YnKivyjPw42Mzul57zi3XRl8gNWh9SMZFwiLfWEfM8=.c6759245-2395-4794-9313-f65f2b4f6d3d@github.com>

On Mon, 22 Apr 2024 16:24:06 GMT, Guoxiong Li <gli at openjdk.org> wrote:

> Hi all,
> 
> This patch removes the class `TenuredSpace` and adjusts its usages. After removing `TenuredSpace`, the file `space.inline.hpp` is empty, so I remove this file and change the included header file to `space.hpp`.
> 
> The test `make test-tier1_gc` passed locally. Thanks for taking the time to review.
> 
> Best Regards,
> -- Guoxiong

This pull request has now been integrated.

Changeset: 2bb5cf5f
Author:    Guoxiong Li <gli at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/2bb5cf5f33337b2cc40aca3bdd36400dc4af5723
Stats:     162 lines in 21 files changed: 11 ins; 127 del; 24 mod

8330155: Serial: Remove TenuredSpace

Reviewed-by: ayang, cjplummer, tschatzl

-------------

PR: https://git.openjdk.org/jdk/pull/18894

From pminborg at openjdk.org  Wed Apr 24 12:00:59 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Wed, 24 Apr 2024 12:00:59 GMT
Subject: Integrated: 8314592: Add shortcut to SymbolLookup::find
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <bk3WBv4WulMnN77g5mkOQyS-2cOiugC00NKt6oUQr2E=.053e25c8-7bbe-4f9c-935b-189664448965@github.com>

On Mon, 25 Mar 2024 14:56:23 GMT, Per Minborg <pminborg at openjdk.org> wrote:

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

This pull request has now been integrated.

Changeset: e923dfe4
Author:    Per Minborg <pminborg at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/e923dfe4c51291099d9b7411e6c9f20be79b9a53
Stats:     151 lines in 22 files changed: 88 ins; 0 del; 63 mod

8314592: Add shortcut to SymbolLookup::find

Reviewed-by: jvernee, prr

-------------

PR: https://git.openjdk.org/jdk/pull/18474

From pminborg at openjdk.org  Wed Apr 24 12:00:55 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Wed, 24 Apr 2024 12:00:55 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v8]
In-Reply-To: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
Message-ID: <pPLDryemgOYiiTFVA0-bkRUKQaHhfS0V0wFJrZcXS1E=.2756273a-3b5d-47da-b732-a6136f82a12c@github.com>

> While `SymbolLookup` correctly uses an `Optional` return to denote whether a symbol has been found by the lookup or not (which enables composition of symbol lookups), many clients end up just calling `Optional::get`, or `Optional::orElseThrow()` on the result.
> 
> This PR proposes to add a convenience method `SymbolLookup::findOrThrow` that will do a lookup and, if no symbol can be found, throws an `IllegalArgumentException` with a relevant exception message.

Per Minborg has updated the pull request incrementally with one additional commit since the last revision:

  Remove redundant test

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18474/files
  - new: https://git.openjdk.org/jdk/pull/18474/files/0e06e0d6..31d92589

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18474&range=06-07

  Stats: 7 lines in 1 file changed: 0 ins; 7 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/18474.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18474/head:pull/18474

PR: https://git.openjdk.org/jdk/pull/18474

From pminborg at openjdk.org  Wed Apr 24 12:00:58 2024
From: pminborg at openjdk.org (Per Minborg)
Date: Wed, 24 Apr 2024 12:00:58 GMT
Subject: RFR: 8314592: Add shortcut to SymbolLookup::find [v7]
In-Reply-To: <YqMHwwkz2Lezr2Mvaav4QLyOAaO1OrjMCq8R3nhBpdA=.6ad77975-989f-42eb-9644-f3b576b68f00@github.com>
References: <WVW1qJmVoiZsdz9o_obJRkBUzRpIl58L7eLXzC4iJIs=.34b25ae5-7098-4459-986d-4223fc5b49ab@github.com>
 <lj7O4Bv1QYMArWAyHkuzZ8zbXQ7mj2WF7b_fDEt8hpk=.e22d8bb2-5dc3-4f8a-b4e1-261aab5c8b00@github.com>
 <YqMHwwkz2Lezr2Mvaav4QLyOAaO1OrjMCq8R3nhBpdA=.6ad77975-989f-42eb-9644-f3b576b68f00@github.com>
Message-ID: <BIdnoOqZEAdovGv_MHq69c-Woe2IUEeMuXdcVbB7rK4=.6ba4fd05-f162-473c-8a49-74b26a51d427@github.com>

On Mon, 22 Apr 2024 13:46:59 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Per Minborg has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 12 additional commits since the last revision:
>> 
>>  - Simplify tests
>>  - Add a test for null arg
>>  - Add a test for findOrThrow()
>>  - Merge branch 'master' into symbol-lookup-findorthrow
>>  - Change exception type
>>  - Update src/java.base/share/classes/java/lang/foreign/package-info.java
>>    
>>    Co-authored-by: Jorn Vernee <JornVernee at users.noreply.github.com>
>>  - Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
>>    
>>    Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>
>>  - Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java
>>    
>>    Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>
>>  - Fix typo
>>  - Update after PR comments
>>  - ... and 2 more: https://git.openjdk.org/jdk/compare/099a64e8...0e06e0d6
>
> test/jdk/java/foreign/loaderLookup/TestSymbolLookupFindOrThrow.java line 41:
> 
>> 39: 
>> 40:     static {
>> 41:         System.loadLibrary("Foo");
> 
> Where is this lib defined?

it is defined in the sub-folder `lookup` in the file `libFoo.c`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18474#discussion_r1577755855

From wkemper at openjdk.org  Wed Apr 24 16:50:52 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 24 Apr 2024 16:50:52 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v14]
In-Reply-To: <8sf2btk2yl1awurhyjbetvDdPUBmtckXH-rnBHsrQZI=.c5973e64-c16c-4b13-8890-fda553830c7e@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <8sf2btk2yl1awurhyjbetvDdPUBmtckXH-rnBHsrQZI=.c5973e64-c16c-4b13-8890-fda553830c7e@github.com>
Message-ID: <_GD6pfz0uGQAa0t7i5UbLslD9PCT23rMtRIw1CyPBzc=.3a30a644-3fd0-4020-92db-90827c7abcfb@github.com>

On Tue, 23 Apr 2024 21:15:07 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
>> 
>> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
>> 
>> 
>>    Control: shenandoah-x86-template
>> Experiment: enable-old-growth-triggers-gh-x86
>> 
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>>                                Genshen/specjbb2015 |                               context_switch_count
>>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>>                             Shenandoah/specjbb2015 |                                      trigger_learn
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
>> 
>> Genshen
>> -------------------------------------------------------------------------------------------------------
>> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>>   Control:      2.500   (+/-  0.68  )         30
>>   Test:        19.625   (+/-  4.79  )         10
>> 
>> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>>   Control:      2.625   (+/-  0.92  )         30
>>   Test:        17.375   (+/-  3.89  )         10
>> 
>> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>>   Control:      9.833   (+/-  3.48  )         30
>>   Test:        32.000   (+/-  2.58  )       ...
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix typo

Looks good. Minor nit: unnecessary cast in shHeap.cpp:2838

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2838:

> 2836:     ShenandoahGenerationalHeap* gen_heap = ShenandoahGenerationalHeap::heap();
> 2837:     ShenandoahOldGeneration* old_gen = gen_heap->old_generation();
> 2838:     ShenandoahOldHeuristics* old_heuristics = (ShenandoahOldHeuristics*) old_gen->heuristics();

Shouldn't need to cast here.

-------------

Changes requested by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/409#pullrequestreview-2020419160
PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1578215530

From kdnilsen at openjdk.org  Wed Apr 24 19:25:08 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 24 Apr 2024 19:25:08 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v15]
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <BvQEMmgHFAjaQdD5DfGpszvB0L9j-kqOcaSVLNc4OO8=.32c50e37-4e35-42fb-9761-7a75d80ae164@github.com>

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Remove redundant cast

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/409/files
  - new: https://git.openjdk.org/shenandoah/pull/409/files/c2adbbec..4c4cbeb1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=14
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=409&range=13-14

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/409.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/409/head:pull/409

PR: https://git.openjdk.org/shenandoah/pull/409

From kdnilsen at openjdk.org  Wed Apr 24 19:25:09 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 24 Apr 2024 19:25:09 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v14]
In-Reply-To: <_GD6pfz0uGQAa0t7i5UbLslD9PCT23rMtRIw1CyPBzc=.3a30a644-3fd0-4020-92db-90827c7abcfb@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <8sf2btk2yl1awurhyjbetvDdPUBmtckXH-rnBHsrQZI=.c5973e64-c16c-4b13-8890-fda553830c7e@github.com>
 <_GD6pfz0uGQAa0t7i5UbLslD9PCT23rMtRIw1CyPBzc=.3a30a644-3fd0-4020-92db-90827c7abcfb@github.com>
Message-ID: <PdSpVaHHw0spGo8WIYlh8IxvyGTWJaXgtmNxxpY1RcE=.5f02a4e5-b9b8-4e49-8a1e-63e0b98fa00c@github.com>

On Wed, 24 Apr 2024 16:46:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix typo
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2838:
> 
>> 2836:     ShenandoahGenerationalHeap* gen_heap = ShenandoahGenerationalHeap::heap();
>> 2837:     ShenandoahOldGeneration* old_gen = gen_heap->old_generation();
>> 2838:     ShenandoahOldHeuristics* old_heuristics = (ShenandoahOldHeuristics*) old_gen->heuristics();
> 
> Shouldn't need to cast here.

Thanks.  Fixed.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/409#discussion_r1578399434

From rehn at openjdk.org  Thu Apr 25 07:24:36 2024
From: rehn at openjdk.org (Robbin Ehn)
Date: Thu, 25 Apr 2024 07:24:36 GMT
Subject: RFR: 8326306: RISC-V: Re-structure MASM calls and jumps
Message-ID: <elBsX6R_0gGr53kHy0smBJESKKIgWaMtmc6yEMHNVkU=.1480e9f6-c889-47e1-a256-667bcbf464f8@github.com>

Hi, please consider.

We have code that directly use the asm for call/jumps instead masm.
Our masm have a bit odd naming, and we don't use 'proper' pseudoinstructions/mnemonics.
Suggested by [riscv-asm-manual](https://github.com/riscv-non-isa/riscv-asm-manual/tree/master)

j offset	jal x0, offset	Jump
jal offset	jal x1, offset	Jump and link
jr rs	        jalr x0, rs, 0	Jump register
jalr rs	        jalr x1, rs, 0	Jump and link register
ret	        jalr x0, x1, 0	Return from subroutine
call offset	auipc x1, offset[31:12]; jalr x1, x1, offset[11:0]	Call far-away subroutine	
tail offset	auipc x6, offset[31:12]; jalr x0, x6, offset[11:0]	Tail call far-away subroutine

But these can only be implemented like this if you have small enough application.
The fallback of these is to use GOT (your C compiler should place a copy of GOT every 2G so it's always reachable).
We don't have GOT, instead we materialize, so there is still differences between these and ours.

This patch:
- Tries to follow these suggested mappings as good we can.
- Make sure all jumps/calls go through MASM. (so we get control and can easily change for sites using a certain calling convention)
- To avoid confusion between MASM public/private methods and ASM methods and the mnemonics there are some renaming.
  E.g. the mnemonics jal means call offset, as we can't use that so there is no 'jal'.
- I enabled c.j, but right now we never generate it.
- As always the macro does no good and are legacy from when code base did not use templates. (also the x-macros screws up my IDE (vim+rtags))

I started down this path due to I have followup patch on top of this which removes trampoline in favor for load-n-jump.
(WIP: https://github.com/robehn/jdk/compare/jal-fixes...robehn:jdk:load-n-link?expand=1)
While looking into our calls it was a bit confusing, this helps. 

Done a couple of t1-3 slightly different version of this patch, and as part of the followup, no issues found. (VF2, qemu, LP4)
Re-running tests, had some last minute changes.

Thanks, Robbin

-------------

Commit messages:
 - Missed a ws
 - JALR

Changes: https://git.openjdk.org/jdk/pull/18942/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18942&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8326306
  Stats: 362 lines in 9 files changed: 104 ins; 106 del; 152 mod
  Patch: https://git.openjdk.org/jdk/pull/18942.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18942/head:pull/18942

PR: https://git.openjdk.org/jdk/pull/18942

From wkemper at openjdk.org  Fri Apr 26 14:17:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 26 Apr 2024 14:17:44 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <qfuc9l7Qq1T2hZUJ6M6dW_YZk48Q6yMjJbg7N5tjzWA=.902a7f3b-9cae-4652-b4a5-7682feb19fef@github.com>

Merges tag jdk-23+20

-------------

Commit messages:
 - 8328703: Illegal accesses in Java_jdk_internal_org_jline_terminal_impl_jna_linux_CLibraryImpl_ioctl0
 - 8331097: Tests build is broken after pr/18914
 - 8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file)
 - 8325373: Improve StackCounter error reporting for bad switch cases
 - 8330815: Use pattern matching for instanceof in KeepAliveCache
 - 8331030: langtools/tools/javac/tree tests fail with SOE with fastdebug and -Xcomp
 - 8322135: Printing JTable in Windows L&F throws InternalError: HTHEME is null
 - 8331018: Clean up non-standard use of /** comments in `jdk.jshell`
 - 8330853: Add missing checks for ConnectionGraph::can_reduce_cmp() call
 - 8330611: AES-CTR vector intrinsic may read out of bounds (x86_64, AVX-512)
 - ... and 190 more: https://git.openjdk.org/shenandoah/compare/b04b3047...87e864bf

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=426&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/426/files
  Stats: 64203 lines in 955 files changed: 31624 ins; 27372 del; 5207 mod
  Patch: https://git.openjdk.org/shenandoah/pull/426.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/426/head:pull/426

PR: https://git.openjdk.org/shenandoah/pull/426

From wkemper at openjdk.org  Fri Apr 26 19:07:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 26 Apr 2024 19:07:54 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <qfuc9l7Qq1T2hZUJ6M6dW_YZk48Q6yMjJbg7N5tjzWA=.902a7f3b-9cae-4652-b4a5-7682feb19fef@github.com>
References: <qfuc9l7Qq1T2hZUJ6M6dW_YZk48Q6yMjJbg7N5tjzWA=.902a7f3b-9cae-4652-b4a5-7682feb19fef@github.com>
Message-ID: <MBSQ0KsxeIEuwFc7Qf2X5c9Bmp6cm8Kt95pKiT9WLyQ=.0dc2635f-4644-480a-9c5b-1cb25e71a3ab@github.com>

> Merges tag jdk-23+20

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 202 commits:

 - Merge tag 'jdk-23+20' into merge-jdk-23+20
   
   Added tag jdk-23+20 for changeset 87e864bf
 - 8328703: Illegal accesses in Java_jdk_internal_org_jline_terminal_impl_jna_linux_CLibraryImpl_ioctl0
   
   Reviewed-by: asotona, shade
 - 8331097: Tests build is broken after pr/18914
   
   Reviewed-by: alanb
 - 8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file)
   
   Reviewed-by: shade, thartmann
 - 8325373: Improve StackCounter error reporting for bad switch cases
   
   Reviewed-by: psandoz
 - 8330815: Use pattern matching for instanceof in KeepAliveCache
   
   Reviewed-by: jpai, djelinski
 - 8331030: langtools/tools/javac/tree tests fail with SOE with fastdebug and -Xcomp
   
   Reviewed-by: jjg, vromero
 - 8322135: Printing JTable in Windows L&F throws InternalError: HTHEME is null
   
   Reviewed-by: abhiscxk, honkar, aivanov
 - 8331018: Clean up non-standard use of /** comments in `jdk.jshell`
   
   Reviewed-by: iris, darcy, jlahoda
 - 8330853: Add missing checks for ConnectionGraph::can_reduce_cmp() call
   
   Reviewed-by: iveresov, dlong, cslucas
 - ... and 192 more: https://git.openjdk.org/shenandoah/compare/6d095910...e030975a

-------------

Changes: https://git.openjdk.org/shenandoah/pull/426/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=426&range=01
  Stats: 64885 lines in 957 files changed: 32327 ins; 27371 del; 5187 mod
  Patch: https://git.openjdk.org/shenandoah/pull/426.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/426/head:pull/426

PR: https://git.openjdk.org/shenandoah/pull/426

From wkemper at openjdk.org  Fri Apr 26 19:28:17 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 26 Apr 2024 19:28:17 GMT
Subject: RFR: Merge openjdk/jdk:master
In-Reply-To: <yU5WbOCoZM-iA7KYFKfydf2n0xUqDfSGkJES7sOJxik=.283adb72-e3da-4253-b5d6-fdc490c22e3c@github.com>
References: <yU5WbOCoZM-iA7KYFKfydf2n0xUqDfSGkJES7sOJxik=.283adb72-e3da-4253-b5d6-fdc490c22e3c@github.com>
Message-ID: <E2irOMcrGgEnrpTjSOvEGMrb07s5gMKj3fvZfliiFD4=.607365d2-7089-4367-bf90-87f3218823c3@github.com>

On Fri, 19 Apr 2024 14:10:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+19

Superseded by https://github.com/openjdk/shenandoah/pull/426

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/425#issuecomment-2079999142

From wkemper at openjdk.org  Fri Apr 26 19:28:17 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 26 Apr 2024 19:28:17 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <yU5WbOCoZM-iA7KYFKfydf2n0xUqDfSGkJES7sOJxik=.283adb72-e3da-4253-b5d6-fdc490c22e3c@github.com>
References: <yU5WbOCoZM-iA7KYFKfydf2n0xUqDfSGkJES7sOJxik=.283adb72-e3da-4253-b5d6-fdc490c22e3c@github.com>
Message-ID: <tgFDfPThhn6LOAVUsWeTqm9LrVLPlcXgx-rNMrKKAcM=.3b41aa40-b89e-46c2-8c2d-f125cac1fc5f@github.com>

On Fri, 19 Apr 2024 14:10:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+19

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/425

From jsjolen at openjdk.org  Mon Apr 29 08:46:16 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Mon, 29 Apr 2024 08:46:16 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v5]
In-Reply-To: <SfhnbpVFZHB4JMJoyP_mpPKqBLq1uEYjKIeno5lgj-k=.8ca29f8e-c944-4970-b157-1d67ed2c9f87@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <asRwE8-BGk8hnVyAK7lHj0f0Esb5kq_Nx9fu0txB2Pc=.144e81bd-2d5d-4d57-9b85-1a281b6ca163@github.com>
 <SfhnbpVFZHB4JMJoyP_mpPKqBLq1uEYjKIeno5lgj-k=.8ca29f8e-c944-4970-b157-1d67ed2c9f87@github.com>
Message-ID: <lIqbF1congxDnliQwPy9Egf6IdBZDorMJ0ecl0NAdpk=.8a69f2b0-ccb8-4255-8ee9-c3a9a9e5c1ae@github.com>

On Sat, 13 Apr 2024 05:38:11 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   mtCode and mtMetaspace were missed from System Dump map
>
> Just a thought: one (manual) test I would do would be that several JVMs run with the same conditions (I would do at least one with -Xmx=Xms and AlwaysPreTouch) accumulate the same NMT numbers, current, and peak. Just to make sure we use the same flags before and after.

Hi @tstuefe, are you OK with the changes as they are now?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2082173662

From wkemper at openjdk.org  Mon Apr 29 16:00:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 16:00:32 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v15]
In-Reply-To: <BvQEMmgHFAjaQdD5DfGpszvB0L9j-kqOcaSVLNc4OO8=.32c50e37-4e35-42fb-9761-7a75d80ae164@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
 <BvQEMmgHFAjaQdD5DfGpszvB0L9j-kqOcaSVLNc4OO8=.32c50e37-4e35-42fb-9761-7a75d80ae164@github.com>
Message-ID: <KdVTEm7To4FZRauZSkfkfE-HbcbBMkRDNQY4hdqDpKA=.71425a01-7e3c-45d4-a536-c8673ac820af@github.com>

On Wed, 24 Apr 2024 19:25:08 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
>> 
>> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
>> 
>> 
>>    Control: shenandoah-x86-template
>> Experiment: enable-old-growth-triggers-gh-x86
>> 
>>                           Most impacted benchmarks |                              Most impacted metrics
>> -------------------------------------------------------------------------------------------------------
>>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>>                                Genshen/specjbb2015 |                               context_switch_count
>>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>>                             Shenandoah/specjbb2015 |                                      trigger_learn
>> 
>> 
>>                                 Only in experiment |                                    Only in control
>> -------------------------------------------------------------------------------------------------------
>>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
>> 
>> Genshen
>> -------------------------------------------------------------------------------------------------------
>> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>>   Control:      2.500   (+/-  0.68  )         30
>>   Test:        19.625   (+/-  4.79  )         10
>> 
>> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>>   Control:      2.625   (+/-  0.92  )         30
>>   Test:        17.375   (+/-  3.89  )         10
>> 
>> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>>   Control:      9.833   (+/-  3.48  )         30
>>   Test:        32.000   (+/-  2.58  )       ...
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove redundant cast

Thanks for fixing the nit!

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/409#pullrequestreview-2028920002

From kdnilsen at openjdk.org  Mon Apr 29 16:08:38 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 29 Apr 2024 16:08:38 GMT
Subject: Integrated: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC
In-Reply-To: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
References: <V2UNOek4Z21vTS_GBk5oi3MfJyC8hkELnTcGc-OxpGU=.09e42b4e-5b0c-493f-8292-6bf78658c553@github.com>
Message-ID: <zYrgTmG32ghgWA6BUL0r6FpiH71UuAwBfSkXQeVWt4I=.b90274b7-a224-4801-96d4-44463e7fa9c8@github.com>

On Mon, 18 Mar 2024 13:32:16 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Enable old-gen growth triggers, which were inadvertantly disabled.  This passes the internal regression pipeline tests.
> 
> As would be expected, we see an increase in mixed-evacuation triggers.  We also see significant improvement on certain extremem workloads due to improved clearing of old-gen.
> 
> 
>    Control: shenandoah-x86-template
> Experiment: enable-old-growth-triggers-gh-x86
> 
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                            Genshen/extremem-phased |                             trigger_expedite_mixed
>                 Genshen/specjbb2015_weak_ref_patch |                                    trigger_failure
>                                Genshen/specjbb2015 |                               context_switch_count
>                     Genshen/hyperalloc_a3072_o4096 |                                     sla_25000_jops
>                             Shenandoah/specjbb2015 |                                      trigger_learn
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>      hyperalloc_a2048_o2048/trigger_expedite_mixed |                   compress/concurrent_thread_roots
>      hyperalloc_a2048_o4096/trigger_expedite_mixed |                        crypto.rsa/ctr_thread_roots
>      hyperalloc_a3072_o2048/trigger_expedite_mixed |                               crypto.rsa/ctr_total
>      hyperalloc_a3072_o4096/trigger_expedite_mixed |               extremem-large-31g/trigger_expansion
>               extremem-large-31g/trigger_overgrown |                  extremem-phased/trigger_expansion
> 
> Genshen
> -------------------------------------------------------------------------------------------------------
> +685.00% specjbb2015_weak_ref_patch/trigger_expedite_mixed p=0.00002
>   Control:      2.500   (+/-  0.68  )         30
>   Test:        19.625   (+/-  4.79  )         10
> 
> +561.90% specjbb2015/trigger_expedite_mixed p=0.00001
>   Control:      2.625   (+/-  0.92  )         30
>   Test:        17.375   (+/-  3.89  )         10
> 
> +225.42% extremem-phased/trigger_expedite_mixed p=0.00000
>   Control:      9.833   (+/-  3.48  )         30
>   Test:        32.000   (+/-  2.58  )         10
> 
> +63.84% hyperalloc_a3072_o4096/evacuation p=0.02662
>   Control:     37....

This pull request has now been integrated.

Changeset: bfb798a6
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/bfb798a685b6953f342e304c971f9b3416a67706
Stats:     268 lines in 6 files changed: 199 ins; 57 del; 12 mod

8328307: GenShen: Re-enable old-has-grown trigger for old-generation GC

Reviewed-by: wkemper, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/409

From wkemper at openjdk.org  Mon Apr 29 16:22:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 16:22:34 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <qfuc9l7Qq1T2hZUJ6M6dW_YZk48Q6yMjJbg7N5tjzWA=.902a7f3b-9cae-4652-b4a5-7682feb19fef@github.com>
References: <qfuc9l7Qq1T2hZUJ6M6dW_YZk48Q6yMjJbg7N5tjzWA=.902a7f3b-9cae-4652-b4a5-7682feb19fef@github.com>
Message-ID: <Mnvz_cn5kBDqeJJdIzPrRWFKF34J3NpMewkLHfjiWsM=.cc382c59-b4f3-4abf-928b-b4834df96d16@github.com>

On Fri, 26 Apr 2024 14:09:55 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-23+20

This pull request has now been integrated.

Changeset: 9ad51580
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/9ad5158012978cd4346c4ae1ba9c53480107bbf6
Stats:     64885 lines in 957 files changed: 32327 ins; 27371 del; 5187 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/426

From wkemper at openjdk.org  Mon Apr 29 18:23:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 18:23:41 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <gGX8XGhIrkE1lWhag_jqV3Fl7hF6upA9EFuZ4IOA7bk=.35c9b576-cb7a-409b-9bcf-dbda25c22259@github.com>
References: <gGX8XGhIrkE1lWhag_jqV3Fl7hF6upA9EFuZ4IOA7bk=.35c9b576-cb7a-409b-9bcf-dbda25c22259@github.com>
Message-ID: <IlbN2wr0qryYkEah5Y9PswxtpUUa3pX6U_ysHMKiar0=.47f71a9d-e3ab-4bdc-8c8a-9d0d73a55064@github.com>

> Merges tag jdk-21.0.3-ga

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/36/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/36/files/05f18b10..05f18b10

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=36&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=36&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/36.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/36/head:pull/36

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/36

From wkemper at openjdk.org  Mon Apr 29 18:23:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 18:23:41 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <gGX8XGhIrkE1lWhag_jqV3Fl7hF6upA9EFuZ4IOA7bk=.35c9b576-cb7a-409b-9bcf-dbda25c22259@github.com>
References: <gGX8XGhIrkE1lWhag_jqV3Fl7hF6upA9EFuZ4IOA7bk=.35c9b576-cb7a-409b-9bcf-dbda25c22259@github.com>
Message-ID: <moL1AAs09emuZVE38J1HWrvMn9_XyTC6iHKdAwUhQgs=.c0c9f894-fdf8-4f58-838a-a71c13ab4607@github.com>

On Thu, 18 Apr 2024 14:16:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.3-ga

This pull request has now been integrated.

Changeset: f3d8e5cd
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/f3d8e5cdb835bf682f8134094c2c103e5aebc7b0
Stats:     173 lines in 19 files changed: 48 ins; 51 del; 74 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/36

From wkemper at openjdk.org  Mon Apr 29 18:38:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 18:38:43 GMT
Subject: RFR: 8329790: GenShen: Old regions that are pinned during final mark
 are not made parseable
Message-ID: <QnWhczTI8W-NXAOTI6g-wN9rJJNiF2AvOwYLuJIPOsY=.9b824891-f65b-48bb-a735-60dd52f2f37e@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 64b44a8c0e3a296d23905ac9e2233fc237057bab

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/38/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=38&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329790
  Stats: 40 lines in 3 files changed: 17 ins; 8 del; 15 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/38.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/38/head:pull/38

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/38

From wkemper at openjdk.org  Mon Apr 29 18:39:53 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 18:39:53 GMT
Subject: RFR: 8329699: GenShen: Move promotion logic out of shHeap and
 shHeapRegion
Message-ID: <5SViV-GJuBfGIfPQr4F4XGfMHpqEhut4eCpRTW_QzpY=.98ed4323-15ec-4917-901b-251020d4d8b5@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 9d869ca1b4d5b210a93dbb4d209189d913c92446

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/39/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=39&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8329699
  Stats: 1876 lines in 36 files changed: 968 ins; 791 del; 117 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/39.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/39/head:pull/39

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/39

From wkemper at openjdk.org  Mon Apr 29 19:53:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 19:53:21 GMT
Subject: Integrated: 8329699: GenShen: Move promotion logic out of shHeap and
 shHeapRegion
In-Reply-To: <5SViV-GJuBfGIfPQr4F4XGfMHpqEhut4eCpRTW_QzpY=.98ed4323-15ec-4917-901b-251020d4d8b5@github.com>
References: <5SViV-GJuBfGIfPQr4F4XGfMHpqEhut4eCpRTW_QzpY=.98ed4323-15ec-4917-901b-251020d4d8b5@github.com>
Message-ID: <aAA4P-5pMZDSjxUfn2wXbOV73rVhNi5aqJb8rZmp9Js=.a657041b-7cdf-441c-a8a8-7e61b982d482@github.com>

On Mon, 29 Apr 2024 18:26:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: 40d77f56
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/40d77f5611166349197f95774443e32694882e0b
Stats:     1876 lines in 36 files changed: 968 ins; 791 del; 117 mod

8329699: GenShen: Move promotion logic out of shHeap and shHeapRegion

Backport-of: 9d869ca1b4d5b210a93dbb4d209189d913c92446

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/39

From wkemper at openjdk.org  Mon Apr 29 20:46:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 20:46:25 GMT
Subject: Integrated: 8329790: GenShen: Old regions that are pinned during final
 mark are not made parseable
In-Reply-To: <QnWhczTI8W-NXAOTI6g-wN9rJJNiF2AvOwYLuJIPOsY=.9b824891-f65b-48bb-a735-60dd52f2f37e@github.com>
References: <QnWhczTI8W-NXAOTI6g-wN9rJJNiF2AvOwYLuJIPOsY=.9b824891-f65b-48bb-a735-60dd52f2f37e@github.com>
Message-ID: <GxfCFbxdu9TtNQEU-MkAw2P3qA15UTskJo-IzJVJVis=.483e1eec-e5ac-4074-98cd-484888964ce9@github.com>

On Mon, 29 Apr 2024 18:23:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: 10daed19
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/10daed195ac558993a5c499f5b3b4a8a97857981
Stats:     40 lines in 3 files changed: 17 ins; 8 del; 15 mod

8329790: GenShen: Old regions that are pinned during final mark are not made parseable

Backport-of: 64b44a8c0e3a296d23905ac9e2233fc237057bab

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/38

From wkemper at openjdk.org  Mon Apr 29 21:04:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 21:04:37 GMT
Subject: RFR: 8330414: GenShen: Class unloading requires old regions be made
 parseable
Message-ID: <lsUButKl5EhjdqINPqfjjuGIEJrP9rDkDHpMHU3kE84=.1b33d563-0114-40f0-abb8-8abfdf67da4f@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 6d09591099fa78a98a1f963987637138089f702e

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/40/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=40&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8330414
  Stats: 339 lines in 19 files changed: 222 ins; 58 del; 59 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/40.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/40/head:pull/40

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/40

From wkemper at openjdk.org  Mon Apr 29 21:09:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 29 Apr 2024 21:09:21 GMT
Subject: Integrated: 8330414: GenShen: Class unloading requires old regions be
 made parseable
In-Reply-To: <lsUButKl5EhjdqINPqfjjuGIEJrP9rDkDHpMHU3kE84=.1b33d563-0114-40f0-abb8-8abfdf67da4f@github.com>
References: <lsUButKl5EhjdqINPqfjjuGIEJrP9rDkDHpMHU3kE84=.1b33d563-0114-40f0-abb8-8abfdf67da4f@github.com>
Message-ID: <qWZw7bL5aci2ajxoZbjJzuyJ44w4dPo4hDK0vETFfNs=.4f8ef46c-3ae0-42ae-a23d-17d8abc4eaee@github.com>

On Mon, 29 Apr 2024 20:51:07 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: e5f55d1f
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/e5f55d1fd605d7c10ceb7723293486204ba7ac18
Stats:     339 lines in 19 files changed: 222 ins; 58 del; 59 mod

8330414: GenShen: Class unloading requires old regions be made parseable

Backport-of: 6d09591099fa78a98a1f963987637138089f702e

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/40

From kdnilsen at openjdk.org  Mon Apr 29 23:17:33 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 29 Apr 2024 23:17:33 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v46]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <xW-70C365dyCLXx8GCl5f0_8nd5QUF3M92BwNFx9Ews=.b2494c4a-5f40-4925-bc4e-060665183fcd@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Fix whitespace
 - Add include for standard defs

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/24ccf880..b69a4bf7

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=45
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=44-45

  Stats: 3 lines in 2 files changed: 2 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 30 01:05:02 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 01:05:02 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC
Message-ID: <DKYYYui8zmVtGZCscDQfc5Gxio5NFVvMbxMbGeYSVLg=.e99b513e-354a-47c3-bb6a-73ad6d4e96fe@github.com>

Reviewed-by: wkemper, ysr

This pull request contains a backport of commit bfb798a6 from the openjdk/shenandoah repository.

The commit being backported was authored by Kelvin Nilsen on 29 Apr 2024 and was reviewed by William Kemper and
Y. Srinivas Ramakrishna.

-------------

Commit messages:
 - 8328307: GenShen: Re-enable old-has-grown trigger for old-generation GC

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/41/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=41&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8328307
  Stats: 268 lines in 6 files changed: 199 ins; 57 del; 12 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/41.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/41/head:pull/41

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/41

From kdnilsen at openjdk.org  Tue Apr 30 01:54:35 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 01:54:35 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v47]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <SCs5ddrRxqQddmAqwlF97Oaw3CJ20nUBpNewilnyr6o=.ae712129-0dee-46e1-b665-8398f645c895@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Refinements to support zero-build compiles

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/b69a4bf7..3bf3df2d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=46
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=45-46

  Stats: 28 lines in 2 files changed: 16 ins; 9 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 30 02:07:15 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 02:07:15 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
 <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
Message-ID: <Ifi5E6xyJI1Eap6WU0fNOSXmo9EXCfuq1hLwf9nHjhw=.0908f699-2d6b-4034-b7a4-fe841e6b0ceb@github.com>

On Tue, 23 Apr 2024 14:08:51 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix typo
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 130:
> 
>> 128: }
>> 129: 
>> 130: inline idx_t ShenandoahRegionPartitions:: leftmost(ShenandoahFreeSetPartitionId which_partition) const {
> 
> There is an excess whitespace after ::

fixed.  thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1584031899

From kdnilsen at openjdk.org  Tue Apr 30 02:36:27 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 02:36:27 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v48]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <qCeFOq16ySlcmlJm7gJzQ0QLh7VTCaZfPcOZ50IRbVY=.82b851f0-76c3-4350-a694-66d1dda08c7d@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix NumPartition type
  
  Beautify the code by changing type of NumPartitions and adding Int and
  UInt forms of NumPartitions.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/3bf3df2d..06909bad

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=47
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=46-47

  Stats: 36 lines in 2 files changed: 2 ins; 0 del; 34 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 30 02:36:28 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 02:36:28 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
 <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
Message-ID: <e7s9U0JQoKzEdVdWRcyVBJdk-6Ggx8lEJ9lkmV61PRo=.c21a8fb6-21f3-4a08-ad66-34bfceaedd07@github.com>

On Tue, 23 Apr 2024 15:39:57 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix typo
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 142:
> 
>> 140: 
>> 141: inline idx_t ShenandoahRegionPartitions::rightmost(ShenandoahFreeSetPartitionId which_partition) const {
>> 142:   assert (int(which_partition) < NumPartitions, "selected free partition must be valid");
> 
> Is the cast needed because NumPartitions is int? You could also make NumPartitions be uintx_8 or make the enum class a intx_8 instead, and avoid some casts? Also seems more natural to have NumPartitions be the same type as the enum.

Good idea.  I've made some refinements as suggested above.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1584051461

From kdnilsen at openjdk.org  Tue Apr 30 02:36:28 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 02:36:28 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <fPhnZ450ekb1VmnlqGfqD-fSsbnYMaqMx9WLLu2cqkc=.c532ea4d-aa69-48b1-85b7-1b69b3f24a80@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
 <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
 <fPhnZ450ekb1VmnlqGfqD-fSsbnYMaqMx9WLLu2cqkc=.c532ea4d-aa69-48b1-85b7-1b69b3f24a80@github.com>
Message-ID: <lJ-QQQhHOBXtlbpRw7owNyr6VEdHCb222cgT-4P1mEk=.ca9eebf6-5ee3-44c2-a44f-80ecf5205a84@github.com>

On Tue, 23 Apr 2024 15:59:52 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 164:
>> 
>>> 162:                                                      idx_t mutator_leftmost_empty, idx_t mutator_rightmost_empty,
>>> 163:                                                      size_t mutator_region_count, size_t mutator_used) {
>>> 164:   _region_counts[int(ShenandoahFreeSetPartitionId::Mutator)] = mutator_region_count;
>> 
>> uhh all those casts are kinda ugly. I wasn't aware that this is necessary when I suggested enum classes. But, afaict, this is the least intrusive way to use the enum as indices, short of going back to C-style-enums. Too bad (and sorry).
>
> maybe it'd indeed be cleaner to just use C-style-enums. I leave that up to you.

Thanks.  I'm ok with current form.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1584051891

From kdnilsen at openjdk.org  Tue Apr 30 02:50:34 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 02:50:34 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v49]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <c395WlU3Vj1VEeXzNtd-IPKFpzq9UV4omYtkwXVOxHI=.2a2782f6-4754-4caa-817a-a7bb3408437b@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Simplify by combining implemnetations of shrink_interval functions

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/06909bad..5c973232

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=48
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=47-48

  Stats: 34 lines in 2 files changed: 3 ins; 30 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 30 02:50:35 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 02:50:35 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
 <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
Message-ID: <6yp0vcdUHaOAAFQpDqlbBYllzI1jgklmY2-LuK0vDYs=.c441622a-f6ed-4339-9a04-c6d51a9392e1@github.com>

On Tue, 23 Apr 2024 16:05:34 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix typo
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 227:
> 
>> 225: }
>> 226: 
>> 227: inline void ShenandoahRegionPartitions::shrink_interval_if_boundary_modified(ShenandoahFreeSetPartitionId partition, idx_t idx) {
> 
> This almost looks like a variant of shrink_interval_if_range_modifies_either_boundary(...), maybe it could just call shrink_interval_if_range_modifies_either_boundary(partition, idx, idx)?

Good suggestion.  Thanks.  Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1584060588

From kdnilsen at openjdk.org  Tue Apr 30 02:58:47 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 02:58:47 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v2]
In-Reply-To: <DKYYYui8zmVtGZCscDQfc5Gxio5NFVvMbxMbGeYSVLg=.e99b513e-354a-47c3-bb6a-73ad6d4e96fe@github.com>
References: <DKYYYui8zmVtGZCscDQfc5Gxio5NFVvMbxMbGeYSVLg=.e99b513e-354a-47c3-bb6a-73ad6d4e96fe@github.com>
Message-ID: <BJ-F8QnBugVcMZ8FKsBe3UeQexHTHBVBpHPr2ILE_S8=.d3e22219-60af-49da-a28a-d4b156af4722@github.com>

> Reviewed-by: wkemper, ysr
> 
> This pull request contains a backport of commit bfb798a6 from the openjdk/shenandoah repository.
> 
> The commit being backported was authored by Kelvin Nilsen on 29 Apr 2024 and was reviewed by William Kemper and
> Y. Srinivas Ramakrishna.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits:

 - Merge branch 'master' of https://git.openjdk.org/shenandoah-jdk21u into backport-kdnilsen-bfb798a6
 - 8328307: GenShen: Re-enable old-has-grown trigger for old-generation GC
   
   Reviewed-by: wkemper, ysr

-------------

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/41/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=41&range=01
  Stats: 268 lines in 6 files changed: 199 ins; 57 del; 12 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/41.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/41/head:pull/41

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/41

From kdnilsen at openjdk.org  Tue Apr 30 03:12:41 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 03:12:41 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v50]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <ZsnsTbo1daQIL7DklKLhxJEy6czJa7vBnfaj5PeWWc8=.0110b61c-963d-4a2f-b519-79c856404087@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Simplify partition_membership_name by code reuse

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/5c973232..2a83e0aa

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=49
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=48-49

  Stats: 13 lines in 2 files changed: 2 ins; 10 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 30 03:12:41 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 03:12:41 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
 <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
Message-ID: <ap9Ea_AgTnOZ4Iku4vhPYaEykL-0eitZ5INGE5gf398=.9794eb65-581b-4b02-82b7-6eb678b737ee@github.com>

On Tue, 23 Apr 2024 17:01:18 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix typo
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 365:
> 
>> 363:   assert (idx < _max, "index is sane: " SIZE_FORMAT " < " SIZE_FORMAT, idx, _max);
>> 364:   ShenandoahFreeSetPartitionId result = ShenandoahFreeSetPartitionId::NotFree;
>> 365:   for (uint partition_id = 0; partition_id < NumPartitions; partition_id++) {
> 
> Looks to me like you could just call the below membership() method (and make membership() available outside of ASSERT), and then pass the result to partition_name()?

Thanks.  I'll make this change.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1584071528

From kdnilsen at openjdk.org  Tue Apr 30 03:22:31 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 03:22:31 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v51]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <Yr1X_qMy5eSO7DdRiS7MXmOkoOja80r0tCTTzb93Dio=.d1c0d292-df08-488c-a003-6573fda43c84@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Assert progress on find_next and find_prev

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/2a83e0aa..d4c90720

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=50
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=49-50

  Stats: 20 lines in 1 file changed: 16 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 30 03:22:32 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 03:22:32 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
 <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
Message-ID: <3y6jiU4dCNrC9uJmQfuUJVviw1Nxltb6RQ8Qfc_BbVQ=.c4ffc6f8-0776-402f-9b8f-7b864abc200b@github.com>

On Tue, 23 Apr 2024 17:11:04 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix typo
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 401:
> 
>> 399: }
>> 400: 
>> 401: inline idx_t ShenandoahRegionPartitions::find_index_of_next_available_region(
> 
> If I read this correctly, calling code assumes that the result is >= start_index, right? Otherwise loops might not terminate. Maybe assert that this holds?

Good idea.  Done.

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 413:
> 
>> 411: }
>> 412: 
>> 413: inline idx_t ShenandoahRegionPartitions::find_index_of_previous_available_region(
> 
> Same as above but other way around?

Also done.
Thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1584076252
PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1584076320

From kdnilsen at openjdk.org  Tue Apr 30 03:44:42 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 03:44:42 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v52]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <fkkCXexQ1gkLYmZbPdEI1ulZieYuY3k56idnH8AFD1g=.4a1b0999-c968-4819-bd23-f1ebff59cc6d@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Remove unnecessary call to update_watermark

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17561/files
  - new: https://git.openjdk.org/jdk/pull/17561/files/d4c90720..8f5ea038

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=51
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=50-51

  Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From kdnilsen at openjdk.org  Tue Apr 30 03:44:42 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 03:44:42 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v45]
In-Reply-To: <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
 <svr8sTxNP64x3_PeMgKZTiF23SffVn4o1D1RocjE40o=.e5cfd422-2cf9-4bdd-9b96-6eaa98899455@github.com>
 <nzgpCuS-hWwpmG4B-kJWV8vbojt1auDOzIjnEPYpCOM=.383a2eb6-9446-469c-a5c9-9fdf32ff5109@github.com>
Message-ID: <8qHQaFq2SP0AnM3H5FJ0vpllonXy4S2XPbQ32Ltin4E=.a76ac516-0786-4dc2-b4c9-5d29ca7cf965@github.com>

On Tue, 23 Apr 2024 17:17:30 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix typo
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 906:
> 
>> 904:     }
>> 905: 
>> 906:     r->set_update_watermark(r->bottom());
> 
> Why is that here? It means we would update-refs in this humongous object, but why is it needed in a new object?

Good catch.  I suspect this code was accidentally introduced by GenShen engineers before we had a good grasp of the update_watermark role.  I'm removing this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17561#discussion_r1584090307

From stuefe at openjdk.org  Tue Apr 30 06:03:14 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 30 Apr 2024 06:03:14 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
Message-ID: <1BNwYTHgU-eHN44HHfYcnfw3XY_BS43XDnqcgDfNPQo=.afd63b8b-4927-4f89-85b4-35e9794acedd@github.com>

On Fri, 19 Apr 2024 09:49:33 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removed extra blank line.

Okay, had another look. Good work. Mostly nits now, but a number of mine and @stefank 's remarks were not addressed. You wrote "Fixed" though. Are you sure you committed your last changes?

I still find it regrettable that we are getting rid of default arguments. This makes many of these APIs rather unwieldy. The patch is massive, which will be a heck of a pain for backporters because patches following this patch will be less likely to be clean backports. 

In a perfect world, one would not call the "raw" os::xxx functions so much and rather use something like ReservedSpace, which then can carry information about the mapping, e.g. the flag. But I never liked ReservedSpace; it feels like old C++ code and often shows surprising behavior. I would love it if someone were to improve that. For example, rewrite initialization to make it more conform to modern C++, and maybe to make it mostly immutable (e.g. Do we really need something like clear_members? Most of the places using clear_members looked to me like they should rely on automatic destructors instead). ReservedSpace is also copied by value, without having a clear semantic of ownership of the underlying mapping.

src/hotspot/share/memory/virtualspace.cpp line 613:

> 611:     }
> 612:   }
> 613: }

stray

-------------

Changes requested by stuefe (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2030234166
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584156869

From stuefe at openjdk.org  Tue Apr 30 06:03:17 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 30 Apr 2024 06:03:17 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <CIhg1QFIpSZudH254R9aMQJhSf1we6xmqkR4uEFJftc=.079e39bd-aef4-42ed-8f66-d686c741beef@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <4p0uq_t37Fkj9fxqD1QC8TOkgAyyW1PVmTknURCquG4=.22b762b8-dea4-4fe3-a19f-d6a3f26c9f27@github.com>
 <CIhg1QFIpSZudH254R9aMQJhSf1we6xmqkR4uEFJftc=.079e39bd-aef4-42ed-8f66-d686c741beef@github.com>
Message-ID: <U8ZdkZMB89bwIRN7gm7A4WDZ4LnBjfcpimwcU1ojHfA=.6b5cfa16-b034-4bc7-bcde-51b4f769845c@github.com>

On Thu, 18 Apr 2024 08:39:46 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/classfile/compactHashtable.cpp line 243:
>> 
>>> 241:     quit("Unable to open hashtable dump file", filename);
>>> 242:   }
>>> 243:   _base = os::map_memory(_fd, filename, 0, nullptr, _size, mtInternal, true, false);
>> 
>> Isn't this CDS code. Should ths be mtClassShared or something else that indicates that this is CDS code?
>
> Fixed.

I don't see the fix.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584163987

From stuefe at openjdk.org  Tue Apr 30 06:03:16 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 30 Apr 2024 06:03:16 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
Message-ID: <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>

On Mon, 15 Apr 2024 16:11:13 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   alignment in coding style changed.

src/hotspot/share/cds/metaspaceShared.cpp line 1322:

> 1320:                                        os::vm_page_size(), mtClassShared, (char*)base_address);
> 1321:       class_space_rs   = ReservedSpace(class_space_size, class_space_alignment,
> 1322:                                        os::vm_page_size(), mtClass, (char*)ccs_base);

Note that here, we place two spaces atop of a region that has been previously mapped with mtClass (see e.g. src/hotspot/cpu/aarch64/compressedKlass_aarch64.cpp). I assume this is not a problem?

src/hotspot/share/memory/virtualspace.cpp line 57:

> 55: }
> 56: 
> 57: ReservedSpace::ReservedSpace(size_t size, size_t preferred_page_size, MEMFLAGS flag) : _fd_for_heap(-1), _nmt_flag(flag) {

Small nit: Mixture of styles. As much as I dislike it, current style is to initialize things via dedicated initialize methods. I'd rather stay consistent.

That said, I would be more than happy for someone to give these classes a once-over and convert them to the more usual style - using initializer lists. Then, we also can make members const that should be const, e.g. _nmt_flags. Not in this PR though.

src/hotspot/share/memory/virtualspace.cpp line 623:

> 621:   }
> 622:   // _nmt_flag is used internally by initialize_compressed_heap
> 623:   _nmt_flag = mtJavaHeap;

Nit, we use a mixture of directly accessing _nmt_flag and accessing it via getter. Hotspot seems to prefer getters/setters. Can we use setters here?

src/hotspot/share/memory/virtualspace.hpp line 46:

> 44:   int      _fd_for_heap;
> 45:   bool     _executable;
> 46:   MEMFLAGS _nmt_flag;

See my remark below. This member, and probably others (e.g. page size and size) could and should probably be const. Food for follow up PRs.

src/hotspot/share/memory/virtualspace.hpp line 71:

> 69:  public:
> 70: 
> 71:   MEMFLAGS nmt_flag() { return _nmt_flag; }

const method

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584163635
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584153176
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584158038
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584154689
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584150480

From stuefe at openjdk.org  Tue Apr 30 06:03:18 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 30 Apr 2024 06:03:18 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <kQ-4TZZW7fQOB7fMPXrlpblrlg5pjuNveFlwrh6VSUI=.6b2ad00e-5be5-489c-a8d4-e2b6e7957ac2@github.com>
 <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>
Message-ID: <3_SnilNMBOpzwdkyaOW4w4QyMfqIjAlR99N0dTBsksc=.d2c0f5dd-4e16-4fd6-9c04-eb8e6ae395ba@github.com>

On Tue, 23 Apr 2024 08:54:24 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 112:
>> 
>>> 110: 
>>> 111:   // Commit...
>>> 112:   if (os::commit_memory((char*)p, word_size * BytesPerWord, !ExecMem, mtMetaspace) == false) {
>> 
>> Not sure if I suggested something different in my first review, but thinking this over, this is wrong. Please don't hardwire mtMetaspace; take the flag from the ReservedSpace member of VirtualSpaceNode.
>> 
>> The reason is that metaspace can be used for at least two different flags, and may later be expanded for more.
>
> Done.

Where? I still see mtMetaspace.

>> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 191:
>> 
>>> 189: 
>>> 190:   // Uncommit...
>>> 191:   if (os::uncommit_memory((char*)p, word_size * BytesPerWord, !ExecMem, mtMetaspace) == false) {
>> 
>> Same here.
>
> Done.

I still see mtMetaspace.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584158963
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584159189

From stuefe at openjdk.org  Tue Apr 30 06:03:19 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 30 Apr 2024 06:03:19 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v12]
In-Reply-To: <5yX9I1JoQY9gmxbIvTDPxuxQSu37KHG0LzlL7cq-3iQ=.38c06bf3-699b-466c-b934-aefedb37b17b@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <J9Obsgl6HgXQ5_WOH0kzGj1YJg5DL02_hXJR14y51mc=.f9c01938-2512-4a37-b983-5568349abdf8@github.com>
 <cXTT4BOiwmgEVGihlR3zHDPaJ8qhyqUJMlKP95_Niig=.38d72eff-ec99-4e46-a3d2-e062f6631485@github.com>
 <5yX9I1JoQY9gmxbIvTDPxuxQSu37KHG0LzlL7cq-3iQ=.38c06bf3-699b-466c-b934-aefedb37b17b@github.com>
Message-ID: <uOzHfQoQV3EWulfWAxpZk-6hcj-sPMS0aOXkeSWCfZM=.05a32d08-7ad7-4874-a258-d5285efb919d@github.com>

On Fri, 19 Apr 2024 09:46:39 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/virtualspace.cpp line 709:
>> 
>>> 707:   assert(max_commit_granularity > 0, "Granularity must be non-zero.");
>>> 708: 
>>> 709: 
>> 
>> This blankline should be reverted.
>
> Done.

Still a stray line.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584158376

From azafari at openjdk.org  Tue Apr 30 08:58:18 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 30 Apr 2024 08:58:18 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
Message-ID: <6wS8eeO2KoYhRkkDxB4YhWStEfLrU2FRtT8CMwYkI74=.bf05a80b-f10b-417e-ba2d-76a86f0a3122@github.com>

On Tue, 30 Apr 2024 05:31:05 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   alignment in coding style changed.
>
> src/hotspot/share/memory/virtualspace.cpp line 57:
> 
>> 55: }
>> 56: 
>> 57: ReservedSpace::ReservedSpace(size_t size, size_t preferred_page_size, MEMFLAGS flag) : _fd_for_heap(-1), _nmt_flag(flag) {
> 
> Small nit: Mixture of styles. As much as I dislike it, current style is to initialize things via dedicated initialize methods. I'd rather stay consistent.
> 
> That said, I would be more than happy for someone to give these classes a once-over and convert them to the more usual style - using initializer lists. Then, we also can make members const that should be const, e.g. _nmt_flags. Not in this PR though.

Do you mean to move the initializations on line 57 (and others in the files) to the `initialize` method?

> src/hotspot/share/memory/virtualspace.hpp line 46:
> 
>> 44:   int      _fd_for_heap;
>> 45:   bool     _executable;
>> 46:   MEMFLAGS _nmt_flag;
> 
> See my remark below. This member, and probably others (e.g. page size and size) could and should probably be const. Food for follow up PRs.

The getter method made as `const`.

> src/hotspot/share/memory/virtualspace.hpp line 71:
> 
>> 69:  public:
>> 70: 
>> 71:   MEMFLAGS nmt_flag() { return _nmt_flag; }
> 
> const method

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584392835
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584393529
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584389450

From azafari at openjdk.org  Tue Apr 30 08:58:20 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 30 Apr 2024 08:58:20 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <1BNwYTHgU-eHN44HHfYcnfw3XY_BS43XDnqcgDfNPQo=.afd63b8b-4927-4f89-85b4-35e9794acedd@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <1BNwYTHgU-eHN44HHfYcnfw3XY_BS43XDnqcgDfNPQo=.afd63b8b-4927-4f89-85b4-35e9794acedd@github.com>
Message-ID: <WMf_grVBcJLLHVu5VzmdD_dr-JKTfG0nBdzhZSU_0cc=.9f811300-4219-4311-b2cb-563339a9cba9@github.com>

On Tue, 30 Apr 2024 05:37:12 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removed extra blank line.
>
> src/hotspot/share/memory/virtualspace.cpp line 613:
> 
>> 611:     }
>> 612:   }
>> 613: }
> 
> stray

There should be no stray now.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584394133

From azafari at openjdk.org  Tue Apr 30 09:04:17 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 30 Apr 2024 09:04:17 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <3_SnilNMBOpzwdkyaOW4w4QyMfqIjAlR99N0dTBsksc=.d2c0f5dd-4e16-4fd6-9c04-eb8e6ae395ba@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <kQ-4TZZW7fQOB7fMPXrlpblrlg5pjuNveFlwrh6VSUI=.6b2ad00e-5be5-489c-a8d4-e2b6e7957ac2@github.com>
 <OJ6NDsT_QHaQ0NgFPFB1mDomjgB0va8HYR3UJqqGTkM=.9aac6a50-78e7-4026-b638-5a4acfe62ec5@github.com>
 <3_SnilNMBOpzwdkyaOW4w4QyMfqIjAlR99N0dTBsksc=.d2c0f5dd-4e16-4fd6-9c04-eb8e6ae395ba@github.com>
Message-ID: <WB6bhEBWtJNdCOoS0QNKaNx0Ebr9fjk9snuVNN_zc0o=.c5e5b373-6553-4e59-b19e-99b3719b2638@github.com>

On Tue, 30 Apr 2024 05:40:15 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Done.
>
> Where? I still see mtMetaspace.

It should be seen now.

>> Done.
>
> I still see mtMetaspace.

It should be seen now.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584403814
PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584402987

From azafari at openjdk.org  Tue Apr 30 09:04:19 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 30 Apr 2024 09:04:19 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
Message-ID: <refC7AZ2N2CUliGi9vFbyBGUBa4DUHfq2uQprflQkyc=.3e0b6a9d-af1d-4b31-b977-3abe9e068049@github.com>

On Tue, 30 Apr 2024 05:38:53 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   alignment in coding style changed.
>
> src/hotspot/share/memory/virtualspace.cpp line 623:
> 
>> 621:   }
>> 622:   // _nmt_flag is used internally by initialize_compressed_heap
>> 623:   _nmt_flag = mtJavaHeap;
> 
> Nit, we use a mixture of directly accessing _nmt_flag and accessing it via getter. Hotspot seems to prefer getters/setters. Can we use setters here?

The flag is not set/changed in other classes, so there is no need to have a `public set_nmt_flag()` member for it.
All the changes to the flag can be done internally using the member directly.
P.S.: There was already a setter but removed after a review comment.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584399496

From azafari at openjdk.org  Tue Apr 30 09:18:13 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 30 Apr 2024 09:18:13 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
Message-ID: <M2IGp3VQO7mV2CSCFZ_fILVj_rS85eN1KRPr42SeaeY=.59cc27b3-665a-4cae-a1c5-52d83a5ec323@github.com>

On Tue, 30 Apr 2024 05:46:45 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   alignment in coding style changed.
>
> src/hotspot/share/cds/metaspaceShared.cpp line 1322:
> 
>> 1320:                                        os::vm_page_size(), mtClassShared, (char*)base_address);
>> 1321:       class_space_rs   = ReservedSpace(class_space_size, class_space_alignment,
>> 1322:                                        os::vm_page_size(), mtClass, (char*)ccs_base);
> 
> Note that here, we place two spaces atop of a region that has been previously mapped with mtClass (see e.g. src/hotspot/cpu/aarch64/compressedKlass_aarch64.cpp). I assume this is not a problem?

It should not be a problem. This PR does not change the functionalities at all. Only the MEMFLAGS is passed down to be given to `MemTracker` API. 
If the above code worked before this PR, it should still work now. 
For NMT point of view, reserving `mtClassShared` and `mtJavaHeap` regions are accepted to overlap with previously reserved regions. Ans, if a whole region is re-reserved but with a different flag, it is also acceptable  and just the accountings are moved from former flag to the new one.
I trust in that all the NMT tests passed where checked these cases.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584424486

From azafari at openjdk.org  Tue Apr 30 09:30:31 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 30 Apr 2024 09:30:31 GMT
Subject: RFR: 8330076: [NMT] add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v14]
In-Reply-To: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
Message-ID: <KgHgK3df3JiH1TZXsY2D4fP9RdAeV_mDew59NW6svus=.9532f39a-bd32-4603-8031-ffad117b2844@github.com>

> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
> 
> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.

Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:

  comments applied.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/18745/files
  - new: https://git.openjdk.org/jdk/pull/18745/files/fa350261..72467f68

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=13
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18745&range=12-13

  Stats: 36 lines in 8 files changed: 0 ins; 3 del; 33 mod
  Patch: https://git.openjdk.org/jdk/pull/18745.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18745/head:pull/18745

PR: https://git.openjdk.org/jdk/pull/18745

From azafari at openjdk.org  Tue Apr 30 09:50:18 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 30 Apr 2024 09:50:18 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v5]
In-Reply-To: <SfhnbpVFZHB4JMJoyP_mpPKqBLq1uEYjKIeno5lgj-k=.8ca29f8e-c944-4970-b157-1d67ed2c9f87@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <asRwE8-BGk8hnVyAK7lHj0f0Esb5kq_Nx9fu0txB2Pc=.144e81bd-2d5d-4d57-9b85-1a281b6ca163@github.com>
 <SfhnbpVFZHB4JMJoyP_mpPKqBLq1uEYjKIeno5lgj-k=.8ca29f8e-c944-4970-b157-1d67ed2c9f87@github.com>
Message-ID: <StFoeEBnCjVeuOnxdbEwTDbbDEsGbVqvUK3plkSmqxc=.588ef12b-571b-445e-8649-841f1d534f1e@github.com>

On Sat, 13 Apr 2024 05:38:11 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   mtCode and mtMetaspace were missed from System Dump map
>
> Just a thought: one (manual) test I would do would be that several JVMs run with the same conditions (I would do at least one with -Xmx=Xms and AlwaysPreTouch) accumulate the same NMT numbers, current, and peak. Just to make sure we use the same flags before and after.

Thank you @tstuefe for your review. Some changes were missed and/or not pushed. So, you should be b able to see them now.
The comment on removing MEMFLAGS from params of the `uncommit_memory` family of API is not applied since existence of the flag makes the VMATree operations more efficient.
I gather the following future PRs that you mentioned, to have individual threads of comments: 
- unify style of initialization in virtualspace.?pp code.
- check if any member can be `const`
- re-designing ReservedSpace

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2084855905

From shade at openjdk.org  Tue Apr 30 11:36:13 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 30 Apr 2024 11:36:13 GMT
Subject: RFR: 8331405: Shenandoah: Optimize ShenandoahLock with TTAS
Message-ID: <UDTvGn-vj89MGIceH4coaImzWYarTRENxxCXaDc3xGI=.2771ae59-73a8-481a-a49f-b6321eebd5e5@github.com>

JDK-8325587 made the contended lock acquisition as the CAS loop with backoffs. Unfortunately, for `SpinPause` path, we always do the CAS and then spin. This is not efficient, as CAS likely requests memory for ownership, which exacerbates the contention and makes the spin loop run longer before blocking.

We should really use TTAS pattern there, like `Thread::SpinAcquire` does it.

Additional testing:
 - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah`
 - [ ] Linux AArch64 server fastdebug, `all` with `-XX:+UseShenandoahGC`

-------------

Commit messages:
 - Fix

Changes: https://git.openjdk.org/jdk/pull/19018/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19018&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8331405
  Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/19018.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19018/head:pull/19018

PR: https://git.openjdk.org/jdk/pull/19018

From stuefe at openjdk.org  Tue Apr 30 12:06:15 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 30 Apr 2024 12:06:15 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <6wS8eeO2KoYhRkkDxB4YhWStEfLrU2FRtT8CMwYkI74=.bf05a80b-f10b-417e-ba2d-76a86f0a3122@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
 <6wS8eeO2KoYhRkkDxB4YhWStEfLrU2FRtT8CMwYkI74=.bf05a80b-f10b-417e-ba2d-76a86f0a3122@github.com>
Message-ID: <My6HX9bdgQqZ-z1lqe2mTWpSFOaPUo8i7URUHC286wc=.cac28875-69b5-46df-b775-3089f72d77e8@github.com>

On Tue, 30 Apr 2024 08:55:31 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/virtualspace.cpp line 57:
>> 
>>> 55: }
>>> 56: 
>>> 57: ReservedSpace::ReservedSpace(size_t size, size_t preferred_page_size, MEMFLAGS flag) : _fd_for_heap(-1), _nmt_flag(flag) {
>> 
>> Small nit: Mixture of styles. As much as I dislike it, current style is to initialize things via dedicated initialize methods. I'd rather stay consistent.
>> 
>> That said, I would be more than happy for someone to give these classes a once-over and convert them to the more usual style - using initializer lists. Then, we also can make members const that should be const, e.g. _nmt_flags. Not in this PR though.
>
> Do you mean to move the initializations on line 57 (and others in the files) to the `initialize` method?

Or, just remove it. The initialise function already initialises the member, right?

But it's really a small nit. I like your variant more if it were uniquely applied.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584684694

From stefank at openjdk.org  Tue Apr 30 13:35:16 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 30 Apr 2024 13:35:16 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <My6HX9bdgQqZ-z1lqe2mTWpSFOaPUo8i7URUHC286wc=.cac28875-69b5-46df-b775-3089f72d77e8@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
 <6wS8eeO2KoYhRkkDxB4YhWStEfLrU2FRtT8CMwYkI74=.bf05a80b-f10b-417e-ba2d-76a86f0a3122@github.com>
 <My6HX9bdgQqZ-z1lqe2mTWpSFOaPUo8i7URUHC286wc=.cac28875-69b5-46df-b775-3089f72d77e8@github.com>
Message-ID: <OCiI_pQRDLMdLyRG2vaSx9Qod7q3PI-hTUV5HchN7ho=.696a17c0-d8e9-4910-b6ca-843de861c074@github.com>

On Tue, 30 Apr 2024 12:03:47 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Do you mean to move the initializations on line 57 (and others in the files) to the `initialize` method?
>
> Or, just remove it. The initialise function already initialises the member, right?
> 
> But it's really a small nit. I like your variant more if it were uniquely applied.

FWIW, in an earlier comment I also mentioned that we really should take a pass over this class and clean the code in various ways.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584830726

From stefank at openjdk.org  Tue Apr 30 13:35:17 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 30 Apr 2024 13:35:17 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v7]
In-Reply-To: <refC7AZ2N2CUliGi9vFbyBGUBa4DUHfq2uQprflQkyc=.3e0b6a9d-af1d-4b31-b977-3abe9e068049@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <7TW9a7Vmnz0nIKq83rYx_VN13PXM9_9nD5iSMzGDfNw=.127fd0ff-ee60-40cf-9994-9a1e81bb5b27@github.com>
 <cUarXdCb6-oYJGuHFc6FRfca6QAtK0XTgGF4VSHRf8w=.05407d27-44f7-40fe-a0d5-56cfe87a6995@github.com>
 <refC7AZ2N2CUliGi9vFbyBGUBa4DUHfq2uQprflQkyc=.3e0b6a9d-af1d-4b31-b977-3abe9e068049@github.com>
Message-ID: <cRq_nZ9TCRp13GcideEjCD02aIudJU1akQqdblA3zLQ=.bf0caa26-815a-4384-83dc-f6c22cf38321@github.com>

On Tue, 30 Apr 2024 08:59:21 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/memory/virtualspace.cpp line 623:
>> 
>>> 621:   }
>>> 622:   // _nmt_flag is used internally by initialize_compressed_heap
>>> 623:   _nmt_flag = mtJavaHeap;
>> 
>> Nit, we use a mixture of directly accessing _nmt_flag and accessing it via getter. Hotspot seems to prefer getters/setters. Can we use setters here?
>
> The flag is not set/changed in other classes, so there is no need to have a `public set_nmt_flag()` member for it.
> All the changes to the flag can be done internally using the member directly.
> P.S.: There was already a setter but removed after a review comment.

> Hotspot seems to prefer getters/setters.

I don't think this is true. Maybe in some places, but I don't think we prefer to use setters/getters from inside classes. Maybe if we want to add some verification code, but otherwise I tend to prefer using the members directly.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/18745#discussion_r1584827626

From zgu at openjdk.org  Tue Apr 30 13:46:04 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Tue, 30 Apr 2024 13:46:04 GMT
Subject: RFR: 8331405: Shenandoah: Optimize ShenandoahLock with TTAS
In-Reply-To: <UDTvGn-vj89MGIceH4coaImzWYarTRENxxCXaDc3xGI=.2771ae59-73a8-481a-a49f-b6321eebd5e5@github.com>
References: <UDTvGn-vj89MGIceH4coaImzWYarTRENxxCXaDc3xGI=.2771ae59-73a8-481a-a49f-b6321eebd5e5@github.com>
Message-ID: <YzN5wh0A1u0ZkcJAaK_EmwnFgk7ui1012_ZWEjmg9kc=.01304cf7-4286-4089-8c0d-5943263d9e2c@github.com>

On Tue, 30 Apr 2024 10:29:53 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> JDK-8325587 made the contended lock acquisition as the CAS loop with backoffs. Unfortunately, for `SpinPause` path, we always do the CAS and then spin. This is not efficient, as CAS likely requests memory for ownership, which exacerbates the contention and makes the spin loop run longer before blocking.
> 
> We should really use TTAS pattern there, like `Thread::SpinAcquire` does it.
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah`
>  - [ ] Linux AArch64 server fastdebug, `all` with `-XX:+UseShenandoahGC`

LGTM

-------------

Marked as reviewed by zgu (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19018#pullrequestreview-2031412587

From stuefe at openjdk.org  Tue Apr 30 14:26:15 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 30 Apr 2024 14:26:15 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v14]
In-Reply-To: <KgHgK3df3JiH1TZXsY2D4fP9RdAeV_mDew59NW6svus=.9532f39a-bd32-4603-8031-ffad117b2844@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <KgHgK3df3JiH1TZXsY2D4fP9RdAeV_mDew59NW6svus=.9532f39a-bd32-4603-8031-ffad117b2844@github.com>
Message-ID: <n5jwCNKO5zCJKKyo5NR7MyTE8G7Gt7OWK8umJtsefeE=.c31f2174-40be-4f9d-bd93-85a35ce0f4db@github.com>

On Tue, 30 Apr 2024 09:30:31 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> `MEMFLAGS flag` is used to hold/show the type of the memory regions in NMT. Each call of NMT API requires a search through the list of memory regions.
>> The Hotspot code reserves/commits/uncommits memory regions and later calls explicitly NMT API with a specific memory type (e.g., `mtGC`, `mtJavaHeap`) for that region.  Therefore, there are two search in the list of regions per reserve/commit/uncommit operations, one for the operation and another for setting the type of the region.  
>> When the memory type is passed in during reserve/commit/uncommit operations, NMT can use it and avoid the extra search for setting the memory type.
>> 
>> Tests: tiers1-5 passed on linux-x64, macosx-aarch64 and windows-x64 for debug and non-debug builds.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   comments applied.

Very good, lets ship this. Any remaining concerns, if there are any, we can address in subsequent RFEs.

-------------

Marked as reviewed by stuefe (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/18745#pullrequestreview-2031528992

From azafari at openjdk.org  Tue Apr 30 15:22:21 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 30 Apr 2024 15:22:21 GMT
Subject: RFR: 8330076: NMT: add/make a mandatory MEMFLAGS argument to
 family of os::reserve/commit/uncommit memory API [v13]
In-Reply-To: <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
References: <5GDKVVPITIzIcyfm-0tKOFzFIEPBgzOe-or1eX_POns=.a5205641-139b-4749-afcc-57ddbc85e6be@github.com>
 <xHdkCGFU8qlQG3YxOw0RXoJE1KGx5fZSWNhRL9LVnqU=.30a49f81-662d-49ad-b959-a05969f4be04@github.com>
 <oMvmwKAc7V3dY4yHcYEBG-nf2fFwetwEJJCx7gqbsfo=.080f0521-2db0-4211-b884-d565ec6bf80c@github.com>
Message-ID: <2qU9ixg7RaT5D5_Ct2ecqd4t5TS9ybjVBus8yEHeubo=.10078568-91ca-42e3-8b73-fede80eab78b@github.com>

On Tue, 23 Apr 2024 06:31:30 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removed extra blank line.
>
> This is a big change, but the pattern of the changes is quite easy to follow.
> 
> I do have a couple of queries below.
> 
> Thanks

@dholmes-ora, I am not sure if you got all your comments addressed. Would you please, have a look at here? 
Thanks.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18745#issuecomment-2085638740

From kdnilsen at openjdk.org  Tue Apr 30 17:40:47 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 30 Apr 2024 17:40:47 GMT
Subject: RFR: 8324649: Shenandoah: replace implementation of free set [v53]
In-Reply-To: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
References: <VAkn0SrKYZWmnflpM8ZwVAXjUF1LjA0_FCBmRJJ4VSc=.c2f979bd-2957-434e-95ca-7b11b1573900@github.com>
Message-ID: <JbXmdpPQXWR9HxvnZrLfYpeMmrIfuGUvFiGXGnREKYY=.be90f6c2-14d0-4c4c-aec5-787835015111@github.com>

> Several objectives:
> 1. Reduce humongous allocation failures by segregating regular regions from humongous regions
> 2. Do not retire regions just because an allocation failed within the region if the memory remaining within the region is large enough to represent a LAB
> 3. Track range of empty regions in addition to range of available regions in order to expedite humongous allocations
> 4. Treat collector reserves as available for Mutator allocations after evacuation completes
> 5. Improve encapsulation so as to enable an OldCollector reserve for future integration of generational Shenandoah
> 
> We have compared performance of existing FreeSet implementation with the proposed PR over a broad set of performance workloads and see that the impact is mostly neutral.
> 
> Comparing 105235.0 metrics from control, 220638.0 from experiment.
> Compare: 0.589s
>                           Most impacted benchmarks |                              Most impacted metrics
> -------------------------------------------------------------------------------------------------------
>                                  Shenandoah/jython |                                          cwr_total
> 
> 
>                                 Only in experiment |                                    Only in control
> -------------------------------------------------------------------------------------------------------
>                  crypto.signverify/trigger_failure |                        crypto.rsa/cmr_thread_roots
>                 extremem-large-31g/adjust_pointers |       scimark.sparse.small/concurrent_thread_roots
>             extremem-large-31g/calculate_addresses |              xml.transform/concurrent_thread_roots
>       crypto.signverify/class_unloading_rendezvous |                    mpegaudio/concurrent_weak_roots
>                                   serial/cmr_total |                        crypto.rsa/ctr_thread_roots
> 
> Shenandoah
> -------------------------------------------------------------------------------------------------------
> +5.64% jython/cwr_total p=0.00037
>   Control:      1.928ms (+/-272.40us)        170
>   Test:         2.037ms (+/-322.73us)        344

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 114 commits:

 - Merge remote-tracking branch 'origin/master' into restructure-free-set
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Remove unnecessary call to update_watermark
 - Assert progress on find_next and find_prev
 - Simplify partition_membership_name by code reuse
 - Simplify by combining implemnetations of shrink_interval functions
 - Fix NumPartition type
   
   Beautify the code by changing type of NumPartitions and adding Int and
   UInt forms of NumPartitions.
 - Refinements to support zero-build compiles
 - Fix whitespace
 - ... and 104 more: https://git.openjdk.org/jdk/compare/a863ef5d...d6e3546c

-------------

Changes: https://git.openjdk.org/jdk/pull/17561/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17561&range=52
  Stats: 2523 lines in 8 files changed: 2139 ins; 189 del; 195 mod
  Patch: https://git.openjdk.org/jdk/pull/17561.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17561/head:pull/17561

PR: https://git.openjdk.org/jdk/pull/17561

From wkemper at openjdk.org  Tue Apr 30 18:18:51 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 30 Apr 2024 18:18:51 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <yU5WbOCoZM-iA7KYFKfydf2n0xUqDfSGkJES7sOJxik=.283adb72-e3da-4253-b5d6-fdc490c22e3c@github.com>
References: <yU5WbOCoZM-iA7KYFKfydf2n0xUqDfSGkJES7sOJxik=.283adb72-e3da-4253-b5d6-fdc490c22e3c@github.com>
Message-ID: <8XcFJVZa1yGIwu869zFzWs9NZ4-FU8lc9thGlLP_WgE=.f2d19666-aeb4-4d2d-bfb6-b65f0bd29683@github.com>

> Merges tag jdk-23+19

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/425/files
  - new: https://git.openjdk.org/shenandoah/pull/425/files/706b421c..706b421c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=425&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=425&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/425.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/425/head:pull/425

PR: https://git.openjdk.org/shenandoah/pull/425

From ysr at openjdk.org  Tue Apr 30 19:59:02 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 30 Apr 2024 19:59:02 GMT
Subject: RFR: 8331405: Shenandoah: Optimize ShenandoahLock with TTAS
In-Reply-To: <UDTvGn-vj89MGIceH4coaImzWYarTRENxxCXaDc3xGI=.2771ae59-73a8-481a-a49f-b6321eebd5e5@github.com>
References: <UDTvGn-vj89MGIceH4coaImzWYarTRENxxCXaDc3xGI=.2771ae59-73a8-481a-a49f-b6321eebd5e5@github.com>
Message-ID: <amhKGnHWJ3yuMV0UztstDW0nZE4XqJDHlBFbml8013k=.93124e7e-bf28-42e7-b144-95d1d336dbba@github.com>

On Tue, 30 Apr 2024 10:29:53 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> JDK-8325587 made the contended lock acquisition as the CAS loop with backoffs. Unfortunately, for `SpinPause` path, we always do the CAS and then spin. This is not efficient, as CAS likely requests memory for ownership, which exacerbates the contention and makes the spin loop run longer before blocking.
> 
> We should really use TTAS pattern there, like `Thread::SpinAcquire` does it.
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, `hotspot_gc_shenandoah`
>  - [ ] Linux AArch64 server fastdebug, `all` with `-XX:+UseShenandoahGC`

Change looks good; reviewed.

Can you share any comparative performance data, may be?

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19018#pullrequestreview-2032408851

From wkemper at openjdk.org  Tue Apr 30 21:41:17 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 30 Apr 2024 21:41:17 GMT
Subject: RFR: 8328307: GenShen: Re-enable old-has-grown trigger for
 old-generation GC [v2]
In-Reply-To: <BJ-F8QnBugVcMZ8FKsBe3UeQexHTHBVBpHPr2ILE_S8=.d3e22219-60af-49da-a28a-d4b156af4722@github.com>
References: <DKYYYui8zmVtGZCscDQfc5Gxio5NFVvMbxMbGeYSVLg=.e99b513e-354a-47c3-bb6a-73ad6d4e96fe@github.com>
 <BJ-F8QnBugVcMZ8FKsBe3UeQexHTHBVBpHPr2ILE_S8=.d3e22219-60af-49da-a28a-d4b156af4722@github.com>
Message-ID: <Z-AthYNIBd2JMalrpOyONbiksh9iEMkB0dxa2i55CD8=.dd77225f-d4df-4752-a573-d6ca2609490f@github.com>

On Tue, 30 Apr 2024 02:58:47 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Reviewed-by: wkemper, ysr
>> 
>> This pull request contains a backport of commit bfb798a6 from the openjdk/shenandoah repository.
>> 
>> The commit being backported was authored by Kelvin Nilsen on 29 Apr 2024 and was reviewed by William Kemper and
>> Y. Srinivas Ramakrishna.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits:
> 
>  - Merge branch 'master' of https://git.openjdk.org/shenandoah-jdk21u into backport-kdnilsen-bfb798a6
>  - 8328307: GenShen: Re-enable old-has-grown trigger for old-generation GC
>    
>    Reviewed-by: wkemper, ysr

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/41#pullrequestreview-2032565854