From ysr at openjdk.org  Fri Sep  1 00:38:17 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 1 Sep 2023 00:38:17 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v4]
In-Reply-To: <bRtf0tMn5iUcB2UkfVwyrL6JoEf0BzQjy3mvM5vh3So=.329ff53b-0234-407f-a2d2-c2c618506b87@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
 <bRtf0tMn5iUcB2UkfVwyrL6JoEf0BzQjy3mvM5vh3So=.329ff53b-0234-407f-a2d2-c2c618506b87@github.com>
Message-ID: <hSFYfozfB2SLdOh28L1qgBX_OKgYlGsnsi8jKAhRimo=.9e458cef-d98f-4751-b1d7-52cfababddfa@github.com>

On Wed, 30 Aug 2023 18:51:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - Replace is_generational with ShenandoahCardBarrier test
>  - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
>  - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
>  - Also update card table when moving discovered References to global list
>  - Fix whitespace
>  - Mark card as dirty if discovered reference list has interesting pointer

changes look good; minor suggestion on some potential refactor/consolidation with existing card marking code.

src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 387:

> 385:       T* addr = reinterpret_cast<T*>(java_lang_ref_Reference::discovered_addr_raw(reference));
> 386:       if (heap->is_in_old(addr) && heap->is_in_young(discovered_head)) {
> 387:         heap->mark_card_as_dirty(addr);

Can you instead call the existing `card_mark_barrier` method?
At that time, could also look at the existing uses of that method and refactor them to use the `ShenandoahCardBarrier` test like you did above, and elide the generational check in the body of the method itself, replacing it with an assertion (if you want, that the heap is generational).

(PS: I was also thinking that in the fullness of time it might be worth doing an audit of the uses of `cas` and `set_oop` and making sure they would call the card-marking code instead of exposing the functionality at the call-sites like done here. Callers that want to avoid card-marking would need to call the respective `_raw` versions then. We should makr sure to conform to naming conventions for those methods that are consistent with those used in other parts of jvm/gc code. This can be done separately and might affect legacy Shenandoah code as well. Just leaving this comment here so as not to lose the thought, but I doubt we want to do that locally here in this PR.)

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/314#pullrequestreview-1605940006
PR Review Comment: https://git.openjdk.org/shenandoah/pull/314#discussion_r1312414190

From kdnilsen at openjdk.org  Fri Sep  1 16:41:34 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 1 Sep 2023 16:41:34 GMT
Subject: RFR: JDK-8315478: GenShen: Tolerate round-off errors in preselected
 promotion budget
Message-ID: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>

An assert failure identified an error in previous implementation.  See JBS ticket for description of issue and reproducer.

-------------

Commit messages:
 - Remove instrumentation in place for debugging
 - Add instrumentation to confirm root cause of problem

Changes: https://git.openjdk.org/shenandoah/pull/317/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=317&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315478
  Stats: 20 lines in 1 file changed: 19 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/317.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/317/head:pull/317

PR: https://git.openjdk.org/shenandoah/pull/317

From kdnilsen at openjdk.org  Fri Sep  1 17:19:47 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 1 Sep 2023 17:19:47 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting collection
 set during GLOBAL GC
Message-ID: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>

With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.

Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.

However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.

-------------

Commit messages:
 - Fix whitespace
 - Remove instrumentation
 - Keep promo budget at zero during GLOBAL GC
 - Let choose_collection_set() for GLOBAL gc expand old on demand

Changes: https://git.openjdk.org/shenandoah/pull/318/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=318&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315479
  Stats: 81 lines in 2 files changed: 66 ins; 3 del; 12 mod
  Patch: https://git.openjdk.org/shenandoah/pull/318.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/318/head:pull/318

PR: https://git.openjdk.org/shenandoah/pull/318

From kdnilsen at openjdk.org  Fri Sep  1 20:15:17 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 1 Sep 2023 20:15:17 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v4]
In-Reply-To: <hSFYfozfB2SLdOh28L1qgBX_OKgYlGsnsi8jKAhRimo=.9e458cef-d98f-4751-b1d7-52cfababddfa@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
 <bRtf0tMn5iUcB2UkfVwyrL6JoEf0BzQjy3mvM5vh3So=.329ff53b-0234-407f-a2d2-c2c618506b87@github.com>
 <hSFYfozfB2SLdOh28L1qgBX_OKgYlGsnsi8jKAhRimo=.9e458cef-d98f-4751-b1d7-52cfababddfa@github.com>
Message-ID: <n6mjB576TXKLpd6cZgdCNlFUpdhNFYOHIDib46R3XKE=.b76242e0-de14-45c9-b97d-308ab1439736@github.com>

On Fri, 1 Sep 2023 00:34:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - Replace is_generational with ShenandoahCardBarrier test
>>  - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
>>  - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
>>  - Also update card table when moving discovered References to global list
>>  - Fix whitespace
>>  - Mark card as dirty if discovered reference list has interesting pointer
>
> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 387:
> 
>> 385:       T* addr = reinterpret_cast<T*>(java_lang_ref_Reference::discovered_addr_raw(reference));
>> 386:       if (heap->is_in_old(addr) && heap->is_in_young(discovered_head)) {
>> 387:         heap->mark_card_as_dirty(addr);
> 
> Can you instead call the existing `card_mark_barrier` method?
> At that time, could also look at the existing uses of that method and refactor them to use the `ShenandoahCardBarrier` test like you did above, and elide the generational check in the body of the method itself, replacing it with an assertion (if you want, that the heap is generational).
> 
> (PS: I was also thinking that in the fullness of time it might be worth doing an audit of the uses of `cas` and `set_oop` and making sure they would call the card-marking code instead of exposing the functionality at the call-sites like done here. Callers that want to avoid card-marking would need to call the respective `_raw` versions then. We should makr sure to conform to naming conventions for those methods that are consistent with those used in other parts of jvm/gc code. This can be done separately and might affect legacy Shenandoah code as well. Just leaving this comment here so as not to lose the thought, but I doubt we want to do that locally here in this PR.)

Made this change and am testing result.

I agree that audit/refactoring in the future would be a good idea.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/314#discussion_r1313473253

From kdnilsen at openjdk.org  Fri Sep  1 20:33:25 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 1 Sep 2023 20:33:25 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v5]
In-Reply-To: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
Message-ID: <FDGcQp4nPHNDIGmroaKnax0xiWdiB8M6uBxYKMWlFoE=.d861446c-8757-4eeb-8cd2-195a534f4225@github.com>

> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Use card_mark_barrier() method and ShenandoahCardBarrier tests

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/314/files
  - new: https://git.openjdk.org/shenandoah/pull/314/files/f375d571..2618ff3c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=04
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=03-04

  Stats: 11 lines in 1 file changed: 5 ins; 2 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/314.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/314/head:pull/314

PR: https://git.openjdk.org/shenandoah/pull/314

From kdnilsen at openjdk.org  Fri Sep  1 20:36:20 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 1 Sep 2023 20:36:20 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v4]
In-Reply-To: <n6mjB576TXKLpd6cZgdCNlFUpdhNFYOHIDib46R3XKE=.b76242e0-de14-45c9-b97d-308ab1439736@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
 <bRtf0tMn5iUcB2UkfVwyrL6JoEf0BzQjy3mvM5vh3So=.329ff53b-0234-407f-a2d2-c2c618506b87@github.com>
 <hSFYfozfB2SLdOh28L1qgBX_OKgYlGsnsi8jKAhRimo=.9e458cef-d98f-4751-b1d7-52cfababddfa@github.com>
 <n6mjB576TXKLpd6cZgdCNlFUpdhNFYOHIDib46R3XKE=.b76242e0-de14-45c9-b97d-308ab1439736@github.com>
Message-ID: <xwttojgF4GqqPP4-rUTnK7eX0EDkB5FW6McDa51a3t0=.6a236e8c-c7df-410b-a3d5-2b008b2e4f5f@github.com>

On Fri, 1 Sep 2023 20:12:58 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 387:
>> 
>>> 385:       T* addr = reinterpret_cast<T*>(java_lang_ref_Reference::discovered_addr_raw(reference));
>>> 386:       if (heap->is_in_old(addr) && heap->is_in_young(discovered_head)) {
>>> 387:         heap->mark_card_as_dirty(addr);
>> 
>> Can you instead call the existing `card_mark_barrier` method?
>> At that time, could also look at the existing uses of that method and refactor them to use the `ShenandoahCardBarrier` test like you did above, and elide the generational check in the body of the method itself, replacing it with an assertion (if you want, that the heap is generational).
>> 
>> (PS: I was also thinking that in the fullness of time it might be worth doing an audit of the uses of `cas` and `set_oop` and making sure they would call the card-marking code instead of exposing the functionality at the call-sites like done here. Callers that want to avoid card-marking would need to call the respective `_raw` versions then. We should makr sure to conform to naming conventions for those methods that are consistent with those used in other parts of jvm/gc code. This can be done separately and might affect legacy Shenandoah code as well. Just leaving this comment here so as not to lose the thought, but I doubt we want to do that locally here in this PR.)
>
> Made this change and am testing result.
> 
> I agree that audit/refactoring in the future would be a good idea.

I've made your suggested changes and am sending through the test pipeline again.

I agree that it would be good to eventually audit/refactor to have more consistent method names and conventions.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/314#discussion_r1313492965

From wkemper at openjdk.org  Fri Sep  1 21:00:17 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 1 Sep 2023 21:00:17 GMT
Subject: RFR: JDK-8315478: GenShen: Tolerate round-off errors in
 preselected promotion budget
In-Reply-To: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
References: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
Message-ID: <e3u5s0zDpep4WyjH2eCtChYBxgS2llObTxQCIkKcKLE=.f84b35a6-5bd5-4943-ad4f-47aab9260125@github.com>

On Fri, 1 Sep 2023 16:33:20 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An assert failure identified an error in previous implementation.  See JBS ticket for description of issue and reproducer.

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/317#pullrequestreview-1607660004

From wkemper at openjdk.org  Fri Sep  1 21:05:17 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 1 Sep 2023 21:05:17 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC
In-Reply-To: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
Message-ID: <8nOgmDvTu53PFVqQMtvP6enxkWDpw94yKBV64AV8fLQ=.4679c314-1cff-4e8f-8088-bef3e3a93f91@github.com>

On Fri, 1 Sep 2023 16:49:35 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
> 
> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
> 
> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.

A couple of minor nits.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahGlobalHeuristics.cpp line 128:

> 126:     ShenandoahHeapRegion* r = data[idx]._region;
> 127:     if (cset->is_preselected(r->index())) {
> 128:       assert(false, "There should be no preselected regions during GLOBAL GC");

Slightly more readable:

 fatal("There should be no preselected regions during GLOBAL GC");

src/hotspot/share/gc/shenandoah/heuristics/shenandoahGlobalHeuristics.cpp line 146:

> 144:       }
> 145:     } else {
> 146:       // r->is_young() && (r->age() < tenuring_threshold)

`assert`?

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/318#pullrequestreview-1607661498
PR Review Comment: https://git.openjdk.org/shenandoah/pull/318#discussion_r1313517316
PR Review Comment: https://git.openjdk.org/shenandoah/pull/318#discussion_r1313518478

From ysr at openjdk.org  Fri Sep  1 21:26:11 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 1 Sep 2023 21:26:11 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v5]
In-Reply-To: <FDGcQp4nPHNDIGmroaKnax0xiWdiB8M6uBxYKMWlFoE=.d861446c-8757-4eeb-8cd2-195a534f4225@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
 <FDGcQp4nPHNDIGmroaKnax0xiWdiB8M6uBxYKMWlFoE=.d861446c-8757-4eeb-8cd2-195a534f4225@github.com>
Message-ID: <kcdV0SP9yf2GLtVamcjETr_-WWJXlCd5nKP_BwteTPM=.e8beb36d-a061-436e-a058-2ca8f4888105@github.com>

On Fri, 1 Sep 2023 20:33:25 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use card_mark_barrier() method and ShenandoahCardBarrier tests

One more change, I think? See comment in-line.

May be there is an issue with it though that I am not seeing which perhaps you might have tried already, but doesn't work? 

Rest looks good.

src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 590:

> 588:     if (heap->is_in_old(_pending_list_tail) && heap->is_in_young(former_head_of_global_list)) {
> 589:       heap->mark_card_as_dirty(_pending_list_tail);
> 590:     }

Can we just replace this section with `set_oop_field(_pending_list_tail, former_head_of_global_list)` ?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/314#pullrequestreview-1607677926
PR Review Comment: https://git.openjdk.org/shenandoah/pull/314#discussion_r1313537864

From ysr at openjdk.org  Sat Sep  2 00:45:07 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 2 Sep 2023 00:45:07 GMT
Subject: RFR: JDK-8315478: GenShen: Tolerate round-off errors in
 preselected promotion budget
In-Reply-To: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
References: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
Message-ID: <e37ayM724jNwzG6-VEYhF3_YzBmtFWvyQ2CsTrwWy78=.cce352ac-a374-4bce-a836-b5fb1436b92f@github.com>

On Fri, 1 Sep 2023 16:33:20 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An assert failure identified an error in previous implementation.  See JBS ticket for description of issue and reproducer.

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 409:

> 407: 
> 408:   assert(old_available >= old_consumed, "Cannot consume (" SIZE_FORMAT ") more than is available (" SIZE_FORMAT ")",
> 409:          old_consumed, old_available);

So you are just adding debugging code to catch the issue more easily? (as well as the change further below)

I don't see any change in the control flow or the state here.

Is the intention to add this change and the one below to aid debugging of a very difficult to reproduce problem?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/317#discussion_r1313661996

From ysr at openjdk.org  Sat Sep  2 00:50:14 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 2 Sep 2023 00:50:14 GMT
Subject: RFR: JDK-8315478: GenShen: Tolerate round-off errors in
 preselected promotion budget
In-Reply-To: <e37ayM724jNwzG6-VEYhF3_YzBmtFWvyQ2CsTrwWy78=.cce352ac-a374-4bce-a836-b5fb1436b92f@github.com>
References: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
 <e37ayM724jNwzG6-VEYhF3_YzBmtFWvyQ2CsTrwWy78=.cce352ac-a374-4bce-a836-b5fb1436b92f@github.com>
Message-ID: <dwoLb4WXorXx4muY-oAk7BJaYaQIIDV0PmOgJ5or7aQ=.44a7f8a5-a284-47e9-ab45-25562be48a76@github.com>

On Sat, 2 Sep 2023 00:42:04 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> An assert failure identified an error in previous implementation.  See JBS ticket for description of issue and reproducer.
>
> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 409:
> 
>> 407: 
>> 408:   assert(old_available >= old_consumed, "Cannot consume (" SIZE_FORMAT ") more than is available (" SIZE_FORMAT ")",
>> 409:          old_consumed, old_available);
> 
> So you are just adding debugging code to catch the issue more easily? (as well as the change further below)
> 
> I don't see any change in the control flow or the state here.
> 
> Is the intention to add this change and the one below to aid debugging of a very difficult to reproduce problem?

ah, never mind, I see that you are adjusting some values there, which I missed in a first reading of the code.

I don't understand this logic well enough to review this change, so please go ahead an land this with the review you have from William.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/317#discussion_r1313662838

From ysr at openjdk.org  Sat Sep  2 01:02:12 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 2 Sep 2023 01:02:12 GMT
Subject: RFR: JDK-8315478: GenShen: Tolerate round-off errors in
 preselected promotion budget
In-Reply-To: <dwoLb4WXorXx4muY-oAk7BJaYaQIIDV0PmOgJ5or7aQ=.44a7f8a5-a284-47e9-ab45-25562be48a76@github.com>
References: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
 <e37ayM724jNwzG6-VEYhF3_YzBmtFWvyQ2CsTrwWy78=.cce352ac-a374-4bce-a836-b5fb1436b92f@github.com>
 <dwoLb4WXorXx4muY-oAk7BJaYaQIIDV0PmOgJ5or7aQ=.44a7f8a5-a284-47e9-ab45-25562be48a76@github.com>
Message-ID: <cCXCwjpe0WGuIADljLi9GTL5Ub8ojRzOvRjxZns6BU0=.0f84b130-9182-4ce8-9fe1-865e4b45e48a@github.com>

On Sat, 2 Sep 2023 00:46:22 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 409:
>> 
>>> 407: 
>>> 408:   assert(old_available >= old_consumed, "Cannot consume (" SIZE_FORMAT ") more than is available (" SIZE_FORMAT ")",
>>> 409:          old_consumed, old_available);
>> 
>> So you are just adding debugging code to catch the issue more easily? (as well as the change further below)
>> 
>> I don't see any change in the control flow or the state here.
>> 
>> Is the intention to add this change and the one below to aid debugging of a very difficult to reproduce problem?
>
> ah, never mind, I see that you are adjusting some values there, which I missed in a first reading of the code.
> 
> I don't understand this logic well enough to review this change, so please go ahead an land this with the review you have from William.

old_consumed = old_evacuated_committed + old_available - old_evacuated_committed
                             = old_available

So effectively this is saying if `old_available < old_consumed`, make `old_available = old_consumed` so the subsequent assertion is satisfied. You are asserting that this can only happen if the difference is because of round-off errors.

I do have a question about this, though. Can we avoid the round-off errors in the arithmetic entirely, by doing rounding in the "safe direction" (e.g. rounding down or up) at each step where rounding is needed, rather than relying on catching the condition and adjusting the values to re-establish the invariants?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/317#discussion_r1313672240

From ysr at openjdk.org  Sat Sep  2 01:28:46 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 2 Sep 2023 01:28:46 GMT
Subject: RFR: 8314935: Shenandoah: Unable to throw OOME on back-to-back
 Full GCs
In-Reply-To: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
References: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
Message-ID: <Koc9GsByrURarK3FIkG7Vu3YbVt9Oun9t0XO7qbckfA=.c7c2a880-9244-4958-84d8-8d8942ca46cd@github.com>

On Wed, 30 Aug 2023 23:33:57 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Retry allocations until at least one full gc is complete, do not spin unnecessarily when gc is not making progress.

A couple of comments about harmonizing the code in GenShen tip with that here, but otherwise looks good.

src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 46:

> 44:   volatile size_t _success_full_gcs;
> 45:   size_t _alloc_failure_degenerated;
> 46:   size_t _alloc_failure_degenerated_upgrade_to_full;

In GenShen tip, `_alloc_failure_degenerated_upgrade_to_full` is also declared volatile.

src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 87:

> 85:   void print_gc_stats(outputStream* out) const;
> 86: 
> 87:   size_t full_gc_count() const {

Is it worthwhile to harmonize this with tip and rename to `get_fullgc_count()` ?
Unless the idea is to change genshen tip to use `full_gc_count()` like here when you pull these changes down.

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15500#pullrequestreview-1607810441
PR Review Comment: https://git.openjdk.org/jdk/pull/15500#discussion_r1313674186
PR Review Comment: https://git.openjdk.org/jdk/pull/15500#discussion_r1313677385

From lkorinth at openjdk.org  Mon Sep  4 11:04:44 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Mon, 4 Sep 2023 11:04:44 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
Message-ID: <5o5B7LbCQN_C9xzd1EvrvTp04-6Atr0gih5WH69LeK4=.3a977034-8fe9-4da8-a167-f5dad3a97d75@github.com>

On Wed, 30 Aug 2023 09:23:55 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix static import

I have created an alternative that uses enums to force the user to make a decision: https://github.com/openjdk/jdk/compare/master...lkorinth:jdk:+process_tools . Another alternative is to do the same but instead using an enum (I think it is not as good). A third alternative is to use the current pull request with a better name.

What do you prefer? Do you have a better alternative? Do someone still think the current code is good? I think what we have today is inferior to all these improvements, and I would like to make it harder to develop bad test cases.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1705068700

From kdnilsen at openjdk.org  Tue Sep  5 17:18:19 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 5 Sep 2023 17:18:19 GMT
Subject: RFR: JDK-8315478: GenShen: Tolerate round-off errors in
 preselected promotion budget
In-Reply-To: <cCXCwjpe0WGuIADljLi9GTL5Ub8ojRzOvRjxZns6BU0=.0f84b130-9182-4ce8-9fe1-865e4b45e48a@github.com>
References: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
 <e37ayM724jNwzG6-VEYhF3_YzBmtFWvyQ2CsTrwWy78=.cce352ac-a374-4bce-a836-b5fb1436b92f@github.com>
 <dwoLb4WXorXx4muY-oAk7BJaYaQIIDV0PmOgJ5or7aQ=.44a7f8a5-a284-47e9-ab45-25562be48a76@github.com>
 <cCXCwjpe0WGuIADljLi9GTL5Ub8ojRzOvRjxZns6BU0=.0f84b130-9182-4ce8-9fe1-865e4b45e48a@github.com>
Message-ID: <YdaXXC0feeHiwmVwP_KDgXGWBsdXpT2PqBIv93LE21k=.860e5a40-eb03-4ce1-aa25-155269a0ade8@github.com>

On Sat, 2 Sep 2023 00:59:31 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> ah, never mind, I see that you are adjusting some values there, which I missed in a first reading of the code.
>> 
>> I don't understand this logic well enough to review this change, so please go ahead an land this with the review you have from William.
>
> old_consumed = old_evacuated_committed + old_available - old_evacuated_committed
>                              = old_available
> 
> So effectively this is saying if `old_available < old_consumed`, make `old_available = old_consumed` so the subsequent assertion is satisfied. You are asserting that this can only happen if the difference is because of round-off errors.
> 
> I do have a question about this, though. Can we avoid the round-off errors in the arithmetic entirely, by doing rounding in the "safe direction" (e.g. rounding down or up) at each step where rounding is needed, rather than relying on catching the condition and adjusting the values to re-establish the invariants?

It would be possible to do all round-off arithmetic more conservatively, to avoid the need for this check.  The "problem" arises when the sum of individual round-off errors is different than the single round-off error for the sum of promotions.  My preference would be to fix this in a separate PR if we choose to do so.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/317#discussion_r1316175884

From kdnilsen at openjdk.org  Tue Sep  5 17:44:20 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 5 Sep 2023 17:44:20 GMT
Subject: RFR: JDK-8315478: GenShen: Tolerate round-off errors in
 preselected promotion budget
In-Reply-To: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
References: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
Message-ID: <DwH7I8AgMUK7UCemhigwfP2bbhy17lUuKju1rjuP1rw=.2557bbc4-8386-4648-8c24-cc14d7f55354@github.com>

On Fri, 1 Sep 2023 16:33:20 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An assert failure identified an error in previous implementation.  See JBS ticket for description of issue and reproducer.

Though all GHA checks have passed, there are still some failures on this branch in internal CI pipelines.  Specifically:

1. perf_specjbb (2 of 5 runs) encountered:

# A fatal error has been detected by the Java Runtime Environment
#  SIGSEGV (0xb) at pc=0x00007fdc7a8f3939, pid=15605, tid=15607
# JRE version: OpenJDK Runtime Environment (22.0) (build 22-root-tolerate-roundoff-with-preselected-promotion-gh-x86.5328a6)
# Java VM: OpenJDK 64-Bit Server VM (22-root-tolerate-roundoff-with-preselected-promotion-gh-x86.5328a6, mixed mode, sharing, compressed oops, compressed class ptrs, shenandoah gc, linux-amd64)
# Problematic frame:
  void ShenandoahScanRemembered<ShenandoahDirectCardMarkRememberedSet>::process_clusters<ShenandoahMarkRefsClosure<(ShenandoahGenerationType)2> >(unsigned long, unsigned long, HeapWordImpl**, ShenandoahMarkRefsClosure<(ShenandoahGenerationType)2>*, bool, unsigned int)+0x289

2. stress_dacapo (5 of 5 runs) encountered:

# A fatal error has been detected by the Java Runtime Environment:
#  Internal Error (/codebuild/output/src4035/src/s3/00/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp:1276), pid=11604, tid=11612
#  Error: Verify init-mark remembered set violation; clean card should be dirty

We believe the first of these is covered by issue: https://bugs.openjdk.org/browse/JDK-8315560

and the second is covered by issue: https://bugs.openjdk.org/browse/JDK-8315044

I will /integrate now, and address the known failures in separate efforts.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/317#issuecomment-1707038212
PR Comment: https://git.openjdk.org/shenandoah/pull/317#issuecomment-1707038938

From kdnilsen at openjdk.org  Tue Sep  5 17:59:29 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 5 Sep 2023 17:59:29 GMT
Subject: Integrated: JDK-8315478: GenShen: Tolerate round-off errors in
 preselected promotion budget
In-Reply-To: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
References: <qmpvWcwJNKKGSi9aGYou0j-OConO6ENTcyFHTUhNs1w=.813541db-680f-4f1c-a22e-de85b703a452@github.com>
Message-ID: <iquuqWyUywLFKAvPavTqM2t-rK08aBsbQWsgolZ4ALw=.3930a4b6-288c-40b7-a0c4-22375ddf87d8@github.com>

On Fri, 1 Sep 2023 16:33:20 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An assert failure identified an error in previous implementation.  See JBS ticket for description of issue and reproducer.

This pull request has now been integrated.

Changeset: 2d76fb13
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/2d76fb13f3064146b4e74d8518e1f97d33582e80
Stats:     20 lines in 1 file changed: 19 ins; 0 del; 1 mod

8315478: GenShen: Tolerate round-off errors in preselected promotion budget

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/317

From kdnilsen at openjdk.org  Tue Sep  5 18:00:33 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 5 Sep 2023 18:00:33 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC
In-Reply-To: <8nOgmDvTu53PFVqQMtvP6enxkWDpw94yKBV64AV8fLQ=.4679c314-1cff-4e8f-8088-bef3e3a93f91@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
 <8nOgmDvTu53PFVqQMtvP6enxkWDpw94yKBV64AV8fLQ=.4679c314-1cff-4e8f-8088-bef3e3a93f91@github.com>
Message-ID: <9-3_WIrLTWABbVjD3wXNwM_L9ZBiXfejj6evJ1QdHKs=.8d92f93e-8027-4382-b236-aebe2644bf49@github.com>

On Fri, 1 Sep 2023 20:59:09 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
>> 
>> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
>> 
>> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahGlobalHeuristics.cpp line 128:
> 
>> 126:     ShenandoahHeapRegion* r = data[idx]._region;
>> 127:     if (cset->is_preselected(r->index())) {
>> 128:       assert(false, "There should be no preselected regions during GLOBAL GC");
> 
> Slightly more readable:
> 
>  fatal("There should be no preselected regions during GLOBAL GC");

Thanks.  I'll make this change.

> src/hotspot/share/gc/shenandoah/heuristics/shenandoahGlobalHeuristics.cpp line 146:
> 
>> 144:       }
>> 145:     } else {
>> 146:       // r->is_young() && (r->age() < tenuring_threshold)
> 
> `assert`?

I'll make this change also.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/318#discussion_r1316219510
PR Review Comment: https://git.openjdk.org/shenandoah/pull/318#discussion_r1316219739

From kdnilsen at openjdk.org  Tue Sep  5 18:07:27 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 5 Sep 2023 18:07:27 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC
In-Reply-To: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
Message-ID: <lXNNSkq3XurTbu-YOXIa9gvNlr04TpJN5U1yUjQVpUY=.344fd8b5-5c50-4422-b759-cd36e9880cb5@github.com>

On Fri, 1 Sep 2023 16:49:35 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
> 
> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
> 
> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.

Though GHA checks have all passed, our internal CI pipelines have encountered a few problems:
1. stress_dacapo (5 of 5 runs) encounters:

#  Error: Verify init-mark remembered set violation; clean card should be dirty

This issue is being addressed under separate ticket: https://bugs.openjdk.org/browse/JDK-8315044

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/318#issuecomment-1707070009

From rriggs at openjdk.org  Tue Sep  5 18:08:39 2023
From: rriggs at openjdk.org (Roger Riggs)
Date: Tue, 5 Sep 2023 18:08:39 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <5o5B7LbCQN_C9xzd1EvrvTp04-6Atr0gih5WH69LeK4=.3a977034-8fe9-4da8-a167-f5dad3a97d75@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
 <5o5B7LbCQN_C9xzd1EvrvTp04-6Atr0gih5WH69LeK4=.3a977034-8fe9-4da8-a167-f5dad3a97d75@github.com>
Message-ID: <CZcrmy99xK8GvWF8BD1TmmM_Q8F0po6d3zExg2zNtSY=.9d126ffd-f3ad-4257-a0e4-0a83e3e17a47@github.com>

On Mon, 4 Sep 2023 11:01:23 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

> What do you prefer? Do you have a better alternative? Do someone still think the current code is good? I think what we have today is inferior to all these improvements, and I would like to make it harder to develop bad test ca

The current API (name) is fine and fit for purpose; it does not promise or hide extra functionality under a simple name.

There needs to be an explicit intention in the test(s) to support after the fact that arbitrary flags can be added.
@AlanBateman's proposal for naming [above](https://github.com/openjdk/jdk/pull/15452#issuecomment-1700459277) (or similar) would capture more clearly that test options are propagated to the child process.
Every test writer should be aware that additional command line options may be mixed in.

There are many cases in which the ProcessTools APIs are not used to create child processes and do not need to be used in writing tests. They provide some convenience but also add a dependency and another API layer to work through in the case of failures.

As far as I'm aware, there is no general guidance or design pattern outside of hotspot tests to propagate flags or use ProcessTools. Adding that as a requirement will need a different level of communication and change.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1707072375

From kdnilsen at openjdk.org  Tue Sep  5 18:21:04 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 5 Sep 2023 18:21:04 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC [v2]
In-Reply-To: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
Message-ID: <C-TD1oMRsYKieNk_SoetBJlDyFH26LnzgRTVkjGO_3o=.256054f5-5136-4d12-b6c6-cd72c679e159@github.com>

> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
> 
> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
> 
> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Respond to reviewer feedback

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/318/files
  - new: https://git.openjdk.org/shenandoah/pull/318/files/ed9cba6a..dde91e7a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=318&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=318&range=00-01

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/318.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/318/head:pull/318

PR: https://git.openjdk.org/shenandoah/pull/318

From kdnilsen at openjdk.org  Tue Sep  5 19:06:51 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 5 Sep 2023 19:06:51 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v6]
In-Reply-To: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
Message-ID: <Dgm1BvkGdDqo6pfwUfgXwGZQYglw6wZheQEGFHtE4I0=.fb0c16fe-7467-4b77-bb2b-8c6f7f526fd1@github.com>

> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision:

 - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
 - Better abstraction from reviewer feedback
 - Use card_mark_barrier() method and ShenandoahCardBarrier tests
 - Replace is_generational with ShenandoahCardBarrier test
 - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
 - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
 - Also update card table when moving discovered References to global list
 - Fix whitespace
 - Mark card as dirty if discovered reference list has interesting pointer

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/314/files
  - new: https://git.openjdk.org/shenandoah/pull/314/files/2618ff3c..e9db1211

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=05
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=04-05

  Stats: 7883 lines in 280 files changed: 5634 ins; 933 del; 1316 mod
  Patch: https://git.openjdk.org/shenandoah/pull/314.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/314/head:pull/314

PR: https://git.openjdk.org/shenandoah/pull/314

From kdnilsen at openjdk.org  Tue Sep  5 19:09:14 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 5 Sep 2023 19:09:14 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v5]
In-Reply-To: <kcdV0SP9yf2GLtVamcjETr_-WWJXlCd5nKP_BwteTPM=.e8beb36d-a061-436e-a058-2ca8f4888105@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
 <FDGcQp4nPHNDIGmroaKnax0xiWdiB8M6uBxYKMWlFoE=.d861446c-8757-4eeb-8cd2-195a534f4225@github.com>
 <kcdV0SP9yf2GLtVamcjETr_-WWJXlCd5nKP_BwteTPM=.e8beb36d-a061-436e-a058-2ca8f4888105@github.com>
Message-ID: <98rKuUsLELOIP1RnKmfUQbl-0imDAG7o_FEA-MF3-28=.ab9be789-a30c-4996-b2de-234d9ad12149@github.com>

On Fri, 1 Sep 2023 21:21:40 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Use card_mark_barrier() method and ShenandoahCardBarrier tests
>
> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 590:
> 
>> 588:     if (heap->is_in_old(_pending_list_tail) && heap->is_in_young(former_head_of_global_list)) {
>> 589:       heap->mark_card_as_dirty(_pending_list_tail);
>> 590:     }
> 
> Can we just replace this section with `set_oop_field(_pending_list_tail, former_head_of_global_list)` ?

Have made this improvement, and am retesting.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/314#discussion_r1316289561

From wkemper at openjdk.org  Tue Sep  5 20:34:41 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 5 Sep 2023 20:34:41 GMT
Subject: RFR: 8314935: Shenandoah: Unable to throw OOME on back-to-back
 Full GCs
In-Reply-To: <Koc9GsByrURarK3FIkG7Vu3YbVt9Oun9t0XO7qbckfA=.c7c2a880-9244-4958-84d8-8d8942ca46cd@github.com>
References: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
 <Koc9GsByrURarK3FIkG7Vu3YbVt9Oun9t0XO7qbckfA=.c7c2a880-9244-4958-84d8-8d8942ca46cd@github.com>
Message-ID: <MA4HIR1Q7cRGKmLJnRS2bK9Ls8fphGU07rZ2DmZtJ2s=.4ec8c15e-e1e4-4244-8753-fd1202c56c42@github.com>

On Sat, 2 Sep 2023 01:15:46 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Retry allocations until at least one full gc is complete, do not spin unnecessarily when gc is not making progress.
>
> src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 46:
> 
>> 44:   volatile size_t _success_full_gcs;
>> 45:   size_t _alloc_failure_degenerated;
>> 46:   size_t _alloc_failure_degenerated_upgrade_to_full;
> 
> In GenShen tip, `_alloc_failure_degenerated_upgrade_to_full` is also declared volatile.

`_alloc_failure_degenerated_upgrade_to_full` is only changed on a safepoint, so it shouldn't need any additional synchronization.

> src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 87:
> 
>> 85:   void print_gc_stats(outputStream* out) const;
>> 86: 
>> 87:   size_t full_gc_count() const {
> 
> Is it worthwhile to harmonize this with tip and rename to `get_fullgc_count()` ?
> Unless the idea is to change genshen tip to use `full_gc_count()` like here when you pull these changes down.

I went with the style convention established by `cycle_counter`. I don't feel strongly about this. If you prefer `get_full_gc_count`, I can change that here (otherwise, I'll change genshen to use `full_gc_count`).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15500#discussion_r1316376004
PR Review Comment: https://git.openjdk.org/jdk/pull/15500#discussion_r1316377342

From wkemper at openjdk.org  Tue Sep  5 20:36:41 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 5 Sep 2023 20:36:41 GMT
Subject: RFR: 8314935: Shenandoah: Unable to throw OOME on back-to-back
 Full GCs
In-Reply-To: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
References: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
Message-ID: <2U_6_7_jrI859incG0EojY5W_e3lYJSHmLtXzzO2sUQ=.8355958e-6077-46ff-a02f-46360b76efa1@github.com>

On Wed, 30 Aug 2023 23:33:57 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Retry allocations until at least one full gc is complete, do not spin unnecessarily when gc is not making progress.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 894:

> 892:     }
> 893: 
> 894:     while (result == nullptr && tries <= ShenandoahFullGCThreshold) {

Just want to point out, the description of `ShenandoahFullGCThreshold` does not cover this use case (also, the original code here was not concerned with full GCs):
>           "How many back-to-back Degenerated GCs should happen before going to a Full GC."

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15500#discussion_r1316379342

From msheppar at openjdk.org  Tue Sep  5 22:25:39 2023
From: msheppar at openjdk.org (Mark Sheppard)
Date: Tue, 5 Sep 2023 22:25:39 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
Message-ID: <Rr1FNFUPl3iuZz7bCujdhv3jOHvKx_ruMOE-55E_xOM=.3eca3dc7-aa3b-481c-8b16-cf16363e3c51@github.com>

On Wed, 30 Aug 2023 09:23:55 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix static import

> From talking to other HotSpot devs it is quite clear that the different names lead to mistakes when writing (copy-n-pasting) tests, so I'm happy that we try to figure out some way to make it more obvious when we prepend the extra test options and when we don't.
> 
> I agree with @msheppar that `createTestJvm` isn't a good name and I wouldn't be opposed to changing that name instead of `createJavaProcessBuilder`. However, if we do rename that function I strongly urge us to also rename the corresponding `executeTestJvm` function.
> 
> I also think it is too obscure that functions with _Test_ in the name prepend the extra test options, and those without _Test_ don't, so I'd like to get rid of that convention.
> 
> I wouldn't be opposed to a change that:
> 
>     * Keeps the `createJavaProcessBuilder` name
> 
>     * Renames `createTestJvm` to `createJavaProcessBuilderPrependTestOpts`
> 
>     * Renames `executeTestJvm` to `executeJavaPrependTestOpts`
> 
>     * Removes `createTestJava`
> 
>     * Removes `executeTestJava`

I think @stefank made a reasonable suggestion which was endorsed by @AlanBateman 

which would remove the misconception hurdle

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1707391042

From wkemper at openjdk.org  Wed Sep  6 15:40:38 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 6 Sep 2023 15:40:38 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v6]
In-Reply-To: <Dgm1BvkGdDqo6pfwUfgXwGZQYglw6wZheQEGFHtE4I0=.fb0c16fe-7467-4b77-bb2b-8c6f7f526fd1@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
 <Dgm1BvkGdDqo6pfwUfgXwGZQYglw6wZheQEGFHtE4I0=.fb0c16fe-7467-4b77-bb2b-8c6f7f526fd1@github.com>
Message-ID: <VguuIYJJ1_nXPQb2UWqEF5VmQyzeUG9rxyX_17DskoA=.5a060d07-3310-45c3-aee8-75e8b98dd695@github.com>

On Tue, 5 Sep 2023 19:06:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
>  - Better abstraction from reviewer feedback
>  - Use card_mark_barrier() method and ShenandoahCardBarrier tests
>  - Replace is_generational with ShenandoahCardBarrier test
>  - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
>  - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
>  - Also update card table when moving discovered References to global list
>  - Fix whitespace
>  - Mark card as dirty if discovered reference list has interesting pointer

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/314#pullrequestreview-1613659964

From kdnilsen at openjdk.org  Wed Sep  6 15:41:07 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 6 Sep 2023 15:41:07 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC [v3]
In-Reply-To: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
Message-ID: <7kZhie_qWMm1rUF7d7KdbPgseIXIXsV3pF0HZDy-qvQ=.f91618e8-f327-4e2f-9e64-9cf0e2dc278e@github.com>

> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
> 
> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
> 
> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - More typos in comments
 - Fix typos in comment

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/318/files
  - new: https://git.openjdk.org/shenandoah/pull/318/files/dde91e7a..10f8e145

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=318&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=318&range=01-02

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/318.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/318/head:pull/318

PR: https://git.openjdk.org/shenandoah/pull/318

From kdnilsen at openjdk.org  Wed Sep  6 15:46:38 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 6 Sep 2023 15:46:38 GMT
Subject: Integrated: 8315044: GenShen: Verifier detects clean card should be
 dirty
In-Reply-To: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
Message-ID: <ExXnkKT3riKONKFVqzX0JtkhADdFcgVWUk781CklaOQ=.a5ce407d-8082-4370-a1b4-e68f99d70c4a@github.com>

On Mon, 28 Aug 2023 23:35:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.

This pull request has now been integrated.

Changeset: c4d0a247
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/c4d0a2478ea5b0e03ed72608aac3c9fd1b8a41e6
Stats:     39 lines in 1 file changed: 34 ins; 0 del; 5 mod

8315044: GenShen: Verifier detects clean card should be dirty

Reviewed-by: ysr, wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/314

From wkemper at openjdk.org  Wed Sep  6 15:47:39 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 6 Sep 2023 15:47:39 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC [v3]
In-Reply-To: <7kZhie_qWMm1rUF7d7KdbPgseIXIXsV3pF0HZDy-qvQ=.f91618e8-f327-4e2f-9e64-9cf0e2dc278e@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
 <7kZhie_qWMm1rUF7d7KdbPgseIXIXsV3pF0HZDy-qvQ=.f91618e8-f327-4e2f-9e64-9cf0e2dc278e@github.com>
Message-ID: <s6Sywra6kto-6sa7AldRNUH0JqHOCJ03-0mIxSt8pwU=.9cb37514-962d-4daa-89c0-b2481b34ad55@github.com>

On Wed, 6 Sep 2023 15:41:07 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
>> 
>> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
>> 
>> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.
>
> Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - More typos in comments
>  - Fix typos in comment

src/hotspot/share/gc/shenandoah/heuristics/shenandoahGlobalHeuristics.cpp line 146:

> 144:       }
> 145:     } else {
> 146:       assert(r->is_young() && (r->age() < tenuring_threshold), "DeMorgan\'s law");

Not quite DeMorgan's law because `r` could be unaffiliated (it isn't _supposed_ to be, of course). Does the straight quote needs to be escaped?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/318#discussion_r1317491456

From wkemper at openjdk.org  Wed Sep  6 15:50:51 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 6 Sep 2023 15:50:51 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC [v3]
In-Reply-To: <7kZhie_qWMm1rUF7d7KdbPgseIXIXsV3pF0HZDy-qvQ=.f91618e8-f327-4e2f-9e64-9cf0e2dc278e@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
 <7kZhie_qWMm1rUF7d7KdbPgseIXIXsV3pF0HZDy-qvQ=.f91618e8-f327-4e2f-9e64-9cf0e2dc278e@github.com>
Message-ID: <VrCphXvPifQ4fbn7jMsfHvMrdbCU3Nia23yADwLfDK0=.624be1ce-5bc0-4b01-9f39-9473ab39e643@github.com>

On Wed, 6 Sep 2023 15:41:07 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
>> 
>> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
>> 
>> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.
>
> Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - More typos in comments
>  - Fix typos in comment

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/318#pullrequestreview-1613681207

From lkorinth at openjdk.org  Wed Sep  6 16:24:45 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Wed, 6 Sep 2023 16:24:45 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
Message-ID: <cAbrAW1-EzSMgJMLOD90ZAgUNU7LcbYF3GYZKPp7sgw=.5f71207b-0641-4b7d-9033-d35e25fd90ef@github.com>

On Wed, 30 Aug 2023 09:23:55 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix static import

I think you are missing the point. If you take a look at [the parent bug of the sub task](https://bugs.openjdk.org/browse/JDK-8314823) you can see that the problem described is *not* that people are using `createTestJvm` in error. The problem is that they are (or possibly are) using `createJavaProcessBuilder` in error. Thus renaming `createTestJvm` might help a little at most for this specific problem. Renaming `createJavaProcessBuilder` most probably helps *more*. I guess the alternative of forcing the user to make a choice using an enum value will help even more.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1708705105

From kdnilsen at openjdk.org  Wed Sep  6 16:40:08 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 6 Sep 2023 16:40:08 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC [v4]
In-Reply-To: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
Message-ID: <jE4PfqSIAYdGPaA_Jc9NA23YZRSYjcNFyBsi44Y60ms=.4d229021-e1bc-4ba9-bbec-b8a4c84123b4@github.com>

> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
> 
> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
> 
> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fixup comment

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/318/files
  - new: https://git.openjdk.org/shenandoah/pull/318/files/10f8e145..867fcf07

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=318&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=318&range=02-03

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/318.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/318/head:pull/318

PR: https://git.openjdk.org/shenandoah/pull/318

From kdnilsen at openjdk.org  Wed Sep  6 16:40:08 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 6 Sep 2023 16:40:08 GMT
Subject: RFR: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC [v3]
In-Reply-To: <s6Sywra6kto-6sa7AldRNUH0JqHOCJ03-0mIxSt8pwU=.9cb37514-962d-4daa-89c0-b2481b34ad55@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
 <7kZhie_qWMm1rUF7d7KdbPgseIXIXsV3pF0HZDy-qvQ=.f91618e8-f327-4e2f-9e64-9cf0e2dc278e@github.com>
 <s6Sywra6kto-6sa7AldRNUH0JqHOCJ03-0mIxSt8pwU=.9cb37514-962d-4daa-89c0-b2481b34ad55@github.com>
Message-ID: <fxNWBmnrnU6Fe-am759pQj_2l5Fo9HLYnRgpB6h-8Xs=.c8415847-8215-44ec-8f2d-2268a4dab342@github.com>

On Wed, 6 Sep 2023 15:43:22 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - More typos in comments
>>  - Fix typos in comment
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahGlobalHeuristics.cpp line 146:
> 
>> 144:       }
>> 145:     } else {
>> 146:       assert(r->is_young() && (r->age() < tenuring_threshold), "DeMorgan\'s law");
> 
> Not quite DeMorgan's law because `r` could be unaffiliated (it isn't _supposed_ to be, of course). Does the straight quote needs to be escaped?

I'm fixing the message here.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/318#discussion_r1317550653

From kdnilsen at openjdk.org  Wed Sep  6 16:40:09 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 6 Sep 2023 16:40:09 GMT
Subject: Integrated: JDK-8315479: GenShen: Expand old-gen while selecting
 collection set during GLOBAL GC
In-Reply-To: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
References: <NUKgfCFKmaiyNp4zzDot-vvb94um0gcrlH5L2HL4V4w=.7c7458b1-55bc-43bf-bde7-2e95065d678e@github.com>
Message-ID: <7WS_XP2y-TWg4U7oMORuIymJ5k9l5xDnTNUfW6FHRAw=.3a4108fb-ff91-4100-ae29-7f5f591e88dd@github.com>

On Fri, 1 Sep 2023 16:49:35 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> With GenShen, the size of old generation is as small as possible in order to maximize memory available to young gen.  This generally allows less frequent (and therefore more efficient) young-gen collections.
> 
> Under normal operation (concurrent marking of old followed by a series of mixed evacuations), old-gen is expanded at the end of a previous GC cycle in order to accommodate evacuation of mixed-evacuation candidates.
> 
> However, when we do a GLOBAL collection, we need to expand OLD gen during the selection of the collection set, depending on how many old-gen regions are selected to be evacuated during the GC evacuation phase.  This PR makes the change.

This pull request has now been integrated.

Changeset: 33a8085e
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/33a8085e044b8a2dfdc909211945a83d89c04afd
Stats:     81 lines in 2 files changed: 66 ins; 3 del; 12 mod

8315479: GenShen: Expand old-gen while selecting collection set during GLOBAL GC

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/318

From ysr at openjdk.org  Wed Sep  6 17:49:40 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 6 Sep 2023 17:49:40 GMT
Subject: RFR: 8314935: Shenandoah: Unable to throw OOME on back-to-back
 Full GCs
In-Reply-To: <MA4HIR1Q7cRGKmLJnRS2bK9Ls8fphGU07rZ2DmZtJ2s=.4ec8c15e-e1e4-4244-8753-fd1202c56c42@github.com>
References: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
 <Koc9GsByrURarK3FIkG7Vu3YbVt9Oun9t0XO7qbckfA=.c7c2a880-9244-4958-84d8-8d8942ca46cd@github.com>
 <MA4HIR1Q7cRGKmLJnRS2bK9Ls8fphGU07rZ2DmZtJ2s=.4ec8c15e-e1e4-4244-8753-fd1202c56c42@github.com>
Message-ID: <S_WyUmetLklpOXuRuUf5pzUQ8vXj3kyZhRNlLd16sxw=.3e1d5402-6769-4ebe-a407-8a07a1a2c59c@github.com>

On Tue, 5 Sep 2023 20:30:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 46:
>> 
>>> 44:   volatile size_t _success_full_gcs;
>>> 45:   size_t _alloc_failure_degenerated;
>>> 46:   size_t _alloc_failure_degenerated_upgrade_to_full;
>> 
>> In GenShen tip, `_alloc_failure_degenerated_upgrade_to_full` is also declared volatile.
>
> `_alloc_failure_degenerated_upgrade_to_full` is only changed on a safepoint, so it shouldn't need any additional synchronization.

OK, as long as this resolves in genshen tip.

>> src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 87:
>> 
>>> 85:   void print_gc_stats(outputStream* out) const;
>>> 86: 
>>> 87:   size_t full_gc_count() const {
>> 
>> Is it worthwhile to harmonize this with tip and rename to `get_fullgc_count()` ?
>> Unless the idea is to change genshen tip to use `full_gc_count()` like here when you pull these changes down.
>
> I went with the style convention established by `cycle_counter`. I don't feel strongly about this. If you prefer `get_full_gc_count`, I can change that here (otherwise, I'll change genshen to use `full_gc_count`).

full_gc_count is fine; so long as these eventually get resolved in genshen tip.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15500#discussion_r1317633732
PR Review Comment: https://git.openjdk.org/jdk/pull/15500#discussion_r1317633706

From wkemper at openjdk.org  Thu Sep  7 18:46:54 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 7 Sep 2023 18:46:54 GMT
Subject: RFR: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger
Message-ID: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>

There are some additional improvements to the init logger that should be easy to upstream.

-------------

Commit messages:
 - Merge branch 'shenandoah-master' into isolate-shenandoah-heap
 - Prototype isolation of shenandoah heap with gc init logger

Changes: https://git.openjdk.org/shenandoah/pull/320/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=320&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315875
  Stats: 179 lines in 7 files changed: 130 ins; 39 del; 10 mod
  Patch: https://git.openjdk.org/shenandoah/pull/320.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/320/head:pull/320

PR: https://git.openjdk.org/shenandoah/pull/320

From wkemper at openjdk.org  Thu Sep  7 18:47:40 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 7 Sep 2023 18:47:40 GMT
Subject: RFR: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger
In-Reply-To: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
References: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
Message-ID: <hZAHIhZdJp2TlNozYr-8BiALyENf1oyzIL0y-R8ZDdo=.29f9ce75-6e64-4f27-acf1-7fde76f36a80@github.com>

On Thu, 7 Sep 2023 18:39:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

> There are some additional improvements to the init logger that should be easy to upstream.

src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp line 55:

> 53:   ShenandoahHeap* heap = ShenandoahHeap::heap();
> 54:   log_info(gc, init)("Mode: %s", heap->mode()->name());
> 55:   log_info(gc, init)("Heuristics: %s", heap->global_generation()->heuristics()->name());

Note - we still reference the `global_generation` abstraction in this file and we won't have that upstream. I wonder if we should push that abstraction back into `ShenandoahHeap` so that this code in `ShenandoahInitLogger` could call `ShenandoahHeap::heuristics` directly, with an implementation like:
```C++
ShenandoahHeuristics* ShenandoahHeap::heuristics() {
  return global_generation()->heuristics();
}

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/320#discussion_r1318992590

From ysr at openjdk.org  Thu Sep  7 21:16:49 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 7 Sep 2023 21:16:49 GMT
Subject: RFR: 8314935: Shenandoah: Unable to throw OOME on back-to-back
 Full GCs
In-Reply-To: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
References: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
Message-ID: <Jo9kb0huyGv1Pg1gHmlbhLOGAKE_sIOjcsS2UQIo1j0=.decd026a-b272-477b-913e-eee9ffc4a201@github.com>

On Wed, 30 Aug 2023 23:33:57 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Retry allocations until at least one full gc is complete, do not spin unnecessarily when gc is not making progress.

Reviewed!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15500#issuecomment-1710778068

From wkemper at openjdk.org  Thu Sep  7 21:16:49 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 7 Sep 2023 21:16:49 GMT
Subject: Integrated: 8314935: Shenandoah: Unable to throw OOME on back-to-back
 Full GCs
In-Reply-To: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
References: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
Message-ID: <N9beUXbpVN_NkKqoCq4FdH0d7CqFQkrzSIjeTu2vcC8=.d8b25f5c-73a7-4d76-8b40-4dffac19b204@github.com>

On Wed, 30 Aug 2023 23:33:57 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Retry allocations until at least one full gc is complete, do not spin unnecessarily when gc is not making progress.

This pull request has now been integrated.

Changeset: 716201c7
Author:    William Kemper <wkemper at openjdk.org>
Committer: Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/716201c77d160dc78db61957aa002eef71641688
Stats:     22 lines in 2 files changed: 5 ins; 11 del; 6 mod

8314935: Shenandoah: Unable to throw OOME on back-to-back Full GCs

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/jdk/pull/15500

From wkemper at openjdk.org  Thu Sep  7 21:43:11 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 7 Sep 2023 21:43:11 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <_q1sIEQJzL1da3Dhp_oAlYYigP0ID10SLlhNnGutvKE=.85044f91-cd2c-49ec-b804-888aed0fe4e7@github.com>

Merges tag jdk-22+14

-------------

Commit messages:
 - 8311964: Some jtreg tests failing on x86 with error 'unrecognized VM options' (C2 flags)
 - 8315689: G1: Remove unused init_hash_seed
 - 8315579: SPARC64 builds are broken after JDK-8304913
 - 8314949: linux PPC64 Big Endian: Implementation of Foreign Function & Memory API
 - 8288660: JavaDoc should be more helpful if it doesn't recognize a tag
 - 8315377: C2: assert(u->find_out_with(Op_AddP) == nullptr) failed: more than 2 chained AddP nodes?
 - 8315648: Add test for JDK-8309979 changes
 - 8315612: RISC-V: intrinsic for unsignedMultiplyHigh
 - 8315406: [REDO] serviceability/jdwp/AllModulesCommandTest.java ignores VM flags
 - 8313575: Refactor PKCS11Test tests
 - ... and 76 more: https://git.openjdk.org/shenandoah/compare/cb3f9680...024133b0

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/321/files
  Stats: 13570 lines in 348 files changed: 9697 ins; 2411 del; 1462 mod
  Patch: https://git.openjdk.org/shenandoah/pull/321.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/321/head:pull/321

PR: https://git.openjdk.org/shenandoah/pull/321

From kdnilsen at openjdk.org  Thu Sep  7 21:48:12 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 7 Sep 2023 21:48:12 GMT
Subject: RFR: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger
In-Reply-To: <hZAHIhZdJp2TlNozYr-8BiALyENf1oyzIL0y-R8ZDdo=.29f9ce75-6e64-4f27-acf1-7fde76f36a80@github.com>
References: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
 <hZAHIhZdJp2TlNozYr-8BiALyENf1oyzIL0y-R8ZDdo=.29f9ce75-6e64-4f27-acf1-7fde76f36a80@github.com>
Message-ID: <qSYAMMFRoGrKwUm--yEwb3KwTOU56HOygc8rb_4f7Wo=.2ceabf42-7c12-402f-b7d4-c11dd992efb0@github.com>

On Thu, 7 Sep 2023 18:45:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are some additional improvements to the init logger that should be easy to upstream.
>
> src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp line 55:
> 
>> 53:   ShenandoahHeap* heap = ShenandoahHeap::heap();
>> 54:   log_info(gc, init)("Mode: %s", heap->mode()->name());
>> 55:   log_info(gc, init)("Heuristics: %s", heap->global_generation()->heuristics()->name());
> 
> Note - we still reference the `global_generation` abstraction in this file and we won't have that upstream. I wonder if we should push that abstraction back into `ShenandoahHeap` so that this code in `ShenandoahInitLogger` could call `ShenandoahHeap::heuristics` directly, with an implementation like:
> ```C++
> ShenandoahHeuristics* ShenandoahHeap::heuristics() {
>   return global_generation()->heuristics();
> }

Yeah.  Your proposed change might make the abstraction a bit cleaner.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/320#discussion_r1319146050

From kdnilsen at openjdk.org  Thu Sep  7 21:48:12 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 7 Sep 2023 21:48:12 GMT
Subject: RFR: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger
In-Reply-To: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
References: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
Message-ID: <cJuFBxZiWrhzIGA0V2j-ageIylB8DK4gGV_zspJDWL8=.184aa072-c5c5-4c15-b623-67eba6574f81@github.com>

On Thu, 7 Sep 2023 18:39:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

> There are some additional improvements to the init logger that should be easy to upstream.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/320#pullrequestreview-1616280810

From wkemper at openjdk.org  Thu Sep  7 23:32:59 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 7 Sep 2023 23:32:59 GMT
Subject: RFR: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger [v2]
In-Reply-To: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
References: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
Message-ID: <zOQr7B4B5cN8gJCwR35DREjBnNilZTlFLn-8fEhsYsk=.2a807573-0d8f-4347-9778-d5cde5c9db95@github.com>

> There are some additional improvements to the init logger that should be easy to upstream.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Hide global generation behind heap

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/320/files
  - new: https://git.openjdk.org/shenandoah/pull/320/files/9ef0a6d6..9960fe7d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=320&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=320&range=00-01

  Stats: 6 lines in 3 files changed: 4 ins; 1 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/320.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/320/head:pull/320

PR: https://git.openjdk.org/shenandoah/pull/320

From kdnilsen at openjdk.org  Thu Sep  7 23:55:25 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 7 Sep 2023 23:55:25 GMT
Subject: Integrated: JDK-8312115: GenShen: fix log messages
Message-ID: <F3bj8_FCM17MVQo80Rj4CQPP4jAKqOpTkpYHBFDajYE=.355dd9e8-1cd6-4b02-bcd4-1dae46ffda36@github.com>

This PR addresses observed shortcoming in GC log messages for bootstrap GCs

-------------

Commit messages:
 - Fix whitespace
 - Merge remote-tracking branch 'origin/master' into fix-bootstrap-logs
 - Merge remote-tracking branch 'origin/master' into fix-bootstrap-logs
 - Merge remote-tracking branch 'origin/master' into fix-bootstrap-logs
 - Merge remote-tracking branch 'origin/master' into fix-bootstrap-logs-gh
 - Fix handling of Concurrent GLOBAL cycles
 - Fix bootstrap log messages

Changes: https://git.openjdk.org/shenandoah/pull/319/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=319&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8312115
  Stats: 28 lines in 4 files changed: 15 ins; 0 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah/pull/319.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/319/head:pull/319

PR: https://git.openjdk.org/shenandoah/pull/319

From wkemper at openjdk.org  Thu Sep  7 23:55:26 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 7 Sep 2023 23:55:26 GMT
Subject: Integrated: JDK-8312115: GenShen: fix log messages
In-Reply-To: <F3bj8_FCM17MVQo80Rj4CQPP4jAKqOpTkpYHBFDajYE=.355dd9e8-1cd6-4b02-bcd4-1dae46ffda36@github.com>
References: <F3bj8_FCM17MVQo80Rj4CQPP4jAKqOpTkpYHBFDajYE=.355dd9e8-1cd6-4b02-bcd4-1dae46ffda36@github.com>
Message-ID: <c-erlqrxEK-G6BiiDXlKA5khBDDoty8nWjKGT49QUZk=.69cb33c6-4296-4be0-8805-6055b97f438b@github.com>

On Thu, 7 Sep 2023 15:43:25 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This PR addresses observed shortcoming in GC log messages for bootstrap GCs

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/319#pullrequestreview-1616319726

From kdnilsen at openjdk.org  Thu Sep  7 23:55:27 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 7 Sep 2023 23:55:27 GMT
Subject: Integrated: JDK-8312115: GenShen: fix log messages
In-Reply-To: <F3bj8_FCM17MVQo80Rj4CQPP4jAKqOpTkpYHBFDajYE=.355dd9e8-1cd6-4b02-bcd4-1dae46ffda36@github.com>
References: <F3bj8_FCM17MVQo80Rj4CQPP4jAKqOpTkpYHBFDajYE=.355dd9e8-1cd6-4b02-bcd4-1dae46ffda36@github.com>
Message-ID: <zmOZm3pNUf8OOch0kez32R08BIG3lihxh6knzr4Jrp0=.3d2003d7-bd35-4dd1-a377-c78ae572165f@github.com>

On Thu, 7 Sep 2023 15:43:25 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This PR addresses observed shortcoming in GC log messages for bootstrap GCs

I'll leave this in draft mode until we get some CI testing results.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/319#issuecomment-1710387330

From kdnilsen at openjdk.org  Thu Sep  7 23:55:28 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 7 Sep 2023 23:55:28 GMT
Subject: Integrated: JDK-8312115: GenShen: fix log messages
In-Reply-To: <F3bj8_FCM17MVQo80Rj4CQPP4jAKqOpTkpYHBFDajYE=.355dd9e8-1cd6-4b02-bcd4-1dae46ffda36@github.com>
References: <F3bj8_FCM17MVQo80Rj4CQPP4jAKqOpTkpYHBFDajYE=.355dd9e8-1cd6-4b02-bcd4-1dae46ffda36@github.com>
Message-ID: <g6QlCcvS8CAJKp8yQuvAQpdh10zaF4VCO2niN8ciYmM=.811230f9-5b0e-4d03-b764-9772f09c832a@github.com>

On Thu, 7 Sep 2023 15:43:25 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This PR addresses observed shortcoming in GC log messages for bootstrap GCs

This pull request has now been integrated.

Changeset: e9015756
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/e9015756ddd4ad034ac254453dfcfbcb55b000e2
Stats:     28 lines in 4 files changed: 15 ins; 0 del; 13 mod

8312115: GenShen: fix log messages

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/319

From shade at openjdk.org  Fri Sep  8 09:35:31 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 8 Sep 2023 09:35:31 GMT
Subject: RFR: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger [v2]
In-Reply-To: <zOQr7B4B5cN8gJCwR35DREjBnNilZTlFLn-8fEhsYsk=.2a807573-0d8f-4347-9778-d5cde5c9db95@github.com>
References: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
 <zOQr7B4B5cN8gJCwR35DREjBnNilZTlFLn-8fEhsYsk=.2a807573-0d8f-4347-9778-d5cde5c9db95@github.com>
Message-ID: <owcsSt2z88XNYzhxt6zT6scTMPrEjo7-HPfEZBVrmoM=.26d6969d-2e4e-48e0-9a8b-3aecc47e3876@github.com>

On Thu, 7 Sep 2023 23:32:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are some additional improvements to the init logger that should be easy to upstream.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Hide global generation behind heap

Looks okay.

-------------

Marked as reviewed by shade (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/320#pullrequestreview-1617070424

From wkemper at openjdk.org  Fri Sep  8 15:32:12 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 8 Sep 2023 15:32:12 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <_q1sIEQJzL1da3Dhp_oAlYYigP0ID10SLlhNnGutvKE=.85044f91-cd2c-49ec-b804-888aed0fe4e7@github.com>
References: <_q1sIEQJzL1da3Dhp_oAlYYigP0ID10SLlhNnGutvKE=.85044f91-cd2c-49ec-b804-888aed0fe4e7@github.com>
Message-ID: <lC2QiZfzk6kDB-dh0ZLzY8a8-CmozuHz2zp_VyBCiS4=.2f38829a-d997-4332-908d-74e0c7db3a37@github.com>

> Merges tag jdk-22+14

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/321/files
  - new: https://git.openjdk.org/shenandoah/pull/321/files/024133b0..024133b0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=321&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=321&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/321.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/321/head:pull/321

PR: https://git.openjdk.org/shenandoah/pull/321

From wkemper at openjdk.org  Fri Sep  8 15:32:13 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 8 Sep 2023 15:32:13 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <_q1sIEQJzL1da3Dhp_oAlYYigP0ID10SLlhNnGutvKE=.85044f91-cd2c-49ec-b804-888aed0fe4e7@github.com>
References: <_q1sIEQJzL1da3Dhp_oAlYYigP0ID10SLlhNnGutvKE=.85044f91-cd2c-49ec-b804-888aed0fe4e7@github.com>
Message-ID: <uK4n-cVkjw4-F3DaTwYtnROn0Ba8rfv58uGvo3QkRD0=.f2d9357d-689e-43e7-af12-83fa124ea518@github.com>

On Thu, 7 Sep 2023 21:37:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+14

This pull request has now been integrated.

Changeset: 496addb0
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/496addb080921244913eab476c34fc3787f45a52
Stats:     13570 lines in 348 files changed: 9697 ins; 2411 del; 1462 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/321

From wkemper at openjdk.org  Fri Sep  8 17:18:07 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 8 Sep 2023 17:18:07 GMT
Subject: RFR: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger [v3]
In-Reply-To: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
References: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
Message-ID: <r0DeagdF6oqLKzmtAL3NEFPM88pKSF7jW95PDH8M03I=.d26e4c85-2fdd-48d3-b9a3-2b11f9c5b6df@github.com>

> There are some additional improvements to the init logger that should be easy to upstream.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix zero build

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/320/files
  - new: https://git.openjdk.org/shenandoah/pull/320/files/9960fe7d..6e1d7bc8

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=320&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=320&range=01-02

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/320.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/320/head:pull/320

PR: https://git.openjdk.org/shenandoah/pull/320

From ysr at openjdk.org  Fri Sep  8 17:24:31 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 8 Sep 2023 17:24:31 GMT
Subject: RFR: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger [v3]
In-Reply-To: <r0DeagdF6oqLKzmtAL3NEFPM88pKSF7jW95PDH8M03I=.d26e4c85-2fdd-48d3-b9a3-2b11f9c5b6df@github.com>
References: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
 <r0DeagdF6oqLKzmtAL3NEFPM88pKSF7jW95PDH8M03I=.d26e4c85-2fdd-48d3-b9a3-2b11f9c5b6df@github.com>
Message-ID: <EpyCDqvg35Vs3KEhEQs4NNL1yggD06_XHHwTmOtoCP4=.cc7765b4-6d2d-47f5-bab0-d762700fcff9@github.com>

On Fri, 8 Sep 2023 17:18:07 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are some additional improvements to the init logger that should be easy to upstream.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix zero build

perfect, thank you!

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/320#pullrequestreview-1617947289

From wkemper at openjdk.org  Fri Sep  8 17:43:33 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 8 Sep 2023 17:43:33 GMT
Subject: Integrated: 8315875: GenShen: Remove heap mode check from
 ShenandoahInitLogger
In-Reply-To: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
References: <z8_On5Y4imhL1Pkj21GJovvSHFdJsFCoYwLIl9eIQks=.08269eda-a93b-488a-9a53-8783ec561842@github.com>
Message-ID: <lII5AAC4hBzOQrfr0-bz6kjUBmYieGZafpXdEETupfw=.82c38d9b-1377-4489-979a-cc7c0218155f@github.com>

On Thu, 7 Sep 2023 18:39:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

> There are some additional improvements to the init logger that should be easy to upstream.

This pull request has now been integrated.

Changeset: 69aa9719
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/69aa97198c4e71051492f368e37aa50fc75834ee
Stats:     183 lines in 7 files changed: 134 ins; 39 del; 10 mod

8315875: GenShen: Remove heap mode check from ShenandoahInitLogger

Reviewed-by: kdnilsen, shade, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/320

From wkemper at openjdk.org  Fri Sep  8 21:17:55 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 8 Sep 2023 21:17:55 GMT
Subject: RFR: 8315950: Shenandoah: Improvements to ShenandoahInitLogger
Message-ID: <KAOiKOZsl7X_JS4A3EBXdhalL7kZg8s8Hv2BqNXnbP8=.cde8da04-daa7-4914-acc2-1450249c5a5e@github.com>

Changes to ShenandoahInitLogger:
* Use macro to simplify reporting of sizes
* Separate heap reporting from gc-specific reporting

Call the init logger from a virtual method to facilitate isolation of different gc modes.

-------------

Commit messages:
 - Improve ShenandoahInitLogger

Changes: https://git.openjdk.org/jdk/pull/15646/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15646&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315950
  Stats: 29 lines in 4 files changed: 5 ins; 11 del; 13 mod
  Patch: https://git.openjdk.org/jdk/pull/15646.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15646/head:pull/15646

PR: https://git.openjdk.org/jdk/pull/15646

From kdnilsen at openjdk.org  Fri Sep  8 23:16:37 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 8 Sep 2023 23:16:37 GMT
Subject: RFR: 8315950: Shenandoah: Improvements to ShenandoahInitLogger
In-Reply-To: <KAOiKOZsl7X_JS4A3EBXdhalL7kZg8s8Hv2BqNXnbP8=.cde8da04-daa7-4914-acc2-1450249c5a5e@github.com>
References: <KAOiKOZsl7X_JS4A3EBXdhalL7kZg8s8Hv2BqNXnbP8=.cde8da04-daa7-4914-acc2-1450249c5a5e@github.com>
Message-ID: <F1yVslWm78qgI-_GhKzYvTSVf_Liv8NnDOPKk7Cj9v0=.6588e82d-27fb-431e-8a5d-7080e1479d71@github.com>

On Fri, 8 Sep 2023 21:11:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Changes to ShenandoahInitLogger:
> * Use macro to simplify reporting of sizes
> * Separate heap reporting from gc-specific reporting
> 
> Call the init logger from a virtual method to facilitate isolation of different gc modes.

Marked as reviewed by kdnilsen (no project role).

-------------

PR Review: https://git.openjdk.org/jdk/pull/15646#pullrequestreview-1618506805

From wkemper at openjdk.org  Mon Sep 11 16:26:04 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 11 Sep 2023 16:26:04 GMT
Subject: RFR: 8315950: Shenandoah: Improvements to ShenandoahInitLogger
 [v2]
In-Reply-To: <KAOiKOZsl7X_JS4A3EBXdhalL7kZg8s8Hv2BqNXnbP8=.cde8da04-daa7-4914-acc2-1450249c5a5e@github.com>
References: <KAOiKOZsl7X_JS4A3EBXdhalL7kZg8s8Hv2BqNXnbP8=.cde8da04-daa7-4914-acc2-1450249c5a5e@github.com>
Message-ID: <3IPreJDnZnlPQkYfnMHs0kdkHC5FntAVHrEPDqWklU8=.6b96eaab-d82b-42d4-bab7-9ca401640ab8@github.com>

> Changes to ShenandoahInitLogger:
> * Use macro to simplify reporting of sizes
> * Separate heap reporting from gc-specific reporting
> 
> Call the init logger from a virtual method to facilitate isolation of different gc modes.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:

 - Merge branch 'jdk/master' into init-logger-improvements
 - Improve ShenandoahInitLogger

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15646/files
  - new: https://git.openjdk.org/jdk/pull/15646/files/e4f2c107..1428fded

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15646&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15646&range=00-01

  Stats: 5823 lines in 132 files changed: 3755 ins; 1311 del; 757 mod
  Patch: https://git.openjdk.org/jdk/pull/15646.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15646/head:pull/15646

PR: https://git.openjdk.org/jdk/pull/15646

From shade at openjdk.org  Mon Sep 11 18:28:02 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 11 Sep 2023 18:28:02 GMT
Subject: RFR: 8299614: Shenandoah: STW mark should keep nmethod/oops referenced
 from stack chunk alive
Message-ID: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>

See the investigation in the bug. Basically, with Loom we have to enable nmethod barriers to get the proper marking of nmethods/oops from stack chunks. See `ShenandoahMarkRefsSuperClosure::do_nmethod` callers and callees for more details. This IMO makes `ShenandoahNMethodBarrier` flag non-optional, as we have to enable nmethod barriers when Loom is enabled. The major part of PR deals with that.

Additional testing:
 - [x] Original reproducer does not fail anymore
 - [x] MacOS AArch64 fastdebug, `hotspot_gc_shenandoah`
 - [x] Linux AArch64 fastdebug, `hotspot_gc_shenandoah`
 - [x] Linux x86_64 fastdebug, `hotspot_gc_shenandoah`
 - [ ] Linux AArch64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
 - [ ] Linux x86_64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`

-------------

Commit messages:
 - ShenandoahNMethodBarrier flag is no longer optional
 - Fix

Changes: https://git.openjdk.org/jdk/pull/15669/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15669&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8299614
  Stats: 56 lines in 15 files changed: 26 ins; 17 del; 13 mod
  Patch: https://git.openjdk.org/jdk/pull/15669.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15669/head:pull/15669

PR: https://git.openjdk.org/jdk/pull/15669

From ysr at openjdk.org  Mon Sep 11 20:30:40 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 11 Sep 2023 20:30:40 GMT
Subject: RFR: 8315950: Shenandoah: Improvements to ShenandoahInitLogger
 [v2]
In-Reply-To: <3IPreJDnZnlPQkYfnMHs0kdkHC5FntAVHrEPDqWklU8=.6b96eaab-d82b-42d4-bab7-9ca401640ab8@github.com>
References: <KAOiKOZsl7X_JS4A3EBXdhalL7kZg8s8Hv2BqNXnbP8=.cde8da04-daa7-4914-acc2-1450249c5a5e@github.com>
 <3IPreJDnZnlPQkYfnMHs0kdkHC5FntAVHrEPDqWklU8=.6b96eaab-d82b-42d4-bab7-9ca401640ab8@github.com>
Message-ID: <hqUFO9ZUU_DY85fnSCivjDwDhMXzvyixdEYrwMuc1is=.ff5a4013-4d9e-442b-92cf-2485fab49953@github.com>

On Mon, 11 Sep 2023 16:26:04 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Changes to ShenandoahInitLogger:
>> * Use macro to simplify reporting of sizes
>> * Separate heap reporting from gc-specific reporting
>> 
>> Call the init logger from a virtual method to facilitate isolation of different gc modes.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
> 
>  - Merge branch 'jdk/master' into init-logger-improvements
>  - Improve ShenandoahInitLogger

Marked as reviewed by ysr (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/15646#pullrequestreview-1620780948

From shade at openjdk.org  Mon Sep 11 21:50:07 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 11 Sep 2023 21:50:07 GMT
Subject: RFR: 8299614: Shenandoah: STW mark should keep nmethod/oops
 referenced from stack chunk alive [v2]
In-Reply-To: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
References: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
Message-ID: <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>

> See the investigation in the bug. Basically, with Loom we have to enable nmethod barriers to get the proper marking of nmethods/oops from stack chunks. See `ShenandoahMarkRefsSuperClosure::do_nmethod` callers and callees for more details. This IMO makes `ShenandoahNMethodBarrier` flag non-optional, as we have to enable nmethod barriers when Loom is enabled. The major part of PR deals with that.
> 
> Additional testing:
>  - [x] Original reproducer does not fail anymore
>  - [x] MacOS AArch64 fastdebug, `hotspot_gc_shenandoah`
>  - [x] Linux AArch64 fastdebug, `hotspot_gc_shenandoah`
>  - [x] Linux x86_64 fastdebug, `hotspot_gc_shenandoah`
>  - [ ] Linux AArch64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>  - [ ] Linux x86_64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Should arm the nmethods for thread root processing

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15669/files
  - new: https://git.openjdk.org/jdk/pull/15669/files/8a753e9f..ca09b71f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15669&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15669&range=00-01

  Stats: 16 lines in 6 files changed: 5 ins; 0 del; 11 mod
  Patch: https://git.openjdk.org/jdk/pull/15669.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15669/head:pull/15669

PR: https://git.openjdk.org/jdk/pull/15669

From wkemper at openjdk.org  Mon Sep 11 22:11:38 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 11 Sep 2023 22:11:38 GMT
Subject: RFR: 8299614: Shenandoah: STW mark should keep nmethod/oops
 referenced from stack chunk alive [v2]
In-Reply-To: <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
References: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
 <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
Message-ID: <ot-StIF_veTIe32gHXOgv8SFgWjvnL5kYGJ96S3-d_Y=.46243b0f-e810-4187-8d1e-fb1919182d44@github.com>

On Mon, 11 Sep 2023 21:50:07 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> See the investigation in the bug. Basically, with Loom we have to enable nmethod barriers to get the proper marking of nmethods/oops from stack chunks. See `ShenandoahMarkRefsSuperClosure::do_nmethod` callers and callees for more details. This IMO makes `ShenandoahNMethodBarrier` flag non-optional, as we have to enable nmethod barriers when Loom is enabled. The major part of PR deals with that.
>> 
>> Additional testing:
>>  - [x] Original reproducer does not fail anymore
>>  - [x] MacOS AArch64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux x86_64 fastdebug, `hotspot_gc_shenandoah`
>>  - [ ] Linux AArch64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>>  - [ ] Linux x86_64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Should arm the nmethods for thread root processing

I'm not suggesting we keep `ShenandoahNMethodBarrier` flag, but for the sake of my own understanding: the passive mode does not need the nmethod barriers, correct?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15669#issuecomment-1714652233

From shade at openjdk.org  Tue Sep 12 06:17:39 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 12 Sep 2023 06:17:39 GMT
Subject: RFR: 8299614: Shenandoah: STW mark should keep nmethod/oops
 referenced from stack chunk alive [v2]
In-Reply-To: <ot-StIF_veTIe32gHXOgv8SFgWjvnL5kYGJ96S3-d_Y=.46243b0f-e810-4187-8d1e-fb1919182d44@github.com>
References: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
 <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
 <ot-StIF_veTIe32gHXOgv8SFgWjvnL5kYGJ96S3-d_Y=.46243b0f-e810-4187-8d1e-fb1919182d44@github.com>
Message-ID: <fFy-1eGcKVhvHuU2D1dJx7Ga2sysja_PN_qpCsoId-w=.891001fe-0e16-4af9-948c-7825a83874c8@github.com>

On Mon, 11 Sep 2023 22:08:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

> I'm not suggesting we keep `ShenandoahNMethodBarrier` flag, but for the sake of my own understanding: the passive mode does not need the nmethod barriers, correct?

Passive needs nmethod barriers, because STW mark in passive mode needs to visit stackChunks, which in current code requires nmethod barriers to be armed to get them to visit the stackChunk's nmethods. Read through `ShenandoahMarkRefsSuperClosure::do_nmethod`. The reproducer (see bug) fails with passive too.

It is an unfortunate dependency, which we can untie by making `ShenandoahMarkRefsSuperClosure::do_nmethod` scan the nmethods directly when nmethod barriers are disabled, like this:


diff --git a/src/hotspot/share/gc/shenandoah/shenandoahClosures.inline.hpp b/src/hotspot/share/gc/shenandoah/shenandoahClosures.inline.hpp
index 1c8daba3d24..fe00fe72220 100644
--- a/src/hotspot/share/gc/shenandoah/shenandoahClosures.inline.hpp
+++ b/src/hotspot/share/gc/shenandoah/shenandoahClosures.inline.hpp
@@ -71,6 +71,7 @@ BoolObjectClosure* ShenandoahIsAliveSelector::is_alive_closure() {
 
 void ShenandoahOopClosureBase::do_nmethod(nmethod* nm) {
   nm->run_nmethod_entry_barrier();
+  nm->follow_nmethod(this);
 }
 
 ShenandoahKeepAliveClosure::ShenandoahKeepAliveClosure() :
diff --git a/src/hotspot/share/gc/shenandoah/shenandoahOopClosures.hpp b/src/hotspot/share/gc/shenandoah/shenandoahOopClosures.hpp
index 11c70f2726a..44f06a26684 100644
--- a/src/hotspot/share/gc/shenandoah/shenandoahOopClosures.hpp
+++ b/src/hotspot/share/gc/shenandoah/shenandoahOopClosures.hpp
@@ -63,6 +63,7 @@ class ShenandoahMarkRefsSuperClosure : public MetadataVisitingOopIterateClosure
   virtual void do_nmethod(nmethod* nm) {
     assert(!is_weak(), "Can't handle weak marking of nmethods");
     nm->run_nmethod_entry_barrier();
+    nm->follow_nmethod(this);
   }
 };


But IMO that is riskier than just delegating to nmethod barriers, like concurrent GC does. ...and it would probably do double work when nmethod barriers are used. 

Further, `ShenandoahNMethodBarrier` is kinda misnomer, because most of the `Shenandoah*Barrier` flags are dealing with emitting the mutator-side barriers. But `ShenandoahNMethodBarrier`, as evident from this PR, only guards arming the already emitted barriers in some conditions. So, counter to other `Shenandoah*Barrier` flags, the nmethod barriers are still emitted in the generated code by interpreter and compilers even with `-ShenandoahNMethodBarrier`. So, this PR resolves this oddity as well.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15669#issuecomment-1715052134

From rkennke at openjdk.org  Tue Sep 12 10:34:43 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 12 Sep 2023 10:34:43 GMT
Subject: RFR: 8299614: Shenandoah: STW mark should keep nmethod/oops
 referenced from stack chunk alive [v2]
In-Reply-To: <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
References: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
 <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
Message-ID: <8jTKFb1CkTAoGtIM_Bryp84ICAgmPcLBbusqUSktxKE=.982398c4-863d-48bf-8cbd-9b28af30bfd5@github.com>

On Mon, 11 Sep 2023 21:50:07 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> See the investigation in the bug. Basically, with Loom we have to enable nmethod barriers to get the proper marking of nmethods/oops from stack chunks. See `ShenandoahMarkRefsSuperClosure::do_nmethod` callers and callees for more details. This IMO makes `ShenandoahNMethodBarrier` flag non-optional, as we have to enable nmethod barriers when Loom is enabled. The major part of PR deals with that.
>> 
>> Additional testing:
>>  - [x] Original reproducer does not fail anymore
>>  - [x] MacOS AArch64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux x86_64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>>  - [x] Linux x86_64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Should arm the nmethods for thread root processing

Looks good to me. Thank you!

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15669#pullrequestreview-1621929358

From ysr at openjdk.org  Tue Sep 12 19:43:53 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 12 Sep 2023 19:43:53 GMT
Subject: RFR: 8316137: GenShen: missing ShenandoahCardBarrier filter
Message-ID: <qV4uOdi0PAIxNMQO_vRu8GomQ6LPdP9XqcDeEV7f4c8=.45a9d4bb-870b-48ce-9141-5bedc902bece@github.com>

A few instances of (inline) calls to the card-barrier code from the VM missed the idiomatic changes in the parent ticket https://bugs.openjdk.org/browse/JDK-8315247. This PR takes care of the cases that I had previously missed where we were doing a generational mode check on the heap.

-------------

Commit messages:
 - More
 - Another instance.
 - Fix assertion message.
 - ShenandoahCardBarrier

Changes: https://git.openjdk.org/shenandoah/pull/322/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=322&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8316137
  Stats: 27 lines in 2 files changed: 12 ins; 3 del; 12 mod
  Patch: https://git.openjdk.org/shenandoah/pull/322.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/322/head:pull/322

PR: https://git.openjdk.org/shenandoah/pull/322

From zgu at openjdk.org  Tue Sep 12 19:59:40 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Tue, 12 Sep 2023 19:59:40 GMT
Subject: RFR: 8299614: Shenandoah: STW mark should keep nmethod/oops
 referenced from stack chunk alive [v2]
In-Reply-To: <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
References: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
 <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
Message-ID: <_ZZ9jlE2VmXcqvWA509iv-0vhKA-P35gVf27ccZq28k=.52285df3-94e7-4c54-baee-6d0a68b005e4@github.com>

On Mon, 11 Sep 2023 21:50:07 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> See the investigation in the bug. Basically, with Loom we have to enable nmethod barriers to get the proper marking of nmethods/oops from stack chunks. See `ShenandoahMarkRefsSuperClosure::do_nmethod` callers and callees for more details. This IMO makes `ShenandoahNMethodBarrier` flag non-optional, as we have to enable nmethod barriers when Loom is enabled. The major part of PR deals with that.
>> 
>> Additional testing:
>>  - [x] Original reproducer does not fail anymore
>>  - [x] MacOS AArch64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux x86_64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>>  - [x] Linux x86_64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Should arm the nmethods for thread root processing

Make senses. LGTM

-------------

Marked as reviewed by zgu (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15669#pullrequestreview-1623020296

From wkemper at openjdk.org  Tue Sep 12 20:25:18 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Sep 2023 20:25:18 GMT
Subject: RFR: 8316137: GenShen: missing ShenandoahCardBarrier filter
In-Reply-To: <qV4uOdi0PAIxNMQO_vRu8GomQ6LPdP9XqcDeEV7f4c8=.45a9d4bb-870b-48ce-9141-5bedc902bece@github.com>
References: <qV4uOdi0PAIxNMQO_vRu8GomQ6LPdP9XqcDeEV7f4c8=.45a9d4bb-870b-48ce-9141-5bedc902bece@github.com>
Message-ID: <EqSqHLKHlQgef68C8p1BomlEWnj_ht1e8CT9ALkZiSk=.0f66e6e6-a129-4b4f-8bf9-6ec04037b2bb@github.com>

On Tue, 12 Sep 2023 19:08:39 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> A few instances of (inline) calls to the card-barrier code from the VM missed the idiomatic changes in the parent ticket https://bugs.openjdk.org/browse/JDK-8315247. This PR takes care of the cases that I had previously missed where we were doing a generational mode check on the heap.

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/322#pullrequestreview-1623060142

From ysr at openjdk.org  Wed Sep 13 01:08:17 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 13 Sep 2023 01:08:17 GMT
Subject: Integrated: 8316137: GenShen: missing ShenandoahCardBarrier filter
In-Reply-To: <qV4uOdi0PAIxNMQO_vRu8GomQ6LPdP9XqcDeEV7f4c8=.45a9d4bb-870b-48ce-9141-5bedc902bece@github.com>
References: <qV4uOdi0PAIxNMQO_vRu8GomQ6LPdP9XqcDeEV7f4c8=.45a9d4bb-870b-48ce-9141-5bedc902bece@github.com>
Message-ID: <3jrKZ0cE61oBTITbPvJCgYOZy0xEUFQOTQjg6wsCZ6A=.6d689da2-451b-4b18-bd9d-0b1b299fe4eb@github.com>

On Tue, 12 Sep 2023 19:08:39 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> A few instances of (inline) calls to the card-barrier code from the VM missed the idiomatic changes in the parent ticket https://bugs.openjdk.org/browse/JDK-8315247. This PR takes care of the cases that I had previously missed where we were doing a generational mode check on the heap.

This pull request has now been integrated.

Changeset: ba88eb1d
Author:    Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/ba88eb1dfeb0a5bb5abe51eb0fd4ac70357e6b5d
Stats:     27 lines in 2 files changed: 12 ins; 3 del; 12 mod

8316137: GenShen: missing ShenandoahCardBarrier filter

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/322

From kdnilsen at openjdk.org  Wed Sep 13 01:52:16 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 13 Sep 2023 01:52:16 GMT
Subject: RFR: 8316137: GenShen: missing ShenandoahCardBarrier filter
In-Reply-To: <qV4uOdi0PAIxNMQO_vRu8GomQ6LPdP9XqcDeEV7f4c8=.45a9d4bb-870b-48ce-9141-5bedc902bece@github.com>
References: <qV4uOdi0PAIxNMQO_vRu8GomQ6LPdP9XqcDeEV7f4c8=.45a9d4bb-870b-48ce-9141-5bedc902bece@github.com>
Message-ID: <50ukLg_Xaq3bwvtFr5JOeam0kAHEEllgBJBu_flj3pA=.7c0e87f0-9f91-418e-89ef-70dd5d7ea5ee@github.com>

On Tue, 12 Sep 2023 19:08:39 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> A few instances of (inline) calls to the card-barrier code from the VM missed the idiomatic changes in the parent ticket https://bugs.openjdk.org/browse/JDK-8315247. This PR takes care of the cases that I had previously missed where we were doing a generational mode check on the heap.

Thanks.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/322#pullrequestreview-1623506837

From shade at openjdk.org  Wed Sep 13 06:06:54 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 13 Sep 2023 06:06:54 GMT
Subject: RFR: 8299614: Shenandoah: STW mark should keep nmethod/oops
 referenced from stack chunk alive [v2]
In-Reply-To: <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
References: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
 <Fr5mzoH7gwNf6xpBT5wE47iZ6Fr83IYtAsithqVsvNQ=.5dec00b6-8920-4b60-a58c-ab17bb12c0df@github.com>
Message-ID: <lcC0ARs4VJyZKAQ61Y3pULUgBm8pUb8-YQ5gMkZUocw=.c5a785e3-800d-4d49-be88-d3fb9f879069@github.com>

On Mon, 11 Sep 2023 21:50:07 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> See the investigation in the bug. Basically, with Loom we have to enable nmethod barriers to get the proper marking of nmethods/oops from stack chunks. See `ShenandoahMarkRefsSuperClosure::do_nmethod` callers and callees for more details. This IMO makes `ShenandoahNMethodBarrier` flag non-optional, as we have to enable nmethod barriers when Loom is enabled. The major part of PR deals with that.
>> 
>> Additional testing:
>>  - [x] Original reproducer does not fail anymore
>>  - [x] MacOS AArch64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux x86_64 fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>>  - [x] Linux x86_64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Should arm the nmethods for thread root processing

Thanks all!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15669#issuecomment-1716997940

From shade at openjdk.org  Wed Sep 13 06:06:54 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 13 Sep 2023 06:06:54 GMT
Subject: Integrated: 8299614: Shenandoah: STW mark should keep nmethod/oops
 referenced from stack chunk alive
In-Reply-To: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
References: <hhcKmsfB2zw_Xph77DDEgH1ARY90Bq2EAS4Zqolibf4=.665fe3cc-3672-4ef6-8802-acd6b928beb0@github.com>
Message-ID: <gO9jbs5iaisRJadoxnzxCtvuMhvkRquwdm7yw4i8qqc=.06f28d79-a200-46f3-a19a-09a1aa5dbf31@github.com>

On Mon, 11 Sep 2023 18:21:28 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> See the investigation in the bug. Basically, with Loom we have to enable nmethod barriers to get the proper marking of nmethods/oops from stack chunks. See `ShenandoahMarkRefsSuperClosure::do_nmethod` callers and callees for more details. This IMO makes `ShenandoahNMethodBarrier` flag non-optional, as we have to enable nmethod barriers when Loom is enabled. The major part of PR deals with that.
> 
> Additional testing:
>  - [x] Original reproducer does not fail anymore
>  - [x] MacOS AArch64 fastdebug, `hotspot_gc_shenandoah`
>  - [x] Linux AArch64 fastdebug, `hotspot_gc_shenandoah`
>  - [x] Linux x86_64 fastdebug, `hotspot_gc_shenandoah`
>  - [x] Linux AArch64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`
>  - [x] Linux x86_64 fastdebug, `tier1 tier2 tier3` with `-XX:+UseShenandoahGC`

This pull request has now been integrated.

Changeset: ece9bdfc
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/ece9bdfc838a5c419c789319ff794f1ae29b0256
Stats:     63 lines in 15 files changed: 31 ins; 17 del; 15 mod

8299614: Shenandoah: STW mark should keep nmethod/oops referenced from stack chunk alive

Reviewed-by: rkennke, zgu

-------------

PR: https://git.openjdk.org/jdk/pull/15669

From wkemper at openjdk.org  Thu Sep 14 14:25:17 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 14 Sep 2023 14:25:17 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <whPi4kQskos00QeV1ZvSuOuhXeFtK1IBcIWGah10gX4=.3f8c2aa7-b493-41c5-90b9-9717cc0cdee9@github.com>

Merges tag jdk-22+15

-------------

Commit messages:
 - 6228794: java.text.ChoiceFormat pattern behavior is not well documented.
 - 8314226: Series of colon-style fallthrough switch cases with guards compiled incorrectly
 - 8316087: Test SignedLoggerFinderTest.java is still failing
 - 8315934: RISC-V: Disable conservative fences per vendor
 - 8316021: Serial: Remove unused Generation::post_compact
 - 8316050: Use hexadecimal encoding in MemorySegment::toString
 - 8314612: TestUnorderedReduction.java fails with -XX:MaxVectorSize=32 and -XX:+AlignVector
 - 8311207: Cleanup for Optimization for UUID.toString
 - 8315898: Open source swing JMenu tests
 - 8315761: Open source few swing JList and JMenuBar tests
 - ... and 88 more: https://git.openjdk.org/shenandoah/compare/024133b0...ce93d27f

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=323&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/323/files
  Stats: 24498 lines in 1388 files changed: 12058 ins; 8350 del; 4090 mod
  Patch: https://git.openjdk.org/shenandoah/pull/323.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/323/head:pull/323

PR: https://git.openjdk.org/shenandoah/pull/323

From kdnilsen at openjdk.org  Thu Sep 14 14:38:42 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 14 Sep 2023 14:38:42 GMT
Subject: RFR: 8316297: GenShen: Degenerated GCs fail to make progress
Message-ID: <jAuhHSnbWdp9E-7JIGAd8CKIVIJHASGxd5NdrBCMTCE=.21816c11-5cc6-4da1-959e-81073d9e3a4e@github.com>

A recent change to the implementation of ShenandoahYoungGeneration::available() resulted in a budgeting shortfall during degenerated GC.  The problem is that we degenerate after mutator allocations fail, at which time young available memory is in very short supply.  The change to available() caused us to only see the mutator budget, which has been depleted.  This resulted in a very high probability that degenerated GC would fail to make progress, forcing degenerated GC to upgrade to full GC.

This PR makes the Collector reserve memory visible to the function that computes evacuation budgets, thereby increasing the likelihood of a successful degenerated cycle and obviating the need for a Full GC.

-------------

Commit messages:
 - Include memory reserved for Collector when computing evacuation budget

Changes: https://git.openjdk.org/shenandoah/pull/324/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=324&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8316297
  Stats: 18 lines in 3 files changed: 14 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/324.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/324/head:pull/324

PR: https://git.openjdk.org/shenandoah/pull/324

From kdnilsen at openjdk.org  Thu Sep 14 15:29:50 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 14 Sep 2023 15:29:50 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC
Message-ID: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>

We have found that expedited GCs too frequently yield very little benefit because they occur so soon after the previous GC that there has been no accumulation of garbage.

The primary motivation for expediting GC is to avoid a situation under which there is so much "promotion" work to be performed that the urgent need to reclaim garbage from young-generation is obstructed by this promotion work.

This PR ends the practice of expediting to support promotion in place.  The effort required to promote a region in place is minimal and unlikely to contend in a major way with young collection efforts.

This PR also reduces the likelihood that we will expedite for promotion by evacuation, because it requires the amount of accumulated promo potential to exceed a particular threshold.

-------------

Commit messages:
 - Reduce frequence of expedited GC

Changes: https://git.openjdk.org/shenandoah/pull/325/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=325&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8316299
  Stats: 14 lines in 1 file changed: 2 ins; 11 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/325.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/325/head:pull/325

PR: https://git.openjdk.org/shenandoah/pull/325

From shade at openjdk.org  Thu Sep 14 15:29:52 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 14 Sep 2023 15:29:52 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC
In-Reply-To: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
Message-ID: <ZUWtSfmGSDQhO3SK--VKDsLJleZNeq7qNQHW17YY3mQ=.7f4dc2d8-05b9-4ce9-b82a-127f153a7039@github.com>

On Thu, 14 Sep 2023 15:21:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> We have found that expedited GCs too frequently yield very little benefit because they occur so soon after the previous GC that there has been no accumulation of garbage.
> 
> The primary motivation for expediting GC is to avoid a situation under which there is so much "promotion" work to be performed that the urgent need to reclaim garbage from young-generation is obstructed by this promotion work.
> 
> This PR ends the practice of expediting to support promotion in place.  The effort required to promote a region in place is minimal and unlikely to contend in a major way with young collection efforts.
> 
> This PR also reduces the likelihood that we will expedite for promotion by evacuation, because it requires the amount of accumulated promo potential to exceed a particular threshold.

Looks okay, but I have a question:

src/hotspot/share/gc/shenandoah/heuristics/shenandoahYoungHeuristics.cpp line 139:

> 137:   ShenandoahHeap* heap = ShenandoahHeap::heap();
> 138: 
> 139:   size_t promo_expedite_threshold = (heap->young_generation()->max_capacity() * ShenandoahEvacReserve) / 512;

What's the reasoning behind `512` divisor here? Since `ShenandoahEvacReserve` is percents of young/total heap size, this makes `promo_expedite_threshold` about 20% of heap size. Is that what we want?

-------------

Marked as reviewed by shade (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/325#pullrequestreview-1627172704
PR Review Comment: https://git.openjdk.org/shenandoah/pull/325#discussion_r1326128445

From shade at openjdk.org  Thu Sep 14 15:35:42 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 14 Sep 2023 15:35:42 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC
In-Reply-To: <ZUWtSfmGSDQhO3SK--VKDsLJleZNeq7qNQHW17YY3mQ=.7f4dc2d8-05b9-4ce9-b82a-127f153a7039@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
 <ZUWtSfmGSDQhO3SK--VKDsLJleZNeq7qNQHW17YY3mQ=.7f4dc2d8-05b9-4ce9-b82a-127f153a7039@github.com>
Message-ID: <up4hoSo4YcFtJs-W29fkyGGtUt7Ac6vta9oNvxTuA0Y=.b0ba2133-1a58-49f0-9c2a-82e19364554a@github.com>

On Thu, 14 Sep 2023 15:25:22 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We have found that expedited GCs too frequently yield very little benefit because they occur so soon after the previous GC that there has been no accumulation of garbage.
>> 
>> The primary motivation for expediting GC is to avoid a situation under which there is so much "promotion" work to be performed that the urgent need to reclaim garbage from young-generation is obstructed by this promotion work.
>> 
>> This PR ends the practice of expediting to support promotion in place.  The effort required to promote a region in place is minimal and unlikely to contend in a major way with young collection efforts.
>> 
>> This PR also reduces the likelihood that we will expedite for promotion by evacuation, because it requires the amount of accumulated promo potential to exceed a particular threshold.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahYoungHeuristics.cpp line 139:
> 
>> 137:   ShenandoahHeap* heap = ShenandoahHeap::heap();
>> 138: 
>> 139:   size_t promo_expedite_threshold = (heap->young_generation()->max_capacity() * ShenandoahEvacReserve) / 512;
> 
> What's the reasoning behind `512` divisor here? Since `ShenandoahEvacReserve` is percents of young/total heap size, this makes `promo_expedite_threshold` about 20% of heap size. Is that what we want?

Err, miscomputed. That would be only `1%` of heap size.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/325#discussion_r1326138138

From kdnilsen at openjdk.org  Thu Sep 14 15:40:39 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 14 Sep 2023 15:40:39 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC
In-Reply-To: <up4hoSo4YcFtJs-W29fkyGGtUt7Ac6vta9oNvxTuA0Y=.b0ba2133-1a58-49f0-9c2a-82e19364554a@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
 <ZUWtSfmGSDQhO3SK--VKDsLJleZNeq7qNQHW17YY3mQ=.7f4dc2d8-05b9-4ce9-b82a-127f153a7039@github.com>
 <up4hoSo4YcFtJs-W29fkyGGtUt7Ac6vta9oNvxTuA0Y=.b0ba2133-1a58-49f0-9c2a-82e19364554a@github.com>
Message-ID: <GmQGynaTy6dwo378o0agKINu2ua9t3k9UVfsf3plegg=.b938d98d-ff58-4cbf-8600-a51a1fc9d491@github.com>

On Thu, 14 Sep 2023 15:32:20 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahYoungHeuristics.cpp line 139:
>> 
>>> 137:   ShenandoahHeap* heap = ShenandoahHeap::heap();
>>> 138: 
>>> 139:   size_t promo_expedite_threshold = (heap->young_generation()->max_capacity() * ShenandoahEvacReserve) / 512;
>> 
>> What's the reasoning behind `512` divisor here? Since `ShenandoahEvacReserve` is percents of young/total heap size, this makes `promo_expedite_threshold` about 20% of heap size. Is that what we want?
>
> Err, miscomputed. That would be only `1%` of heap size.

Yes.  20% of 5%.  The idea: In the rare scenario that we anticipate that promotion will consume over 20% of the evacuation reserve, we'll expedite the cycle so as to improve the likelihood that GC will replenish young allocation pool before it becomes exhausted.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/325#discussion_r1326142487

From shade at openjdk.org  Thu Sep 14 15:40:40 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 14 Sep 2023 15:40:40 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC
In-Reply-To: <GmQGynaTy6dwo378o0agKINu2ua9t3k9UVfsf3plegg=.b938d98d-ff58-4cbf-8600-a51a1fc9d491@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
 <ZUWtSfmGSDQhO3SK--VKDsLJleZNeq7qNQHW17YY3mQ=.7f4dc2d8-05b9-4ce9-b82a-127f153a7039@github.com>
 <up4hoSo4YcFtJs-W29fkyGGtUt7Ac6vta9oNvxTuA0Y=.b0ba2133-1a58-49f0-9c2a-82e19364554a@github.com>
 <GmQGynaTy6dwo378o0agKINu2ua9t3k9UVfsf3plegg=.b938d98d-ff58-4cbf-8600-a51a1fc9d491@github.com>
Message-ID: <CPTu_YevKd0a7RG55HaLfGD_j5mg2L8NfDzkl_5RRDw=.049a8c52-ac2b-4fcb-a0d2-dd299112eace@github.com>

On Thu, 14 Sep 2023 15:35:43 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Err, miscomputed. That would be only `1%` of heap size.
>
> Yes.  20% of 5%.  The idea: In the rare scenario that we anticipate that promotion will consume over 20% of the evacuation reserve, we'll expedite the cycle so as to improve the likelihood that GC will replenish young allocation pool before it becomes exhausted.

All right. Feels like it needs to be a tunable, but hardcoded is okay for now.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/325#discussion_r1326146116

From wkemper at openjdk.org  Thu Sep 14 16:28:28 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 14 Sep 2023 16:28:28 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC
In-Reply-To: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
Message-ID: <cN6HlyP0vOhg1zWMW6pGQNpsgdxExXUcRGdxRSY2cDY=.2c70fd82-3be3-44e7-abdc-579ead7261f1@github.com>

On Thu, 14 Sep 2023 15:21:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> We have found that expedited GCs too frequently yield very little benefit because they occur so soon after the previous GC that there has been no accumulation of garbage.
> 
> The primary motivation for expediting GC is to avoid a situation under which there is so much "promotion" work to be performed that the urgent need to reclaim garbage from young-generation is obstructed by this promotion work.
> 
> This PR ends the practice of expediting to support promotion in place.  The effort required to promote a region in place is minimal and unlikely to contend in a major way with young collection efforts.
> 
> This PR also reduces the likelihood that we will expedite for promotion by evacuation, because it requires the amount of accumulated promo potential to exceed a particular threshold.

Changes requested by wkemper (Committer).

src/hotspot/share/gc/shenandoah/heuristics/shenandoahYoungHeuristics.cpp line 149:

> 147:   }
> 148: 
> 149:   size_t promo_in_place_potential = heap->get_promotion_in_place_potential();

Can we also now delete `ShenandoahHeap::get_promotion_in_place_potential` and all the attendant code that maintains the value? This heuristic was the only place reading it.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/325#pullrequestreview-1627317452
PR Review Comment: https://git.openjdk.org/shenandoah/pull/325#discussion_r1326223128

From wkemper at openjdk.org  Thu Sep 14 16:30:18 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 14 Sep 2023 16:30:18 GMT
Subject: RFR: 8316297: GenShen: Degenerated GCs fail to make progress
In-Reply-To: <jAuhHSnbWdp9E-7JIGAd8CKIVIJHASGxd5NdrBCMTCE=.21816c11-5cc6-4da1-959e-81073d9e3a4e@github.com>
References: <jAuhHSnbWdp9E-7JIGAd8CKIVIJHASGxd5NdrBCMTCE=.21816c11-5cc6-4da1-959e-81073d9e3a4e@github.com>
Message-ID: <NZ32De8Z2CzKZNmEB16slJaEKRgFBu2-Rq1jz6RHtTY=.e69698f0-834e-4dd0-8aae-b3db97797724@github.com>

On Thu, 14 Sep 2023 14:32:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> A recent change to the implementation of ShenandoahYoungGeneration::available() resulted in a budgeting shortfall during degenerated GC.  The problem is that we degenerate after mutator allocations fail, at which time young available memory is in very short supply.  The change to available() caused us to only see the mutator budget, which has been depleted.  This resulted in a very high probability that degenerated GC would fail to make progress, forcing degenerated GC to upgrade to full GC.
> 
> This PR makes the Collector reserve memory visible to the function that computes evacuation budgets, thereby increasing the likelihood of a successful degenerated cycle and obviating the need for a Full GC.

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/324#pullrequestreview-1627320170

From kdnilsen at openjdk.org  Thu Sep 14 17:22:15 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 14 Sep 2023 17:22:15 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC
In-Reply-To: <cN6HlyP0vOhg1zWMW6pGQNpsgdxExXUcRGdxRSY2cDY=.2c70fd82-3be3-44e7-abdc-579ead7261f1@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
 <cN6HlyP0vOhg1zWMW6pGQNpsgdxExXUcRGdxRSY2cDY=.2c70fd82-3be3-44e7-abdc-579ead7261f1@github.com>
Message-ID: <DQ_7YE2oHXd4Jj8F9Aq5xH1b5IggWzg8DnNCGrOThm8=.f3f77d5d-8769-471c-ab0c-826aacb55047@github.com>

On Thu, 14 Sep 2023 16:25:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> We have found that expedited GCs too frequently yield very little benefit because they occur so soon after the previous GC that there has been no accumulation of garbage.
>> 
>> The primary motivation for expediting GC is to avoid a situation under which there is so much "promotion" work to be performed that the urgent need to reclaim garbage from young-generation is obstructed by this promotion work.
>> 
>> This PR ends the practice of expediting to support promotion in place.  The effort required to promote a region in place is minimal and unlikely to contend in a major way with young collection efforts.
>> 
>> This PR also reduces the likelihood that we will expedite for promotion by evacuation, because it requires the amount of accumulated promo potential to exceed a particular threshold.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahYoungHeuristics.cpp line 149:
> 
>> 147:   }
>> 148: 
>> 149:   size_t promo_in_place_potential = heap->get_promotion_in_place_potential();
> 
> Can we also now delete `ShenandoahHeap::get_promotion_in_place_potential` and all the attendant code that maintains the value? This heuristic was the only place reading it.

Good point.  I'll do that.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/325#discussion_r1326296597

From kdnilsen at openjdk.org  Thu Sep 14 17:40:52 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 14 Sep 2023 17:40:52 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC [v2]
In-Reply-To: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
Message-ID: <_0lovBTL2p5XQ0_Q6nuZ8IIZFHsHwEAS1JI5xJyZ9Dk=.8f599043-bf5a-4588-9ab8-98e280cc5cd4@github.com>

> We have found that expedited GCs too frequently yield very little benefit because they occur so soon after the previous GC that there has been no accumulation of garbage.
> 
> The primary motivation for expediting GC is to avoid a situation under which there is so much "promotion" work to be performed that the urgent need to reclaim garbage from young-generation is obstructed by this promotion work.
> 
> This PR ends the practice of expediting to support promotion in place.  The effort required to promote a region in place is minimal and unlikely to contend in a major way with young collection efforts.
> 
> This PR also reduces the likelihood that we will expedite for promotion by evacuation, because it requires the amount of accumulated promo potential to exceed a particular threshold.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Remove ShenandoahHeap::[gs]et_promotion_in_place_potential() everywhere

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/325/files
  - new: https://git.openjdk.org/shenandoah/pull/325/files/44b203a0..de00edc3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=325&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=325&range=00-01

  Stats: 17 lines in 5 files changed: 0 ins; 17 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/325.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/325/head:pull/325

PR: https://git.openjdk.org/shenandoah/pull/325

From wkemper at openjdk.org  Thu Sep 14 18:24:17 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 14 Sep 2023 18:24:17 GMT
Subject: RFR: 8316299: GenShen: Reduce frequency of expedited GC [v2]
In-Reply-To: <_0lovBTL2p5XQ0_Q6nuZ8IIZFHsHwEAS1JI5xJyZ9Dk=.8f599043-bf5a-4588-9ab8-98e280cc5cd4@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
 <_0lovBTL2p5XQ0_Q6nuZ8IIZFHsHwEAS1JI5xJyZ9Dk=.8f599043-bf5a-4588-9ab8-98e280cc5cd4@github.com>
Message-ID: <DRgWU8_Wth_EWBgElOE97_AOtGKJ3WgyDH24oiOffSM=.8f235782-9fd5-4481-a313-33442dda2047@github.com>

On Thu, 14 Sep 2023 17:40:52 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> We have found that expedited GCs too frequently yield very little benefit because they occur so soon after the previous GC that there has been no accumulation of garbage.
>> 
>> The primary motivation for expediting GC is to avoid a situation under which there is so much "promotion" work to be performed that the urgent need to reclaim garbage from young-generation is obstructed by this promotion work.
>> 
>> This PR ends the practice of expediting to support promotion in place.  The effort required to promote a region in place is minimal and unlikely to contend in a major way with young collection efforts.
>> 
>> This PR also reduces the likelihood that we will expedite for promotion by evacuation, because it requires the amount of accumulated promo potential to exceed a particular threshold.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove ShenandoahHeap::[gs]et_promotion_in_place_potential() everywhere

Looks good - thanks!

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/325#pullrequestreview-1627514315

From wkemper at openjdk.org  Thu Sep 14 21:10:31 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 14 Sep 2023 21:10:31 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <whPi4kQskos00QeV1ZvSuOuhXeFtK1IBcIWGah10gX4=.3f8c2aa7-b493-41c5-90b9-9717cc0cdee9@github.com>
References: <whPi4kQskos00QeV1ZvSuOuhXeFtK1IBcIWGah10gX4=.3f8c2aa7-b493-41c5-90b9-9717cc0cdee9@github.com>
Message-ID: <6F9odrfdkvD0THPMCqJm30I8lkRtEaTzGEt5twRpLzk=.4dc47bfc-cec2-481f-8636-ceed44cc3ae7@github.com>

> Merges tag jdk-22+15

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 99 commits:

 - Merge tag 'jdk-22+15' into merge-jdk-22+15
   
   Added tag jdk-22+15 for changeset ce93d27f
 - 6228794: java.text.ChoiceFormat pattern behavior is not well documented.
   
   Reviewed-by: naoto
 - 8314226: Series of colon-style fallthrough switch cases with guards compiled incorrectly
   
   Reviewed-by: mcimadamore, vromero
 - 8316087: Test SignedLoggerFinderTest.java is still failing
   
   Reviewed-by: dfuchs
 - 8315934: RISC-V: Disable conservative fences per vendor
   
   Reviewed-by: rehn, mli, fyang
 - 8316021: Serial: Remove unused Generation::post_compact
   
   Reviewed-by: tschatzl
 - 8316050: Use hexadecimal encoding in MemorySegment::toString
   
   Reviewed-by: rriggs, mcimadamore
 - 8314612: TestUnorderedReduction.java fails with -XX:MaxVectorSize=32 and -XX:+AlignVector
   
   Reviewed-by: chagedorn, kvn
 - 8311207: Cleanup for Optimization for UUID.toString
   
   Reviewed-by: liach, rriggs
 - 8315898: Open source swing JMenu tests
   
   Reviewed-by: serb
 - ... and 89 more: https://git.openjdk.org/shenandoah/compare/ba88eb1d...94bb1e88

-------------

Changes: https://git.openjdk.org/shenandoah/pull/323/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=323&range=01
  Stats: 24505 lines in 1389 files changed: 12059 ins; 8356 del; 4090 mod
  Patch: https://git.openjdk.org/shenandoah/pull/323.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/323/head:pull/323

PR: https://git.openjdk.org/shenandoah/pull/323

From cslucas at openjdk.org  Thu Sep 14 22:16:39 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 14 Sep 2023 22:16:39 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <GxzTnW0VOuKOfPP9vUaguO8cRjrhKDXLcduc0uHElG4=.34679448-aefc-4cfe-bf90-c0febfe617f5@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
 <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
 <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>
 <3_ThxcuU3e_hPvWi4lJBfXsyG4Ky_eyyifbkZ2izlKQ=.0070b59a-31ae-4ede-9625-a9e4bf3b7a16@github.com>
 <jqZc_VR69436ChXcaqwMMgYzIMYtiOFVLVG2QoQUAJc=.0ee1ff3b-0cfa-4594-84e8-508cde6703b5@github.com>
 <GxzTnW0VOuKOfPP9vUaguO8cRjrhKDXLcduc0uHElG4=.34679448-aefc-4cfe-bf90-c0febfe617f5@github.com>
Message-ID: <bnXt9FaproaunGvlxW2ktp4l5Nve738SI1wuVtZcfMU=.54f21489-0882-4bfc-a388-b11ac1d541e4@github.com>

On Thu, 31 Aug 2023 20:42:39 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

>> Thanks for clarifying @iwanowww . I think I see your point now. My original intent was to just move these methods out of PhaseMacroExpand and not much else. 
>> 
>> I'm going to do some more refactoring and patch all users of these make methods to just use this single method: `static Node* make(PhaseIterGVN& igvn, Node* base, Node* ptr, Node* offset)`. What do you think?
>
> Sounds good. 
> 
> But in the future I'd like to see `PhaseMacroExpand` and `PhaseIdealLoop` migrated to `GraphKit` instead.

@iwanowww do you think I should just withdraw this PR and close the associated RFE?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1326567520

From kdnilsen at openjdk.org  Fri Sep 15 00:13:17 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 15 Sep 2023 00:13:17 GMT
Subject: Integrated: 8316297: GenShen: Degenerated GCs fail to make progress
In-Reply-To: <jAuhHSnbWdp9E-7JIGAd8CKIVIJHASGxd5NdrBCMTCE=.21816c11-5cc6-4da1-959e-81073d9e3a4e@github.com>
References: <jAuhHSnbWdp9E-7JIGAd8CKIVIJHASGxd5NdrBCMTCE=.21816c11-5cc6-4da1-959e-81073d9e3a4e@github.com>
Message-ID: <q6Biop2-915a-QIFIeGcK3dZaW1g4XHXNxOmO-waMn8=.fb0b705b-75c4-4892-955e-4822e863458a@github.com>

On Thu, 14 Sep 2023 14:32:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> A recent change to the implementation of ShenandoahYoungGeneration::available() resulted in a budgeting shortfall during degenerated GC.  The problem is that we degenerate after mutator allocations fail, at which time young available memory is in very short supply.  The change to available() caused us to only see the mutator budget, which has been depleted.  This resulted in a very high probability that degenerated GC would fail to make progress, forcing degenerated GC to upgrade to full GC.
> 
> This PR makes the Collector reserve memory visible to the function that computes evacuation budgets, thereby increasing the likelihood of a successful degenerated cycle and obviating the need for a Full GC.

This pull request has now been integrated.

Changeset: 88365d18
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/88365d181181e9d3fb267229691843e0c2971c00
Stats:     18 lines in 3 files changed: 14 ins; 0 del; 4 mod

8316297: GenShen: Degenerated GCs fail to make progress

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/324

From kdnilsen at openjdk.org  Fri Sep 15 03:08:24 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 15 Sep 2023 03:08:24 GMT
Subject: Integrated: 8316299: GenShen: Reduce frequency of expedited GC
In-Reply-To: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
References: <IdVxVPHyWAUmDPIA7x34AyFaxM8LkqgjnClqAx5-fpg=.0e8bcb53-6998-4cbc-bd69-44a2b681d2f4@github.com>
Message-ID: <jr5yqN2ni_txrL1BB5nxavAM6TWMjcRDjErTfy1UlzI=.e0b5157f-ac16-4adf-929d-3b1d11dc81a3@github.com>

On Thu, 14 Sep 2023 15:21:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> We have found that expedited GCs too frequently yield very little benefit because they occur so soon after the previous GC that there has been no accumulation of garbage.
> 
> The primary motivation for expediting GC is to avoid a situation under which there is so much "promotion" work to be performed that the urgent need to reclaim garbage from young-generation is obstructed by this promotion work.
> 
> This PR ends the practice of expediting to support promotion in place.  The effort required to promote a region in place is minimal and unlikely to contend in a major way with young collection efforts.
> 
> This PR also reduces the likelihood that we will expedite for promotion by evacuation, because it requires the amount of accumulated promo potential to exceed a particular threshold.

This pull request has now been integrated.

Changeset: aa1063ff
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/aa1063ff5eac93b2ddf71f3ef71cb4ea92521427
Stats:     31 lines in 6 files changed: 2 ins; 28 del; 1 mod

8316299: GenShen: Reduce frequency of expedited GC

Reviewed-by: shade, wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/325

From wkemper at openjdk.org  Fri Sep 15 16:35:37 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Sep 2023 16:35:37 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <whPi4kQskos00QeV1ZvSuOuhXeFtK1IBcIWGah10gX4=.3f8c2aa7-b493-41c5-90b9-9717cc0cdee9@github.com>
References: <whPi4kQskos00QeV1ZvSuOuhXeFtK1IBcIWGah10gX4=.3f8c2aa7-b493-41c5-90b9-9717cc0cdee9@github.com>
Message-ID: <ZcPBIRT5bW2pRfPae1Wd_xUlGzbd7G3bCqaGCaijgYk=.775763aa-b24c-4814-9800-b41c1c99965a@github.com>

On Thu, 14 Sep 2023 14:16:50 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+15

This pull request has now been integrated.

Changeset: cb662b93
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/cb662b93b2c8dd0ed404ad24084bf9befd5f191b
Stats:     24505 lines in 1389 files changed: 12059 ins; 8356 del; 4090 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/323

From vlivanov at openjdk.org  Sat Sep 16 00:48:49 2023
From: vlivanov at openjdk.org (Vladimir Ivanov)
Date: Sat, 16 Sep 2023 00:48:49 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <bnXt9FaproaunGvlxW2ktp4l5Nve738SI1wuVtZcfMU=.54f21489-0882-4bfc-a388-b11ac1d541e4@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
 <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
 <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>
 <3_ThxcuU3e_hPvWi4lJBfXsyG4Ky_eyyifbkZ2izlKQ=.0070b59a-31ae-4ede-9625-a9e4bf3b7a16@github.com>
 <jqZc_VR69436ChXcaqwMMgYzIMYtiOFVLVG2QoQUAJc=.0ee1ff3b-0cfa-4594-84e8-508cde6703b5@github.com>
 <GxzTnW0VOuKOfPP9vUaguO8cRjrhKDXLcduc0uHElG4=.34679448-aefc-4cfe-bf90-c0febfe617f5@github.com>
 <bnXt9FaproaunGvlxW2ktp4l5Nve738SI1wuVtZcfMU=.54f21489-0882-4bfc-a388-b11ac1d541e4@github.com>
Message-ID: <eigk-mW2Mnsj47-8nGYP7cE76FOHt00MmAQQQdDdkkk=.13b0bfeb-ff9f-472d-8b99-4fcb58db29a0@github.com>

On Thu, 14 Sep 2023 22:14:05 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> Sounds good. 
>> 
>> But in the future I'd like to see `PhaseMacroExpand` and `PhaseIdealLoop` migrated to `GraphKit` instead.
>
> @iwanowww do you think I should just withdraw this PR and close the associated RFE?

It's up to you, Cesar. I find existing code good enough, but also I don't have anything against your proposal.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1327878747

From wkemper at openjdk.org  Mon Sep 18 16:27:48 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Sep 2023 16:27:48 GMT
Subject: RFR: 8315560: GenShen: assert failed: Object klass pointer must go to
 metaspace
Message-ID: <QK5cXewv_HgDXMNrAjaRHw2IH7rbN5R4qee8G2yJDjE=.fb14e78f-b147-4585-ac85-faac79a92d4f@github.com>

The SATB barrier which is on during concurrent marking of old may cause pointers into the collection set to be held by the SATB buffer. When a concurrent cycle degenerates during or before the root scan, these buffers are purged. However, they must also be purged at the end of a degenerated cycle. The buffers cannot be purged unconditionally at the beginning of a degenerated cycle because it could lose pointers necessary to complete the concurrent mark of the young generation.

-------------

Commit messages:
 - Fix typo in comment
 - Update comment
 - Merge remote-tracking branch 'shenandoah/master' into purge-old-satb-after-degen-update-refs
 - Purge old SATB at end of degen cycle (in case we didn't at the start)

Changes: https://git.openjdk.org/shenandoah/pull/326/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=326&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315560
  Stats: 7 lines in 1 file changed: 7 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/326.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/326/head:pull/326

PR: https://git.openjdk.org/shenandoah/pull/326

From kdnilsen at openjdk.org  Mon Sep 18 17:15:31 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 18 Sep 2023 17:15:31 GMT
Subject: RFR: 8315560: GenShen: assert failed: Object klass pointer must go
 to metaspace
In-Reply-To: <QK5cXewv_HgDXMNrAjaRHw2IH7rbN5R4qee8G2yJDjE=.fb14e78f-b147-4585-ac85-faac79a92d4f@github.com>
References: <QK5cXewv_HgDXMNrAjaRHw2IH7rbN5R4qee8G2yJDjE=.fb14e78f-b147-4585-ac85-faac79a92d4f@github.com>
Message-ID: <BzUNzHlShpvgwelNgKKqyxgg8jB0dUFQAVFzGMbreoM=.ba9ef819-45ab-4424-ad97-1d726a9653c1@github.com>

On Mon, 18 Sep 2023 16:21:22 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The SATB barrier which is on during concurrent marking of old may cause pointers into the collection set to be held by the SATB buffer. When a concurrent cycle degenerates during or before the root scan, these buffers are purged. However, they must also be purged at the end of a degenerated cycle. The buffers cannot be purged unconditionally at the beginning of a degenerated cycle because it could lose pointers necessary to complete the concurrent mark of the young generation.

As discussed off-channel, there may be compounding issues that trigger this same or similar error message.  My rr recording of this symptom might suggest that there's something else at play here, but I agree that this fix is necessary.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/326#pullrequestreview-1631606983

From wkemper at openjdk.org  Mon Sep 18 18:01:28 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Sep 2023 18:01:28 GMT
Subject: Integrated: 8315560: GenShen: assert failed: Object klass pointer must
 go to metaspace
In-Reply-To: <QK5cXewv_HgDXMNrAjaRHw2IH7rbN5R4qee8G2yJDjE=.fb14e78f-b147-4585-ac85-faac79a92d4f@github.com>
References: <QK5cXewv_HgDXMNrAjaRHw2IH7rbN5R4qee8G2yJDjE=.fb14e78f-b147-4585-ac85-faac79a92d4f@github.com>
Message-ID: <GzkbulxIGH0KewcsVGLq3QIF7Ce4WDQqN2V3LdNi6-M=.29a29f0d-0a6b-4787-a4bb-8a3ea99852f7@github.com>

On Mon, 18 Sep 2023 16:21:22 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The SATB barrier which is on during concurrent marking of old may cause pointers into the collection set to be held by the SATB buffer. When a concurrent cycle degenerates during or before the root scan, these buffers are purged. However, they must also be purged at the end of a degenerated cycle. The buffers cannot be purged unconditionally at the beginning of a degenerated cycle because it could lose pointers necessary to complete the concurrent mark of the young generation.

This pull request has now been integrated.

Changeset: 7f9b983d
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/7f9b983d57da8a4416ce7a0d217cf516aaea7cd0
Stats:     7 lines in 1 file changed: 7 ins; 0 del; 0 mod

8315560: GenShen: assert failed: Object klass pointer must go to metaspace

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/326

From kdnilsen at openjdk.org  Tue Sep 19 19:43:41 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 19 Sep 2023 19:43:41 GMT
Subject: RFR: 8312116  GenShen: make instantaneous allocation rate triggers
 more timely
Message-ID: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>

When heuristics fail to trigger because an instantaneous "allocation spike" is not so large as to consume all available memory before GC completes, this assessment is based on an assumption that the allocation rate remains constant, and it ignores the time that will be lost due to the _interval between consecutive allocation spike measurements.

This PR watches for "acceleration" of allocation rates.  When acceleration of allocation is detected in 3 consecutive allocation spike measurements, it calculates a best-fit curve (assuming constant acceleration) and predicts the memory to be consumed during the time that spans both the next sample interval and the GC effort that follows it.  If the memory to be allocated according to anticipated acceleration of allocations during this time span exceeds what is available, we trigger immediately.

-------------

Commit messages:
 - Add every sample to history but only trigger if spiking
 - Change _last_trigger to OTHER for accelerated-rate trigger
 - Fix white space
 - Improve log message and fix white space
 - Fixup errors in computation of accelerated memory consumption
 - Merge remote-tracking branch 'origin' into make-instantaneous-alloc-rate-trigger-quicker
 - Add a new form of allocation spike trigger for accelerating alloc rates

Changes: https://git.openjdk.org/shenandoah/pull/327/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=327&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8312116
  Stats: 306 lines in 5 files changed: 297 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/shenandoah/pull/327.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/327/head:pull/327

PR: https://git.openjdk.org/shenandoah/pull/327

From kdnilsen at openjdk.org  Tue Sep 19 19:43:42 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 19 Sep 2023 19:43:42 GMT
Subject: RFR: 8312116 GenShen: make instantaneous allocation rate triggers
 more timely
In-Reply-To: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>
References: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>
Message-ID: <AlB38St_jDfUX4tHPSYN2A53bPitgYHk2HJ2_az28nk=.ed9dcb65-95ec-44de-b14e-b0ded91695d2@github.com>

On Mon, 18 Sep 2023 17:07:41 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> When heuristics fail to trigger because an instantaneous "allocation spike" is not so large as to consume all available memory before GC completes, this assessment is based on an assumption that the allocation rate remains constant, and it ignores the time that will be lost due to the _interval between consecutive allocation spike measurements.
> 
> This PR watches for "acceleration" of allocation rates.  When acceleration of allocation is detected in 3 consecutive allocation spike measurements, it calculates a best-fit curve (assuming constant acceleration) and predicts the memory to be consumed during the time that spans both the next sample interval and the GC effort that follows it.  If the memory to be allocated according to anticipated acceleration of allocations during this time span exceeds what is available, we trigger immediately.

I'm leaving this in draft mode until I complete functional and performance testing and remove the temporary instrumentation code.

Experiments with various workloads show these changes to decrease the number of degenerated GCs.  In some cases, the greater sensitivity of the acceleration trigger causes more frequent GCs than without it.  I am experimenting with various configuration options on multiple workloads to determine an "optimal" default configuration.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/327#issuecomment-1724018834
PR Comment: https://git.openjdk.org/shenandoah/pull/327#issuecomment-1726363501

From cslucas at openjdk.org  Tue Sep 19 21:51:41 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Tue, 19 Sep 2023 21:51:41 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <eigk-mW2Mnsj47-8nGYP7cE76FOHt00MmAQQQdDdkkk=.13b0bfeb-ff9f-472d-8b99-4fcb58db29a0@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
 <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
 <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>
 <3_ThxcuU3e_hPvWi4lJBfXsyG4Ky_eyyifbkZ2izlKQ=.0070b59a-31ae-4ede-9625-a9e4bf3b7a16@github.com>
 <jqZc_VR69436ChXcaqwMMgYzIMYtiOFVLVG2QoQUAJc=.0ee1ff3b-0cfa-4594-84e8-508cde6703b5@github.com>
 <GxzTnW0VOuKOfPP9vUaguO8cRjrhKDXLcduc0uHElG4=.34679448-aefc-4cfe-bf90-c0febfe617f5@github.com>
 <bnXt9FaproaunGvlxW2ktp4l5Nve738SI1wuVtZcfMU=.54f21489-0882-4bfc-a388-b11ac1d541e4@github.com>
 <eigk-mW2Mnsj47-8nGYP7cE76FOHt00MmAQQQdDdkkk=.13b0bfeb-ff9f-472d-8b99-4fcb58db29a0@github.com>
Message-ID: <dg318KW7REYpT8basBffiq8HB3ZEwxB2vBBODvSIiAk=.2ffdc013-0af8-4453-b17e-184641198bdb@github.com>

On Sat, 16 Sep 2023 00:45:56 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

>> @iwanowww do you think I should just withdraw this PR and close the associated RFE?
>
> It's up to you, Cesar. I find existing code good enough, but also I don't have anything against your proposal.

Thanks, Vladimir. Since the work is already done and it may improve things slightly, I'll keep the PR open.
I created this RFE for migrating PhaseMacroExpand, PhaseIdealLoop to GraphKit: https://bugs.openjdk.org/browse/JDK-8316560

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1330741302

From kvn at openjdk.org  Tue Sep 19 22:39:41 2023
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Tue, 19 Sep 2023 22:39:41 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v4]
In-Reply-To: <sNp-9t72PQRHZL_AjB8NqE8Ji5aJNubHqvtBRjBSunI=.d488ae9f-ff3e-453e-8851-1fd08dbd2526@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <sNp-9t72PQRHZL_AjB8NqE8Ji5aJNubHqvtBRjBSunI=.d488ae9f-ff3e-453e-8851-1fd08dbd2526@github.com>
Message-ID: <oTslu7rJlYIn7ZN8N6c8Vvzfy1mB26hINFlpao-AIP4=.497c9ff5-925c-4d70-8400-3516c0fa6c6f@github.com>

On Thu, 31 Aug 2023 21:56:07 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
>> 
>> Tested with tier1-3.
>
> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Convert uses of AddPNode::make to new AddPNode

I don't like the last update. Now you may create AddP node even if offset is 0. Yes, AddPNode::Identity() will fix that but it will happen only later during IGVN transform. The only code which checks offset in BarrierSetC2::obj_allocate is hard to read.

The duplicated code was factored into `basic_plus_adr()` for that reason - to simplify code in uses.

May be we should put these changes on hold until after you look on JDK-8316560.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15480#issuecomment-1726631114

From cslucas at openjdk.org  Wed Sep 20 16:53:55 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Wed, 20 Sep 2023 16:53:55 GMT
Subject: Withdrawn: JDK-8315279: Factor 'basic_plus_adr' out of
 PhaseMacroExpand and delete make_load/store
In-Reply-To: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
Message-ID: <Hf15gIl09JJswiaRVTGa1hXSLJnvIRidCfVoODs3pvo=.e366a488-4e26-4db3-8b78-f8346209b447@github.com>

On Wed, 30 Aug 2023 03:13:08 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
> 
> Tested with tier1-3.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/15480

From wkemper at openjdk.org  Wed Sep 20 18:14:53 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Sep 2023 18:14:53 GMT
Subject: Withdrawn: 8315950: Shenandoah: Improvements to ShenandoahInitLogger
In-Reply-To: <KAOiKOZsl7X_JS4A3EBXdhalL7kZg8s8Hv2BqNXnbP8=.cde8da04-daa7-4914-acc2-1450249c5a5e@github.com>
References: <KAOiKOZsl7X_JS4A3EBXdhalL7kZg8s8Hv2BqNXnbP8=.cde8da04-daa7-4914-acc2-1450249c5a5e@github.com>
Message-ID: <MhBW8XQ7yniKtJcqOBsaxqJ_YgNuvCOq6SCtsWizrbI=.ddc385b9-2403-429f-9319-256ebac8690b@github.com>

On Fri, 8 Sep 2023 21:11:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Changes to ShenandoahInitLogger:
> * Use macro to simplify reporting of sizes
> * Separate heap reporting from gc-specific reporting
> 
> Call the init logger from a virtual method to facilitate isolation of different gc modes.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/15646

From wkemper at openjdk.org  Wed Sep 20 22:49:50 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Sep 2023 22:49:50 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
Message-ID: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>

Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

-------------

Commit messages:
 - Fix 32-bit build error
 - Do not use atomics in header
 - Signal gc threshold exceeded when appropriate

Changes: https://git.openjdk.org/jdk/pull/15852/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8316632
  Stats: 55 lines in 6 files changed: 42 ins; 3 del; 10 mod
  Patch: https://git.openjdk.org/jdk/pull/15852.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15852/head:pull/15852

PR: https://git.openjdk.org/jdk/pull/15852

From kdnilsen at openjdk.org  Wed Sep 20 23:06:41 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 20 Sep 2023 23:06:41 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <q9-uNs7BNVFQLm0-1ho0VbL_rKYTfwTCkQPN_Erd_6g=.3adb2769-0729-4022-921c-a0a9dd0c3053@github.com>

On Wed, 20 Sep 2023 22:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

Looks good to me.

-------------

Marked as reviewed by kdnilsen (no project role).

PR Review: https://git.openjdk.org/jdk/pull/15852#pullrequestreview-1636619801

From shade at openjdk.org  Thu Sep 21 08:00:42 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Sep 2023 08:00:42 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <ipTPp67VBeVboYM-HhnzsTH4Q_u_bIXn_nZ5tg0TLsI=.b13f1f8d-d981-4b1c-873d-ad2f1d100c9e@github.com>

On Wed, 20 Sep 2023 22:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 940:

> 938:   *gc_overhead_limit_was_exceeded = false;
> 939:   ShenandoahAllocRequest req = ShenandoahAllocRequest::for_shared(size);
> 940:   if (get_gc_no_progress_count() > ShenandoahNoProgressThreshold) {

Move this block to `ShenandoahHeap::allocate_memory`? This would require adding a (default) argument to that method, but it is a good tradeoff for putting all mechanics around this into a single method.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1332636658

From shade at openjdk.org  Thu Sep 21 08:31:42 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Sep 2023 08:31:42 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <DHaBwzvt2pjlhHCmGb_AKZVCB906jVsLq6iO6M_BNAw=.a7bf1655-86a9-442f-a811-e10eaf2e149e@github.com>

On Wed, 20 Sep 2023 22:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

Please also run `tier1 tier2 tier3` with Shenandoah? In `hotspot:tier1`, I am seeing the failure here:


TEST: gc/InfiniteList.java

STDERR:
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.base/java.util.jar.Manifest$FastInputStream.<init>(Manifest.java:421)
        at java.base/java.util.jar.Manifest$FastInputStream.<init>(Manifest.java:416)
        at java.base/java.util.jar.Manifest.read(Manifest.java:287)
        at java.base/java.util.jar.Manifest.<init>(Manifest.java:101)
        at java.base/java.util.jar.Manifest.<init>(Manifest.java:88)
        at java.base/java.util.jar.JarFile.getManifestFromReference(JarFile.java:431)
        at java.base/java.util.jar.JarFile.getManifest(JarFile.java:405)
        at java.base/jdk.internal.loader.URLClassPath$JarLoader$2.getManifest(URLClassPath.java:853)
        at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:848)
        at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
        at com.sun.javatest.regtest.agent.MainWrapper.main(MainWrapper.java:94)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15852#issuecomment-1729099298

From wkemper at openjdk.org  Thu Sep 21 14:23:47 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Sep 2023 14:23:47 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <fLS6taGyP2pOsW5wl6Hx_6wGUbFXb5gvxhIpbx-skw4=.76ea649f-e411-4235-a260-1883541accd2@github.com>

Merges tag jdk-22+16

-------------

Commit messages:
 - 8316627: JViewport Test headless failure
 - 8316156: ByteArrayInputStream.transferTo causes MaxDirectMemorySize overflow
 - 8316532: Native library copying in BuildMicrobenchmark.gmk cause dups on macOS
 - 8315869: UseHeavyMonitors not used
 - 8316562: serviceability/sa/jmap-hprof/JMapHProfLargeHeapTest.java times out after JDK-8314829
 - 8296246: Update Unicode Data Files to Version 15.1.0
 - 8316149: Open source several Swing JTree JViewport KeyboardManager tests
 - 8315880: change LockingMode default from LM_LEGACY to LM_LIGHTWEIGHT
 - 8316427: Duplicated code for {obj,type}ArrayKlass::array_klass
 - 8315981: Opensource five more random Swing tests
 - ... and 101 more: https://git.openjdk.org/shenandoah/compare/ce93d27f...c04c9ea3

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/328/files
  Stats: 59801 lines in 1086 files changed: 16824 ins; 11605 del; 31372 mod
  Patch: https://git.openjdk.org/shenandoah/pull/328.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/328/head:pull/328

PR: https://git.openjdk.org/shenandoah/pull/328

From wkemper at openjdk.org  Thu Sep 21 16:10:40 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Sep 2023 16:10:40 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <ipTPp67VBeVboYM-HhnzsTH4Q_u_bIXn_nZ5tg0TLsI=.b13f1f8d-d981-4b1c-873d-ad2f1d100c9e@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <ipTPp67VBeVboYM-HhnzsTH4Q_u_bIXn_nZ5tg0TLsI=.b13f1f8d-d981-4b1c-873d-ad2f1d100c9e@github.com>
Message-ID: <JaSuL0-E8LwvKVMvq7AgCRh69-LsCj42ouh-f00GMxo=.36229503-b5cb-4574-bded-ffcedbfa85d8@github.com>

On Thu, 21 Sep 2023 07:56:17 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 940:
> 
>> 938:   *gc_overhead_limit_was_exceeded = false;
>> 939:   ShenandoahAllocRequest req = ShenandoahAllocRequest::for_shared(size);
>> 940:   if (get_gc_no_progress_count() > ShenandoahNoProgressThreshold) {
> 
> Move this block to `ShenandoahHeap::allocate_memory`? This would require adding a (default) argument to that method, but it is a good tradeoff for putting all mechanics around this into a single method.

I put it here to keep it on the slow(er) path so that allocations for LABs won't have additional checks on the progress counter. Also, a failed LAB allocation will come back to `mem_allocate` to try for a shared allocation.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333308520

From wkemper at openjdk.org  Thu Sep 21 16:13:45 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Sep 2023 16:13:45 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <tZkx9aMolB-lkukbq6ZZB0Zgr7tkROHN_qJxgf1agek=.50b41ecf-de42-459c-846b-c8b1a52839ad@github.com>

On Wed, 20 Sep 2023 22:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

I will run additional tests with Shenandoah enabled.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15852#issuecomment-1729887100

From wkemper at openjdk.org  Thu Sep 21 19:38:49 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Sep 2023 19:38:49 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <fLS6taGyP2pOsW5wl6Hx_6wGUbFXb5gvxhIpbx-skw4=.76ea649f-e411-4235-a260-1883541accd2@github.com>
References: <fLS6taGyP2pOsW5wl6Hx_6wGUbFXb5gvxhIpbx-skw4=.76ea649f-e411-4235-a260-1883541accd2@github.com>
Message-ID: <m10F4NBbzOcZy15GfapXFHMdR_xNrN0Lkq_bshH77fw=.12856794-d3f7-4edd-91fb-5de6e62a0d75@github.com>

> Merges tag jdk-22+16

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/328/files
  - new: https://git.openjdk.org/shenandoah/pull/328/files/c04c9ea3..c04c9ea3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=328&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=328&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/328.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/328/head:pull/328

PR: https://git.openjdk.org/shenandoah/pull/328

From wkemper at openjdk.org  Thu Sep 21 19:38:52 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Sep 2023 19:38:52 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <fLS6taGyP2pOsW5wl6Hx_6wGUbFXb5gvxhIpbx-skw4=.76ea649f-e411-4235-a260-1883541accd2@github.com>
References: <fLS6taGyP2pOsW5wl6Hx_6wGUbFXb5gvxhIpbx-skw4=.76ea649f-e411-4235-a260-1883541accd2@github.com>
Message-ID: <xDkQOCEOeXQYsCruDEBVqe4V1hxH4KeG0MhPAT-EuFM=.3ecf8741-2620-4b40-9033-18bc6fbda30d@github.com>

On Thu, 21 Sep 2023 14:16:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+16

This pull request has now been integrated.

Changeset: 88295152
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/88295152c4dfa0efba94c9209efecd3c6fff5a43
Stats:     59801 lines in 1086 files changed: 16824 ins; 11605 del; 31372 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/328

From ysr at openjdk.org  Thu Sep 21 21:51:12 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 21 Sep 2023 21:51:12 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <m2bTUSJx89KdQKDQoEH13JY1kTv1Us2gPjLmIvb9QBE=.5fe5af2e-0245-4802-a7e7-0e6f5e2aab2b@github.com>

On Wed, 20 Sep 2023 22:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

I have a few questions; thanks for clearing up my confusion! :-)

src/hotspot/share/gc/shenandoah/shenandoahControlThread.hpp line 124:

> 122:   ~ShenandoahControlThread();
> 123: 
> 124:   // Handle allocation failure from normal allocation.

Re "normal allocation" : what is not a normal (i.e. abnormal?) allocation?

src/hotspot/share/gc/shenandoah/shenandoahControlThread.hpp line 125:

> 123: 
> 124:   // Handle allocation failure from normal allocation.
> 125:   // Optionally blocks while collector is handling the failure.

Who is allowed to call the non-blocking version? What is the semantics of block or don't block? Can the documentation be extended to elaborate on its use?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 947:

> 945:     control_thread()->handle_alloc_failure(req, false);
> 946:     notify_gc_progress();
> 947:     *gc_overhead_limit_was_exceeded = true;

I am confused. We have noted that we did more than `ShenandoahNoProgressThreshold` collections at this point. So we will fail the allocation request, set `gc_overhead_limit_was_exceeded`, and return `null` to the requester.

What happens at the caller, in response to this set of conditions?

Why then does it make sense to `notify_gc_progress()`, which clears the counter of `_gc_no_progress_count`? Where was the ostensible progress in this case?

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 292:

> 290:   ShenandoahSharedFlag   _concurrent_strong_root_in_progress;
> 291: 
> 292:   size_t _no_gc_progress_count;

Can we call this `_gc_no_progress_count` instead, so it's consistent with the corresponding `get_` method? The difference in naming is unnecessary at best and potentially confusing for maintenance in the long run, at worst.

-------------

PR Review: https://git.openjdk.org/jdk/pull/15852#pullrequestreview-1638713120
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333628772
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333629455
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333633859
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333608047

From wkemper at openjdk.org  Thu Sep 21 21:57:10 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Sep 2023 21:57:10 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <m2bTUSJx89KdQKDQoEH13JY1kTv1Us2gPjLmIvb9QBE=.5fe5af2e-0245-4802-a7e7-0e6f5e2aab2b@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <m2bTUSJx89KdQKDQoEH13JY1kTv1Us2gPjLmIvb9QBE=.5fe5af2e-0245-4802-a7e7-0e6f5e2aab2b@github.com>
Message-ID: <fIDCCApdrdxKXQMxa7ovnnKFMBNdQjJQtecMyjMjUxs=.2d4be08a-bdee-4642-aae4-21b605da2d58@github.com>

On Thu, 21 Sep 2023 21:39:22 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).
>
> src/hotspot/share/gc/shenandoah/shenandoahControlThread.hpp line 124:
> 
>> 122:   ~ShenandoahControlThread();
>> 123: 
>> 124:   // Handle allocation failure from normal allocation.
> 
> Re "normal allocation" : what is not a normal (i.e. abnormal?) allocation?

I believe 'normal' here is a mutator allocation. I can update this comment.

> src/hotspot/share/gc/shenandoah/shenandoahControlThread.hpp line 125:
> 
>> 123: 
>> 124:   // Handle allocation failure from normal allocation.
>> 125:   // Optionally blocks while collector is handling the failure.
> 
> Who is allowed to call the non-blocking version? What is the semantics of block or don't block? Can the documentation be extended to elaborate on its use?

When a mutator allocation fails, it will block on this call and wait to be notified by the control thread to retry the allocation. The notification follows a degenerated or full GC.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333637391
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333638234

From wkemper at openjdk.org  Thu Sep 21 22:00:11 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Sep 2023 22:00:11 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <fIDCCApdrdxKXQMxa7ovnnKFMBNdQjJQtecMyjMjUxs=.2d4be08a-bdee-4642-aae4-21b605da2d58@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <m2bTUSJx89KdQKDQoEH13JY1kTv1Us2gPjLmIvb9QBE=.5fe5af2e-0245-4802-a7e7-0e6f5e2aab2b@github.com>
 <fIDCCApdrdxKXQMxa7ovnnKFMBNdQjJQtecMyjMjUxs=.2d4be08a-bdee-4642-aae4-21b605da2d58@github.com>
Message-ID: <_RnYdoISAUxRvoPwjrjvADl3MDQuZEj3FF68WtkBc-g=.ef09a2b9-1826-4151-88c7-8359533c0791@github.com>

On Thu, 21 Sep 2023 21:54:14 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahControlThread.hpp line 125:
>> 
>>> 123: 
>>> 124:   // Handle allocation failure from normal allocation.
>>> 125:   // Optionally blocks while collector is handling the failure.
>> 
>> Who is allowed to call the non-blocking version? What is the semantics of block or don't block? Can the documentation be extended to elaborate on its use?
>
> When a mutator allocation fails, it will block on this call and wait to be notified by the control thread to retry the allocation. The notification follows a degenerated or full GC.

With this change, the mutator is also allowed to call the non-blocking version when it cannot allocate because the GC overhead has been exceeded. I didn't make sense to me to have the mutator block and wait for a GC cycle when the GC threshold is already exceeded.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333640109

From wkemper at openjdk.org  Thu Sep 21 22:12:13 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Sep 2023 22:12:13 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <m2bTUSJx89KdQKDQoEH13JY1kTv1Us2gPjLmIvb9QBE=.5fe5af2e-0245-4802-a7e7-0e6f5e2aab2b@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <m2bTUSJx89KdQKDQoEH13JY1kTv1Us2gPjLmIvb9QBE=.5fe5af2e-0245-4802-a7e7-0e6f5e2aab2b@github.com>
Message-ID: <HZH1n319_XKi4iQlJ7seIVYf0NfBWR1aUIxaqmWFT-k=.3e328814-5f5a-4eab-a0de-dc9d3d903127@github.com>

On Thu, 21 Sep 2023 21:47:19 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 947:
> 
>> 945:     control_thread()->handle_alloc_failure(req, false);
>> 946:     notify_gc_progress();
>> 947:     *gc_overhead_limit_was_exceeded = true;
> 
> I am confused. We have noted that we did more than `ShenandoahNoProgressThreshold` collections at this point. So we will fail the allocation request, set `gc_overhead_limit_was_exceeded`, and return `null` to the requester.
> 
> What happens at the caller, in response to this set of conditions?
> 
> Why then does it make sense to `notify_gc_progress()`, which clears the counter of `_gc_no_progress_count`? Where was the ostensible progress in this case?

Ultimately, the caller will have an OOME raised. The `_gc_no_progress_count` is cleared to allow subsequent mutator allocations to resume the slower block-and-retry loop in `ShenandoahHeap::allocate_memory`.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 292:
> 
>> 290:   ShenandoahSharedFlag   _concurrent_strong_root_in_progress;
>> 291: 
>> 292:   size_t _no_gc_progress_count;
> 
> Can we call this `_gc_no_progress_count` instead, so it's consistent with the corresponding `get_` method? The difference in naming is unnecessary at best and potentially confusing for maintenance in the long run, at worst.

Yes, I'll rename this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333646726
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333646943

From duke at openjdk.org  Fri Sep 22 15:07:52 2023
From: duke at openjdk.org (duke)
Date: Fri, 22 Sep 2023 15:07:52 GMT
Subject: Withdrawn: 8305896: Alternative full GC forwarding
In-Reply-To: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
References: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
Message-ID: <bF3h1r0s9vNgSkVEKwSb2ciRFypjIzv8rBYP14m6lIs=.def8415d-c9f7-43b8-bbde-63b778371c75@github.com>

On Fri, 21 Apr 2023 15:17:36 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> Currently, the full-GC modes of Serial, Shenandoah and G1 GCs are forwarding objects by over-writing the object header with the new object location. Unfortunately, for compact object headers ([JDK-8294992](https://bugs.openjdk.org/browse/JDK-8294992)) this would not work, because the crucial class information is also stored in the header, and we could no longer iterate over objects until the headers would be restored. Also, the preserved-headers tables would grow quite large.
> 
> I propose to use an alternative algorithm for full-GC (sliding-GC) forwarding that uses a special encoding so that the forwarding information fits into the lowest 32 bits of the header.
> 
> It exploits the insight that, with sliding GCs, objects from one region will only ever be forwarded to one of two possible target regions. For this to work, we need to divide the heap into equal-sized regions. This is already the case for Shenandoah and G1, and can easily be overlaid for Serial GC, by assuming either the whole heap as a single region (if it fits) or by using SpaceAlignment-sized virtual regions.
> 
> We also build and maintain a table that has N elements, where N is the number of regions. Each entry is two addresses, which are the start-address of the possible target regions for each source region.
> 
> With this, forwarding information would be encoded like this:
>  - Bits 0 and 1: same as before, we put in '11' to indicate that the object is forwarded.
>  - Bit 2: Used for 'fallback'-forwarding (see below)
>  - Bit 3: Selects the target region 0 or 1. Look up the base address in the table (see above)
>  - Bits 4..31 The number of heap words from the target base address
> 
> This works well for all sliding GCs in Serial, G1 and Shenandoah. The exception is in G1, there is a special mode called 'serial compaction' which acts as a last-last-ditch effort to squeeze more space out of the heap by re-forwarding the tails of the compaction chains. Unfortunately, this breaks the assumption of the sliding-forwarding-table. When that happens, we initialize a fallback table, which is a simple open hash-table, and set the Bit 2 in the forwarding to indicate that we shall look up the forwardee in the fallback-table.
> 
> All the table accesses can be done unsynchronized because:
> - Serial GC is single-threaded anyway
> - In G1 and Shenandoah, GC worker threads divide up the work such that each worker does disjoint sets of regions.
> - G1 serial compaction is single-threaded
> 
> The change introduces a new (experimental) flag -...

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/13582

From wkemper at openjdk.org  Fri Sep 22 20:53:38 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Sep 2023 20:53:38 GMT
Subject: RFR: 8316755: GenShen: Serviceability needs to know about
 ShenandoahGenerationalHeap
Message-ID: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>

Plumb `ShenandoahGenerationalHeap` into `vmStructs`.  

Testing: 

make test TEST=hotspot_serviceability TEST_VM_OPTS="-Xlog:gc*=info:file=/tmp/servicability.log -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational" CONF=fastdebug

...

Test results: passed: 319; failed: 0

-------------

Commit messages:
 - Add vm struct bindings for ShenandoahGenerationalHeap

Changes: https://git.openjdk.org/shenandoah/pull/329/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=329&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8316755
  Stats: 37 lines in 3 files changed: 37 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/329.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/329/head:pull/329

PR: https://git.openjdk.org/shenandoah/pull/329

From ysr at openjdk.org  Sat Sep 23 01:50:22 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 23 Sep 2023 01:50:22 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <fJz4ketqxJfNZOqPtRyvO6VupFXNpXQ_c2XOXx6QM1U=.5249b20e-efa7-4983-b5c8-2e16d339dc37@github.com>

On Wed, 20 Sep 2023 22:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

LGTM, modulo a couple of clarifying documentation comments, as discussed.

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15852#pullrequestreview-1640764029

From ysr at openjdk.org  Sat Sep 23 01:50:23 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 23 Sep 2023 01:50:23 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <_RnYdoISAUxRvoPwjrjvADl3MDQuZEj3FF68WtkBc-g=.ef09a2b9-1826-4151-88c7-8359533c0791@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <m2bTUSJx89KdQKDQoEH13JY1kTv1Us2gPjLmIvb9QBE=.5fe5af2e-0245-4802-a7e7-0e6f5e2aab2b@github.com>
 <fIDCCApdrdxKXQMxa7ovnnKFMBNdQjJQtecMyjMjUxs=.2d4be08a-bdee-4642-aae4-21b605da2d58@github.com>
 <_RnYdoISAUxRvoPwjrjvADl3MDQuZEj3FF68WtkBc-g=.ef09a2b9-1826-4151-88c7-8359533c0791@github.com>
Message-ID: <hHKavqIpGt_wHC30SfkZW-vd3_O4iXvLFe_Zo6FXCSo=.ad299aa2-802f-48e3-85dd-b4ae1eb2f9fe@github.com>

On Thu, 21 Sep 2023 21:57:16 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> When a mutator allocation fails, it will block on this call and wait to be notified by the control thread to retry the allocation. The notification follows a degenerated or full GC.
>
> With this change, the mutator is also allowed to call the non-blocking version when it cannot allocate because the GC overhead has been exceeded. I didn't make sense to me to have the mutator block and wait for a GC cycle when the GC threshold is already exceeded.

OK, thanks, that makes sense.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1334903148

From ysr at openjdk.org  Sat Sep 23 01:50:23 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 23 Sep 2023 01:50:23 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
In-Reply-To: <HZH1n319_XKi4iQlJ7seIVYf0NfBWR1aUIxaqmWFT-k=.3e328814-5f5a-4eab-a0de-dc9d3d903127@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <m2bTUSJx89KdQKDQoEH13JY1kTv1Us2gPjLmIvb9QBE=.5fe5af2e-0245-4802-a7e7-0e6f5e2aab2b@github.com>
 <HZH1n319_XKi4iQlJ7seIVYf0NfBWR1aUIxaqmWFT-k=.3e328814-5f5a-4eab-a0de-dc9d3d903127@github.com>
Message-ID: <bIQhITsQhVeRSBGbESkEUZXYXelM5NOGeCps5fHMvGA=.087ec8f8-0d05-4a6c-858c-445c8e700157@github.com>

On Thu, 21 Sep 2023 22:08:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 947:
>> 
>>> 945:     control_thread()->handle_alloc_failure(req, false);
>>> 946:     notify_gc_progress();
>>> 947:     *gc_overhead_limit_was_exceeded = true;
>> 
>> I am confused. We have noted that we did more than `ShenandoahNoProgressThreshold` collections at this point. So we will fail the allocation request, set `gc_overhead_limit_was_exceeded`, and return `null` to the requester.
>> 
>> What happens at the caller, in response to this set of conditions?
>> 
>> Why then does it make sense to `notify_gc_progress()`, which clears the counter of `_gc_no_progress_count`? Where was the ostensible progress in this case?
>
> Ultimately, the caller will have an OOME raised. The `_gc_no_progress_count` is cleared to allow subsequent mutator allocations to resume the slower block-and-retry loop in `ShenandoahHeap::allocate_memory`.

OK, thanks, that makes sense. Can you add a comment line to that effect?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1334903081

From kdnilsen at openjdk.org  Mon Sep 25 01:42:27 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 25 Sep 2023 01:42:27 GMT
Subject: RFR: 8312116 GenShen: make instantaneous allocation rate triggers
 more timely [v2]
In-Reply-To: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>
References: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>
Message-ID: <7jr2jKxm4mpWKh6exILLgnKq6KDYosI0I3STp0hbau4=.85ac80db-92b6-4dd3-ac67-4f4d97410ec4@github.com>

> When heuristics fail to trigger because an instantaneous "allocation spike" is not so large as to consume all available memory before GC completes, this assessment is based on an assumption that the allocation rate remains constant, and it ignores the time that will be lost due to the _interval between consecutive allocation spike measurements.
> 
> This PR watches for "acceleration" of allocation rates.  When acceleration of allocation is detected in 3 consecutive allocation spike measurements, it calculates a best-fit curve (assuming constant acceleration) and predicts the memory to be consumed during the time that spans both the next sample interval and the GC effort that follows it.  If the memory to be allocated according to anticipated acceleration of allocations during this time span exceeds what is available, we trigger immediately.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fine tune and fix the heuristics

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/327/files
  - new: https://git.openjdk.org/shenandoah/pull/327/files/5050e39f..734ca370

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=327&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=327&range=00-01

  Stats: 315 lines in 3 files changed: 210 ins; 16 del; 89 mod
  Patch: https://git.openjdk.org/shenandoah/pull/327.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/327/head:pull/327

PR: https://git.openjdk.org/shenandoah/pull/327

From shade at openjdk.org  Mon Sep 25 07:32:01 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 25 Sep 2023 07:32:01 GMT
Subject: RFR: 8316755: GenShen: Serviceability needs to know about
 ShenandoahGenerationalHeap
In-Reply-To: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
References: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
Message-ID: <wlRELskBXmitCmItFNJ3C25AJGP_cTdaH0nWbZY5Ff0=.d092d78e-8a3f-4215-8885-e3f6cdac6dcf@github.com>

On Fri, 22 Sep 2023 20:45:26 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Plumb `ShenandoahGenerationalHeap` into `vmStructs`.  
> 
> Testing: 
> 
> make test TEST=hotspot_serviceability TEST_VM_OPTS="-Xlog:gc*=info:file=/tmp/servicability.log -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational" CONF=fastdebug
> 
> ...
> 
> Test results: passed: 319; failed: 0

Good, with minor nit.

src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahGenerationalHeap.java line 33:

> 31:         super(addr);
> 32:     }
> 33: }

Missing newline here, I think.

-------------

Marked as reviewed by shade (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/329#pullrequestreview-1641498034
PR Review Comment: https://git.openjdk.org/shenandoah/pull/329#discussion_r1335490484

From ysr at openjdk.org  Mon Sep 25 18:02:49 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 25 Sep 2023 18:02:49 GMT
Subject: RFR: 8316755: GenShen: Serviceability needs to know about
 ShenandoahGenerationalHeap
In-Reply-To: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
References: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
Message-ID: <e6t4DFXtRZocx_K24LdhGJR7Z_vZrBI3jl7bWXYb-gA=.dc00a728-4cc9-420c-b001-c2fd9919360a@github.com>

On Fri, 22 Sep 2023 20:45:26 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Plumb `ShenandoahGenerationalHeap` into `vmStructs`.  
> 
> Testing: 
> 
> make test TEST=hotspot_serviceability TEST_VM_OPTS="-Xlog:gc*=info:file=/tmp/servicability.log -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational" CONF=fastdebug
> 
> ...
> 
> Test results: passed: 319; failed: 0

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/329#pullrequestreview-1642726562

From kdnilsen at openjdk.org  Mon Sep 25 18:13:56 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 25 Sep 2023 18:13:56 GMT
Subject: RFR: 8316755: GenShen: Serviceability needs to know about
 ShenandoahGenerationalHeap
In-Reply-To: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
References: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
Message-ID: <FYBwGKfEtXWb-FK2Zv_yL6h-4MK81fW1YxZaXYGvWVw=.2a94be81-aad5-433d-81dd-46e88e3e1766@github.com>

On Fri, 22 Sep 2023 20:45:26 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Plumb `ShenandoahGenerationalHeap` into `vmStructs`.  
> 
> Testing: 
> 
> make test TEST=hotspot_serviceability TEST_VM_OPTS="-Xlog:gc*=info:file=/tmp/servicability.log -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational" CONF=fastdebug
> 
> ...
> 
> Test results: passed: 319; failed: 0

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/329#pullrequestreview-1642743306

From wkemper at openjdk.org  Mon Sep 25 18:35:42 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Sep 2023 18:35:42 GMT
Subject: RFR: 8316755: GenShen: Serviceability needs to know about
 ShenandoahGenerationalHeap [v2]
In-Reply-To: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
References: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
Message-ID: <Go1lT6IgTHjO90YGwqKckqwnzZtFgLSlSmYqeCyxCwQ=.c10a13a2-10ee-4d5f-959a-cfd3ca956d3d@github.com>

> Plumb `ShenandoahGenerationalHeap` into `vmStructs`.  
> 
> Testing: 
> 
> make test TEST=hotspot_serviceability TEST_VM_OPTS="-Xlog:gc*=info:file=/tmp/servicability.log -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational" CONF=fastdebug
> 
> ...
> 
> Test results: passed: 319; failed: 0

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Fix copyright comment
 - Add missing newline

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/329/files
  - new: https://git.openjdk.org/shenandoah/pull/329/files/2e30feea..0baf1a16

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=329&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=329&range=00-01

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/329.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/329/head:pull/329

PR: https://git.openjdk.org/shenandoah/pull/329

From wkemper at openjdk.org  Mon Sep 25 18:45:09 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Sep 2023 18:45:09 GMT
Subject: Integrated: 8316755: GenShen: Serviceability needs to know about
 ShenandoahGenerationalHeap
In-Reply-To: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
References: <dAwcgQwzo3dr5nip52WkKgXP2IiIlEldgnvZCZCwdXk=.d88163bf-310c-45d2-80f9-9ef52a987940@github.com>
Message-ID: <WFlxSU0xjHhpILW-nhdVE2bjzVLHUEJ2snGamIzIaf4=.d03a758d-d9ca-4e6c-b212-9f18967704b6@github.com>

On Fri, 22 Sep 2023 20:45:26 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Plumb `ShenandoahGenerationalHeap` into `vmStructs`.  
> 
> Testing: 
> 
> make test TEST=hotspot_serviceability TEST_VM_OPTS="-Xlog:gc*=info:file=/tmp/servicability.log -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational" CONF=fastdebug
> 
> ...
> 
> Test results: passed: 319; failed: 0

This pull request has now been integrated.

Changeset: a23e366c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/a23e366ce650464b4711bab1f028dee6174f0b7f
Stats:     37 lines in 3 files changed: 37 ins; 0 del; 0 mod

8316755: GenShen: Serviceability needs to know about ShenandoahGenerationalHeap

Reviewed-by: shade, ysr, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/329

From wkemper at openjdk.org  Mon Sep 25 21:08:18 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Sep 2023 21:08:18 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v2]
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <xm4-Ff5C6JsZc261VoUfbvkcYCZzJMQ_TUwxO_dvN4M=.7fc4b9be-dbca-41fa-88fb-082fcdf474bb@github.com>

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

William Kemper has updated the pull request incrementally with three additional commits since the last revision:

 - Improve comment, increase default for no progress threshold
 - Allocator should not reset bad progress count
 - Allocator should not reset bad progress count

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15852/files
  - new: https://git.openjdk.org/jdk/pull/15852/files/f68a942b..f864be67

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=00-01

  Stats: 36 lines in 6 files changed: 16 ins; 12 del; 8 mod
  Patch: https://git.openjdk.org/jdk/pull/15852.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15852/head:pull/15852

PR: https://git.openjdk.org/jdk/pull/15852

From zgu at openjdk.org  Tue Sep 26 12:54:32 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Tue, 26 Sep 2023 12:54:32 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC need
 to cleanup old OopMapCache entries
Message-ID: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>

During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.

Test:
  hotspot_gc_shenandoah (fastdebug and release on MacOSX)

-------------

Commit messages:
 - 8316929: Shenandoah: Shenandoah degenerated GC and full GC need to cleanup old OopMapCache entries

Changes: https://git.openjdk.org/jdk/pull/15921/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15921&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8316929
  Stats: 18 lines in 2 files changed: 13 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/15921.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15921/head:pull/15921

PR: https://git.openjdk.org/jdk/pull/15921

From kdnilsen at openjdk.org  Wed Sep 27 13:38:21 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 27 Sep 2023 13:38:21 GMT
Subject: RFR: 8312116 GenShen: make instantaneous allocation rate triggers
 more timely [v3]
In-Reply-To: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>
References: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>
Message-ID: <bP1bkxw5SZtjbdTAf68FfvAqlYiioBnLA1GA0aJbwik=.6036d000-04da-4488-9f3d-60bf95e6833e@github.com>

> When heuristics fail to trigger because an instantaneous "allocation spike" is not so large as to consume all available memory before GC completes, this assessment is based on an assumption that the allocation rate remains constant, and it ignores the time that will be lost due to the _interval between consecutive allocation spike measurements.
> 
> This PR watches for "acceleration" of allocation rates.  When acceleration of allocation is detected in 3 consecutive allocation spike measurements, it calculates a best-fit curve (assuming constant acceleration) and predicts the memory to be consumed during the time that spans both the next sample interval and the GC effort that follows it.  If the memory to be allocated according to anticipated acceleration of allocations during this time span exceeds what is available, we trigger immediately.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Checkpoint this code so we can pursue suspected deadlock JBS issue
  
  I'm filing a new ticket to explore deadlock issue related to
  ShenandoahControlThread::_regulator_lock.  The issue was last observed
  on this branch.  New changes that we are about to integrate will likely
  make it much more difficult to reproduce the issue.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/327/files
  - new: https://git.openjdk.org/shenandoah/pull/327/files/734ca370..d6bc97eb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=327&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=327&range=01-02

  Stats: 53 lines in 4 files changed: 48 ins; 1 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/327.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/327/head:pull/327

PR: https://git.openjdk.org/shenandoah/pull/327

From kdnilsen at openjdk.org  Wed Sep 27 15:45:33 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 27 Sep 2023 15:45:33 GMT
Subject: RFR: 8312116 GenShen: make instantaneous allocation rate triggers
 more timely [v4]
In-Reply-To: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>
References: <3kCLM3psFvTGFmMUovwVQoxC_3ByduhCfwccYyd-SYg=.78d2f69d-e06c-46db-9053-d17076bffc33@github.com>
Message-ID: <47KsJQLA0PRDq8AilLor_DLeboeUbgIai7NGBLpFctM=.07635aaf-be70-436e-87ab-6176dec8dbe8@github.com>

> When heuristics fail to trigger because an instantaneous "allocation spike" is not so large as to consume all available memory before GC completes, this assessment is based on an assumption that the allocation rate remains constant, and it ignores the time that will be lost due to the _interval between consecutive allocation spike measurements.
> 
> This PR watches for "acceleration" of allocation rates.  When acceleration of allocation is detected in 3 consecutive allocation spike measurements, it calculates a best-fit curve (assuming constant acceleration) and predicts the memory to be consumed during the time that spans both the next sample interval and the GC effort that follows it.  If the memory to be allocated according to anticipated acceleration of allocations during this time span exceeds what is available, we trigger immediately.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Give higher priority to young over old and disable instrumentation

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/327/files
  - new: https://git.openjdk.org/shenandoah/pull/327/files/d6bc97eb..7315c76c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=327&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=327&range=02-03

  Stats: 47 lines in 4 files changed: 37 ins; 0 del; 10 mod
  Patch: https://git.openjdk.org/shenandoah/pull/327.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/327/head:pull/327

PR: https://git.openjdk.org/shenandoah/pull/327

From wkemper at openjdk.org  Wed Sep 27 17:20:06 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 27 Sep 2023 17:20:06 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v3]
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <mFCZNYew3b5-eFhumy8hnEX-C-9uw7X1bihJeVzZ7Gc=.2a63e9ab-79a6-4ae6-b2fd-fc25fc201c54@github.com>

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:

 - Merge remote-tracking branch 'openjdk/master' into shenandoah-oome-redux
 - Extend exemption for EATests that rely on timely OOME to Shenandoah
 - Improve comment, increase default for no progress threshold
 - Allocator should not reset bad progress count
 - Allocator should not reset bad progress count
 - Fix 32-bit build error
 - Do not use atomics in header
 - Signal gc threshold exceeded when appropriate

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15852/files
  - new: https://git.openjdk.org/jdk/pull/15852/files/f864be67..cd4989e7

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=01-02

  Stats: 77835 lines in 2231 files changed: 29102 ins; 15056 del; 33677 mod
  Patch: https://git.openjdk.org/jdk/pull/15852.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15852/head:pull/15852

PR: https://git.openjdk.org/jdk/pull/15852

From wkemper at openjdk.org  Wed Sep 27 18:36:12 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 27 Sep 2023 18:36:12 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries
In-Reply-To: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
Message-ID: <HBras_qM3VzB6hpBRATgpWswFkD9d0_Tru4ITUAgqGg=.50e070d6-e3de-4b34-a2ee-3b27f42d63d3@github.com>

On Tue, 26 Sep 2023 12:48:12 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
> 
> Test:
>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)

This looks good to me. Does the issue also affect 17?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15921#issuecomment-1737890063

From zgu at openjdk.org  Thu Sep 28 05:15:21 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Thu, 28 Sep 2023 05:15:21 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries
In-Reply-To: <HBras_qM3VzB6hpBRATgpWswFkD9d0_Tru4ITUAgqGg=.50e070d6-e3de-4b34-a2ee-3b27f42d63d3@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
 <HBras_qM3VzB6hpBRATgpWswFkD9d0_Tru4ITUAgqGg=.50e070d6-e3de-4b34-a2ee-3b27f42d63d3@github.com>
Message-ID: <feEiHReaaxgMMaq18wp3HcGknE9IF_RL9Lu-8OSHIEk=.3a0c0ee9-b9d6-4c85-b564-66c1f2827a1e@github.com>

On Wed, 27 Sep 2023 18:33:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This looks good to me. Does the issue also affect 17?

Yes. I updated affected versions in bug.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15921#issuecomment-1738441225

From wkemper at openjdk.org  Thu Sep 28 16:46:39 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 28 Sep 2023 16:46:39 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <45fGfm8zQY7thdKC2BeN8oC7CGllx2VssiqtRsbTzK0=.ecab59fe-1565-416a-8915-845263ffc8f3@github.com>

Merges tag jdk-22+17

-------------

Commit messages:
 - 8316661: CompilerThread leaks CodeBlob memory when dynamically stopping compiler thread in non-product
 - 8315721: CloseRace.java#id0 fails transiently on libgraal
 - 8315966: Relativize initial_sp in interpreter frames
 - 8316924: java/lang/Thread/virtual/stress/ParkALot.java times out
 - 8316710: Exclude java/awt/font/Rotate/RotatedTextTest.java
 - 8299915: Remove ArrayAllocatorMallocLimit and associated code
 - 8316417: ObjectMonitorIterator does not return the most recent monitor and is incorrect if no monitors exists
 - 8293176: SSLEngine handshaker does not send an alert after a bad parameters
 - 8316895: SeenThread::print_action_queue called on a null pointer
 - 8316933: RISC-V: compiler/vectorapi/VectorCastShape128Test.java fails when using RVV
 - ... and 87 more: https://git.openjdk.org/shenandoah/compare/c04c9ea3...edcc559f

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/331/files
  Stats: 10939 lines in 411 files changed: 8536 ins; 995 del; 1408 mod
  Patch: https://git.openjdk.org/shenandoah/pull/331.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/331/head:pull/331

PR: https://git.openjdk.org/shenandoah/pull/331

From rkennke at openjdk.org  Thu Sep 28 16:53:46 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 28 Sep 2023 16:53:46 GMT
Subject: RFR: 8305896: Alternative full GC forwarding [v53]
In-Reply-To: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
References: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
Message-ID: <vwYOiDEZgh-lUaM7iNG7cbw5nProOfIEdfFigWI5jRQ=.ecd91402-7222-4146-982a-5371ed73ba67@github.com>

> Currently, the full-GC modes of Serial, Shenandoah and G1 GCs are forwarding objects by over-writing the object header with the new object location. Unfortunately, for compact object headers ([JDK-8294992](https://bugs.openjdk.org/browse/JDK-8294992)) this would not work, because the crucial class information is also stored in the header, and we could no longer iterate over objects until the headers would be restored. Also, the preserved-headers tables would grow quite large.
> 
> I propose to use an alternative algorithm for full-GC (sliding-GC) forwarding that uses a special encoding so that the forwarding information fits into the lowest 32 bits of the header.
> 
> It exploits the insight that, with sliding GCs, objects from one region will only ever be forwarded to one of two possible target regions. For this to work, we need to divide the heap into equal-sized regions. This is already the case for Shenandoah and G1, and can easily be overlaid for Serial GC, by assuming either the whole heap as a single region (if it fits) or by using SpaceAlignment-sized virtual regions.
> 
> We also build and maintain a table that has N elements, where N is the number of regions. Each entry is two addresses, which are the start-address of the possible target regions for each source region.
> 
> With this, forwarding information would be encoded like this:
>  - Bits 0 and 1: same as before, we put in '11' to indicate that the object is forwarded.
>  - Bit 2: Used for 'fallback'-forwarding (see below)
>  - Bit 3: Selects the target region 0 or 1. Look up the base address in the table (see above)
>  - Bits 4..31 The number of heap words from the target base address
> 
> This works well for all sliding GCs in Serial, G1 and Shenandoah. The exception is in G1, there is a special mode called 'serial compaction' which acts as a last-last-ditch effort to squeeze more space out of the heap by re-forwarding the tails of the compaction chains. Unfortunately, this breaks the assumption of the sliding-forwarding-table. When that happens, we initialize a fallback table, which is a simple open hash-table, and set the Bit 2 in the forwarding to indicate that we shall look up the forwardee in the fallback-table.
> 
> All the table accesses can be done unsynchronized because:
> - Serial GC is single-threaded anyway
> - In G1 and Shenandoah, GC worker threads divide up the work such that each worker does disjoint sets of regions.
> - G1 serial compaction is single-threaded
> 
> The change introduces a new (experimental) flag -...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 120 commits:

 - Merge remote-tracking branch 'upstream/master' into JDK-8305896
 - Merge remote-tracking branch 'upstream/master' into JDK-8305896
 - Merge branch 'master' into JDK-8305896
 - Various cleanups
 - Some @shipilev comments from downstream review
 - Further templatize Serial GC's adjust_pointers()
 - Merge branch 'master' into JDK-8305896
 - Specialize full-GC loops to get UseAltGCForwarding flag check out of hot paths
 - Remove G1-only assert for fallback forwarding, and comment with explanation
 - Merge branch 'master' into JDK-8305896
 - ... and 110 more: https://git.openjdk.org/jdk/compare/3481a485...95c2bd6e

-------------

Changes: https://git.openjdk.org/jdk/pull/13582/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13582&range=52
  Stats: 1200 lines in 39 files changed: 1077 ins; 10 del; 113 mod
  Patch: https://git.openjdk.org/jdk/pull/13582.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/13582/head:pull/13582

PR: https://git.openjdk.org/jdk/pull/13582

From kdnilsen at openjdk.org  Thu Sep 28 17:12:52 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 28 Sep 2023 17:12:52 GMT
Subject: RFR: 8317049: GenShen: Very rare loss of function for
 ShenandoahControlThread::_regulator_lock
Message-ID: <yullOkPUQLqsUHeDukVzLER0HOya70A_6uZYh72G9TE=.c9656218-7fb4-482f-b125-e7228041b41d@github.com>

We have observed that we very rarely miss _regulator_lock notifications.  We have identified the root cause to be a race, which occurs when the ShenandoahControlThread notifies the _regulator_lock before the ShenandoahRegulatorThread has begun to wait for the notification.

This change forces the ShenandoahControlThread to wait until the ShenandoahRegulatorThread is inside its wait() invocation.  It does so by acquiring the Mutator lock for _regulator_thread before notifying the control thread.  The control thread must acquire this lock before it can notify.  The control thread will not be able to acquire the lock until the regulator thread releases the lock by invoking ml.wait().

-------------

Commit messages:
 - Force control thread to wait until regulator ready to receive notify

Changes: https://git.openjdk.org/shenandoah/pull/332/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=332&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8317049
  Stats: 11 lines in 1 file changed: 8 ins; 3 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/332.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/332/head:pull/332

PR: https://git.openjdk.org/shenandoah/pull/332

From wkemper at openjdk.org  Fri Sep 29 03:24:51 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 29 Sep 2023 03:24:51 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <45fGfm8zQY7thdKC2BeN8oC7CGllx2VssiqtRsbTzK0=.ecab59fe-1565-416a-8915-845263ffc8f3@github.com>
References: <45fGfm8zQY7thdKC2BeN8oC7CGllx2VssiqtRsbTzK0=.ecab59fe-1565-416a-8915-845263ffc8f3@github.com>
Message-ID: <q-L9Stm07in-yzFdVRjHUdKNr2zw2XO9elR--t-Ef4I=.8707188a-125b-469f-a3df-156fd234f11d@github.com>

> Merges tag jdk-22+17

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/331/files
  - new: https://git.openjdk.org/shenandoah/pull/331/files/edcc559f..edcc559f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=331&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=331&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/331.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/331/head:pull/331

PR: https://git.openjdk.org/shenandoah/pull/331

From wkemper at openjdk.org  Fri Sep 29 03:24:55 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 29 Sep 2023 03:24:55 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <45fGfm8zQY7thdKC2BeN8oC7CGllx2VssiqtRsbTzK0=.ecab59fe-1565-416a-8915-845263ffc8f3@github.com>
References: <45fGfm8zQY7thdKC2BeN8oC7CGllx2VssiqtRsbTzK0=.ecab59fe-1565-416a-8915-845263ffc8f3@github.com>
Message-ID: <ItcFw0NUUij4JWf0jaCeHzQ2siK1BGMM3SvPeoCbges=.4c1cdfcb-4535-4d5e-90c2-753fdb98e47e@github.com>

On Thu, 28 Sep 2023 16:41:20 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+17

This pull request has now been integrated.

Changeset: 9b70aab3
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/9b70aab3ed7d69d0493797573644071bc1280c4d
Stats:     10939 lines in 411 files changed: 8536 ins; 995 del; 1408 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/331

From rkennke at openjdk.org  Fri Sep 29 11:38:39 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 29 Sep 2023 11:38:39 GMT
Subject: RFR: 8305896: Alternative full GC forwarding [v54]
In-Reply-To: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
References: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
Message-ID: <OMtU9q-rdB185OsIK9IEGmK4KTBKAX0KTWcNeAeXeiA=.ae7d11e3-1482-4894-8ff3-a7a75c1e73ac@github.com>

> Currently, the full-GC modes of Serial, Shenandoah and G1 GCs are forwarding objects by over-writing the object header with the new object location. Unfortunately, for compact object headers ([JDK-8294992](https://bugs.openjdk.org/browse/JDK-8294992)) this would not work, because the crucial class information is also stored in the header, and we could no longer iterate over objects until the headers would be restored. Also, the preserved-headers tables would grow quite large.
> 
> I propose to use an alternative algorithm for full-GC (sliding-GC) forwarding that uses a special encoding so that the forwarding information fits into the lowest 32 bits of the header.
> 
> It exploits the insight that, with sliding GCs, objects from one region will only ever be forwarded to one of two possible target regions. For this to work, we need to divide the heap into equal-sized regions. This is already the case for Shenandoah and G1, and can easily be overlaid for Serial GC, by assuming either the whole heap as a single region (if it fits) or by using SpaceAlignment-sized virtual regions.
> 
> We also build and maintain a table that has N elements, where N is the number of regions. Each entry is two addresses, which are the start-address of the possible target regions for each source region.
> 
> With this, forwarding information would be encoded like this:
>  - Bits 0 and 1: same as before, we put in '11' to indicate that the object is forwarded.
>  - Bit 2: Used for 'fallback'-forwarding (see below)
>  - Bit 3: Selects the target region 0 or 1. Look up the base address in the table (see above)
>  - Bits 4..31 The number of heap words from the target base address
> 
> This works well for all sliding GCs in Serial, G1 and Shenandoah. The exception is in G1, there is a special mode called 'serial compaction' which acts as a last-last-ditch effort to squeeze more space out of the heap by re-forwarding the tails of the compaction chains. Unfortunately, this breaks the assumption of the sliding-forwarding-table. When that happens, we initialize a fallback table, which is a simple open hash-table, and set the Bit 2 in the forwarding to indicate that we shall look up the forwardee in the fallback-table.
> 
> All the table accesses can be done unsynchronized because:
> - Serial GC is single-threaded anyway
> - In G1 and Shenandoah, GC worker threads divide up the work such that each worker does disjoint sets of regions.
> - G1 serial compaction is single-threaded
> 
> The change introduces a new (experimental) flag -...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Fix build

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/13582/files
  - new: https://git.openjdk.org/jdk/pull/13582/files/95c2bd6e..2d068012

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=13582&range=53
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13582&range=52-53

  Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/13582.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/13582/head:pull/13582

PR: https://git.openjdk.org/jdk/pull/13582

From shade at openjdk.org  Fri Sep 29 16:46:29 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 29 Sep 2023 16:46:29 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v3]
In-Reply-To: <mFCZNYew3b5-eFhumy8hnEX-C-9uw7X1bihJeVzZ7Gc=.2a63e9ab-79a6-4ae6-b2fd-fc25fc201c54@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <mFCZNYew3b5-eFhumy8hnEX-C-9uw7X1bihJeVzZ7Gc=.2a63e9ab-79a6-4ae6-b2fd-fc25fc201c54@github.com>
Message-ID: <2bXcUWi2pXLKmpiPTNIT2_xu2fGBwPHB5YUMInN-OFo=.92656fc9-7549-41fa-bb9d-445cee3db5f8@github.com>

On Wed, 27 Sep 2023 17:20:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'openjdk/master' into shenandoah-oome-redux
>  - Extend exemption for EATests that rely on timely OOME to Shenandoah
>  - Improve comment, increase default for no progress threshold
>  - Allocator should not reset bad progress count
>  - Allocator should not reset bad progress count
>  - Fix 32-bit build error
>  - Do not use atomics in header
>  - Signal gc threshold exceeded when appropriate

Apologies, but I have more comments.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 877:

> 875:     }
> 876: 
> 877:     if (result == nullptr && !req.is_lab_alloc() && get_gc_no_progress_count() > ShenandoahNoProgressThreshold) {

Can this be moved to the block that already does allocation-after-gc retries? That block already exits with `nullptr` (implies delivering OOME), and it already calls `handle_alloc_failure` (thus triggering GC). We "only" need it to specialize for `is_lab_alloc` and `ShenandoahNoProgressThreshold`?

This PR changes that block anyway...

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 902:

> 900:       ResourceMark rm;
> 901:       log_debug(gc, alloc)("Thread: %s, Result: " PTR_FORMAT ", Shared: %s, Size: " SIZE_FORMAT ", Original: " SIZE_FORMAT ", Latest: " SIZE_FORMAT,
> 902:                            Thread::current()->name(), p2i(result), BOOL_TO_STR(!req.is_lab_alloc()), req.size(), original_count, get_gc_no_progress_count());

There is `type_string()` that can be used instead of `is_lab_alloc()` here.

test/hotspot/jtreg/TEST.groups line 612:

> 610:   gtest/NMTGtests.java
> 611: 
> 612: hotspot_oome = \

Adding new test groups require additional attention. I think this should be removed.

Note that during debugging/testing, you can run the same by:


$ make test TEST="runtime/reflect/ReflectOutOfMemoryError.java gc/InfiniteList.java runtime/ClassInitErrors/TestOutOfMemoryDuringInit.java"

-------------

PR Review: https://git.openjdk.org/jdk/pull/15852#pullrequestreview-1651130171
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1341588940
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1341590508
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1341577573

From zgu at openjdk.org  Fri Sep 29 21:37:34 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Fri, 29 Sep 2023 21:37:34 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries
In-Reply-To: <HBras_qM3VzB6hpBRATgpWswFkD9d0_Tru4ITUAgqGg=.50e070d6-e3de-4b34-a2ee-3b27f42d63d3@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
 <HBras_qM3VzB6hpBRATgpWswFkD9d0_Tru4ITUAgqGg=.50e070d6-e3de-4b34-a2ee-3b27f42d63d3@github.com>
Message-ID: <qJjWnbHcvGjX7vtpYKz9ln7Dw37QfIIKykkjODtPFhc=.4057e0c0-67af-47fb-9055-ec3ac994c6b6@github.com>

On Wed, 27 Sep 2023 18:33:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
>> 
>> Test:
>>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)
>
> This looks good to me. Does the issue also affect 17?

@earthling-amzn @rkennke @shipilev Could you please review? Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15921#issuecomment-1741513712