From zgu at openjdk.org  Sat Feb  1 16:42:50 2025
From: zgu at openjdk.org (Zhengyu Gu)
Date: Sat, 1 Feb 2025 16:42:50 GMT
Subject: RFR: 8348171: Refactor GenerationCounters and its subclasses [v4]
In-Reply-To: <bRZ7PVXUDU31NoQxDYS42gmlcvvhBwMn2dxSvZYQH3g=.2c6361ce-cd4e-4909-8014-fe7712ed7c94@github.com>
References: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
 <bRZ7PVXUDU31NoQxDYS42gmlcvvhBwMn2dxSvZYQH3g=.2c6361ce-cd4e-4909-8014-fe7712ed7c94@github.com>
Message-ID: <zBqpje4bU8Itm234zb8PKge_Tc-YN1sfA8RGCqwlsUM=.04ccd8a8-b137-429f-ba1c-e3037ae88ec5@github.com>

On Fri, 31 Jan 2025 12:30:30 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Simple refactoring of removing the use of `virtual` method and use concrete subclasses when needed.
>> 
>> Test: tier1-5
>
> Albert Mingkun Yang has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - review
>  - * some more refactoring

src/hotspot/share/gc/shenandoah/shenandoahMonitoringSupport.cpp line 39:

> 37:           GenerationCounters("Young", 0, 0, 0, (size_t)0, (size_t)0) {};
> 38: 
> 39:   void update_all() {

Shenandoah looks a bit odd now. @kdnilsen @wkemper and @ysramakrishna may want to take a look?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23209#discussion_r1938303987

From zgu at openjdk.org  Sat Feb  1 16:47:52 2025
From: zgu at openjdk.org (Zhengyu Gu)
Date: Sat, 1 Feb 2025 16:47:52 GMT
Subject: RFR: 8348171: Refactor GenerationCounters and its subclasses [v4]
In-Reply-To: <bRZ7PVXUDU31NoQxDYS42gmlcvvhBwMn2dxSvZYQH3g=.2c6361ce-cd4e-4909-8014-fe7712ed7c94@github.com>
References: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
 <bRZ7PVXUDU31NoQxDYS42gmlcvvhBwMn2dxSvZYQH3g=.2c6361ce-cd4e-4909-8014-fe7712ed7c94@github.com>
Message-ID: <m_o9Exg2DzAtCsu_ZOgmwOhChPt0zoX4F2sYe8SdHWo=.dc8a2c54-4412-4312-be0f-21714175c9f5@github.com>

On Fri, 31 Jan 2025 12:30:30 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Simple refactoring of removing the use of `virtual` method and use concrete subclasses when needed.
>> 
>> Test: tier1-5
>
> Albert Mingkun Yang has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - review
>  - * some more refactoring

`Shenandoah` code no longer aligns to others. Other than that, LGTM.

-------------

Marked as reviewed by zgu (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23209#pullrequestreview-2588364961

From tschatzl at openjdk.org  Mon Feb  3 09:25:50 2025
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Mon, 3 Feb 2025 09:25:50 GMT
Subject: RFR: 8348171: Refactor GenerationCounters and its subclasses [v4]
In-Reply-To: <bRZ7PVXUDU31NoQxDYS42gmlcvvhBwMn2dxSvZYQH3g=.2c6361ce-cd4e-4909-8014-fe7712ed7c94@github.com>
References: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
 <bRZ7PVXUDU31NoQxDYS42gmlcvvhBwMn2dxSvZYQH3g=.2c6361ce-cd4e-4909-8014-fe7712ed7c94@github.com>
Message-ID: <7k73_VjUmBq7-G2reVDOlB7-vSazUekr8Q3Ez3houa0=.61d2baf5-72c6-41cd-aa74-c49a5b5e9ce1@github.com>

On Fri, 31 Jan 2025 12:30:30 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Simple refactoring of removing the use of `virtual` method and use concrete subclasses when needed.
>> 
>> Test: tier1-5
>
> Albert Mingkun Yang has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - review
>  - * some more refactoring

Lgtm. Related to @zhengyu123 's comment, not sure right now what is meant with "looking odd" here as the previous code did not update the counters either, but it might be useful to wait on Shenandoah team's input anyway.

-------------

Marked as reviewed by tschatzl (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23209#pullrequestreview-2589340090

From wkemper at openjdk.org  Mon Feb  3 20:36:09 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 3 Feb 2025 20:36:09 GMT
Subject: RFR: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact: fatal
 error: Before Updating References: Thread C2 CompilerThread1: expected
 gc-state 9, actual 21
Message-ID: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>

Non-java threads were not having their gc-state configured when they attach. If they were created before the verifier's safepoint, but after the iteration over non-java threads, they would not have the correct state.

-------------

Commit messages:
 - Set gc state for all attached threads (not just java threads).

Changes: https://git.openjdk.org/jdk/pull/23428/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23428&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8348268
  Stats: 4 lines in 1 file changed: 3 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23428.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23428/head:pull/23428

PR: https://git.openjdk.org/jdk/pull/23428

From wkemper at openjdk.org  Mon Feb  3 22:34:13 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 3 Feb 2025 22:34:13 GMT
Subject: [jdk24] RFR: 8349002: GenShen: Deadlock during shutdown
Message-ID: <6a_6G1g93RUACyYcHG5B9HtqFBaqdETRRdhvFWwrfi8=.e88c6252-f6c4-4e3f-972a-2c4495d27127@github.com>

Clean backport. Fixes bug introduced by [JDK-8345970](https://bugs.openjdk.org/browse/JDK-8345970).

-------------

Commit messages:
 - Backport 06ebb170bac3879dc1e378b48b1c7ef006070c86

Changes: https://git.openjdk.org/jdk/pull/23429/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23429&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8349002
  Stats: 5 lines in 2 files changed: 4 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/23429.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23429/head:pull/23429

PR: https://git.openjdk.org/jdk/pull/23429

From wkemper at openjdk.org  Mon Feb  3 22:34:26 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 3 Feb 2025 22:34:26 GMT
Subject: RFR: 8349002: GenShen: Deadlock during shutdown
Message-ID: <D14C2C6K9KfySMH_Wnsax5V5VjjLaP4T6PIt_SBKpTI=.2a242bc8-689a-4203-a97f-a18257b52201@github.com>

Clean backport. Fixes bug introduced by [JDK-8345970](https://bugs.openjdk.org/browse/JDK-8345970).

-------------

Commit messages:
 - Backport 06ebb170bac3879dc1e378b48b1c7ef006070c86

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/153/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=153&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8349002
  Stats: 5 lines in 2 files changed: 4 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/153.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/153/head:pull/153

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/153

From kvn at openjdk.org  Mon Feb  3 23:51:17 2025
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Mon, 3 Feb 2025 23:51:17 GMT
Subject: [jdk24] RFR: 8349002: GenShen: Deadlock during shutdown
In-Reply-To: <6a_6G1g93RUACyYcHG5B9HtqFBaqdETRRdhvFWwrfi8=.e88c6252-f6c4-4e3f-972a-2c4495d27127@github.com>
References: <6a_6G1g93RUACyYcHG5B9HtqFBaqdETRRdhvFWwrfi8=.e88c6252-f6c4-4e3f-972a-2c4495d27127@github.com>
Message-ID: <OAiWVEmLseEETfWZk0QJxPd-EHCgTLlpeh6zU83g7dU=.874e5f8c-05e0-4100-8f10-669d804f0a0e@github.com>

On Mon, 3 Feb 2025 22:27:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport. Fixes bug introduced by [JDK-8345970](https://bugs.openjdk.org/browse/JDK-8345970).

We are in RDP2 phase of JDK 24 release. Only P1 and P2 are allowed to be pushed with approval: https://openjdk.org/jeps/3#rdp-2

Consider backporting the fix into JDK 24 Update release.

-------------

Changes requested by kvn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23429#pullrequestreview-2591384756

From phh at openjdk.org  Tue Feb  4 15:23:17 2025
From: phh at openjdk.org (Paul Hohensee)
Date: Tue, 4 Feb 2025 15:23:17 GMT
Subject: RFR: 8348610: GenShen: TestShenandoahEvacuationInformationEvent
 failed with setRegions >= regionsFreed: expected 1 >= 57
In-Reply-To: <M-8yf6fmPVUPLubCfG83quv9jTAHkqhcrgy_dHy288M=.70835c1f-be6c-4ff7-b2a1-1eb2b96ba69f@github.com>
References: <M-8yf6fmPVUPLubCfG83quv9jTAHkqhcrgy_dHy288M=.70835c1f-be6c-4ff7-b2a1-1eb2b96ba69f@github.com>
Message-ID: <sYWloWwhkEIPgbITxgOjGQoG0wcAchomsvYKVuJLBPA=.e0042123-eec6-4719-ad35-42e27207ee11@github.com>

On Thu, 30 Jan 2025 04:43:59 GMT, Satyen Subramaniam <ssubramaniam at openjdk.org> wrote:

> Renaming `ShenandoahEvacuationInformation.freedRegions` to `ShenandoahEvacuationInformation.freeRegions` for clarity, and fixing incorrect assertion in  TestShenandoahEvacuationInformationEvent.cpp
> 
> Tested with tier 1, tier 2, and tier 3 tests.

Marked as reviewed by phh (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23362#pullrequestreview-2593210592

From phh at openjdk.org  Tue Feb  4 15:53:13 2025
From: phh at openjdk.org (Paul Hohensee)
Date: Tue, 4 Feb 2025 15:53:13 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check
In-Reply-To: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
Message-ID: <o2bgv1zSGO79lVvAk1CfeYKGPl0XI7wA6nKAwvk9ylI=.a8b0ffaf-720f-4ae0-a8bc-8aaf9013bb01@github.com>

On Fri, 24 Jan 2025 18:30:02 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
> 
> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
> 
> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.

src/hotspot/share/gc/shenandoah/shenandoahMetrics.cpp line 52:

> 50:   size_t free_actual   = free_set->available();
> 51:   // The sum of free_set->capacity() and ->reserved represents capacity of young in generational, heap in non-generational.
> 52:   size_t free_expected = ((free_set->capacity() + free_set->reserved()) / 100) * ShenandoahCriticalFreeThreshold;

As an outsider, the units involved and what exactly is being calculated is pretty opaque. Why would we divide by 100 to compute free_expected and not do the same for free_actual? Do we care about integer division truncation? The default value of ShenandoahCriticalFreeThreshold is 1, so multiplying by it is a nop by default, which seems strange.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1941436272

From shade at openjdk.org  Tue Feb  4 16:05:10 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 4 Feb 2025 16:05:10 GMT
Subject: RFR: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact:
 fatal error: Before Updating References: Thread C2 CompilerThread1: expected
 gc-state 9, actual 21
In-Reply-To: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
References: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
Message-ID: <W_tKubqPMhpIKr7jBXx7mVaT_qb90sHc6t2O9EK_oNs=.fdf2009d-8480-43c1-ad0f-93a6e6125a2d@github.com>

On Mon, 3 Feb 2025 20:28:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Non-java threads were not having their gc-state configured when they attach. If they were created before the verifier's safepoint, but after the iteration over non-java threads, they would not have the correct state.

It looks generally okay, but I am confused how this fixes a bad state in `C2 CompilerThread1`, since compiler threads are Java threads? https://github.com/openjdk/jdk/blob/beb43e2633900bb9ab3c975376fe5860b6d054e0/src/hotspot/share/compiler/compilerThread.hpp#L42

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23428#issuecomment-2634411610

From phh at openjdk.org  Tue Feb  4 16:19:20 2025
From: phh at openjdk.org (Paul Hohensee)
Date: Tue, 4 Feb 2025 16:19:20 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v2]
In-Reply-To: <IHu0084TdPHzsXBBxYqfGcmur84YSZe87_IoJ_ppCC4=.80f116b2-6589-45ea-8e25-e75409d03d6e@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
 <IHu0084TdPHzsXBBxYqfGcmur84YSZe87_IoJ_ppCC4=.80f116b2-6589-45ea-8e25-e75409d03d6e@github.com>
Message-ID: <H4zAHYOUbgurlFElsE3NFr2Fz0g6xLa7dmHMppLkdN8=.5fb926a6-d7b6-47e5-8eee-16d0c42ffc3c@github.com>

On Mon, 27 Jan 2025 02:05:02 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
>> 
>> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
>> 
>> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
>> 
>> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Respond to reviewer feedback

src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 318:

> 316: 
> 317:   if (ShenandoahHeuristics::should_start_gc()) {
> 318:     _start_gc_is_pending = true;

I assume there's no race here, i.e., only one thread reads/writes _start_gc_is_pending. If there's a race, make sure it's benign. In either case, _start_gc_is_pending is made "sticky" by this code.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.cpp line 261:

> 259: 
> 260: void ShenandoahHeuristics::record_success_concurrent() {
> 261:   _start_gc_is_pending = false;

The name _start_gc_is_pending implies that it should be set false as soon as a gc cycle starts, not when it finishes. Maybe _gc_pending? Or maybe setting it false at the end of a gc cycle is a bug? :)

src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.hpp line 87:

> 85:   size_t _declined_trigger_count;         // This counts how many times since previous GC finished that this
> 86:                                           //  heuristic has answered false to should_start_gc().
> 87:   size_t _previous_trigger_declinations;  // This represents the value of _declined_trigger_count as captured at the

Maybe the name should be _most_recent_declined_trigger_count, which relates it directly to _declined_trigger_count.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23305#discussion_r1941486248
PR Review Comment: https://git.openjdk.org/jdk/pull/23305#discussion_r1941462312
PR Review Comment: https://git.openjdk.org/jdk/pull/23305#discussion_r1941468695

From duke at openjdk.org  Tue Feb  4 16:52:10 2025
From: duke at openjdk.org (duke)
Date: Tue, 4 Feb 2025 16:52:10 GMT
Subject: RFR: 8348610: GenShen: TestShenandoahEvacuationInformationEvent
 failed with setRegions >= regionsFreed: expected 1 >= 57
In-Reply-To: <M-8yf6fmPVUPLubCfG83quv9jTAHkqhcrgy_dHy288M=.70835c1f-be6c-4ff7-b2a1-1eb2b96ba69f@github.com>
References: <M-8yf6fmPVUPLubCfG83quv9jTAHkqhcrgy_dHy288M=.70835c1f-be6c-4ff7-b2a1-1eb2b96ba69f@github.com>
Message-ID: <Y682gpBURzE6KT-KgazZKHdiU-vBk0dsL6MZC3oUiVo=.9e3cc60c-38a3-406a-b77a-c9318cc3b9e8@github.com>

On Thu, 30 Jan 2025 04:43:59 GMT, Satyen Subramaniam <ssubramaniam at openjdk.org> wrote:

> Renaming `ShenandoahEvacuationInformation.freedRegions` to `ShenandoahEvacuationInformation.freeRegions` for clarity, and fixing incorrect assertion in  TestShenandoahEvacuationInformationEvent.cpp
> 
> Tested with tier 1, tier 2, and tier 3 tests.

@satyenme 
Your change (at version 923a29d84a06315bfde7d3d1d8b48ff27fef8f9e) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23362#issuecomment-2634531314

From wkemper at openjdk.org  Tue Feb  4 17:25:20 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 4 Feb 2025 17:25:20 GMT
Subject: RFR: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact:
 fatal error: Before Updating References: Thread C2 CompilerThread1: expected
 gc-state 9, actual 21
In-Reply-To: <W_tKubqPMhpIKr7jBXx7mVaT_qb90sHc6t2O9EK_oNs=.fdf2009d-8480-43c1-ad0f-93a6e6125a2d@github.com>
References: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
 <W_tKubqPMhpIKr7jBXx7mVaT_qb90sHc6t2O9EK_oNs=.fdf2009d-8480-43c1-ad0f-93a6e6125a2d@github.com>
Message-ID: <G_RWRFu89V4SKQ8NbxJNMozoss0MX3mT3ePrn6n_GuU=.bbe1a219-733d-44a3-a9f3-6c69a32fde98@github.com>

On Tue, 4 Feb 2025 16:03:02 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Non-java threads were not having their gc-state configured when they attach. If they were created before the verifier's safepoint, but after the iteration over non-java threads, they would not have the correct state.
>
> It looks generally okay, but I am confused how this fixes a bad state in `C2 CompilerThread1`, since compiler threads are Java threads? https://github.com/openjdk/jdk/blob/beb43e2633900bb9ab3c975376fe5860b6d054e0/src/hotspot/share/compiler/compilerThread.hpp#L42

@shipilev , that is a good point. Will take a closer look.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23428#issuecomment-2634609075

From ssubramaniam at openjdk.org  Tue Feb  4 17:22:25 2025
From: ssubramaniam at openjdk.org (Satyen Subramaniam)
Date: Tue, 4 Feb 2025 17:22:25 GMT
Subject: Integrated: 8348610: GenShen: TestShenandoahEvacuationInformationEvent
 failed with setRegions >= regionsFreed: expected 1 >= 57
In-Reply-To: <M-8yf6fmPVUPLubCfG83quv9jTAHkqhcrgy_dHy288M=.70835c1f-be6c-4ff7-b2a1-1eb2b96ba69f@github.com>
References: <M-8yf6fmPVUPLubCfG83quv9jTAHkqhcrgy_dHy288M=.70835c1f-be6c-4ff7-b2a1-1eb2b96ba69f@github.com>
Message-ID: <DM9z6PkzitGwOVKzfTsFfQrZKnE_DtnsRwbfbGKlXks=.f029b5e1-6a1a-4250-98b5-cbeaf30b8bf5@github.com>

On Thu, 30 Jan 2025 04:43:59 GMT, Satyen Subramaniam <ssubramaniam at openjdk.org> wrote:

> Renaming `ShenandoahEvacuationInformation.freedRegions` to `ShenandoahEvacuationInformation.freeRegions` for clarity, and fixing incorrect assertion in  TestShenandoahEvacuationInformationEvent.cpp
> 
> Tested with tier 1, tier 2, and tier 3 tests.

This pull request has now been integrated.

Changeset: bad39b6d
Author:    Satyen Subramaniam <ssubramaniam at openjdk.org>
Committer: Paul Hohensee <phh at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/bad39b6d8892ba9b86bc81bf01108a1df617defb
Stats:     15 lines in 5 files changed: 2 ins; 1 del; 12 mod

8348610: GenShen: TestShenandoahEvacuationInformationEvent failed with setRegions >= regionsFreed: expected 1 >= 57

Reviewed-by: wkemper, phh

-------------

PR: https://git.openjdk.org/jdk/pull/23362

From wkemper at openjdk.org  Wed Feb  5 19:06:20 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 5 Feb 2025 19:06:20 GMT
Subject: [jdk24] Withdrawn: 8349002: GenShen: Deadlock during shutdown
In-Reply-To: <6a_6G1g93RUACyYcHG5B9HtqFBaqdETRRdhvFWwrfi8=.e88c6252-f6c4-4e3f-972a-2c4495d27127@github.com>
References: <6a_6G1g93RUACyYcHG5B9HtqFBaqdETRRdhvFWwrfi8=.e88c6252-f6c4-4e3f-972a-2c4495d27127@github.com>
Message-ID: <wZterSJOfWd72jwsBtM5te6qUsNI_CyClmgaK2wkFFE=.56462344-b0b0-4f3b-b984-6a158120042b@github.com>

On Mon, 3 Feb 2025 22:27:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport. Fixes bug introduced by [JDK-8345970](https://bugs.openjdk.org/browse/JDK-8345970).

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/23429

From wkemper at openjdk.org  Wed Feb  5 19:06:20 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 5 Feb 2025 19:06:20 GMT
Subject: [jdk24] RFR: 8349002: GenShen: Deadlock during shutdown
In-Reply-To: <6a_6G1g93RUACyYcHG5B9HtqFBaqdETRRdhvFWwrfi8=.e88c6252-f6c4-4e3f-972a-2c4495d27127@github.com>
References: <6a_6G1g93RUACyYcHG5B9HtqFBaqdETRRdhvFWwrfi8=.e88c6252-f6c4-4e3f-972a-2c4495d27127@github.com>
Message-ID: <uBVN5xuNC6ySQ8AqYuqG-tTt9wrhxuc6MNK6FtBBvKc=.c7e895c7-7d49-45ae-a1e9-63ca7399d140@github.com>

On Mon, 3 Feb 2025 22:27:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport. Fixes bug introduced by [JDK-8345970](https://bugs.openjdk.org/browse/JDK-8345970).

Understood. Will target JDK24 update release.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23429#issuecomment-2637789977

From wkemper at openjdk.org  Wed Feb  5 19:09:59 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 5 Feb 2025 19:09:59 GMT
Subject: Integrated: 8349002: GenShen: Deadlock during shutdown
In-Reply-To: <D14C2C6K9KfySMH_Wnsax5V5VjjLaP4T6PIt_SBKpTI=.2a242bc8-689a-4203-a97f-a18257b52201@github.com>
References: <D14C2C6K9KfySMH_Wnsax5V5VjjLaP4T6PIt_SBKpTI=.2a242bc8-689a-4203-a97f-a18257b52201@github.com>
Message-ID: <IA4Hu8lSnTTgcvZnp86L286hPqHiMcwCqWwt7zEemxo=.4f0c6415-4d90-4f3e-9665-8e306acddb1b@github.com>

On Mon, 3 Feb 2025 22:27:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport. Fixes bug introduced by [JDK-8345970](https://bugs.openjdk.org/browse/JDK-8345970).

This pull request has now been integrated.

Changeset: ceaeb7b4
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/ceaeb7b4271f52af9e73e97a5d19bd441fbcd96a
Stats:     5 lines in 2 files changed: 4 ins; 0 del; 1 mod

8349002: GenShen: Deadlock during shutdown

Backport-of: 06ebb170bac3879dc1e378b48b1c7ef006070c86

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/153

From wkemper at openjdk.org  Wed Feb  5 22:35:21 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 5 Feb 2025 22:35:21 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads may
 violate assertions
Message-ID: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>

There are several changes to the operation of Shenandoah's control threads here.
* The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
  *  The cancellation handling is driven entirely by the cancellation cause
  * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
  * The shutdown sequence is simpler
* The generational control thread uses a lock to coordinate updates to the requested cause and generation
  * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
  * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
  *  The control thread doesn't loop on its own (unless the pacer is enabled).

-------------

Commit messages:
 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - Simplify shControlThread
 - Revert unnecessary changes
 - Fix interrupted old cycle handling
 - Restore reporting allocations to pacer
 - Better names, better comments
 - WIP: Simplify shutdown protocol
 - WIP: Don't need request.mode anymore
 - WIP: Simplify degenerated cycle handling
 - WIP: Passes tier1, mostly passes tier2
 - ... and 4 more: https://git.openjdk.org/jdk/compare/b499c827...f97f257b

Changes: https://git.openjdk.org/jdk/pull/23475/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8349094
  Stats: 817 lines in 14 files changed: 241 ins; 286 del; 290 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Thu Feb  6 14:25:15 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 6 Feb 2025 14:25:15 GMT
Subject: RFR: Merge openjdk/jdk21u:master
Message-ID: <RyGHA8rSeFhJHbDEmgr1dCx6tiKb68IbsukYnosB_Wk=.9d54deef-5f29-44f4-93a0-6cf4598a0530@github.com>

Merges tag jdk-21.0.7+1

-------------

Commit messages:
 - 8345468: test/jdk/javax/swing/JScrollBar/4865918/bug4865918.java fails in ubuntu22.04
 - 8348625: [21u, 17u] Revert JDK-8185862 to restore old java.awt.headless behavior on Windows
 - 8347427: JTabbedPane/8134116/Bug8134116.java has no license header
 - 8332494: java/util/zip/EntryCount64k.java failing with java.lang.RuntimeException: '\\A\\Z' missing from stderr
 - 8343378: Exceptions in javax/management DeadLockTest.java do not cause test failure
 - 8327986: ASAN reports use-after-free in DirectivesParserTest.empty_object_vm
 - 8328387: Convert java/awt/Frame/FrameStateTest/FrameStateTest.html applet test to main
 - 8327098: GTest needs larger combination limit
 - 8315486: vmTestbase/nsk/jdwp/ThreadReference/ForceEarlyReturn/forceEarlyReturn002/forceEarlyReturn002.java timed out
 - 8347129: cpuset cgroups controller is required for no good reason
 - ... and 157 more: https://git.openjdk.org/shenandoah-jdk21u/compare/7069f193...d2cbada0

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=154&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/154/files
  Stats: 61790 lines in 579 files changed: 15357 ins; 6745 del; 39688 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/154.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/154/head:pull/154

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/154

From wkemper at openjdk.org  Thu Feb  6 17:20:52 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 6 Feb 2025 17:20:52 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v2]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <9vqH905wEy_k3MoOq-wmpzFWuniRKpiDAu6en7bOSr4=.a8fee870-a8fc-4532-acc7-c37975e8a948@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Remove invalid assert, alloc waiters wait until allocation failure is clear

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/f97f257b..a7a6eea1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=00-01

  Stats: 5 lines in 2 files changed: 2 ins; 1 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From kdnilsen at openjdk.org  Thu Feb  6 23:34:15 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 6 Feb 2025 23:34:15 GMT
Subject: RFR: 8343468: GenShen: Enable relocation of remembered set card
 tables [v3]
In-Reply-To: <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
References: <e5SIIH1XtQ-EFSNZk2-rUzYP_HFku9C8LK_O-nVMSzI=.9d639980-5a54-41dc-989d-253669d3c392@github.com>
 <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
Message-ID: <VzwBg_BVEkJ0xzjo-HxNtEOz7Suv1gFq_55GdWEeWEE=.08bcb2f6-23a4-40ad-ac85-5739eed39c6a@github.com>

On Thu, 23 Jan 2025 05:45:43 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> In the current Generational Shenandoah implementation, the pointers to the read and write card tables are established at JVM launch time and fixed during the whole of the application execution. Because they are considered constants, they are embedded as such in JIT-compiled code. 
>> 
>> The cleaning of dirty cards in the read card table is performed during the `init-mark` pause, and our experiments show that it represents a sizable portion of that phase's duration. This pull request makes the addresses of the read and write card tables dynamic, with the end goal of reducing the duration of the `init-mark` pause by moving the cleaning of the dirty cards in the read card table to the `reset` concurrent phase.
>> 
>> The idea is quite simple. Instead of using distinct read and write card tables for the entire duration of the JVM execution, we alternate which card table serves as the read/write table during each GC cycle. In the `reset` phase we concurrently clean the cards in the the current _read_ table so that when the cycle reaches the next `init-mark` phase we have a version of the card table totally clear. In the next `init-mark` pause we swap the pointers to the base of the read and write tables. When the `init-mark` finishes the mutator threads will operate on the table just cleaned in the `reset` phase; the GC will operate on the table that just turned the new _read_ table.
>> 
>> Most of the changes in the patch account for the fact that the write card table is no longer at a fixed address.
>> 
>> The primary benefit of this change is that it eliminates the need to copy and zero the remembered set during the init-mark Safepoint. A secondary benefit is that it allows us to replace the init-mark Safepoint with an `init-mark` handshake?something we plan to work on after this PR is merged.
>> 
>> Our internal performance testing showed a significant reduction in the duration of `init-mark` pauses and no statistically significant regression due to the dynamic loading of the card table address in JIT-compiled code.
>> 
>> Functional testing was performed on Linux, macOS, Windows running on x64, AArch64, and their respective 32-bit versions. I?d appreciate it if someone with access to RISC-V (@luhenry  ?) and PowerPC (@TheRealMDoerr ?)  platforms could review and test the changes for those platforms, as I have limited access to running tests on them.
>
> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
> 
>  - Merge master
>  - Addressing PR comments: some refactorings, ppc fix, off-by-one fix.
>  - Relocation of Card Tables

Thanks for pulling this together.  Looks great.

-------------

Marked as reviewed by kdnilsen (Author).

PR Review: https://git.openjdk.org/jdk/pull/23170#pullrequestreview-2600251495

From wkemper at openjdk.org  Fri Feb  7 02:04:29 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Feb 2025 02:04:29 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v3]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <UQaK2zw8go-_73bVEZuZwCOfThx1vc4ww86Xtck1AV8=.a11ce26b-b213-4f3a-9cd3-ddaf107c2ea8@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request incrementally with three additional commits since the last revision:

 - Resuming an old cycle should not preempt a young cycle
 - Use logging tag 'thread' to help control debug volume
 - Do not stomp on pending requests when running a degenerated cycle

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/a7a6eea1..ae207480

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=01-02

  Stats: 64 lines in 8 files changed: 35 ins; 12 del; 17 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Fri Feb  7 22:21:45 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Feb 2025 22:21:45 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v4]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <uFt5WqIFxS4qLunXjHvh7TRkv2U0VOMlvVL2ei4QMRM=.d0206312-b3f0-46de-9a28-0fc25c2b5658@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Simplify locking protocol
 - Make shutdown more robust, make better use of request lock

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/ae207480..a6513bcb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=02-03

  Stats: 133 lines in 5 files changed: 54 ins; 39 del; 40 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Fri Feb  7 22:28:25 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Feb 2025 22:28:25 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v5]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <KWHUgYWu50v1-loQMIfw59z1kBIuIbkzCE6DkW78Iqg=.10fafc82-4518-4480-8b25-244dc174beb2@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix includes

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/a6513bcb..d16f6fd0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=03-04

  Stats: 3 lines in 1 file changed: 1 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Fri Feb  7 22:46:08 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Feb 2025 22:46:08 GMT
Subject: RFR: Merge openjdk/jdk21u:master [v2]
In-Reply-To: <RyGHA8rSeFhJHbDEmgr1dCx6tiKb68IbsukYnosB_Wk=.9d54deef-5f29-44f4-93a0-6cf4598a0530@github.com>
References: <RyGHA8rSeFhJHbDEmgr1dCx6tiKb68IbsukYnosB_Wk=.9d54deef-5f29-44f4-93a0-6cf4598a0530@github.com>
Message-ID: <uUxPWpyynmzP2TMufqObv4xtpEw3gAocXTL-qToGXL8=.ae16d250-4242-41ee-9472-36aeb7cbbe9d@github.com>

> Merges tag jdk-21.0.7+1

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 168 commits:

 - Merge branch 'master' into merge-jdk-21.0.7+1
 - 8345468: test/jdk/javax/swing/JScrollBar/4865918/bug4865918.java fails in ubuntu22.04
   
   Backport-of: 6f307623568efe4d90942cd22ec9a26b2e1ca1b1
 - 8348625: [21u, 17u] Revert JDK-8185862 to restore old java.awt.headless behavior on Windows
   
   Reviewed-by: sgehwolf
 - 8347427: JTabbedPane/8134116/Bug8134116.java has no license header
   
   Backport-of: f67b703625afa2e049c572978d29ac00d8c956d3
 - 8332494: java/util/zip/EntryCount64k.java failing with java.lang.RuntimeException: '\\A\\Z' missing from stderr
   
   Backport-of: f5ab7dff402a3152f5d5736cc6521b4be617eccf
 - 8343378: Exceptions in javax/management DeadLockTest.java do not cause test failure
   
   Backport-of: 4a70c83bd0c563185123ce9d8a34e006c62db7cc
 - 8327986: ASAN reports use-after-free in DirectivesParserTest.empty_object_vm
   
   Reviewed-by: rrich
   Backport-of: 47f33a59eaaffc74881fcc9e29d13ff9b2538c2a
 - 8328387: Convert java/awt/Frame/FrameStateTest/FrameStateTest.html applet test to main
   
   Backport-of: 269163d509ec3c80983f55c5b47f472fa76be26c
 - 8327098: GTest needs larger combination limit
   
   Backport-of: c901da48e30d53cb8e4e3c1f0584c5f2d3d095f1
 - 8315486: vmTestbase/nsk/jdwp/ThreadReference/ForceEarlyReturn/forceEarlyReturn002/forceEarlyReturn002.java timed out
   
   Backport-of: 041510dc21df36d9860f4f0048241c2cabb55ee7
 - ... and 158 more: https://git.openjdk.org/shenandoah-jdk21u/compare/ceaeb7b4...c2a2d6ce

-------------

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/154/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=154&range=01
  Stats: 61769 lines in 574 files changed: 15344 ins; 6744 del; 39681 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/154.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/154/head:pull/154

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/154

From wkemper at openjdk.org  Fri Feb  7 22:46:10 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Feb 2025 22:46:10 GMT
Subject: Integrated: Merge openjdk/jdk21u:master
In-Reply-To: <RyGHA8rSeFhJHbDEmgr1dCx6tiKb68IbsukYnosB_Wk=.9d54deef-5f29-44f4-93a0-6cf4598a0530@github.com>
References: <RyGHA8rSeFhJHbDEmgr1dCx6tiKb68IbsukYnosB_Wk=.9d54deef-5f29-44f4-93a0-6cf4598a0530@github.com>
Message-ID: <sq4Dn0RhqF4U4gJuQl0h5d-5sYppRLRopHeZGrKdUs4=.b3568246-5c97-4c3d-8fc7-69b6cecf15b0@github.com>

On Thu, 6 Feb 2025 14:18:40 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.7+1

This pull request has now been integrated.

Changeset: c9889dbe
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/c9889dbe139cf78b77a72122b65905eb389294ed
Stats:     61769 lines in 574 files changed: 15344 ins; 6744 del; 39681 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/154

From kdnilsen at openjdk.org  Fri Feb  7 23:59:52 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 7 Feb 2025 23:59:52 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
Message-ID: <zAm49fM7Tb8MIYYzf3SAzAwopT6Vmm4HeLf0ADbWxXc=.e698174c-fff2-4d1b-89bd-b060996c8e5b@github.com>

> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
> 
> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
> 
> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Respond to reviewer feedback
  
  In testing suggested refinements, I discovered a bug in original
  implementation.  ShenandoahFreeSet::capacity() does not represent the
  size of young generation.  It represents the total size of the young
  regions that had available memory at the time we most recently rebuilt
  the ShenandoahFreeSet.
  
  I am rerunning the performance tests following this suggested change.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23306/files
  - new: https://git.openjdk.org/jdk/pull/23306/files/a850e484..7969515d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23306&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23306&range=00-01

  Stats: 13 lines in 5 files changed: 4 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/23306.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23306/head:pull/23306

PR: https://git.openjdk.org/jdk/pull/23306

From kdnilsen at openjdk.org  Fri Feb  7 23:59:52 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 7 Feb 2025 23:59:52 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <BUeokbLWxbV2laEURYDBv2jFqmM2dS7Q0MiN5COnGhg=.d37408d4-ac5b-4271-bf16-6a097a0b80d1@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <BUeokbLWxbV2laEURYDBv2jFqmM2dS7Q0MiN5COnGhg=.d37408d4-ac5b-4271-bf16-6a097a0b80d1@github.com>
Message-ID: <DMBBzTtS0VG2VT_Sj-H1ciomyBTt9y5h-1rmFcaYLUk=.bf5c8675-3b06-4b82-a8ce-8a18a767c67f@github.com>

On Fri, 31 Jan 2025 01:15:01 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Respond to reviewer feedback
>>   
>>   In testing suggested refinements, I discovered a bug in original
>>   implementation.  ShenandoahFreeSet::capacity() does not represent the
>>   size of young generation.  It represents the total size of the young
>>   regions that had available memory at the time we most recently rebuilt
>>   the ShenandoahFreeSet.
>>   
>>   I am rerunning the performance tests following this suggested change.
>
> src/hotspot/share/gc/shenandoah/shenandoahMetrics.cpp line 52:
> 
>> 50:   size_t free_actual   = free_set->available();
>> 51:   // The sum of free_set->capacity() and ->reserved represents capacity of young in generational, heap in non-generational.
>> 52:   size_t free_expected = ((free_set->capacity() + free_set->reserved()) / 100) * ShenandoahCriticalFreeThreshold;
> 
> We may pass ShenandoahGeneration as parameter to `is_good_progress` to simplify the calculation of free_expected, it should be like:
> `
> generation->max_capacity() / 100 * ShenandoahCriticalFreeThreshold
> `
> Good part is, free_expected might be more accurate in Full GC/Degen for global cycle, e.g. Full GC collects memory for global, `free_expected` should be calculated using the metrics from global generation. But either way, `free_expected` is not clearly defined in generational mode now, current code also works.

Thanks for this suggestion.  I've made change.  It turns out there was actually a bug in the original implementation, so I am retesting the performance results.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1947334711

From kdnilsen at openjdk.org  Fri Feb  7 23:59:52 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 7 Feb 2025 23:59:52 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <o2bgv1zSGO79lVvAk1CfeYKGPl0XI7wA6nKAwvk9ylI=.a8b0ffaf-720f-4ae0-a8bc-8aaf9013bb01@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <o2bgv1zSGO79lVvAk1CfeYKGPl0XI7wA6nKAwvk9ylI=.a8b0ffaf-720f-4ae0-a8bc-8aaf9013bb01@github.com>
Message-ID: <T47Dxx2PdnYwhi7QdDf3DSRshKvbf3otG6tVWpxmWk0=.729a5c73-cfd9-4c1f-8a4c-94e874b03ad0@github.com>

On Tue, 4 Feb 2025 15:50:59 GMT, Paul Hohensee <phh at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Respond to reviewer feedback
>>   
>>   In testing suggested refinements, I discovered a bug in original
>>   implementation.  ShenandoahFreeSet::capacity() does not represent the
>>   size of young generation.  It represents the total size of the young
>>   regions that had available memory at the time we most recently rebuilt
>>   the ShenandoahFreeSet.
>>   
>>   I am rerunning the performance tests following this suggested change.
>
> src/hotspot/share/gc/shenandoah/shenandoahMetrics.cpp line 52:
> 
>> 50:   size_t free_actual   = free_set->available();
>> 51:   // The sum of free_set->capacity() and ->reserved represents capacity of young in generational, heap in non-generational.
>> 52:   size_t free_expected = ((free_set->capacity() + free_set->reserved()) / 100) * ShenandoahCriticalFreeThreshold;
> 
> As an outsider, the units involved and what exactly is being calculated is pretty opaque. Why would we divide by 100 to compute free_expected and not do the same for free_actual? Do we care about integer division truncation? The default value of ShenandoahCriticalFreeThreshold is 1, so multiplying by it is a nop by default, which seems strange.

ShenandoahCriticalFreeThreshold represents a percentage of the "total size".  To calculate N% of the young generation size, we divide the generation size by 100 and then multiply by ShenandoahCriticalFreeThreshold.  This code is a bit different in the most recent revision.  Do you think it needs a comment?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1947335561

From xpeng at openjdk.org  Sat Feb  8 02:11:21 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Sat, 8 Feb 2025 02:11:21 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <DMBBzTtS0VG2VT_Sj-H1ciomyBTt9y5h-1rmFcaYLUk=.bf5c8675-3b06-4b82-a8ce-8a18a767c67f@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <BUeokbLWxbV2laEURYDBv2jFqmM2dS7Q0MiN5COnGhg=.d37408d4-ac5b-4271-bf16-6a097a0b80d1@github.com>
 <DMBBzTtS0VG2VT_Sj-H1ciomyBTt9y5h-1rmFcaYLUk=.bf5c8675-3b06-4b82-a8ce-8a18a767c67f@github.com>
Message-ID: <VzAdY2hbY0nzMULrW3uWEXBdKnimP0a6NrdyqpXxM9g=.5eb1c3ca-95aa-42c3-85b1-b278eef78806@github.com>

On Fri, 7 Feb 2025 23:54:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahMetrics.cpp line 52:
>> 
>>> 50:   size_t free_actual   = free_set->available();
>>> 51:   // The sum of free_set->capacity() and ->reserved represents capacity of young in generational, heap in non-generational.
>>> 52:   size_t free_expected = ((free_set->capacity() + free_set->reserved()) / 100) * ShenandoahCriticalFreeThreshold;
>> 
>> We may pass ShenandoahGeneration as parameter to `is_good_progress` to simplify the calculation of free_expected, it should be like:
>> `
>> generation->max_capacity() / 100 * ShenandoahCriticalFreeThreshold
>> `
>> Good part is, free_expected might be more accurate in Full GC/Degen for global cycle, e.g. Full GC collects memory for global, `free_expected` should be calculated using the metrics from global generation. But either way, `free_expected` is not clearly defined in generational mode now, current code also works.
>
> Thanks for this suggestion.  I've made change.  It turns out there was actually a bug in the original implementation, so I am retesting the performance results.

Thanks, honest I didn't understand that why `(free_set->capacity() + free_set->reserved()` represents capacity of young in generational, is it the bug you found? `free_set->capacity()` excludes the regions doesn't have enough capacity(it is calculated when rebuild free set)

I thought a bit more, it makes more sense to calculate free_expected in `snap_before`, max_capacity of generations may change after collection, the free_expected should be calculated before the cycle.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1947405475

From duke at openjdk.org  Sat Feb  8 16:03:19 2025
From: duke at openjdk.org (duke)
Date: Sat, 8 Feb 2025 16:03:19 GMT
Subject: Withdrawn: 8344880: AArch64: Add compile time check for class offsets
In-Reply-To: <axMGQNFft8YRtpwLxP8a6DgDiRzIetPwr2lRN78pTbk=.2c614059-a00a-4379-8d53-145ac832fe92@github.com>
References: <axMGQNFft8YRtpwLxP8a6DgDiRzIetPwr2lRN78pTbk=.2c614059-a00a-4379-8d53-145ac832fe92@github.com>
Message-ID: <Lw99zpnw80Zh9PLX1AoEatFGdMBOFGYH8pbzJrFzxMY=.cee6faed-8062-4561-b395-53e128294f83@github.com>

On Fri, 6 Dec 2024 23:57:41 GMT, Chad Rakoczy <duke at openjdk.org> wrote:

> [JDK-8344880](https://bugs.openjdk.org/browse/JDK-8344880)
> 
> Adds compile time checks for str/ldr instructions to verify that the immediate offset will fit. This adds static_assert for constant offsets that are checked at compile time. The macro offset_of is not constexpr so instead the class size is checked. If the size of a class fits into a memory instructions then any offset in it will fit.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/22623

From wkemper at openjdk.org  Mon Feb 10 17:52:20 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Feb 2025 17:52:20 GMT
Subject: RFR: 8343468: GenShen: Enable relocation of remembered set card
 tables [v3]
In-Reply-To: <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
References: <e5SIIH1XtQ-EFSNZk2-rUzYP_HFku9C8LK_O-nVMSzI=.9d639980-5a54-41dc-989d-253669d3c392@github.com>
 <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
Message-ID: <xRgq4-I4LMM6_erzwBPLw8LEgzgO22opHTZBYhNR8L4=.6e68d5c4-d096-455e-abdc-116527323939@github.com>

On Thu, 23 Jan 2025 05:45:43 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> In the current Generational Shenandoah implementation, the pointers to the read and write card tables are established at JVM launch time and fixed during the whole of the application execution. Because they are considered constants, they are embedded as such in JIT-compiled code. 
>> 
>> The cleaning of dirty cards in the read card table is performed during the `init-mark` pause, and our experiments show that it represents a sizable portion of that phase's duration. This pull request makes the addresses of the read and write card tables dynamic, with the end goal of reducing the duration of the `init-mark` pause by moving the cleaning of the dirty cards in the read card table to the `reset` concurrent phase.
>> 
>> The idea is quite simple. Instead of using distinct read and write card tables for the entire duration of the JVM execution, we alternate which card table serves as the read/write table during each GC cycle. In the `reset` phase we concurrently clean the cards in the the current _read_ table so that when the cycle reaches the next `init-mark` phase we have a version of the card table totally clear. In the next `init-mark` pause we swap the pointers to the base of the read and write tables. When the `init-mark` finishes the mutator threads will operate on the table just cleaned in the `reset` phase; the GC will operate on the table that just turned the new _read_ table.
>> 
>> Most of the changes in the patch account for the fact that the write card table is no longer at a fixed address.
>> 
>> The primary benefit of this change is that it eliminates the need to copy and zero the remembered set during the init-mark Safepoint. A secondary benefit is that it allows us to replace the init-mark Safepoint with an `init-mark` handshake?something we plan to work on after this PR is merged.
>> 
>> Our internal performance testing showed a significant reduction in the duration of `init-mark` pauses and no statistically significant regression due to the dynamic loading of the card table address in JIT-compiled code.
>> 
>> Functional testing was performed on Linux, macOS, Windows running on x64, AArch64, and their respective 32-bit versions. I?d appreciate it if someone with access to RISC-V (@luhenry  ?) and PowerPC (@TheRealMDoerr ?)  platforms could review and test the changes for those platforms, as I have limited access to running tests on them.
>
> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
> 
>  - Merge master
>  - Addressing PR comments: some refactorings, ppc fix, off-by-one fix.
>  - Relocation of Card Tables

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23170#pullrequestreview-2606716056

From phh at openjdk.org  Mon Feb 10 18:44:13 2025
From: phh at openjdk.org (Paul Hohensee)
Date: Mon, 10 Feb 2025 18:44:13 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <T47Dxx2PdnYwhi7QdDf3DSRshKvbf3otG6tVWpxmWk0=.729a5c73-cfd9-4c1f-8a4c-94e874b03ad0@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <o2bgv1zSGO79lVvAk1CfeYKGPl0XI7wA6nKAwvk9ylI=.a8b0ffaf-720f-4ae0-a8bc-8aaf9013bb01@github.com>
 <T47Dxx2PdnYwhi7QdDf3DSRshKvbf3otG6tVWpxmWk0=.729a5c73-cfd9-4c1f-8a4c-94e874b03ad0@github.com>
Message-ID: <wkcah9cik6P2Pt3Of0c-gYIed6GLUPwedTBAAgEGL2c=.42050f75-1043-46c6-8ef1-918705a9daae@github.com>

On Fri, 7 Feb 2025 23:56:56 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahMetrics.cpp line 52:
>> 
>>> 50:   size_t free_actual   = free_set->available();
>>> 51:   // The sum of free_set->capacity() and ->reserved represents capacity of young in generational, heap in non-generational.
>>> 52:   size_t free_expected = ((free_set->capacity() + free_set->reserved()) / 100) * ShenandoahCriticalFreeThreshold;
>> 
>> As an outsider, the units involved and what exactly is being calculated is pretty opaque. Why would we divide by 100 to compute free_expected and not do the same for free_actual? Do we care about integer division truncation? The default value of ShenandoahCriticalFreeThreshold is 1, so multiplying by it is a nop by default, which seems strange.
>
> ShenandoahCriticalFreeThreshold represents a percentage of the "total size".  To calculate N% of the young generation size, we divide the generation size by 100 and then multiply by ShenandoahCriticalFreeThreshold.  This code is a bit different in the most recent revision.  Do you think it needs a comment?

Yes :)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1949689186

From shade at openjdk.org  Mon Feb 10 18:53:12 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 10 Feb 2025 18:53:12 GMT
Subject: RFR: 8343468: GenShen: Enable relocation of remembered set card
 tables [v3]
In-Reply-To: <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
References: <e5SIIH1XtQ-EFSNZk2-rUzYP_HFku9C8LK_O-nVMSzI=.9d639980-5a54-41dc-989d-253669d3c392@github.com>
 <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
Message-ID: <5GD87O6WaG7QG9PLlH7ssfGtp1szWUjmosVSl8-TAok=.d04789f7-7f87-4b44-bc76-80676f0f4fc8@github.com>

On Thu, 23 Jan 2025 05:45:43 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> In the current Generational Shenandoah implementation, the pointers to the read and write card tables are established at JVM launch time and fixed during the whole of the application execution. Because they are considered constants, they are embedded as such in JIT-compiled code. 
>> 
>> The cleaning of dirty cards in the read card table is performed during the `init-mark` pause, and our experiments show that it represents a sizable portion of that phase's duration. This pull request makes the addresses of the read and write card tables dynamic, with the end goal of reducing the duration of the `init-mark` pause by moving the cleaning of the dirty cards in the read card table to the `reset` concurrent phase.
>> 
>> The idea is quite simple. Instead of using distinct read and write card tables for the entire duration of the JVM execution, we alternate which card table serves as the read/write table during each GC cycle. In the `reset` phase we concurrently clean the cards in the the current _read_ table so that when the cycle reaches the next `init-mark` phase we have a version of the card table totally clear. In the next `init-mark` pause we swap the pointers to the base of the read and write tables. When the `init-mark` finishes the mutator threads will operate on the table just cleaned in the `reset` phase; the GC will operate on the table that just turned the new _read_ table.
>> 
>> Most of the changes in the patch account for the fact that the write card table is no longer at a fixed address.
>> 
>> The primary benefit of this change is that it eliminates the need to copy and zero the remembered set during the init-mark Safepoint. A secondary benefit is that it allows us to replace the init-mark Safepoint with an `init-mark` handshake?something we plan to work on after this PR is merged.
>> 
>> Our internal performance testing showed a significant reduction in the duration of `init-mark` pauses and no statistically significant regression due to the dynamic loading of the card table address in JIT-compiled code.
>> 
>> Functional testing was performed on Linux, macOS, Windows running on x64, AArch64, and their respective 32-bit versions. I?d appreciate it if someone with access to RISC-V (@luhenry  ?) and PowerPC (@TheRealMDoerr ?)  platforms could review and test the changes for those platforms, as I have limited access to running tests on them.
>
> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
> 
>  - Merge master
>  - Addressing PR comments: some refactorings, ppc fix, off-by-one fix.
>  - Relocation of Card Tables

I'll take a look at this tomorrow, thanks.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23170#issuecomment-2648942403

From kdnilsen at openjdk.org  Mon Feb 10 19:55:12 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 19:55:12 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v2]
In-Reply-To: <H4zAHYOUbgurlFElsE3NFr2Fz0g6xLa7dmHMppLkdN8=.5fb926a6-d7b6-47e5-8eee-16d0c42ffc3c@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
 <IHu0084TdPHzsXBBxYqfGcmur84YSZe87_IoJ_ppCC4=.80f116b2-6589-45ea-8e25-e75409d03d6e@github.com>
 <H4zAHYOUbgurlFElsE3NFr2Fz0g6xLa7dmHMppLkdN8=.5fb926a6-d7b6-47e5-8eee-16d0c42ffc3c@github.com>
Message-ID: <68DeNcSBaX3EJo0OuQI7800ywqaQjhcCMpIjFqwdoao=.0da72a64-afa1-43bc-83bb-d4caf0d62514@github.com>

On Tue, 4 Feb 2025 16:08:02 GMT, Paul Hohensee <phh at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Respond to reviewer feedback
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.hpp line 87:
> 
>> 85:   size_t _declined_trigger_count;         // This counts how many times since previous GC finished that this
>> 86:                                           //  heuristic has answered false to should_start_gc().
>> 87:   size_t _previous_trigger_declinations;  // This represents the value of _declined_trigger_count as captured at the
> 
> Maybe the name should be _most_recent_declined_trigger_count, which relates it directly to _declined_trigger_count.

Thanks for suggestion.  I'm making this change.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23305#discussion_r1949788716

From kdnilsen at openjdk.org  Mon Feb 10 20:02:14 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 20:02:14 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v2]
In-Reply-To: <H4zAHYOUbgurlFElsE3NFr2Fz0g6xLa7dmHMppLkdN8=.5fb926a6-d7b6-47e5-8eee-16d0c42ffc3c@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
 <IHu0084TdPHzsXBBxYqfGcmur84YSZe87_IoJ_ppCC4=.80f116b2-6589-45ea-8e25-e75409d03d6e@github.com>
 <H4zAHYOUbgurlFElsE3NFr2Fz0g6xLa7dmHMppLkdN8=.5fb926a6-d7b6-47e5-8eee-16d0c42ffc3c@github.com>
Message-ID: <fxdIM6L7YnC1Dfqr2mHSUIEewZuJxi5NROXEUIZV1cA=.45fce6cc-f3df-431e-8172-7bc6e39de0b3@github.com>

On Tue, 4 Feb 2025 16:04:34 GMT, Paul Hohensee <phh at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Respond to reviewer feedback
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.cpp line 261:
> 
>> 259: 
>> 260: void ShenandoahHeuristics::record_success_concurrent() {
>> 261:   _start_gc_is_pending = false;
> 
> The name _start_gc_is_pending implies that it should be set false as soon as a gc cycle starts, not when it finishes. Maybe _gc_pending? Or maybe setting it false at the end of a gc cycle is a bug? :)

You make a good point.  I'll change the control flow to cancel the trigger as soon as we start up the GC.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23305#discussion_r1949798178

From kdnilsen at openjdk.org  Mon Feb 10 20:28:54 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 20:28:54 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v3]
In-Reply-To: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
Message-ID: <2v0axonBAvZDKo779TX8POWEXGeMCA5xaKV3KQBQo14=.fbd1e6bc-0e12-4a0c-a9f7-ba1d3c5f728d@github.com>

> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
> 
> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
> 
> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
> 
> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Respond to reviewer feedback
 - Use generation size to determine expected free

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23305/files
  - new: https://git.openjdk.org/jdk/pull/23305/files/ee3cdacc..8a9e4c5e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23305&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23305&range=01-02

  Stats: 27 lines in 8 files changed: 13 ins; 3 del; 11 mod
  Patch: https://git.openjdk.org/jdk/pull/23305.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23305/head:pull/23305

PR: https://git.openjdk.org/jdk/pull/23305

From kdnilsen at openjdk.org  Mon Feb 10 20:28:54 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 20:28:54 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v2]
In-Reply-To: <H4zAHYOUbgurlFElsE3NFr2Fz0g6xLa7dmHMppLkdN8=.5fb926a6-d7b6-47e5-8eee-16d0c42ffc3c@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
 <IHu0084TdPHzsXBBxYqfGcmur84YSZe87_IoJ_ppCC4=.80f116b2-6589-45ea-8e25-e75409d03d6e@github.com>
 <H4zAHYOUbgurlFElsE3NFr2Fz0g6xLa7dmHMppLkdN8=.5fb926a6-d7b6-47e5-8eee-16d0c42ffc3c@github.com>
Message-ID: <hL5KGQy9eaEBOHl0aG_ja94ywRbyR32agDpKPm_kV0Y=.107adf58-177d-483f-9b49-d75112ed89c5@github.com>

On Tue, 4 Feb 2025 16:14:49 GMT, Paul Hohensee <phh at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Respond to reviewer feedback
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 318:
> 
>> 316: 
>> 317:   if (ShenandoahHeuristics::should_start_gc()) {
>> 318:     _start_gc_is_pending = true;
> 
> I assume there's no race here, i.e., only one thread reads/writes _start_gc_is_pending. If there's a race, make sure it's benign. In either case, _start_gc_is_pending is made "sticky" by this code.

There is no race.  A single control thread queries should_start-gc() and that is the same thread that initiates the GC.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23305#discussion_r1949828557

From kdnilsen at openjdk.org  Mon Feb 10 20:41:10 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 20:41:10 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <VzAdY2hbY0nzMULrW3uWEXBdKnimP0a6NrdyqpXxM9g=.5eb1c3ca-95aa-42c3-85b1-b278eef78806@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <BUeokbLWxbV2laEURYDBv2jFqmM2dS7Q0MiN5COnGhg=.d37408d4-ac5b-4271-bf16-6a097a0b80d1@github.com>
 <DMBBzTtS0VG2VT_Sj-H1ciomyBTt9y5h-1rmFcaYLUk=.bf5c8675-3b06-4b82-a8ce-8a18a767c67f@github.com>
 <VzAdY2hbY0nzMULrW3uWEXBdKnimP0a6NrdyqpXxM9g=.5eb1c3ca-95aa-42c3-85b1-b278eef78806@github.com>
Message-ID: <tHIrbdWAxe_P0oKDw0njZCZpbGIQnGoy5hgmWGTdJ94=.0cc6086d-f8f3-413f-bbb5-73867ad93f44@github.com>

On Sat, 8 Feb 2025 02:06:13 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Thanks for this suggestion.  I've made change.  It turns out there was actually a bug in the original implementation, so I am retesting the performance results.
>
> Thanks, honest I didn't understand that why `(free_set->capacity() + free_set->reserved()` represents capacity of young in generational, is it the bug you found? `free_set->capacity()` is the capacity of all mutator regions which also excludes the regions doesn't have capacity for new object alloc(it is calculated when rebuild free set)
> 
> I thought a bit more, it makes more sense to calculate free_expected in `snap_before`, max_capacity of generations may change after collection, the free_expected should be calculated before the cycle.

Interesting thoughts.  So young-generation size will change under these circumstances:

1. There's a lot of young-gen memory to be promoted, or we choose to promote some young-gen regions in place (by relabeling the regions as OLD without evacuating their data).  In both of these cases, we may shrink young in order to expand old.
2. The GC cycle is mixed, so it has the side effect of reclaiming some old regions.  These reclaimed old regions will typically be granted back to young, until such time as we need to expand old in order to hold results of promotion.

While it makes sense for expected to be computed based on "original size" of young generation, the question of how much free remaining in young represents "good progress" should probably be based on the current size of young.  Ultimately, we are trying to figure out if there's enough memory in young to make it worthwhile to attempt another concurrent GC cycle.

I realize this thinking is a bit "fuzzy".  The heuristic was originally designed for non-generational use.

I'm inclined to keep as is currently implemented, but should probably add a comment to explain why.  What do you think?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1949847579

From wkemper at openjdk.org  Mon Feb 10 21:26:59 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Feb 2025 21:26:59 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v6]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <JQPJ9ojE1F6rNO1vHRonnKSU4kR2gq78A20LdyEEVG0=.8c6caae8-704d-4ac4-a53c-7dfda3ebecc8@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Add event for control thread state changes
 - Fix shutdown livelock error

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/d16f6fd0..f11584d5

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=04-05

  Stats: 13 lines in 1 file changed: 6 ins; 3 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Mon Feb 10 21:54:51 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Feb 2025 21:54:51 GMT
Subject: RFR: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact:
 fatal error: Before Updating References: Thread C2 CompilerThread1: expected
 gc-state 9, actual 21 [v2]
In-Reply-To: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
References: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
Message-ID: <73lkaeIWH7aBWahNyU_czTYSnSmCMOURWYDv55-zc4Y=.39398a24-6904-465c-8e47-7bfe32efc9db@github.com>

> Non-java threads were not having their gc-state configured when they attach. If they were created before the verifier's safepoint, but after the iteration over non-java threads, they would not have the correct state.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Hold the thread lock when concurrently changing gc state

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23428/files
  - new: https://git.openjdk.org/jdk/pull/23428/files/f402628e..1a4e3bb1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23428&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23428&range=00-01

  Stats: 13 lines in 1 file changed: 8 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/23428.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23428/head:pull/23428

PR: https://git.openjdk.org/jdk/pull/23428

From xpeng at openjdk.org  Mon Feb 10 22:27:12 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Mon, 10 Feb 2025 22:27:12 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <tHIrbdWAxe_P0oKDw0njZCZpbGIQnGoy5hgmWGTdJ94=.0cc6086d-f8f3-413f-bbb5-73867ad93f44@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <BUeokbLWxbV2laEURYDBv2jFqmM2dS7Q0MiN5COnGhg=.d37408d4-ac5b-4271-bf16-6a097a0b80d1@github.com>
 <DMBBzTtS0VG2VT_Sj-H1ciomyBTt9y5h-1rmFcaYLUk=.bf5c8675-3b06-4b82-a8ce-8a18a767c67f@github.com>
 <VzAdY2hbY0nzMULrW3uWEXBdKnimP0a6NrdyqpXxM9g=.5eb1c3ca-95aa-42c3-85b1-b278eef78806@github.com>
 <tHIrbdWAxe_P0oKDw0njZCZpbGIQnGoy5hgmWGTdJ94=.0cc6086d-f8f3-413f-bbb5-73867ad93f44@github.com>
Message-ID: <7B6iyKusHAIUGeaVJVEiCWQTIe64ZPJKImH-YYUB3K0=.8d7612e4-9e08-4b90-9c60-0f68d3e7c4ad@github.com>

On Mon, 10 Feb 2025 20:38:35 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Thanks, honest I didn't understand that why `(free_set->capacity() + free_set->reserved()` represents capacity of young in generational, is it the bug you found? `free_set->capacity()` is the capacity of all mutator regions which also excludes the regions doesn't have capacity for new object alloc(it is calculated when rebuild free set)
>> 
>> I thought a bit more, it makes more sense to calculate free_expected in `snap_before`, max_capacity of generations may change after collection, the free_expected should be calculated before the cycle.
>
> Interesting thoughts.  So young-generation size will change under these circumstances:
> 
> 1. There's a lot of young-gen memory to be promoted, or we choose to promote some young-gen regions in place (by relabeling the regions as OLD without evacuating their data).  In both of these cases, we may shrink young in order to expand old.
> 2. The GC cycle is mixed, so it has the side effect of reclaiming some old regions.  These reclaimed old regions will typically be granted back to young, until such time as we need to expand old in order to hold results of promotion.
> 
> While it makes sense for expected to be computed based on "original size" of young generation, the question of how much free remaining in young represents "good progress" should probably be based on the current size of young.  Ultimately, we are trying to figure out if there's enough memory in young to make it worthwhile to attempt another concurrent GC cycle.
> 
> I realize this thinking is a bit "fuzzy".  The heuristic was originally designed for non-generational use.
> 
> I'm inclined to keep as is currently implemented, but should probably add a comment to explain why.  What do you think?

Thanks for the explanation, I agree with it is bit "fuzzy".  
I'm not sure we should consider following case:

Degen cycle doesn't reclaim any memory,  but promoted some young regions resulting in young capacity to shrink, in this case we may treat it as "good progress" but actually it is not.

A "good progress" could be `free_actual_after > free_actual_before && free_actual_after > free_expected`, what do you think? I am not sure all cases triggering degen cycle, this might be a false case that never happens.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1949985042

From kdnilsen at openjdk.org  Mon Feb 10 23:19:27 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 23:19:27 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v4]
In-Reply-To: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
Message-ID: <ClCqfVUL3mBY0YUmG9SCA-xUXoSFWmUuegzHXmGdCU4=.01a5ff2b-16ff-431b-be14-29994cbfc386@github.com>

> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
> 
> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
> 
> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
> 
> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Revert "Use generation size to determine expected free"
  
  This reverts commit 94a32ebfe5fefcc0e899e09e6fbfc0585c62b4e0.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23305/files
  - new: https://git.openjdk.org/jdk/pull/23305/files/8a9e4c5e..ee7fe689

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23305&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23305&range=02-03

  Stats: 5 lines in 4 files changed: 0 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/23305.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23305/head:pull/23305

PR: https://git.openjdk.org/jdk/pull/23305

From kdnilsen at openjdk.org  Mon Feb 10 23:32:09 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 23:32:09 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <7B6iyKusHAIUGeaVJVEiCWQTIe64ZPJKImH-YYUB3K0=.8d7612e4-9e08-4b90-9c60-0f68d3e7c4ad@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <BUeokbLWxbV2laEURYDBv2jFqmM2dS7Q0MiN5COnGhg=.d37408d4-ac5b-4271-bf16-6a097a0b80d1@github.com>
 <DMBBzTtS0VG2VT_Sj-H1ciomyBTt9y5h-1rmFcaYLUk=.bf5c8675-3b06-4b82-a8ce-8a18a767c67f@github.com>
 <VzAdY2hbY0nzMULrW3uWEXBdKnimP0a6NrdyqpXxM9g=.5eb1c3ca-95aa-42c3-85b1-b278eef78806@github.com>
 <tHIrbdWAxe_P0oKDw0njZCZpbGIQnGoy5hgmWGTdJ94=.0cc6086d-f8f3-413f-bbb5-73867ad93f44@github.com>
 <7B6iyKusHAIUGeaVJVEiCWQTIe64ZPJKImH-YYUB3K0=.8d7612e4-9e08-4b90-9c60-0f68d3e7c4ad@github.com>
Message-ID: <16WXn9LEVXGdRSeJ98OxomG66UfnruLxo9nnfY52ZJo=.f4acdbb1-c99b-4be8-807b-bdbf9504af81@github.com>

On Mon, 10 Feb 2025 22:24:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Interesting thoughts.  So young-generation size will change under these circumstances:
>> 
>> 1. There's a lot of young-gen memory to be promoted, or we choose to promote some young-gen regions in place (by relabeling the regions as OLD without evacuating their data).  In both of these cases, we may shrink young in order to expand old.
>> 2. The GC cycle is mixed, so it has the side effect of reclaiming some old regions.  These reclaimed old regions will typically be granted back to young, until such time as we need to expand old in order to hold results of promotion.
>> 
>> While it makes sense for expected to be computed based on "original size" of young generation, the question of how much free remaining in young represents "good progress" should probably be based on the current size of young.  Ultimately, we are trying to figure out if there's enough memory in young to make it worthwhile to attempt another concurrent GC cycle.
>> 
>> I realize this thinking is a bit "fuzzy".  The heuristic was originally designed for non-generational use.
>> 
>> I'm inclined to keep as is currently implemented, but should probably add a comment to explain why.  What do you think?
>
> Thanks for the explanation, I agree with it is bit "fuzzy".  
> I'm not sure we should consider following case:
> 
> Degen cycle doesn't reclaim any memory,  but promoted some young regions resulting in young capacity to shrink, in this case we may treat it as "good progress" but actually it is not.
> 
> A "good progress" could be `free_actual_after > free_actual_before && free_actual_after > free_expected`, what do you think? I am not sure all cases triggering degen cycle, this might be a false case that never happens.

If we manage to pass the test "free_actual_after > free_expected" following the degen, even if young has shrunk, I think it is reasonable to pursue concurrent GC.  Passing this exact test at the end of the next GC (assuming no further adjustments to generation sizes) would qualify us to continue with concurrent GC on the next cycle.

In general, it is very rare that "full gc" is the right thing to do.  we're in the process of deprecating it entirely.

I will add a comment to clarify the thinking here.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1950040394

From kdnilsen at openjdk.org  Mon Feb 10 23:43:11 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 23:43:11 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <zAm49fM7Tb8MIYYzf3SAzAwopT6Vmm4HeLf0ADbWxXc=.e698174c-fff2-4d1b-89bd-b060996c8e5b@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <zAm49fM7Tb8MIYYzf3SAzAwopT6Vmm4HeLf0ADbWxXc=.e698174c-fff2-4d1b-89bd-b060996c8e5b@github.com>
Message-ID: <SZ1EFVFauYcLj81aQkD8vO4Cq7oM7-irxrCEFcTx7SU=.547c3ed8-15f2-467c-a625-cf288ae9e7cf@github.com>

On Fri, 7 Feb 2025 23:59:52 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
>> 
>> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
>> 
>> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Respond to reviewer feedback
>   
>   In testing suggested refinements, I discovered a bug in original
>   implementation.  ShenandoahFreeSet::capacity() does not represent the
>   size of young generation.  It represents the total size of the young
>   regions that had available memory at the time we most recently rebuilt
>   the ShenandoahFreeSet.
>   
>   I am rerunning the performance tests following this suggested change.

These are updated performance results after making the change that uses generation size to determine expected.  This change computes a larger expected size, increasing the likelihood that a particular degenerated cycle will be considered "bad progress":

![Screenshot 2025-02-10 at 3 38 18?PM](https://github.com/user-attachments/assets/d0826502-aec1-4e30-88e7-03a4d25e5661)

This represents overall improvement compared to previously reported number.  It would appear that the difference in performance might be the result of "random noise".

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23306#issuecomment-2649499090

From kdnilsen at openjdk.org  Mon Feb 10 23:48:11 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Feb 2025 23:48:11 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <zAm49fM7Tb8MIYYzf3SAzAwopT6Vmm4HeLf0ADbWxXc=.e698174c-fff2-4d1b-89bd-b060996c8e5b@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <zAm49fM7Tb8MIYYzf3SAzAwopT6Vmm4HeLf0ADbWxXc=.e698174c-fff2-4d1b-89bd-b060996c8e5b@github.com>
Message-ID: <Zc3d5WxYqoel_exrIpCbodT0H9CQI70PhpRO4MzlwsM=.4268b971-b35f-43e3-b0a0-09450455d496@github.com>

On Fri, 7 Feb 2025 23:59:52 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
>> 
>> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
>> 
>> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Respond to reviewer feedback
>   
>   In testing suggested refinements, I discovered a bug in original
>   implementation.  ShenandoahFreeSet::capacity() does not represent the
>   size of young generation.  It represents the total size of the young
>   regions that had available memory at the time we most recently rebuilt
>   the ShenandoahFreeSet.
>   
>   I am rerunning the performance tests following this suggested change.

These are the results of combining both proposed PRs into a single execution test:

![Screenshot 2025-02-10 at 3 41 31?PM](https://github.com/user-attachments/assets/c3db758c-41ea-4c0d-a91e-0a44aaefc390)

This result is not as good as what was reported above.  In my judgment, it still represents improvement over tip.  The difference between the two runs may also be signal noise as there is no clear correlation between number of Full GCs and percentile latencies.  The two full GCs reported in the "better both (redo)" run both result from alloc failure during evacuation.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23306#issuecomment-2649506896

From xpeng at openjdk.org  Mon Feb 10 23:51:13 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Mon, 10 Feb 2025 23:51:13 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v2]
In-Reply-To: <zAm49fM7Tb8MIYYzf3SAzAwopT6Vmm4HeLf0ADbWxXc=.e698174c-fff2-4d1b-89bd-b060996c8e5b@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <zAm49fM7Tb8MIYYzf3SAzAwopT6Vmm4HeLf0ADbWxXc=.e698174c-fff2-4d1b-89bd-b060996c8e5b@github.com>
Message-ID: <GJX5UmG60nYTZQzDCkRycfByrzVWcBcfgGr198FrjHg=.09f76131-64f8-4481-94fd-1ad3e0c40fb8@github.com>

On Fri, 7 Feb 2025 23:59:52 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
>> 
>> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
>> 
>> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Respond to reviewer feedback
>   
>   In testing suggested refinements, I discovered a bug in original
>   implementation.  ShenandoahFreeSet::capacity() does not represent the
>   size of young generation.  It represents the total size of the young
>   regions that had available memory at the time we most recently rebuilt
>   the ShenandoahFreeSet.
>   
>   I am rerunning the performance tests following this suggested change.

Thank for the comprehensive tests and explanations, my approve doesn't count though:)

-------------

Marked as reviewed by xpeng (Author).

PR Review: https://git.openjdk.org/jdk/pull/23306#pullrequestreview-2607419434

From wkemper at openjdk.org  Tue Feb 11 00:54:36 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Feb 2025 00:54:36 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v7]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <7vsmPKQNSOx9PxGp2C1yjC5IeEtB2ZWPRybQQ-s4YNE=.1b8ffa7e-cc6d-4885-a9c4-16a503d9d8d9@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Do not accept requests if control thread is terminating
 - Notify waiters when control thread terminates

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/f11584d5..861ed699

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=06
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=05-06

  Stats: 26 lines in 3 files changed: 24 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From kdnilsen at openjdk.org  Tue Feb 11 03:39:41 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Feb 2025 03:39:41 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always need
 full gc
Message-ID: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>

In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

-------------

Commit messages:
 - Be less eager to upgrade degen to full gc

Changes: https://git.openjdk.org/jdk/pull/23552/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23552&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8349766
  Stats: 20 lines in 2 files changed: 17 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/23552.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23552/head:pull/23552

PR: https://git.openjdk.org/jdk/pull/23552

From kdnilsen at openjdk.org  Tue Feb 11 03:39:41 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Feb 2025 03:39:41 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc
In-Reply-To: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
Message-ID: <XZRWWDL1kjQZBICYDrGhlitxOC8RnO9PGQuyW_3rwD8=.4bb72f8f-a526-47c3-b566-410ec033533b@github.com>

On Tue, 11 Feb 2025 03:31:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

Some detailed results running the workload mentioned in JBS ticket on tip:

![Screenshot 2025-02-10 at 7 10 18?PM](https://github.com/user-attachments/assets/c06606a6-ec21-4e40-b117-915ddfc0d1f6)

These are results running the same workload with the changes of this PR:

![Screenshot 2025-02-10 at 7 35 47?PM](https://github.com/user-attachments/assets/432c227e-9bf4-4f21-8099-1b39b5af364a)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23552#issuecomment-2649732684
PR Comment: https://git.openjdk.org/jdk/pull/23552#issuecomment-2649733471

From kdnilsen at openjdk.org  Tue Feb 11 03:53:10 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Feb 2025 03:53:10 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc
In-Reply-To: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
Message-ID: <YZnf91uaYMt1BFuRpATQhdipCUPZlJ4_thybZMpvyD8=.93d0f06c-cd96-4601-b509-741fab505e9f@github.com>

On Tue, 11 Feb 2025 03:31:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

Green represents improvement compared to tip.  For runs 3-5, the new code is notably better.  Run 2 is significantly worse in p50, but about equal to tip average at p99.999 and above.  Run 1 is close to averages of tip at p50, but up to 56% above max at higher percentiles.

Most noteworthy is that we were able to significantly reduce the number of Full GCs without causing a crash or OOM.  On this workload, full GCs are known to require approximately 3 s of pause time.  The average degen cycle required 1.4s (102 out of cycle, 142 at roots, 149 at mark, 1 at evac, 28 at update refs).  Note that an upgraded Full GC results in a pause that is the sum of a Full GC plus a degenerated GC.

While there is reason to be concerned about trial two results on the PR code, I expect that unlucky scenario, whatever it was, will be much less likely in the context of in-flight PRs to advance triggering of GC when allocation rates are accelerating and to surge GC workers whenever there is increased risk of degenerated cycles.  Perhaps, we should wait until those other PRs are integrated and then retest.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23552#issuecomment-2649744182

From kdnilsen at openjdk.org  Tue Feb 11 04:08:48 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Feb 2025 04:08:48 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v3]
In-Reply-To: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
Message-ID: <0bbrstoX8nDMn2Ku_WwSYn_NYSSLi3yXkWdg28imCHo=.ab1661a4-1ea5-4c57-9fde-0ee63ebac027@github.com>

> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
> 
> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
> 
> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Add comments suggested by reviewers

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23306/files
  - new: https://git.openjdk.org/jdk/pull/23306/files/7969515d..8f644cdb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23306&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23306&range=01-02

  Stats: 15 lines in 1 file changed: 14 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/23306.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23306/head:pull/23306

PR: https://git.openjdk.org/jdk/pull/23306

From xpeng at openjdk.org  Tue Feb 11 05:54:11 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 11 Feb 2025 05:54:11 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v3]
In-Reply-To: <0bbrstoX8nDMn2Ku_WwSYn_NYSSLi3yXkWdg28imCHo=.ab1661a4-1ea5-4c57-9fde-0ee63ebac027@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <0bbrstoX8nDMn2Ku_WwSYn_NYSSLi3yXkWdg28imCHo=.ab1661a4-1ea5-4c57-9fde-0ee63ebac027@github.com>
Message-ID: <eJ2tG_0dTqUJGEWAPXiX3xmXul9ER59TN87JrlFSbxw=.c3e67421-f016-4a58-b745-baaefb07e181@github.com>

On Tue, 11 Feb 2025 04:08:48 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
>> 
>> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
>> 
>> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add comments suggested by reviewers

Marked as reviewed by xpeng (Author).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23306#pullrequestreview-2607738788

From shade at openjdk.org  Tue Feb 11 08:50:11 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 11 Feb 2025 08:50:11 GMT
Subject: RFR: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact:
 fatal error: Before Updating References: Thread C2 CompilerThread1: expected
 gc-state 9, actual 21 [v2]
In-Reply-To: <73lkaeIWH7aBWahNyU_czTYSnSmCMOURWYDv55-zc4Y=.39398a24-6904-465c-8e47-7bfe32efc9db@github.com>
References: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
 <73lkaeIWH7aBWahNyU_czTYSnSmCMOURWYDv55-zc4Y=.39398a24-6904-465c-8e47-7bfe32efc9db@github.com>
Message-ID: <Scp0_-fuEZaDz8QvfUeymprit1OymZQ-qTr2j4X8A_8=.1f662526-0498-4cd2-8184-355fcddb8994@github.com>

On Mon, 10 Feb 2025 21:54:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Non-java threads were not having their gc-state configured when they attach. If they were created before the verifier's safepoint, but after the iteration over non-java threads, they would not have the correct state.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Hold the thread lock when concurrently changing gc state

Great find. 

So that means we cannot safely do `ShenandoahHeap::set_gc_state_concurrent`, unless we hold `Threads_lock` and do a handshake afterwards? I think a part of comment that you have near `MutexLocker` can go to `ShenandoahHeap::set_gc_state_concurrent` with the `assert(Threads_lock->is_locked(), ...`.

-------------

PR Review: https://git.openjdk.org/jdk/pull/23428#pullrequestreview-2608045755

From phh at openjdk.org  Tue Feb 11 14:20:12 2025
From: phh at openjdk.org (Paul Hohensee)
Date: Tue, 11 Feb 2025 14:20:12 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v3]
In-Reply-To: <0bbrstoX8nDMn2Ku_WwSYn_NYSSLi3yXkWdg28imCHo=.ab1661a4-1ea5-4c57-9fde-0ee63ebac027@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <0bbrstoX8nDMn2Ku_WwSYn_NYSSLi3yXkWdg28imCHo=.ab1661a4-1ea5-4c57-9fde-0ee63ebac027@github.com>
Message-ID: <2XLAHIk0VEr8Xae-jNqjMZjBtPTrHqm8nl7tn_rigS8=.155e8a5a-193a-49b8-a773-b8e60b4dc3f5@github.com>

On Tue, 11 Feb 2025 04:08:48 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
>> 
>> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
>> 
>> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add comments suggested by reviewers

Looks good.

-------------

Marked as reviewed by phh (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23306#pullrequestreview-2608901033

From kdnilsen at openjdk.org  Tue Feb 11 14:20:13 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Feb 2025 14:20:13 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v3]
In-Reply-To: <wkcah9cik6P2Pt3Of0c-gYIed6GLUPwedTBAAgEGL2c=.42050f75-1043-46c6-8ef1-918705a9daae@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <o2bgv1zSGO79lVvAk1CfeYKGPl0XI7wA6nKAwvk9ylI=.a8b0ffaf-720f-4ae0-a8bc-8aaf9013bb01@github.com>
 <T47Dxx2PdnYwhi7QdDf3DSRshKvbf3otG6tVWpxmWk0=.729a5c73-cfd9-4c1f-8a4c-94e874b03ad0@github.com>
 <wkcah9cik6P2Pt3Of0c-gYIed6GLUPwedTBAAgEGL2c=.42050f75-1043-46c6-8ef1-918705a9daae@github.com>
Message-ID: <pVViI4puLHljKW080YNE-SPv2-cm4Co7bE8akJwq6HY=.577ce30e-2340-4caa-bb6d-92154d74ac07@github.com>

On Mon, 10 Feb 2025 18:41:27 GMT, Paul Hohensee <phh at openjdk.org> wrote:

>> ShenandoahCriticalFreeThreshold represents a percentage of the "total size".  To calculate N% of the young generation size, we divide the generation size by 100 and then multiply by ShenandoahCriticalFreeThreshold.  This code is a bit different in the most recent revision.  Do you think it needs a comment?
>
> Yes :)

I've added a comment here.  Thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23306#discussion_r1950933308

From ayang at openjdk.org  Tue Feb 11 15:28:25 2025
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Tue, 11 Feb 2025 15:28:25 GMT
Subject: RFR: 8348171: Refactor GenerationCounters and its subclasses [v5]
In-Reply-To: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
References: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
Message-ID: <7otkT63ENoyKzZ29CbYpycLLwL89ARajYg36Mstz4tQ=.fd3c7dcf-5a8b-44be-9205-09e3d160d54d@github.com>

> Simple refactoring of removing the use of `virtual` method and use concrete subclasses when needed.
> 
> Test: tier1-5

Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:

 - Merge branch 'master' into gen-counter
 - review
 - * some more refactoring
 - review
 - Merge branch 'master' into gen-counter
 - merge
 - gen-counter

-------------

Changes: https://git.openjdk.org/jdk/pull/23209/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23209&range=04
  Stats: 202 lines in 17 files changed: 6 ins; 160 del; 36 mod
  Patch: https://git.openjdk.org/jdk/pull/23209.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23209/head:pull/23209

PR: https://git.openjdk.org/jdk/pull/23209

From kdnilsen at openjdk.org  Tue Feb 11 18:15:58 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Feb 2025 18:15:58 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v4]
In-Reply-To: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
Message-ID: <FLRF9V0JTKiZw4WXb7N96ea5iXiGILKi4rFwxTLbnfI=.27464268-7856-4295-9175-32e146572f52@github.com>

> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
> 
> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
> 
> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:

 - Merge tag 'jdk-25+9' into fix-generational-no-progress-check
   
   Added tag jdk-25+9 for changeset 30f71622
 - Add comments suggested by reviewers
 - Respond to reviewer feedback
   
   In testing suggested refinements, I discovered a bug in original
   implementation.  ShenandoahFreeSet::capacity() does not represent the
   size of young generation.  It represents the total size of the young
   regions that had available memory at the time we most recently rebuilt
   the ShenandoahFreeSet.
   
   I am rerunning the performance tests following this suggested change.
 - Use freeset to determine goodness of progress
   
   As previously implemented, we used the heap size to measure goodness of
   progress.  However, heap size is only appropriate for non-generational
   Shenandoah.  Freeset abstraction works for both.
 - Use size-of young generation to assess progress
   
   Previously, we were using size of heap to asses progress of generational
   degenerated cycle.  But that is not appropriate, because the collection
   set is chosen based on the size of young generation.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23306/files
  - new: https://git.openjdk.org/jdk/pull/23306/files/8f644cdb..8c610136

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23306&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23306&range=02-03

  Stats: 43531 lines in 2988 files changed: 18658 ins; 14204 del; 10669 mod
  Patch: https://git.openjdk.org/jdk/pull/23306.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23306/head:pull/23306

PR: https://git.openjdk.org/jdk/pull/23306

From kdnilsen at openjdk.org  Tue Feb 11 18:21:09 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Feb 2025 18:21:09 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v5]
In-Reply-To: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
Message-ID: <RHrY14hAilv_keEaYg-9o-XfTOmrQYTWngPw9Rm6c_8=.228ab1f7-bdd4-4402-a912-5d484764fd8e@github.com>

> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
> 
> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
> 
> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
> 
> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains nine additional commits since the last revision:

 - Merge tag 'jdk-25+9' into eliminate-no-fault-degen-penalties
   
   Added tag jdk-25+9 for changeset 30f71622
 - Revert "Use generation size to determine expected free"
   
   This reverts commit 94a32ebfe5fefcc0e899e09e6fbfc0585c62b4e0.
 - Respond to reviewer feedback
 - Use generation size to determine expected free
 - Respond to reviewer feedback
 - Fix white space
 - Remove debug instrumentation
 - Only penalize heuristic if heuristic responsible
   
   If we degenerate through no fault of "late triggering", then do not
   penalize the heuristic.
 - Eliminate no-fault degen penalties
   
   As originally implemented, we apply penalties to the triggering
   heuristic every time we experience a degenerated cycle.  This has the
   effect of forcing GC triggers to spiral out of control.  This commit
   changes the penalty mechanism.  When a degen happens through no fault of
   the heuristic triggering mechanism, we do not pile on additional
   penalties.  Specifically, we consider that heuristic triggering is not
   responsible for a degenerated cycle that is associated with a GC that
   began immediately following the end of the previous GC cycle.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23305/files
  - new: https://git.openjdk.org/jdk/pull/23305/files/ee7fe689..3aabd4db

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23305&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23305&range=03-04

  Stats: 43531 lines in 2988 files changed: 18658 ins; 14204 del; 10669 mod
  Patch: https://git.openjdk.org/jdk/pull/23305.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23305/head:pull/23305

PR: https://git.openjdk.org/jdk/pull/23305

From phh at openjdk.org  Tue Feb 11 18:51:16 2025
From: phh at openjdk.org (Paul Hohensee)
Date: Tue, 11 Feb 2025 18:51:16 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v5]
In-Reply-To: <RHrY14hAilv_keEaYg-9o-XfTOmrQYTWngPw9Rm6c_8=.228ab1f7-bdd4-4402-a912-5d484764fd8e@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
 <RHrY14hAilv_keEaYg-9o-XfTOmrQYTWngPw9Rm6c_8=.228ab1f7-bdd4-4402-a912-5d484764fd8e@github.com>
Message-ID: <8Gt2wkVtRhYtPwLWfkuH8fWrboud7gjBRpCfzT2GeLw=.9e580aa0-34b7-4b7e-9ab7-f49cec2d3a6a@github.com>

On Tue, 11 Feb 2025 18:21:09 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
>> 
>> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
>> 
>> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
>> 
>> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains nine additional commits since the last revision:
> 
>  - Merge tag 'jdk-25+9' into eliminate-no-fault-degen-penalties
>    
>    Added tag jdk-25+9 for changeset 30f71622
>  - Revert "Use generation size to determine expected free"
>    
>    This reverts commit 94a32ebfe5fefcc0e899e09e6fbfc0585c62b4e0.
>  - Respond to reviewer feedback
>  - Use generation size to determine expected free
>  - Respond to reviewer feedback
>  - Fix white space
>  - Remove debug instrumentation
>  - Only penalize heuristic if heuristic responsible
>    
>    If we degenerate through no fault of "late triggering", then do not
>    penalize the heuristic.
>  - Eliminate no-fault degen penalties
>    
>    As originally implemented, we apply penalties to the triggering
>    heuristic every time we experience a degenerated cycle.  This has the
>    effect of forcing GC triggers to spiral out of control.  This commit
>    changes the penalty mechanism.  When a degen happens through no fault of
>    the heuristic triggering mechanism, we do not pile on additional
>    penalties.  Specifically, we consider that heuristic triggering is not
>    responsible for a degenerated cycle that is associated with a GC that
>    began immediately following the end of the previous GC cycle.

Marked as reviewed by phh (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23305#pullrequestreview-2609684439

From wkemper at openjdk.org  Tue Feb 11 19:31:09 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Feb 2025 19:31:09 GMT
Subject: RFR: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact:
 fatal error: Before Updating References: Thread C2 CompilerThread1: expected
 gc-state 9, actual 21 [v2]
In-Reply-To: <73lkaeIWH7aBWahNyU_czTYSnSmCMOURWYDv55-zc4Y=.39398a24-6904-465c-8e47-7bfe32efc9db@github.com>
References: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
 <73lkaeIWH7aBWahNyU_czTYSnSmCMOURWYDv55-zc4Y=.39398a24-6904-465c-8e47-7bfe32efc9db@github.com>
Message-ID: <91rj68CdahMzjrRCIMEH0mR6CxmDQayALIYHXBykJ5c=.a4164dc3-79db-43e8-9a8a-c6216e826f5b@github.com>

On Mon, 10 Feb 2025 21:54:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Non-java threads were not having their gc-state configured when they attach. If they were created before the verifier's safepoint, but after the iteration over non-java threads, they would not have the correct state.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Hold the thread lock when concurrently changing gc state

That's right. The `on_thread_attach` callback and the thread being added to the list of threads _does_ happen under the `Thread_lock`, by the handshake mechanism (and the java thread iterator) do _not_ take the thread lock. In this particular assertion violation, the thread received a stale `gc_state` when it attached (before the control thread even entered `concurrent_prepare_for_update_refs`), however, the control thread executed the handshake _before_ the recently attached thread was actually added to the java thread list.

I will update the comment and add an assert in `set_gc_state_concurrent`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23428#issuecomment-2651866466

From wkemper at openjdk.org  Tue Feb 11 19:39:25 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Feb 2025 19:39:25 GMT
Subject: RFR: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact:
 fatal error: Before Updating References: Thread C2 CompilerThread1: expected
 gc-state 9, actual 21 [v3]
In-Reply-To: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
References: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
Message-ID: <-v8uprH0cQK06apB7HGbrHNO31cCmzOXxMiZB8ipWx4=.7ce5e5da-bcc8-4101-9bf8-23fb899d06c2@github.com>

> Non-java threads were not having their gc-state configured when they attach. If they were created before the verifier's safepoint, but after the iteration over non-java threads, they would not have the correct state.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Update comments, add an assertion

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23428/files
  - new: https://git.openjdk.org/jdk/pull/23428/files/1a4e3bb1..c57bf8a0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23428&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23428&range=01-02

  Stats: 11 lines in 1 file changed: 6 ins; 1 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/23428.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23428/head:pull/23428

PR: https://git.openjdk.org/jdk/pull/23428

From shade at openjdk.org  Tue Feb 11 20:11:10 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 11 Feb 2025 20:11:10 GMT
Subject: RFR: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact:
 fatal error: Before Updating References: Thread C2 CompilerThread1: expected
 gc-state 9, actual 21 [v3]
In-Reply-To: <-v8uprH0cQK06apB7HGbrHNO31cCmzOXxMiZB8ipWx4=.7ce5e5da-bcc8-4101-9bf8-23fb899d06c2@github.com>
References: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
 <-v8uprH0cQK06apB7HGbrHNO31cCmzOXxMiZB8ipWx4=.7ce5e5da-bcc8-4101-9bf8-23fb899d06c2@github.com>
Message-ID: <hMJzi_F9tewrNeJ1E8zzhWDXxZQcF7oTUPrHsMYlStg=.fb35c15c-a58b-44de-bcc1-ffe11b7f26fe@github.com>

On Tue, 11 Feb 2025 19:39:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Non-java threads were not having their gc-state configured when they attach. If they were created before the verifier's safepoint, but after the iteration over non-java threads, they would not have the correct state.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update comments, add an assertion

Marked as reviewed by shade (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23428#pullrequestreview-2609916144

From wkemper at openjdk.org  Tue Feb 11 20:23:14 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Feb 2025 20:23:14 GMT
Subject: Integrated: 8348268: Test gc/shenandoah/TestResizeTLAB.java#compact:
 fatal error: Before Updating References: Thread C2 CompilerThread1:
 expected gc-state 9, actual 21
In-Reply-To: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
References: <tIgZKTAnDVtMoGA1apHRxj-knlEEvZUd62XktLNvNak=.332ac530-ec89-4839-a310-51ea7592d6b4@github.com>
Message-ID: <9AZBJik8xf6tZdYSesYFvrlDs6Z8tDbEZkXvQz7Cm6s=.cb15a767-4988-40df-b87e-2e1868a15752@github.com>

On Mon, 3 Feb 2025 20:28:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Non-java threads were not having their gc-state configured when they attach. Additionally, we need to hold the `Threads_lock` when concurrently changing the gc state to make sure that any stale gc state observed when the thread `attaches` is fixed by the handshake when the thread list is iterated.

This pull request has now been integrated.

Changeset: 8c09d40d
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/8c09d40d6c345fda9fc7b358a53cae3b5965580b
Stats:     22 lines in 2 files changed: 16 ins; 1 del; 5 mod

8348268: Test gc/shenandoah/TestResizeTLAB.java#compact: fatal error: Before Updating References: Thread C2 CompilerThread1: expected gc-state 9, actual 21

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/jdk/pull/23428

From wkemper at openjdk.org  Tue Feb 11 23:01:58 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Feb 2025 23:01:58 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v8]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <YoA3n6JAvpTWMh7IA7kUN1ihi1KWzAnQyA7H44T2ZKY=.0e1fc356-2730-48bb-b345-166f3bb3a33e@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Make shutdown safer for threads requesting (or expecting) gc

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/861ed699..047d6ffa

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=06-07

  Stats: 35 lines in 5 files changed: 9 ins; 18 del; 8 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Wed Feb 12 21:12:39 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 12 Feb 2025 21:12:39 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v9]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <ltreKsahABuHOVOHJcFTKVLxqot3tHA2mH4SPm7waVI=.d4ace7b3-3dac-465b-9c7b-6d9197b7982c@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Improve message for assertion

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/047d6ffa..779492c6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=08
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=07-08

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Thu Feb 13 00:20:40 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 13 Feb 2025 00:20:40 GMT
Subject: RFR: 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id
 && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out
 of range (3131947710)
Message-ID: <umB3nnb782QKrYg33qE5xyBQqg73WELy8VTtf2-Szog=.dca65055-d0be-4f12-ac3a-805545c50283@github.com>

Restore weak roots rendezvous handshake. This is necessary to have mutators complete the LRB before the concurrent GC invalidates any oop handles that may exist in native stacks.

-------------

Commit messages:
 - Restore weak roots rendezvous handshake

Changes: https://git.openjdk.org/jdk/pull/23604/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23604&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8348092
  Stats: 19 lines in 1 file changed: 14 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/23604.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23604/head:pull/23604

PR: https://git.openjdk.org/jdk/pull/23604

From shade at openjdk.org  Thu Feb 13 08:34:14 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 13 Feb 2025 08:34:14 GMT
Subject: RFR: 8348092: Shenandoah: assert(nk >=
 _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed:
 narrowKlass ID out of range (3131947710)
In-Reply-To: <umB3nnb782QKrYg33qE5xyBQqg73WELy8VTtf2-Szog=.dca65055-d0be-4f12-ac3a-805545c50283@github.com>
References: <umB3nnb782QKrYg33qE5xyBQqg73WELy8VTtf2-Szog=.dca65055-d0be-4f12-ac3a-805545c50283@github.com>
Message-ID: <NowW29iRZlwV13OTZvhpNhkJ8FY6TAtO3nWcOg8svhg=.5dfc7276-1b84-4d56-9572-dd5a46c8d987@github.com>

On Thu, 13 Feb 2025 00:15:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Restore weak roots rendezvous handshake. This is necessary to have mutators complete the LRB before the concurrent GC invalidates any oop handles that may exist in native stacks.

Marked as reviewed by shade (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23604#pullrequestreview-2614219213

From ayang at openjdk.org  Thu Feb 13 09:23:27 2025
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Thu, 13 Feb 2025 09:23:27 GMT
Subject: RFR: 8348171: Refactor GenerationCounters and its subclasses [v5]
In-Reply-To: <7otkT63ENoyKzZ29CbYpycLLwL89ARajYg36Mstz4tQ=.fd3c7dcf-5a8b-44be-9205-09e3d160d54d@github.com>
References: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
 <7otkT63ENoyKzZ29CbYpycLLwL89ARajYg36Mstz4tQ=.fd3c7dcf-5a8b-44be-9205-09e3d160d54d@github.com>
Message-ID: <XE5pwm8rCwcmsfkxJ5T5XGDxgaSHv_jcQgnmkhlBUJo=.d5b822d5-29e4-40c7-8dc5-d8c63fd16f9d@github.com>

On Tue, 11 Feb 2025 15:28:25 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Simple refactoring of removing the use of `virtual` method and use concrete subclasses when needed.
>> 
>> Test: tier1-5
>
> Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:
> 
>  - Merge branch 'master' into gen-counter
>  - review
>  - * some more refactoring
>  - review
>  - Merge branch 'master' into gen-counter
>  - merge
>  - gen-counter

Any suggestions/comments/objections from Shenandoah team? I'd like to merge this patch, if none.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23209#issuecomment-2655989267

From wkemper at openjdk.org  Thu Feb 13 14:23:52 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 13 Feb 2025 14:23:52 GMT
Subject: RFR: Merge openjdk/jdk21u:master
Message-ID: <u0BmLUiLr7B-vRiBasLRiY5VRLRdxZLVrbIc6iyAypE=.e7eb407f-0c9a-437d-ab5a-ce36c39f998c@github.com>

Merges tag jdk-21.0.7+2

-------------

Commit messages:
 - 8345368: java/io/File/createTempFile/SpecialTempFile.java fails on Windows Server 2025
 - 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025
 - 8349603: [21u, 17u, 11u] Update GHA JDKs after Jan/25 updates
 - 8348562: ZGC: segmentation fault due to missing node type check in barrier elision analysis
 - 8340411: open source several 2D imaging tests
 - 8330702: Update failure handler to don't generate Error message if cores actions are empty
 - 8334371: [AIX] Beginning with AIX 7.3 TL1 mmap() supports 64K memory pages
 - 8347911: Limit the length of inflated text chunks
 - 8338571: [TestBug] DefaultCloseOperation.java test not working as expected wrt instruction after JDK-8325851 fix
 - 8343491: javax/management/remote/mandatory/connection/DeadLockTest.java failing with NoSuchObjectException: no such object in table
 - ... and 7 more: https://git.openjdk.org/shenandoah-jdk21u/compare/d2cbada0...3e556491

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/155/files
  Stats: 1472 lines in 28 files changed: 1150 ins; 229 del; 93 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/155.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/155/head:pull/155

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/155

From wkemper at openjdk.org  Thu Feb 13 16:37:21 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 13 Feb 2025 16:37:21 GMT
Subject: Integrated: 8348092: Shenandoah: assert(nk >=
 _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed:
 narrowKlass ID out of range (3131947710)
In-Reply-To: <umB3nnb782QKrYg33qE5xyBQqg73WELy8VTtf2-Szog=.dca65055-d0be-4f12-ac3a-805545c50283@github.com>
References: <umB3nnb782QKrYg33qE5xyBQqg73WELy8VTtf2-Szog=.dca65055-d0be-4f12-ac3a-805545c50283@github.com>
Message-ID: <NKrKClqhIKHx9Itqn5VAWcsA_i75huTkhR7ChHxSH8c=.305860de-f496-4cb8-afc1-8d5a7b6eb139@github.com>

On Thu, 13 Feb 2025 00:15:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Restore weak roots rendezvous handshake. This is necessary to have mutators complete the LRB before the concurrent GC invalidates any oop handles that may exist in native stacks.

This pull request has now been integrated.

Changeset: 28e744dc
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/28e744dc642db8ebe376403f28630438a5ee3f44
Stats:     19 lines in 1 file changed: 14 ins; 0 del; 5 mod

8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/jdk/pull/23604

From andrew at openjdk.org  Thu Feb 13 16:46:47 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Thu, 13 Feb 2025 16:46:47 GMT
Subject: RFR: Merge jdk8u:master
Message-ID: <xhEXNUGehPgVRRKHsJXEN5C67tYL0yBqeqro9ab8Ufs=.3550e5e1-4cc9-4c55-a780-013a46be2129@github.com>

Merge jdk8u332-b06

-------------

Commit messages:
 - Merge jdk8u332-b06
 - 8277488: Add expiry exception for Digicert (geotrustglobalca) expiring in May 2022
 - Added tag jdk8u332-b05 for changeset 2a92df021686

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk8u/pull/11/files
  Stats: 4 lines in 2 files changed: 3 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk8u/pull/11.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk8u.git pull/11/head:pull/11

PR: https://git.openjdk.org/shenandoah-jdk8u/pull/11

From wkemper at openjdk.org  Thu Feb 13 16:59:08 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 13 Feb 2025 16:59:08 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v10]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <oXtWoCZfb8tXHdqPYPKfcf4U7iSniNg3M852Y-jJgC8=.fbaaba96-9ea4-4951-85ef-669d5f2d27b9@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 28 additional commits since the last revision:

 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - Improve message for assertion
 - Make shutdown safer for threads requesting (or expecting) gc
 - Do not accept requests if control thread is terminating
 - Notify waiters when control thread terminates
 - Add event for control thread state changes
 - Fix shutdown livelock error
 - Fix includes
 - Simplify locking protocol
 - Make shutdown more robust, make better use of request lock
 - ... and 18 more: https://git.openjdk.org/jdk/compare/06ea83a4...51d09207

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/779492c6..51d09207

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=09
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=08-09

  Stats: 12604 lines in 600 files changed: 8568 ins; 1551 del; 2485 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From andrew at openjdk.org  Thu Feb 13 17:27:25 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Thu, 13 Feb 2025 17:27:25 GMT
Subject: RFR: Merge jdk8u:master
In-Reply-To: <xhEXNUGehPgVRRKHsJXEN5C67tYL0yBqeqro9ab8Ufs=.3550e5e1-4cc9-4c55-a780-013a46be2129@github.com>
References: <xhEXNUGehPgVRRKHsJXEN5C67tYL0yBqeqro9ab8Ufs=.3550e5e1-4cc9-4c55-a780-013a46be2129@github.com>
Message-ID: <W58CIJg9Ap5wC7GoRRy_tu1w5DYVtU4d50S-8rHRyl4=.82265c78-e386-4bb1-b9c4-71faf772312a@github.com>

On Thu, 13 Feb 2025 16:42:13 GMT, Andrew John Hughes <andrew at openjdk.org> wrote:

> Merge jdk8u332-b06

GHA builds will not work until [JDK-8284622](https://bugs.openjdk.org/browse/JDK-8284622) is merged in 8u362-b03

-------------

PR Comment: https://git.openjdk.org/shenandoah-jdk8u/pull/11#issuecomment-2657279534

From andrew at openjdk.org  Thu Feb 13 22:12:58 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Thu, 13 Feb 2025 22:12:58 GMT
Subject: git: openjdk/shenandoah-jdk8u: master: 3 new changesets
Message-ID: <b1ff418e-cd2c-401a-b045-5452f4e4259d@openjdk.org>

Changeset: 0183285d
Branch: master
Author:    Andrew John Hughes <andrew at openjdk.org>
Date:      2022-03-21 20:56:59 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/0183285d00171de3cfb6791ee5eabe9127932a43

Added tag jdk8u332-b05 for changeset 2a92df021686

! .hgtags

Changeset: 9a303aef
Branch: master
Author:    Andrew John Hughes <andrew at openjdk.org>
Date:      2022-03-24 03:03:42 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/9a303aef21f8db21cf6acc9dc91b6ca33819eb01

8277488: Add expiry exception for Digicert (geotrustglobalca) expiring in May 2022

Reviewed-by: weijun, sgehwolf

! jdk/test/sun/security/lib/cacerts/VerifyCACerts.java

Changeset: 5ca34513
Branch: master
Author:    Andrew John Hughes <andrew at openjdk.org>
Date:      2025-02-12 20:18:34 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/5ca34513542f68d6adfe670e500e294b636fc6c2

Merge jdk8u332-b06



From andrew at openjdk.org  Thu Feb 13 22:13:07 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Thu, 13 Feb 2025 22:13:07 GMT
Subject: git: openjdk/shenandoah-jdk8u: Added tag jdk8u332-b06 for changeset
 9a303aef
Message-ID: <2bbee72d-097f-4ec2-b7f3-50e029a05129@openjdk.org>

Tagged by: Andrew John Hughes <andrew at openjdk.org>
Date:      2022-03-29 03:33:24 +0000


Changeset: 9a303aef
Author:    Andrew John Hughes <andrew at openjdk.org>
Date:      2022-03-24 03:03:42 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/9a303aef21f8db21cf6acc9dc91b6ca33819eb01


From andrew at openjdk.org  Thu Feb 13 22:13:10 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Thu, 13 Feb 2025 22:13:10 GMT
Subject: git: openjdk/shenandoah-jdk8u: Added tag shenandoah8u332-b06 for
 changeset 5ca34513
Message-ID: <9e464909-ca3e-47f2-b0c0-31f8f0129991@openjdk.org>

Tagged by: Andrew John Hughes <andrew at openjdk.org>
Date:      2025-02-13 16:39:09 +0000

Added tag shenandoah8u332-b06 for changeset 5ca34513542
Changeset: 5ca34513
Author:    Andrew John Hughes <andrew at openjdk.org>
Date:      2025-02-12 20:18:34 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/5ca34513542f68d6adfe670e500e294b636fc6c2


From andrew at openjdk.org  Thu Feb 13 22:14:51 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Thu, 13 Feb 2025 22:14:51 GMT
Subject: RFR: Merge jdk8u:master [v2]
In-Reply-To: <xhEXNUGehPgVRRKHsJXEN5C67tYL0yBqeqro9ab8Ufs=.3550e5e1-4cc9-4c55-a780-013a46be2129@github.com>
References: <xhEXNUGehPgVRRKHsJXEN5C67tYL0yBqeqro9ab8Ufs=.3550e5e1-4cc9-4c55-a780-013a46be2129@github.com>
Message-ID: <_v1DLgUyUWw1kzgJFpZtHoaMPUeOX6WMKuin7iLlz5Q=.3540a226-d66f-4cf9-9ee7-33bd46cec962@github.com>

> Merge jdk8u332-b06

Andrew John Hughes has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk8u/pull/11/files
  - new: https://git.openjdk.org/shenandoah-jdk8u/pull/11/files/5ca34513..5ca34513

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk8u&pr=11&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk8u&pr=11&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk8u/pull/11.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk8u.git pull/11/head:pull/11

PR: https://git.openjdk.org/shenandoah-jdk8u/pull/11

From iris at openjdk.org  Thu Feb 13 22:14:51 2025
From: iris at openjdk.org (Iris Clark)
Date: Thu, 13 Feb 2025 22:14:51 GMT
Subject: Withdrawn: Merge jdk8u:master
In-Reply-To: <xhEXNUGehPgVRRKHsJXEN5C67tYL0yBqeqro9ab8Ufs=.3550e5e1-4cc9-4c55-a780-013a46be2129@github.com>
References: <xhEXNUGehPgVRRKHsJXEN5C67tYL0yBqeqro9ab8Ufs=.3550e5e1-4cc9-4c55-a780-013a46be2129@github.com>
Message-ID: <E_CV7iczwNCgaZ9O6GOxXNNv_zFmtj0NMW7mlf8p9uE=.83518692-9db0-4f3c-be3e-a55b35b539af@github.com>

On Thu, 13 Feb 2025 16:42:13 GMT, Andrew John Hughes <andrew at openjdk.org> wrote:

> Merge jdk8u332-b06

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah-jdk8u/pull/11

From wkemper at openjdk.org  Thu Feb 13 22:39:08 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 13 Feb 2025 22:39:08 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc
In-Reply-To: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
Message-ID: <e-CR4R2-x1jit6dymbFjHM03V6AkJs2JyBlORCkc-qw=.2833fb3b-7eb5-4ddc-86d3-d40106c22fdb@github.com>

On Tue, 11 Feb 2025 03:31:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

Looks good to me.

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/jdk/pull/23552#pullrequestreview-2616371490

From wkemper at openjdk.org  Thu Feb 13 22:40:00 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 13 Feb 2025 22:40:00 GMT
Subject: RFR: Merge openjdk/jdk21u:master [v2]
In-Reply-To: <u0BmLUiLr7B-vRiBasLRiY5VRLRdxZLVrbIc6iyAypE=.e7eb407f-0c9a-437d-ab5a-ce36c39f998c@github.com>
References: <u0BmLUiLr7B-vRiBasLRiY5VRLRdxZLVrbIc6iyAypE=.e7eb407f-0c9a-437d-ab5a-ce36c39f998c@github.com>
Message-ID: <6xL6YwMmQPLuc9PSWMJ5qMgtcBOQzAzbDozzrvBqhf8=.772326dd-2742-4aef-bb20-a7496d354811@github.com>

> Merges tag jdk-21.0.7+2

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/155/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/155/files/3e556491..3e556491

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=155&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=155&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/155.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/155/head:pull/155

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/155

From wkemper at openjdk.org  Thu Feb 13 22:40:02 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 13 Feb 2025 22:40:02 GMT
Subject: Integrated: Merge openjdk/jdk21u:master
In-Reply-To: <u0BmLUiLr7B-vRiBasLRiY5VRLRdxZLVrbIc6iyAypE=.e7eb407f-0c9a-437d-ab5a-ce36c39f998c@github.com>
References: <u0BmLUiLr7B-vRiBasLRiY5VRLRdxZLVrbIc6iyAypE=.e7eb407f-0c9a-437d-ab5a-ce36c39f998c@github.com>
Message-ID: <V3TBpQK3BXXERKLX33qJZd5SBhNs6z8Q265iSZQ_Y5s=.9e31a08b-adbb-4baf-ab0b-a6e41d04ff62@github.com>

On Thu, 13 Feb 2025 14:19:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.7+2

This pull request has now been integrated.

Changeset: 1fc48d59
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/1fc48d598e1a7c3995dceaf0556d03ba7e313c34
Stats:     1472 lines in 28 files changed: 1150 ins; 229 del; 93 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/155

From kdnilsen at openjdk.org  Thu Feb 13 23:27:48 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 13 Feb 2025 23:27:48 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc [v2]
In-Reply-To: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
Message-ID: <pVoAuzqB-TW1r8UhKMcY8Z939qWvTtFZwvFeAVrC1qA=.46f18e05-5106-40d3-83f9-f62ebc2eec01@github.com>

> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:

 - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
   
   Reviewed-by: shade
 - Merge tag 'jdk-25+10' into defer-generational-full-gc
   
   Added tag jdk-25+10 for changeset a637ccf2
 - Be less eager to upgrade degen to full gc

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23552/files
  - new: https://git.openjdk.org/jdk/pull/23552/files/17e5e919..8d662e10

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23552&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23552&range=00-01

  Stats: 9730 lines in 593 files changed: 6779 ins; 1405 del; 1546 mod
  Patch: https://git.openjdk.org/jdk/pull/23552.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23552/head:pull/23552

PR: https://git.openjdk.org/jdk/pull/23552

From kdnilsen at openjdk.org  Fri Feb 14 01:18:01 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 14 Feb 2025 01:18:01 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v5]
In-Reply-To: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
Message-ID: <DhB7IXHkcseWMjgLWRyvSE791XCSZv_QcTPEXWzaX4c=.46b19bb5-7821-4537-8ef5-22102de95027@github.com>

> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
> 
> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
> 
> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:

 - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
   
   Reviewed-by: shade
 - Merge tag 'jdk-25+10' into fix-generational-no-progress-check
   
   Added tag jdk-25+10 for changeset a637ccf2
 - Merge tag 'jdk-25+9' into fix-generational-no-progress-check
   
   Added tag jdk-25+9 for changeset 30f71622
 - Add comments suggested by reviewers
 - Respond to reviewer feedback
   
   In testing suggested refinements, I discovered a bug in original
   implementation.  ShenandoahFreeSet::capacity() does not represent the
   size of young generation.  It represents the total size of the young
   regions that had available memory at the time we most recently rebuilt
   the ShenandoahFreeSet.
   
   I am rerunning the performance tests following this suggested change.
 - Use freeset to determine goodness of progress
   
   As previously implemented, we used the heap size to measure goodness of
   progress.  However, heap size is only appropriate for non-generational
   Shenandoah.  Freeset abstraction works for both.
 - Use size-of young generation to assess progress
   
   Previously, we were using size of heap to asses progress of generational
   degenerated cycle.  But that is not appropriate, because the collection
   set is chosen based on the size of young generation.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23306/files
  - new: https://git.openjdk.org/jdk/pull/23306/files/8c610136..0e86c5bd

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23306&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23306&range=03-04

  Stats: 12378 lines in 689 files changed: 8313 ins; 1890 del; 2175 mod
  Patch: https://git.openjdk.org/jdk/pull/23306.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23306/head:pull/23306

PR: https://git.openjdk.org/jdk/pull/23306

From kdnilsen at openjdk.org  Fri Feb 14 01:35:51 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 14 Feb 2025 01:35:51 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v6]
In-Reply-To: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
Message-ID: <xrk54IVeSvAYe15JPHpPERMtcZgpoZNMhFiLuD-q6JQ=.353f566a-48c8-42b7-bbb8-79828c8d6553@github.com>

> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
> 
> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
> 
> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
> 
> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:

 - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
   
   Reviewed-by: shade
 - Merge tag 'jdk-25+10' into eliminate-no-fault-degen-penalties
   
   Added tag jdk-25+10 for changeset a637ccf2
 - Merge tag 'jdk-25+9' into eliminate-no-fault-degen-penalties
   
   Added tag jdk-25+9 for changeset 30f71622
 - Revert "Use generation size to determine expected free"
   
   This reverts commit 94a32ebfe5fefcc0e899e09e6fbfc0585c62b4e0.
 - Respond to reviewer feedback
 - Use generation size to determine expected free
 - Respond to reviewer feedback
 - Fix white space
 - Remove debug instrumentation
 - Only penalize heuristic if heuristic responsible
   
   If we degenerate through no fault of "late triggering", then do not
   penalize the heuristic.
 - ... and 1 more: https://git.openjdk.org/jdk/compare/961a87d9...0d85e341

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23305/files
  - new: https://git.openjdk.org/jdk/pull/23305/files/3aabd4db..0d85e341

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23305&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23305&range=04-05

  Stats: 12378 lines in 689 files changed: 8313 ins; 1890 del; 2175 mod
  Patch: https://git.openjdk.org/jdk/pull/23305.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23305/head:pull/23305

PR: https://git.openjdk.org/jdk/pull/23305

From wkemper at openjdk.org  Fri Feb 14 01:48:58 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 14 Feb 2025 01:48:58 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v11]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <aAIUbSqmPIcv-FdB1TOBDVvNHDQoV2sc0MWSDVI3YNo=.1c29bbff-6ea7-43c6-a4dc-1ec07a68e200@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
> 
> ## Testing
> * jtreg hotspot_gc_shenandoah
> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Old gen bootstrap cycle must make it to init mark

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/51d09207..82f96090

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=10
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=09-10

  Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From xpeng at openjdk.org  Fri Feb 14 06:58:37 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 14 Feb 2025 06:58:37 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v8]
In-Reply-To: <6duTgo8vKHyCUnasOsrHp341B2krxcK8jNogKjX09gs=.af63669e-9c8d-4f17-b055-bf3a03a9618e@github.com>
References: <6duTgo8vKHyCUnasOsrHp341B2krxcK8jNogKjX09gs=.af63669e-9c8d-4f17-b055-bf3a03a9618e@github.com>
Message-ID: <nu_PvCNQj9_paWKN3-YiUAezspzeVIvvMbgWVMSBWcI=.9f422e35-7ae9-4804-ad1b-33fd80b09882@github.com>

> Reset marking bitmaps after collection cycle; for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with Shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly since in most case bitmap for young gen should have been reset after pervious concurrent cycle finishes if there is no need to preserve bitmap states.
> 
> GenShen:
> Before:
> 
> [33.342s][info][gc,stats    ] Concurrent Reset               =    0.023 s (a =     1921 us) (n =    12) (lvls, us =      133,      385,     1191,     1836,     8878)
> 
> 
> After:
> 
> [33.597s][info][gc,stats    ] Concurrent Reset               =    0.004 s (a =      317 us) (n =    13) (lvls, us =       58,      119,      217,      410,      670)
> [33.597s][info][gc,stats    ] Concurrent Reset After Collect =    0.018 s (a =     1365 us) (n =    13) (lvls, us =       91,      186,      818,     1836,     3872)
> 
> 
> Shenandoah:
> Before:
> 
> [33.144s][info][gc,stats    ] Concurrent Reset               =    0.014 s (a =     1067 us) (n =    13) (lvls, us =      139,      277,      898,     1328,     2118)
> 
> After:
> 
> [33.128s][info][gc,stats    ] Concurrent Reset               =    0.003 s (a =      225 us) (n =    13) (lvls, us =       32,       92,      137,      295,      542)
> [33.128s][info][gc,stats    ] Concurrent Reset After Collect =    0.009 s (a =      661 us) (n =    13) (lvls, us =       92,      160,      594,      896,     1661)
> 
> 
> Additional changes:
> * Remove `ShenandoahResetBitmapClosure` and  `ShenandoahPrepareForMarkClosure`, merge the code with `ShenandoahResetBitmapClosure`, saving one iteration over all the regions. 
> * Use API `ShenandoahGeneration::parallel_heap_region_iterate_free` to iterate the regions, two benefits from this:
>   -  Underneath it calls `ShenandoahHeap::parallel_heap_region_iterate`, which is faster for very light tasks, see https://bugs.openjdk.org/browse/JDK-8337154
>   -  `ShenandoahGeneration::parallel_heap_region_iterate_free` decorate the closure with `ShenandoahExcludeRegionClosure`, which simplifies the code in closure.
> * When `_do_old_gc_bootstrap is true`, instead of reset mark bitmap for old gen separately, simply reset the global generations, so we don't need walk the all regions twice. 
> * Clean up FullGC code, remove duplicate code.
> 
> Additional tests:
> - [x] CONF=macosx-aarch64-server-fastdebug make test T...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 24 additional commits since the last revision:

 - Merge branch 'openjdk:master' into reset-bitmap
 - Merge branch 'openjdk:master' into reset-bitmap
 - Merge branch 'openjdk:master' into reset-bitmap
 - Merge branch 'openjdk:master' into reset-bitmap
 - Adding condition "!_do_old_gc_bootstrap && !heap->is_concurrent_old_mark_in_progress()" back and address some PR comments
 - Remove entry_reset_after_collect from ShenandoahOldGC
 - Remove condition check !_do_old_gc_bootstrap && !heap->is_concurrent_old_mark_in_progress() from op_reset_after_collect
 - Merge branch 'openjdk:master' into reset-bitmap
 - Address review comments
 - Merge branch 'openjdk:master' into reset-bitmap
 - ... and 14 more: https://git.openjdk.org/jdk/compare/a90afca6...c7e9bff3

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22778/files
  - new: https://git.openjdk.org/jdk/pull/22778/files/92c63159..c7e9bff3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22778&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22778&range=06-07

  Stats: 66728 lines in 3845 files changed: 34690 ins; 16712 del; 15326 mod
  Patch: https://git.openjdk.org/jdk/pull/22778.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22778/head:pull/22778

PR: https://git.openjdk.org/jdk/pull/22778

From phh at openjdk.org  Fri Feb 14 10:41:13 2025
From: phh at openjdk.org (Paul Hohensee)
Date: Fri, 14 Feb 2025 10:41:13 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v6]
In-Reply-To: <xrk54IVeSvAYe15JPHpPERMtcZgpoZNMhFiLuD-q6JQ=.353f566a-48c8-42b7-bbb8-79828c8d6553@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
 <xrk54IVeSvAYe15JPHpPERMtcZgpoZNMhFiLuD-q6JQ=.353f566a-48c8-42b7-bbb8-79828c8d6553@github.com>
Message-ID: <CS4STmlOLiY8ImvMi_45kmAHZkiOJ6n81zUui2bwnpc=.ffd0eceb-2bfd-42e8-b8cd-a3d0add96bc9@github.com>

On Fri, 14 Feb 2025 01:35:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
>> 
>> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
>> 
>> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
>> 
>> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:
> 
>  - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
>    
>    Reviewed-by: shade
>  - Merge tag 'jdk-25+10' into eliminate-no-fault-degen-penalties
>    
>    Added tag jdk-25+10 for changeset a637ccf2
>  - Merge tag 'jdk-25+9' into eliminate-no-fault-degen-penalties
>    
>    Added tag jdk-25+9 for changeset 30f71622
>  - Revert "Use generation size to determine expected free"
>    
>    This reverts commit 94a32ebfe5fefcc0e899e09e6fbfc0585c62b4e0.
>  - Respond to reviewer feedback
>  - Use generation size to determine expected free
>  - Respond to reviewer feedback
>  - Fix white space
>  - Remove debug instrumentation
>  - Only penalize heuristic if heuristic responsible
>    
>    If we degenerate through no fault of "late triggering", then do not
>    penalize the heuristic.
>  - ... and 1 more: https://git.openjdk.org/jdk/compare/0a5fcdaf...0d85e341

Marked as reviewed by phh (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23305#pullrequestreview-2617399579

From phh at openjdk.org  Fri Feb 14 10:46:15 2025
From: phh at openjdk.org (Paul Hohensee)
Date: Fri, 14 Feb 2025 10:46:15 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v5]
In-Reply-To: <DhB7IXHkcseWMjgLWRyvSE791XCSZv_QcTPEXWzaX4c=.46b19bb5-7821-4537-8ef5-22102de95027@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <DhB7IXHkcseWMjgLWRyvSE791XCSZv_QcTPEXWzaX4c=.46b19bb5-7821-4537-8ef5-22102de95027@github.com>
Message-ID: <dzgPR3TpouZLKKKoX_uTkgBS_QTemta0kio90YAWRyE=.883a22ab-940a-4cd0-b958-da98ca0ade38@github.com>

On Fri, 14 Feb 2025 01:18:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
>> 
>> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
>> 
>> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
> 
>  - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
>    
>    Reviewed-by: shade
>  - Merge tag 'jdk-25+10' into fix-generational-no-progress-check
>    
>    Added tag jdk-25+10 for changeset a637ccf2
>  - Merge tag 'jdk-25+9' into fix-generational-no-progress-check
>    
>    Added tag jdk-25+9 for changeset 30f71622
>  - Add comments suggested by reviewers
>  - Respond to reviewer feedback
>    
>    In testing suggested refinements, I discovered a bug in original
>    implementation.  ShenandoahFreeSet::capacity() does not represent the
>    size of young generation.  It represents the total size of the young
>    regions that had available memory at the time we most recently rebuilt
>    the ShenandoahFreeSet.
>    
>    I am rerunning the performance tests following this suggested change.
>  - Use freeset to determine goodness of progress
>    
>    As previously implemented, we used the heap size to measure goodness of
>    progress.  However, heap size is only appropriate for non-generational
>    Shenandoah.  Freeset abstraction works for both.
>  - Use size-of young generation to assess progress
>    
>    Previously, we were using size of heap to asses progress of generational
>    degenerated cycle.  But that is not appropriate, because the collection
>    set is chosen based on the size of young generation.

Marked as reviewed by phh (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23306#pullrequestreview-2617414395

From duke at openjdk.org  Fri Feb 14 15:14:15 2025
From: duke at openjdk.org (duke)
Date: Fri, 14 Feb 2025 15:14:15 GMT
Subject: RFR: 8348595: GenShen: Fix generational free-memory no-progress
 check [v5]
In-Reply-To: <DhB7IXHkcseWMjgLWRyvSE791XCSZv_QcTPEXWzaX4c=.46b19bb5-7821-4537-8ef5-22102de95027@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
 <DhB7IXHkcseWMjgLWRyvSE791XCSZv_QcTPEXWzaX4c=.46b19bb5-7821-4537-8ef5-22102de95027@github.com>
Message-ID: <Juf6BOplhb8GCGK8xSMJxsIFVRlSz1p93zcVNCPws9c=.d36e0f3f-1514-4daa-84de-77e2429bfe41@github.com>

On Fri, 14 Feb 2025 01:18:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
>> 
>> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
>> 
>> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
> 
>  - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
>    
>    Reviewed-by: shade
>  - Merge tag 'jdk-25+10' into fix-generational-no-progress-check
>    
>    Added tag jdk-25+10 for changeset a637ccf2
>  - Merge tag 'jdk-25+9' into fix-generational-no-progress-check
>    
>    Added tag jdk-25+9 for changeset 30f71622
>  - Add comments suggested by reviewers
>  - Respond to reviewer feedback
>    
>    In testing suggested refinements, I discovered a bug in original
>    implementation.  ShenandoahFreeSet::capacity() does not represent the
>    size of young generation.  It represents the total size of the young
>    regions that had available memory at the time we most recently rebuilt
>    the ShenandoahFreeSet.
>    
>    I am rerunning the performance tests following this suggested change.
>  - Use freeset to determine goodness of progress
>    
>    As previously implemented, we used the heap size to measure goodness of
>    progress.  However, heap size is only appropriate for non-generational
>    Shenandoah.  Freeset abstraction works for both.
>  - Use size-of young generation to assess progress
>    
>    Previously, we were using size of heap to asses progress of generational
>    degenerated cycle.  But that is not appropriate, because the collection
>    set is chosen based on the size of young generation.

@kdnilsen 
Your change (at version 0e86c5bd1ae330522daa9652f7843342fef9f83e) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23306#issuecomment-2659585115

From duke at openjdk.org  Fri Feb 14 15:16:25 2025
From: duke at openjdk.org (duke)
Date: Fri, 14 Feb 2025 15:16:25 GMT
Subject: RFR: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic [v6]
In-Reply-To: <xrk54IVeSvAYe15JPHpPERMtcZgpoZNMhFiLuD-q6JQ=.353f566a-48c8-42b7-bbb8-79828c8d6553@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
 <xrk54IVeSvAYe15JPHpPERMtcZgpoZNMhFiLuD-q6JQ=.353f566a-48c8-42b7-bbb8-79828c8d6553@github.com>
Message-ID: <L7FQONS2iV55DHpApfFwvYeoVNFHIBoTH7NkLDOtZMM=.e72d5408-5197-43af-80a0-a265c319a380@github.com>

On Fri, 14 Feb 2025 01:35:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
>> 
>> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
>> 
>> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
>> 
>> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:
> 
>  - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
>    
>    Reviewed-by: shade
>  - Merge tag 'jdk-25+10' into eliminate-no-fault-degen-penalties
>    
>    Added tag jdk-25+10 for changeset a637ccf2
>  - Merge tag 'jdk-25+9' into eliminate-no-fault-degen-penalties
>    
>    Added tag jdk-25+9 for changeset 30f71622
>  - Revert "Use generation size to determine expected free"
>    
>    This reverts commit 94a32ebfe5fefcc0e899e09e6fbfc0585c62b4e0.
>  - Respond to reviewer feedback
>  - Use generation size to determine expected free
>  - Respond to reviewer feedback
>  - Fix white space
>  - Remove debug instrumentation
>  - Only penalize heuristic if heuristic responsible
>    
>    If we degenerate through no fault of "late triggering", then do not
>    penalize the heuristic.
>  - ... and 1 more: https://git.openjdk.org/jdk/compare/a1bdb2da...0d85e341

@kdnilsen 
Your change (at version 0d85e34107d74e471a791e0523cabc403e02178c) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23305#issuecomment-2659590224

From kdnilsen at openjdk.org  Fri Feb 14 16:43:17 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 14 Feb 2025 16:43:17 GMT
Subject: Integrated: 8348594: Shenandoah: Do not penalize for degeneration when
 not the fault of triggering heuristic
In-Reply-To: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
References: <I23rSXtDTr6iBN7meafN7idhMkGOuBdh_kxL5EowYjI=.6d9abfad-482c-493f-9725-c66be896e43c@github.com>
Message-ID: <G2M5bSBCv2TXqWWKRVne8qKmPxXLnAqEm41ZM0xqNnM=.8e0713b3-0a28-4386-83a6-01beda903666@github.com>

On Fri, 24 Jan 2025 18:18:25 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Shenandoah heuristics use a penalty mechanism to cause earlier GC triggers when recent concurrent GC cycles degenerate.  Degeneration is a stop-the-world remediation that allows GC to catch up when mutator allocations fail during concurrent GC.  The fact that we needed to degenerate indicates that we were overly optimistic in delaying the trigger that starts concurrent GC.
> 
> We have observed that it is common for degenerated GC cycles to cascade upon each other.  The condition that caused an initial degenerated cycle is often not fully resolved by the end of that degenerated cycle.  For example, the application may be experiencing a phase change and the GC heuristics are not yet attuned to the new behavior.  Furthermore, a degenerated GC may exacerbate the problem condition.  During the stop-the-world pause imposed by the first degenerated GC, work continues to accumulate in the form of new client requests that are buffered in network sockets until the end of that degenerated GC.
> 
> As originally implemented, each degeneration would "pile on" additional penalties.  These penalties cause the GC frequency to continue to increase.  And the expanding CPU load of GC makes it increasingly difficult for mutator threads to catchup.  The large penalties accumulated while we are trying to resolve the problem linger long after the problem condition has been resolved.
> 
> This change does not add further to the degeneration penalties if a new degenerated cycle occurs through no fault of the triggering mechanism.  We only add the degeneration penalty if the reason we are now degenerating can be attributed to a consciously late trigger by the heuristic.

This pull request has now been integrated.

Changeset: 38322407
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/38322407cd1664115e975c7fd9cb61e40d9557b5
Stats:     82 lines in 12 files changed: 78 ins; 0 del; 4 mod

8348594: Shenandoah: Do not penalize for degeneration when not the fault of triggering heuristic

Reviewed-by: phh, wkemper

-------------

PR: https://git.openjdk.org/jdk/pull/23305

From kdnilsen at openjdk.org  Fri Feb 14 16:44:16 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 14 Feb 2025 16:44:16 GMT
Subject: Integrated: 8348595: GenShen: Fix generational free-memory no-progress
 check
In-Reply-To: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
References: <qxVLivQe7QASskJibcjSy-PmAXScg3r7k4Rdomvzws0=.03982de5-8ae8-4a9a-b940-d0990e27ff84@github.com>
Message-ID: <NVwQ83U9MhNdmYup6-OZ7RJKKWEJTvIjByztAVvrEJc=.c2ec0a26-fb8d-4465-88f7-034834156aca@github.com>

On Fri, 24 Jan 2025 18:30:02 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> At the end of a degenerated GC, we check whether sufficient progress has been made in replenishing the memory available to the mutator. The test for good progress is implemented as a ratio of free memory against the total heap size.
> 
> For generational Shenandoah, the ratio should be computed against the size of the young generation. Note that the size of the generational collection set is based on young generation size rather than total heap size.
> 
> This issue first identified in GenShen GC logs, where a large number of degenerated cycles were upgrading to full GC because the free-set progress was short of desired by 10-25%.

This pull request has now been integrated.

Changeset: ba6c9659
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/ba6c96599aac1a6c08cb66c611474f83bbc9b260
Stats:     27 lines in 5 files changed: 21 ins; 0 del; 6 mod

8348595: GenShen: Fix generational free-memory no-progress check

Reviewed-by: phh, xpeng

-------------

PR: https://git.openjdk.org/jdk/pull/23306

From wkemper at openjdk.org  Fri Feb 14 17:43:48 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 14 Feb 2025 17:43:48 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
> 
> ## Testing
> * jtreg hotspot_gc_shenandoah
> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:

 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - Old gen bootstrap cycle must make it to init mark
 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - Improve message for assertion
 - Make shutdown safer for threads requesting (or expecting) gc
 - Do not accept requests if control thread is terminating
 - Notify waiters when control thread terminates
 - Add event for control thread state changes
 - Fix shutdown livelock error
 - Fix includes
 - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda

-------------

Changes: https://git.openjdk.org/jdk/pull/23475/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=11
  Stats: 892 lines in 18 files changed: 285 ins; 281 del; 326 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From kdnilsen at openjdk.org  Fri Feb 14 18:37:54 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 14 Feb 2025 18:37:54 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc [v3]
In-Reply-To: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
Message-ID: <pPxPemtxTsQgawNmFtugoGpH3raIvF2uwWbqoPDFAbk=.fe173bcb-c344-45a1-9b5b-982dadfc5b58@github.com>

> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Fix typo in merge conflict resolution
 - 8348595: GenShen: Fix generational free-memory no-progress check
   
   Reviewed-by: phh, xpeng

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23552/files
  - new: https://git.openjdk.org/jdk/pull/23552/files/8d662e10..0f5051a1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23552&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23552&range=01-02

  Stats: 27 lines in 5 files changed: 21 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/23552.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23552/head:pull/23552

PR: https://git.openjdk.org/jdk/pull/23552

From kdnilsen at openjdk.org  Fri Feb 14 18:51:31 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 14 Feb 2025 18:51:31 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc [v4]
In-Reply-To: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
Message-ID: <Bm_Sov4wuQuzrwvOLx1Gl129IF5sFjxbYlThJWjbD74=.21aef82b-b0dd-4e71-8b36-668b913243dc@github.com>

> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits:

 - Merge master
 - Fix typo in merge conflict resolution
 - 8348595: GenShen: Fix generational free-memory no-progress check
   
   Reviewed-by: phh, xpeng
 - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
   
   Reviewed-by: shade
 - Merge tag 'jdk-25+10' into defer-generational-full-gc
   
   Added tag jdk-25+10 for changeset a637ccf2
 - Be less eager to upgrade degen to full gc

-------------

Changes: https://git.openjdk.org/jdk/pull/23552/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23552&range=03
  Stats: 20 lines in 2 files changed: 17 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/23552.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23552/head:pull/23552

PR: https://git.openjdk.org/jdk/pull/23552

From kdnilsen at openjdk.org  Fri Feb 14 19:41:16 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 14 Feb 2025 19:41:16 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
Message-ID: <k8CL9OJ6zsVTkWIN5RsYfb-p1vEMoF4GxEsSJe7MUfk=.ee841060-bee0-4184-8169-55bc7884f63e@github.com>

On Fri, 14 Feb 2025 17:43:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are several changes to the operation of Shenandoah's control threads here.
>> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>>   *  The cancellation handling is driven entirely by the cancellation cause
>>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>>   * The shutdown sequence is simpler
>> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
>> 
>> ## Testing
>> * jtreg hotspot_gc_shenandoah
>> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:
> 
>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>  - Old gen bootstrap cycle must make it to init mark
>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>  - Improve message for assertion
>  - Make shutdown safer for threads requesting (or expecting) gc
>  - Do not accept requests if control thread is terminating
>  - Notify waiters when control thread terminates
>  - Add event for control thread state changes
>  - Fix shutdown livelock error
>  - Fix includes
>  - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda

Thank you.  This looks very clean to me.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 98:

> 96:   }
> 97: 
> 98:   // In case any threads are waiting for a cycle to happen, let them know it isn't.

maybe "it isn't happening", or "it won't happen".

-------------

Marked as reviewed by kdnilsen (Author).

PR Review: https://git.openjdk.org/jdk/pull/23475#pullrequestreview-2618641262
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956626274

From ysr at openjdk.org  Sat Feb 15 01:55:20 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 15 Feb 2025 01:55:20 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
Message-ID: <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>

On Fri, 14 Feb 2025 17:43:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are several changes to the operation of Shenandoah's control threads here.
>> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>>   *  The cancellation handling is driven entirely by the cancellation cause
>>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>>   * The shutdown sequence is simpler
>> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
>> 
>> ## Testing
>> * jtreg hotspot_gc_shenandoah
>> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:
> 
>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>  - Old gen bootstrap cycle must make it to init mark
>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>  - Improve message for assertion
>  - Make shutdown safer for threads requesting (or expecting) gc
>  - Do not accept requests if control thread is terminating
>  - Notify waiters when control thread terminates
>  - Add event for control thread state changes
>  - Fix shutdown livelock error
>  - Fix includes
>  - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda

Flushing the comments at EOD; will complete review later.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.hpp line 188:

> 186: 
> 187:   bool should_start_gc() override;
> 188:   bool resume_old_cycle();

Documentation comment please, especially explaining the return value.
For things that may return `false` and not do anything, it's better to use `try_` prefix. In fact, the method doesn't actually resume the cycle, but checks if we are in a state such that we should resume it. So, I'd name it `should_resume_old_cycle()`, consistent with the name `should_start_gc()` for the previous method.

src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 101:

> 99:         || cause == GCCause::_shenandoah_allocation_failure_evac
> 100:         || cause == GCCause::_shenandoah_humongous_allocation_failure;
> 101:   }

Would it make sense to move this implementation also to the .cpp file like the other static `is_...` methods below?

src/hotspot/share/gc/shenandoah/shenandoahController.hpp line 42:

> 40:   volatile size_t _allocs_seen;
> 41:   shenandoah_padding(1);
> 42:   volatile size_t _gc_id;

// A monotonically increasing GC count.

src/hotspot/share/gc/shenandoah/shenandoahController.hpp line 66:

> 64: 
> 65:   // This cancels the collection cycle and has an option to block
> 66:   // until another cycle runs and clears the alloc failure gc flag.

But "the alloc failure gc flag" is gone above. The comment should be updated as well. A public API's description should avoid talking about its internal implementation details here. It's OK to talk about implementation details in the implementation of the method, not in the header spec here.

src/hotspot/share/gc/shenandoah/shenandoahController.hpp line 87:

> 85:   // Returns the internal gc count used by the control thread. Probably
> 86:   // doesn't need to be exposed.
> 87:   size_t get_gc_id();

As far as I can tell, there's a single non-internal (public) use of this, and it's from `ShenandoahOldGeneration::handle_failed_promotion()` where it's being used for reducing logging data. If we do need to expose this through a public API, I'd elide the "Probably doesn't need to be exposed", and update the comment to:


// Return the value of a monotonic increasing GC count, maintained by the control thread.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 64:

> 62: void ShenandoahGenerationalControlThread::run_service() {
> 63: 
> 64:   const int64_t wait_ms = ShenandoahPacing ? ShenandoahControlIntervalMin : 0;

So we are supporting ShenandoahPacing with GenShenm(at least till we pull it in the future), but don't want to enable it by default, correct?

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 64:

> 62: private:
> 63:   // This lock is used to coordinate setting the _requested_gc_cause, _requested generation
> 64:   // and _gc_mode. It is important that these be changed together and have a consistent view.

In that case, for ease of maintenance, I'd move the declaration of all of the 3 data members that this lock protects next to this lock, either immediately preceding or immediately succeeding its declaration in the body of this class.

Are these data members always both read and written under this lock? If so, then `_gc_mode` below doesn't need to be defined `volatile`.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 88:

> 86:   uint _age_period;
> 87: 
> 88:   // The mode is read frequently by requesting threads and only ever written by the control thread.

Do requesting threads lock the mutex when reading? I am trying to square yr comment that it's protected by the mutex with the field being declared `volatile`.

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 450:

> 448: 
> 449:   void cancel_concurrent_mark();
> 450:   bool cancel_gc(GCCause::Cause cause);

// Returns true if and only if cancellation request was successfully communicated.

-------------

PR Review: https://git.openjdk.org/jdk/pull/23475#pullrequestreview-2618968208
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956962731
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956965579
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956944585
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956918529
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956929734
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956981955
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956816268
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956824150
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956952381

From ysr at openjdk.org  Sat Feb 15 01:55:21 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 15 Feb 2025 01:55:21 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
Message-ID: <LmXhohuv__2HFhtSt89EWx63pHiPn1nSdI2OFpzjV38=.30fcd0b9-4c0b-4849-97d6-06a79065c9cf@github.com>

On Sat, 15 Feb 2025 01:10:51 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:
>> 
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - Fix shutdown livelock error
>>  - Fix includes
>>  - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda
>
> src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 101:
> 
>> 99:         || cause == GCCause::_shenandoah_allocation_failure_evac
>> 100:         || cause == GCCause::_shenandoah_humongous_allocation_failure;
>> 101:   }
> 
> Would it make sense to move this implementation also to the .cpp file like the other static `is_...` methods below?

Or is this guaranteeing inlining into the caller's body, which you might prefer for the callers?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956968182

From ysr at openjdk.org  Sat Feb 15 01:55:22 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 15 Feb 2025 01:55:22 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <k8CL9OJ6zsVTkWIN5RsYfb-p1vEMoF4GxEsSJe7MUfk=.ee841060-bee0-4184-8169-55bc7884f63e@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <k8CL9OJ6zsVTkWIN5RsYfb-p1vEMoF4GxEsSJe7MUfk=.ee841060-bee0-4184-8169-55bc7884f63e@github.com>
Message-ID: <xD19CslZfDxxY-cEpYzt4rQJQfNfXtQZiD2ZUHryfvI=.38b1bb1e-98ed-4ba2-a44f-53ccad92f532@github.com>

On Fri, 14 Feb 2025 19:28:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:
>> 
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - Fix shutdown livelock error
>>  - Fix includes
>>  - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 98:
> 
>> 96:   }
>> 97: 
>> 98:   // In case any threads are waiting for a cycle to happen, let them know it isn't.
> 
> maybe "it isn't happening", or "it won't happen".

This is interesting. If GC is stopping prior to shutting down the VM, is there any point in notifying these waiting threads. Why not let them wait, and quietly shut things down? Are there JCK or other tests that would fail in that case?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1956979069

From ssubramaniam at openjdk.org  Tue Feb 18 19:27:02 2025
From: ssubramaniam at openjdk.org (Satyen Subramaniam)
Date: Tue, 18 Feb 2025 19:27:02 GMT
Subject: Withdrawn: 8348610: GenShen: TestShenandoahEvacuationInformationEvent
 failed with setRegions >= regionsFreed: expected 1 >= 57
In-Reply-To: <xREh-BpJoPVPebBRIs8aEsAm8qmI1mcqJjaN8dw2u2Y=.bcf359a2-3148-4d4a-a7a1-0842448efe44@github.com>
References: <xREh-BpJoPVPebBRIs8aEsAm8qmI1mcqJjaN8dw2u2Y=.bcf359a2-3148-4d4a-a7a1-0842448efe44@github.com>
Message-ID: <I4Cpzauo7L75nDDoYYfuF2KPwVJis10PzMtwgUGkEvk=.36dd0f33-8623-47d6-b7b2-ebb3a18178ad@github.com>

On Thu, 30 Jan 2025 00:07:28 GMT, Satyen Subramaniam <ssubramaniam at openjdk.org> wrote:

> Renaming `ShenandoahEvacuationInformation.freedRegions` to `ShenandoahEvacuationInformation.freeRegions` for clarity, and fixing incorrect assertion in  TestShenandoahEvacuationInformationEvent.cpp
> 
> Tested with tier 1, tier 2, and tier 3 tests.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/558

From kdnilsen at openjdk.org  Tue Feb 18 19:28:28 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Feb 2025 19:28:28 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc [v5]
In-Reply-To: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
Message-ID: <SwSImg7B1mdgxMu6tFrrYBYS6194MSObe_4DmZhx4pU=.1139c830-28fe-4e1b-9811-d228f9b24345@github.com>

> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:

 - Merge branch 'master' of https://git.openjdk.org/jdk into defer-generational-full-gc
 - Merge master
 - Fix typo in merge conflict resolution
 - 8348595: GenShen: Fix generational free-memory no-progress check
   
   Reviewed-by: phh, xpeng
 - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
   
   Reviewed-by: shade
 - Merge tag 'jdk-25+10' into defer-generational-full-gc
   
   Added tag jdk-25+10 for changeset a637ccf2
 - Be less eager to upgrade degen to full gc

-------------

Changes: https://git.openjdk.org/jdk/pull/23552/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23552&range=04
  Stats: 20 lines in 2 files changed: 17 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/23552.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23552/head:pull/23552

PR: https://git.openjdk.org/jdk/pull/23552

From kdnilsen at openjdk.org  Tue Feb 18 19:49:54 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Feb 2025 19:49:54 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and small
 Java heap
Message-ID: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>

Original implementation was not robust to overriding of CardSizeInBytes, especially to smaller values.  This fixes that issue.

-------------

Commit messages:
 - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
 - Merge tag 'jdk-25+10' into fix-small-card-size
 - Remove SIZE_FORMAT usage
 - Merge tag 'jdk-25+9' into fix-small-card-size
 - Remove debug instrumentation
 - Fix several bookkeeping errors
 - Revert "Remove debug instrumentation"
 - Remove debug instrumentation
 - Use snprintf instead of sprintf
 - Add a jtreg test for small card size
 - ... and 1 more: https://git.openjdk.org/jdk/compare/a637ccf2...7120cdf3

Changes: https://git.openjdk.org/jdk/pull/23373/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23373&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8347804
  Stats: 107 lines in 6 files changed: 79 ins; 4 del; 24 mod
  Patch: https://git.openjdk.org/jdk/pull/23373.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23373/head:pull/23373

PR: https://git.openjdk.org/jdk/pull/23373

From kdnilsen at openjdk.org  Tue Feb 18 19:49:54 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Feb 2025 19:49:54 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap
In-Reply-To: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
Message-ID: <wk63gxbbjcUgcP2W7ycip8Zc6h2ZqwmOAql6YhpuhuI=.67428bf9-7b60-427e-add7-3eafdf8d63bd@github.com>

On Thu, 30 Jan 2025 18:55:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Original implementation was not robust to overriding of CardSizeInBytes, especially to smaller values.  This fixes that issue.

Internal pipelines reveal a regression.  Changing to draft mode while I chase this down.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23373#issuecomment-2625332964

From duke at openjdk.org  Wed Feb 19 06:21:33 2025
From: duke at openjdk.org (sli-x)
Date: Wed, 19 Feb 2025 06:21:33 GMT
Subject: RFR: 8340434: Excessive Young GCs Triggered by CodeCache GC Threshold
Message-ID: <g22FgTJTATrz5UIKsSy3a98OgJIm8hvZ9KFm4u75PyY=.dea7d218-c0d7-43a9-afff-d167383eeb63@github.com>

The trigger of _codecache_GC_threshold in CodeCache::gc_on_allocation is the key to this problem.

if (used_ratio > threshold) {
    // After threshold is reached, scale it by free_ratio so that more aggressive
    // GC is triggered as we approach code cache exhaustion
    threshold *= free_ratio;
  }
  // If code cache has been allocated without any GC at all, let's make sure
  // it is eventually invoked to avoid trouble.
  if (allocated_since_last_ratio > threshold) {
    // In case the GC is concurrent, we make sure only one thread requests the GC.
    if (Atomic::cmpxchg(&_unloading_threshold_gc_requested, false, true) == false) {
      log_info(codecache)("Triggering threshold (%.3f%%) GC due to allocating %.3f%% since last unloading (%.3f%% used -> %.3f%% used)",
                          threshold * 100.0, allocated_since_last_ratio * 100.0, last_used_ratio * 100.0, used_ratio * 100.0);
      Universe::heap()->collect(GCCause::_codecache_GC_threshold);
    }
  }

Here with the limited codecache size, the free_ratio will get lower and lower (so as the threshold) if no methods can be swept and thus leads to a more and more frequent collection behavior. Since the collection happens in stw, the whole performance of gc will also be degraded. 

So a simple solution is to delete the scaling logic here. However, I think here lies some problems worth further exploring.

There're two options to control a code cache sweeper,  StartAggressiveSweepingAt and SweeperThreshold.  StartAggressiveSweepingAt is a sweeper triggered for little space in codeCache and does little harm. However,  SweeperThreshold, first introduced by [JDK-8244660](https://bugs.openjdk.org/browse/JDK-8244660), was designed for a regular sweep for codecache, when codeCache sweeper and heap collection are actually individual. After [JDK-8290025](https://bugs.openjdk.org/browse/JDK-8290025) and some patches related, the old mechanism of codeCache sweeper is merged into a concurrent heap collection. So the Code cache sweeper heuristics and the unloading behavior will be promised by the concurrent collection. There's no longer any "zombie" methods to be counted. Considering it will introduce lots of useless collection jobs, I think SweeperThreshold should be deleted now.

-------------

Commit messages:
 - remove SweeperThreshold and set it to Obselete

Changes: https://git.openjdk.org/jdk/pull/21084/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21084&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8340434
  Stats: 55 lines in 14 files changed: 1 ins; 53 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21084.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21084/head:pull/21084

PR: https://git.openjdk.org/jdk/pull/21084

From tschatzl at openjdk.org  Wed Feb 19 06:21:33 2025
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Wed, 19 Feb 2025 06:21:33 GMT
Subject: RFR: 8340434: Excessive Young GCs Triggered by CodeCache GC
 Threshold
In-Reply-To: <g22FgTJTATrz5UIKsSy3a98OgJIm8hvZ9KFm4u75PyY=.dea7d218-c0d7-43a9-afff-d167383eeb63@github.com>
References: <g22FgTJTATrz5UIKsSy3a98OgJIm8hvZ9KFm4u75PyY=.dea7d218-c0d7-43a9-afff-d167383eeb63@github.com>
Message-ID: <VhKB5YMijnYFr4aLB6gLnxWH6uVivzdVrP2ydOMLOZo=.2cff75e3-bd13-441a-8c3c-199e66c24e6e@github.com>

On Thu, 19 Sep 2024 08:43:50 GMT, sli-x <duke at openjdk.org> wrote:

> Here with the limited codecache size, the free_ratio will get lower and lower (so as the threshold) if no methods can be swept and thus leads to a more and more frequent collection behavior. Since the collection happens in stw, the whole performance of gc will also be degraded.
>
>So a simple solution is to delete the scaling logic here. However, I think here lies some problems worth further exploring.
>
>There're two options to control a code cache sweeper, StartAggressiveSweepingAt and SweeperThreshold. StartAggressiveSweepingAt is a sweeper triggered for little space in codeCache and does little harm. However, SweeperThreshold, first introduced by [JDK-8244660](https://bugs.openjdk.org/browse/JDK-8244660), was designed for a regular sweep for codecache, when codeCache sweeper and heap collection are actually individual. After [JDK-8290025](https://bugs.openjdk.org/browse/JDK-8290025) and some patches related, the old mechanism of codeCache sweeper is merged into a concurrent heap collection. So the Code cache sweeper heuristics and the unloading behavior will be promised by the concurrent collection. There's no longer any "zombie" methods to be counted. Considering it will introduce lots of useless collection jobs, I think SweeperThreshold should be deleted now.

I think the general concern presented out by the code

>    // After threshold is reached, scale it by free_ratio so that more aggressive
>    // GC is triggered as we approach code cache exhaustion

is still valid.

How this is implemented also makes somewhat sense: changes are the trigger for collections, allow larger changes before trying to clean out the code cache the emptier the code cache is.

It tries to limit code cache memory usage by increasingly doing more frequent collections the more occupied the code cache becomes, i.e. some kind of backpressure on code cache usage.

Your use case of limiting the code cache size (and setting initial == max) seems to be relatively unusual one to me, and apparently does not fit that model as it seems that you set code cache size close to actual max usage.

Removing `SweepingThreshold` would affect the regular case as well in a significant way (allocate until bumping into the `StartAggressiveSweepingAt`) I do not think removing this part of the heuristic isn't good (or desired at all).

Maybe an alternative could be only not doing this heuristic part in your case; and even then  am not sure that waiting until hitting the `StartAggressiveSweepingAt` threshold is a good idea; it may be too late to avoid disabling the compiler at least temporarily. 
And even then, as long as the memory usage keeps being larger larger than the threshold, this will result in continuous code cache sweeps (_every time_ _any_ memory is allocated in the code cache).

>From the  [JDK-8244660](https://bugs.openjdk.org/browse/JDK-8244660) CR:

> This is because users with different sized code caches might want different thresholds. (Otherwise there would be no way to control the sweepers intensity). 

Which means that one could just take that suggestion literally and not only change the initial/max code cache size but also that threshold in your use case.

Stepping back a little, this situation very much resembles issues with G1's `InitiatingHeapOccupancyPercent` pre [JDK-8136677](https://bugs.openjdk.org/browse/JDK-8136677) where a one-size-fits-all value also did not work, and many many people tuned `InitiatingHeapOccupancyPercent` manually in the past.

Maybe a  similar mechanism at least taking actual code cache allocation rate into account ("when will the current watermark will be hit"?) would be preferable to replace both options (note that since I'm not an expert in code cache, there may be other reasons to clean out the code cache than just occupancy threshold)?

Thomas

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21084#issuecomment-2383475220

From robilad at openjdk.org  Wed Feb 19 06:21:33 2025
From: robilad at openjdk.org (Dalibor Topic)
Date: Wed, 19 Feb 2025 06:21:33 GMT
Subject: RFR: 8340434: Excessive Young GCs Triggered by CodeCache GC
 Threshold
In-Reply-To: <g22FgTJTATrz5UIKsSy3a98OgJIm8hvZ9KFm4u75PyY=.dea7d218-c0d7-43a9-afff-d167383eeb63@github.com>
References: <g22FgTJTATrz5UIKsSy3a98OgJIm8hvZ9KFm4u75PyY=.dea7d218-c0d7-43a9-afff-d167383eeb63@github.com>
Message-ID: <OAjOmnQiWi734OrtK9vs9zyx1VJw5xQMDatdylli1ok=.65168c50-9652-4695-b8ae-9672fce40d39@github.com>

On Thu, 19 Sep 2024 08:43:50 GMT, sli-x <duke at openjdk.org> wrote:

> The trigger of _codecache_GC_threshold in CodeCache::gc_on_allocation is the key to this problem.
> 
> if (used_ratio > threshold) {
>     // After threshold is reached, scale it by free_ratio so that more aggressive
>     // GC is triggered as we approach code cache exhaustion
>     threshold *= free_ratio;
>   }
>   // If code cache has been allocated without any GC at all, let's make sure
>   // it is eventually invoked to avoid trouble.
>   if (allocated_since_last_ratio > threshold) {
>     // In case the GC is concurrent, we make sure only one thread requests the GC.
>     if (Atomic::cmpxchg(&_unloading_threshold_gc_requested, false, true) == false) {
>       log_info(codecache)("Triggering threshold (%.3f%%) GC due to allocating %.3f%% since last unloading (%.3f%% used -> %.3f%% used)",
>                           threshold * 100.0, allocated_since_last_ratio * 100.0, last_used_ratio * 100.0, used_ratio * 100.0);
>       Universe::heap()->collect(GCCause::_codecache_GC_threshold);
>     }
>   }
> 
> Here with the limited codecache size, the free_ratio will get lower and lower (so as the threshold) if no methods can be swept and thus leads to a more and more frequent collection behavior. Since the collection happens in stw, the whole performance of gc will also be degraded. 
> 
> So a simple solution is to delete the scaling logic here. However, I think here lies some problems worth further exploring.
> 
> There're two options to control a code cache sweeper,  StartAggressiveSweepingAt and SweeperThreshold.  StartAggressiveSweepingAt is a sweeper triggered for little space in codeCache and does little harm. However,  SweeperThreshold, first introduced by [JDK-8244660](https://bugs.openjdk.org/browse/JDK-8244660), was designed for a regular sweep for codecache, when codeCache sweeper and heap collection are actually individual. After [JDK-8290025](https://bugs.openjdk.org/browse/JDK-8290025) and some patches related, the old mechanism of codeCache sweeper is merged into a concurrent heap collection. So the Code cache sweeper heuristics and the unloading behavior will be promised by the concurrent collection. There's no longer any "zombie" methods to be counted. Considering it will introduce lots of useless collection jobs, I think SweeperThreshold should be deleted now.

Hi, 

please send an e-mail to dalibor.topic at oracle.com so that I can verify your account in Skara.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21084#issuecomment-2427338142

From ayang at openjdk.org  Wed Feb 19 14:18:05 2025
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Wed, 19 Feb 2025 14:18:05 GMT
Subject: RFR: 8348171: Refactor GenerationCounters and its subclasses [v5]
In-Reply-To: <7otkT63ENoyKzZ29CbYpycLLwL89ARajYg36Mstz4tQ=.fd3c7dcf-5a8b-44be-9205-09e3d160d54d@github.com>
References: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
 <7otkT63ENoyKzZ29CbYpycLLwL89ARajYg36Mstz4tQ=.fd3c7dcf-5a8b-44be-9205-09e3d160d54d@github.com>
Message-ID: <bLt4IDuqtzGf8WQd9mP8Lwx6ezjphPUc9H9jI52ttOQ=.aad6139e-e742-480a-ae62-eb0362150c40@github.com>

On Tue, 11 Feb 2025 15:28:25 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Simple refactoring of removing the use of `virtual` method and use concrete subclasses when needed.
>> 
>> Test: tier1-5
>
> Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:
> 
>  - Merge branch 'master' into gen-counter
>  - review
>  - * some more refactoring
>  - review
>  - Merge branch 'master' into gen-counter
>  - merge
>  - gen-counter

Thanks for review.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23209#issuecomment-2668780554

From ayang at openjdk.org  Wed Feb 19 14:18:05 2025
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Wed, 19 Feb 2025 14:18:05 GMT
Subject: Integrated: 8348171: Refactor GenerationCounters and its subclasses
In-Reply-To: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
References: <F_cpBcdlWdfdzK6GeuuauMLuy2aB-QD42k_Opy4XBl8=.5266c44c-e98b-462f-ac85-29ecfb22db1e@github.com>
Message-ID: <jrY5_Yx_kf9cYAfLtrAJRZA5V7CTZ87jzAwyhsFV2sU=.7d0469da-de87-4d1d-ba66-91e48f34269d@github.com>

On Tue, 21 Jan 2025 09:53:07 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

> Simple refactoring of removing the use of `virtual` method and use concrete subclasses when needed.
> 
> Test: tier1-5

This pull request has now been integrated.

Changeset: c6e47fd5
Author:    Albert Mingkun Yang <ayang at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/c6e47fd5812997e3428249be1c77c60e7b05a5df
Stats:     202 lines in 17 files changed: 6 ins; 160 del; 36 mod

8348171: Refactor GenerationCounters and its subclasses

Co-authored-by: Thomas Schatzl <tschatzl at openjdk.org>
Reviewed-by: gli, tschatzl, zgu

-------------

PR: https://git.openjdk.org/jdk/pull/23209

From andrew at openjdk.org  Wed Feb 19 15:49:28 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Wed, 19 Feb 2025 15:49:28 GMT
Subject: RFR: Merge jdk8u:master
Message-ID: <0-KY5icPKpiaB8p261b901PBCh1qzjGBN4xQFGDJPKU=.6f22bf19-fcc8-4aff-92d8-f21498a7d2bd@github.com>

Merge jdk8u332-b07

-------------

Commit messages:
 - Merge jdk8u332-b07
 - 8284548: Invalid XPath expression causes StringIndexOutOfBoundsException
 - 8281388: Change wrapping of EncryptedPrivateKeyInfo
 - 8282300: Throws NamingException instead of InvalidNameException after JDK-8278972
 - 8278972: Improve URL supports
 - 8278805: Enhance BMP image loading
 - 8278449: Improve keychain support
 - 8282397: createTempFile method of java.io.File is failing when called with suffix of spaces character
 - 8278356: Improve file creation
 - 8278008: Improve Santuario processing
 - ... and 10 more: https://git.openjdk.org/shenandoah-jdk8u/compare/5ca34513...0c935e9c

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk8u/pull/12/files
  Stats: 3516 lines in 62 files changed: 2391 ins; 729 del; 396 mod
  Patch: https://git.openjdk.org/shenandoah-jdk8u/pull/12.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk8u.git pull/12/head:pull/12

PR: https://git.openjdk.org/shenandoah-jdk8u/pull/12

From andrew at openjdk.org  Wed Feb 19 15:49:28 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Wed, 19 Feb 2025 15:49:28 GMT
Subject: RFR: Merge jdk8u:master
In-Reply-To: <0-KY5icPKpiaB8p261b901PBCh1qzjGBN4xQFGDJPKU=.6f22bf19-fcc8-4aff-92d8-f21498a7d2bd@github.com>
References: <0-KY5icPKpiaB8p261b901PBCh1qzjGBN4xQFGDJPKU=.6f22bf19-fcc8-4aff-92d8-f21498a7d2bd@github.com>
Message-ID: <HNcrAUeEENYgiZyioC65fvphVrIlkRmKjneYyJTRxLA=.908ff933-1e2d-4d5e-8dda-6147e24adc7e@github.com>

On Wed, 19 Feb 2025 15:44:23 GMT, Andrew John Hughes <andrew at openjdk.org> wrote:

> Merge jdk8u332-b07

GHA builds will not work until [JDK-8284622](https://bugs.openjdk.org/browse/JDK-8284622) is merged in 8u362-b03

-------------

PR Comment: https://git.openjdk.org/shenandoah-jdk8u/pull/12#issuecomment-2669042924

From andrew at openjdk.org  Wed Feb 19 15:55:47 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Wed, 19 Feb 2025 15:55:47 GMT
Subject: RFR: Merge jdk8u:master [v2]
In-Reply-To: <0-KY5icPKpiaB8p261b901PBCh1qzjGBN4xQFGDJPKU=.6f22bf19-fcc8-4aff-92d8-f21498a7d2bd@github.com>
References: <0-KY5icPKpiaB8p261b901PBCh1qzjGBN4xQFGDJPKU=.6f22bf19-fcc8-4aff-92d8-f21498a7d2bd@github.com>
Message-ID: <pt8ImsndnAQaxsQtUFUXpVQTe5AMYvAoXLK4RUoqIO4=.75277baf-8044-4072-bafa-5e96ad3920f8@github.com>

> Merge jdk8u332-b07

Andrew John Hughes has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk8u/pull/12/files
  - new: https://git.openjdk.org/shenandoah-jdk8u/pull/12/files/0c935e9c..0c935e9c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk8u&pr=12&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk8u&pr=12&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk8u/pull/12.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk8u.git pull/12/head:pull/12

PR: https://git.openjdk.org/shenandoah-jdk8u/pull/12

From andrew at openjdk.org  Wed Feb 19 15:55:40 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Wed, 19 Feb 2025 15:55:40 GMT
Subject: git: openjdk/shenandoah-jdk8u: master: 20 new changesets
Message-ID: <73df6b88-1a42-4cab-a12e-3f5c42089dad@openjdk.org>

Changeset: 703cd7c3
Branch: master
Author:    Andrew John Hughes <andrew at openjdk.org>
Date:      2022-03-29 03:33:24 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/703cd7c3247f4122b2f316838a99adbc28454015

Added tag jdk8u332-b06 for changeset 6d5c4e11830c

! .hgtags

Changeset: c957789e
Branch: master
Author:    Martin Balao <mbalao at openjdk.org>
Date:      2022-04-15 01:35:49 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/c957789e68889bc05aeece367779cc427bf4780f

8269938: Enhance XML processing passes redux

Reviewed-by: andrew

! jaxp/src/com/sun/org/apache/xerces/internal/parsers/AbstractDOMParser.java
! jaxp/src/com/sun/org/apache/xml/internal/serializer/ToHTMLStream.java
! jaxp/src/com/sun/org/apache/xml/internal/serializer/ToStream.java
! jaxp/src/com/sun/xml/internal/stream/events/EntityDeclarationImpl.java
! jaxp/src/com/sun/xml/internal/stream/events/NotationDeclarationImpl.java
! jaxp/src/jdk/xml/internal/JdkXmlUtils.java

Changeset: 22ae2b4e
Branch: master
Author:    Yuri Nesterenko <yan at openjdk.org>
Date:      2022-04-15 01:58:03 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/22ae2b4eaf411bcedb292a851fb184a01ceabb9d

8270504: Better Xpath expression handling

Reviewed-by: andrew

! jaxp/src/com/sun/java_cup/internal/runtime/lr_parser.java
! jaxp/src/com/sun/org/apache/xalan/internal/XalanConstants.java
- jaxp/src/com/sun/org/apache/xalan/internal/utils/XMLSecurityManager.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/compiler/Parser.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/compiler/XPathParser.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/compiler/XSLTC.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/compiler/sym.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/compiler/util/ErrorMessages.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/compiler/util/ErrorMsg.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/trax/TransformerFactoryImpl.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/trax/TransformerImpl.java
! jaxp/src/com/sun/org/apache/xalan/internal/xsltc/trax/Util.java
! jaxp/src/com/sun/org/apache/xml/internal/utils/XMLReaderManager.java
! jaxp/src/com/sun/org/apache/xpath/internal/XPath.java
! jaxp/src/com/sun/org/apache/xpath/internal/compiler/Lexer.java
+ jaxp/src/com/sun/org/apache/xpath/internal/compiler/Token.java
! jaxp/src/com/sun/org/apache/xpath/internal/compiler/XPathParser.java
! jaxp/src/com/sun/org/apache/xpath/internal/jaxp/XPathFactoryImpl.java
! jaxp/src/com/sun/org/apache/xpath/internal/jaxp/XPathImpl.java
! jaxp/src/com/sun/org/apache/xpath/internal/res/XPATHErrorResources.java
! jaxp/src/jdk/xml/internal/JdkXmlUtils.java
+ jaxp/src/jdk/xml/internal/XMLLimitAnalyzer.java
+ jaxp/src/jdk/xml/internal/XMLSecurityManager.java

Changeset: d5e4c821
Branch: master
Author:    David Alvarez <alvdavi at openjdk.org>
Date:      2022-03-30 22:50:06 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/d5e4c8211b2cc7c5d5e02f8d5f51b4d048c40836

8272255: Completely handle MIDI files

Reviewed-by: andrew

! jdk/src/share/classes/com/sun/media/sound/AudioFileSoundbankReader.java

Changeset: 0417c4b7
Branch: master
Author:    Oli Gillespie <ogillesp at amazon.com>
Committer: David Alvarez <alvdavi at openjdk.org>
Date:      2022-02-04 18:41:32 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/0417c4b79bb90197b92404b93900f65ccab4d453

8272261: Improve JFR recording file processing

Reviewed-by: andrew

! jdk/src/share/classes/jdk/jfr/internal/tool/JSONWriter.java
! jdk/src/share/classes/jdk/jfr/internal/tool/XMLWriter.java

Changeset: d4a2cb2f
Branch: master
Author:    Martin Balao <mbalao at openjdk.org>
Date:      2022-04-15 02:17:55 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/d4a2cb2fc593a23e8b55242c84210e217b9eefa7

8272594: Better record of recordings

Reviewed-by: andrew

! jdk/src/share/classes/jdk/jfr/consumer/ParserFactory.java
! jdk/src/share/classes/jdk/jfr/internal/MetadataReader.java
! jdk/src/share/classes/jdk/jfr/internal/consumer/RecordingInput.java

Changeset: b44052f1
Branch: master
Author:    David Alvarez <alvdavi at openjdk.org>
Date:      2022-04-15 02:24:16 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/b44052f15b5939c30ee910fd6f8d4c1ddef44a6b

8274221: More definite BER encodings

Reviewed-by: andrew

! jdk/src/share/classes/sun/security/util/DerIndefLenConverter.java

Changeset: 09520f3c
Branch: master
Author:    Aleksei Voitylov <avoitylov at openjdk.org>
Date:      2022-02-18 00:38:29 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/09520f3c48e3656f1b225cda0d4057e3183daa3a

8275151: Improved Object Identification

Reviewed-by: andrew

! jdk/src/share/classes/sun/security/util/ObjectIdentifier.java

Changeset: 399ad9ed
Branch: master
Author:    Aleksei Voitylov <avoitylov at openjdk.org>
Date:      2022-02-18 00:39:20 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/399ad9ed3f097966da84e02415371ed22406e906

8277227: Better identification of OIDs

Reviewed-by: andrew

! jdk/src/share/classes/sun/security/util/ObjectIdentifier.java

Changeset: 89d03b67
Branch: master
Author:    Aleksei Voitylov <avoitylov at openjdk.org>
Date:      2022-02-18 00:41:43 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/89d03b676ae0213239ecd46092d1d9c06d34fc99

8277672: Better invocation handler handling

Reviewed-by: andrew

! jdk/src/share/classes/sun/reflect/annotation/AnnotationInvocationHandler.java

Changeset: c3f39d06
Branch: master
Author:    Ilarion Nakonechnyy <inakonechnyy at openjdk.org>
Date:      2022-03-29 19:21:47 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/c3f39d06604af76645d3503e829a887cf932cceb

8278008: Improve Santuario processing

Backport SANTUARIO-572 & SANTUARIO-566

Reviewed-by: andrew

! jdk/src/share/classes/com/sun/org/apache/xml/internal/security/keys/keyresolver/implementations/KeyInfoReferenceResolver.java
! jdk/src/share/classes/com/sun/org/apache/xml/internal/security/resource/xmlsecurity_en.properties
! jdk/src/share/classes/com/sun/org/apache/xml/internal/security/signature/XMLSignatureInput.java
! jdk/src/share/classes/com/sun/org/apache/xml/internal/security/transforms/implementations/TransformXPath.java
! jdk/src/share/classes/com/sun/org/apache/xml/internal/security/utils/resolver/implementations/ResolverLocalFilesystem.java
! jdk/src/share/classes/org/jcp/xml/dsig/internal/dom/DOMURIDereferencer.java

Changeset: d72d2896
Branch: master
Author:    Martin Balao <mbalao at openjdk.org>
Date:      2022-04-15 02:53:01 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/d72d28967d732ba32e02178b828255378c5a8938

8278356: Improve file creation

Reviewed-by: andrew

! jdk/src/share/classes/java/io/File.java
! jdk/src/share/classes/java/io/FileSystem.java
! jdk/src/solaris/classes/java/io/UnixFileSystem.java
! jdk/src/windows/classes/java/io/WinNTFileSystem.java

Changeset: c09086e8
Branch: master
Author:    Martin Balao <mbalao at openjdk.org>
Date:      2022-04-15 02:55:32 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/c09086e86b77f221129c91f656044f2bacc1a8b3

8282397: createTempFile method of java.io.File is failing when called with suffix of spaces character

Reviewed-by: andrew

! jdk/src/windows/classes/java/io/WinNTFileSystem.java

Changeset: 43ae78b5
Branch: master
Author:    Alexey Bakhtin <abakhtin at openjdk.org>
Date:      2022-04-15 03:25:16 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/43ae78b5a0d4588f83bd7ed372e7845d33f3987c

8278449: Improve keychain support

Reviewed-by: andrew

! jdk/src/macosx/classes/apple/security/KeychainStore.java
! jdk/src/macosx/native/apple/security/KeystoreImpl.m
! jdk/src/share/classes/sun/security/tools/keytool/Main.java
! jdk/test/lib/testlibrary/jdk/testlibrary/SecurityTools.java

Changeset: 3ee4e621
Branch: master
Author:    Andrew Brygin <bae at openjdk.org>
Date:      2022-04-15 03:37:34 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/3ee4e621de17f5078b4ff4730840a14fb1cd2f3b

8278805: Enhance BMP image loading

Reviewed-by: andrew

! jdk/src/share/classes/com/sun/imageio/plugins/bmp/BMPImageReader.java
! jdk/src/share/classes/com/sun/imageio/plugins/common/ReaderUtil.java

Changeset: 718c8a13
Branch: master
Author:    Yuri Nesterenko <yan at openjdk.org>
Date:      2022-03-11 10:38:18 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/718c8a13ea37d9b04610f7f437493d0d5899ae88

8278972: Improve URL supports

Reviewed-by: andrew

! jdk/src/share/classes/com/sun/jndi/dns/DnsUrl.java
! jdk/src/share/classes/com/sun/jndi/ldap/LdapURL.java
! jdk/src/share/classes/com/sun/jndi/toolkit/url/GenericURLContext.java
! jdk/src/share/classes/com/sun/jndi/toolkit/url/Uri.java
! jdk/src/share/classes/com/sun/jndi/url/rmi/rmiURLContext.java

Changeset: 28bdf44d
Branch: master
Author:    Yuri Nesterenko <yan at openjdk.org>
Date:      2022-04-15 03:59:28 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/28bdf44d25c4b759814f9228250f6eddf648fb6c

8282300: Throws NamingException instead of InvalidNameException after JDK-8278972

Reviewed-by: andrew

! jdk/src/share/classes/com/sun/jndi/url/rmi/rmiURLContext.java

Changeset: 1feff780
Branch: master
Author:    Martin Balao <mbalao at openjdk.org>
Date:      2022-04-15 04:04:13 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/1feff780bc8f86be3440b0fef678575914364e49

8281388: Change wrapping of EncryptedPrivateKeyInfo

Reviewed-by: andrew

! jdk/src/share/classes/javax/crypto/EncryptedPrivateKeyInfo.java
! jdk/src/share/classes/sun/security/util/DerValue.java

Changeset: 6a6f2c2f
Branch: master
Author:    Christoph Langer <clanger at openjdk.org>
Date:      2022-04-15 04:19:06 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/6a6f2c2fe2ee99d1ec7918fe5fad2f5c0d61d599

8284548: Invalid XPath expression causes StringIndexOutOfBoundsException

Reviewed-by: andrew

! jaxp/src/com/sun/org/apache/xpath/internal/compiler/Lexer.java
+ jdk/test/javax/xml/jaxp/XPath/8284548/InvalidXPath.java

Changeset: 0c935e9c
Branch: master
Author:    Andrew John Hughes <andrew at openjdk.org>
Date:      2025-02-13 22:23:28 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/0c935e9cf8da1ad98fb3fef68bd0a99cd9bf22de

Merge jdk8u332-b07



From iris at openjdk.org  Wed Feb 19 15:55:48 2025
From: iris at openjdk.org (Iris Clark)
Date: Wed, 19 Feb 2025 15:55:48 GMT
Subject: Withdrawn: Merge jdk8u:master
In-Reply-To: <0-KY5icPKpiaB8p261b901PBCh1qzjGBN4xQFGDJPKU=.6f22bf19-fcc8-4aff-92d8-f21498a7d2bd@github.com>
References: <0-KY5icPKpiaB8p261b901PBCh1qzjGBN4xQFGDJPKU=.6f22bf19-fcc8-4aff-92d8-f21498a7d2bd@github.com>
Message-ID: <FDGCuvCGczFSEPKXbfhws-HfYtaYHYINpmahB_VkdcM=.aa11ab99-0a12-4f7d-afa3-e224693ed4b7@github.com>

On Wed, 19 Feb 2025 15:44:23 GMT, Andrew John Hughes <andrew at openjdk.org> wrote:

> Merge jdk8u332-b07

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah-jdk8u/pull/12

From andrew at openjdk.org  Wed Feb 19 15:55:56 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Wed, 19 Feb 2025 15:55:56 GMT
Subject: git: openjdk/shenandoah-jdk8u: Added tag jdk8u332-b07 for changeset
 6a6f2c2f
Message-ID: <0f235976-a235-4dc4-af12-4ee56fae982a@openjdk.org>

Tagged by: Andrew John Hughes <andrew at openjdk.org>
Date:      2022-04-15 04:34:35 +0000


Changeset: 6a6f2c2f
Author:    Christoph Langer <clanger at openjdk.org>
Date:      2022-04-15 04:19:06 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/6a6f2c2fe2ee99d1ec7918fe5fad2f5c0d61d599


From andrew at openjdk.org  Wed Feb 19 15:56:06 2025
From: andrew at openjdk.org (Andrew John Hughes)
Date: Wed, 19 Feb 2025 15:56:06 GMT
Subject: git: openjdk/shenandoah-jdk8u: Added tag shenandoah8u332-b07 for
 changeset 0c935e9c
Message-ID: <5db5948e-b4ef-43d3-9ffa-5b60fc4aba90@openjdk.org>

Tagged by: Andrew John Hughes <andrew at openjdk.org>
Date:      2025-02-15 14:19:15 +0000

Added tag shenandoah8u332-b07 for changeset 0c935e9cf8d
Changeset: 0c935e9c
Author:    Andrew John Hughes <andrew at openjdk.org>
Date:      2025-02-13 22:23:28 +0000
URL:       https://git.openjdk.org/shenandoah-jdk8u/commit/0c935e9cf8da1ad98fb3fef68bd0a99cd9bf22de


From xpeng at openjdk.org  Wed Feb 19 20:58:06 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Feb 2025 20:58:06 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <5au_4m8XLah7rypwO90JKB5C41b7meh_IVRXYuOYveY=.daf14a11-252e-4dbf-846f-acccae09af18@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <8ggezlgzCLo_ux2lTJ1UJzVD5VgwSlSo35ze0tQ8XcI=.b4d84743-5e40-4478-82ab-b0de3505e6ab@github.com>
 <5au_4m8XLah7rypwO90JKB5C41b7meh_IVRXYuOYveY=.daf14a11-252e-4dbf-846f-acccae09af18@github.com>
Message-ID: <A2xZVG_7HSIgvwOJLK2PgLQ5RcPsAOmzG4E33nKLO5Y=.71f81ab4-8afb-4ace-9731-e5c7bd6caad5@github.com>

On Wed, 19 Feb 2025 17:22:59 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 71:
>> 
>>> 69:         while (SafepointSynchronize::is_synchronizing() &&
>>> 70:                !SafepointMechanism::local_poll_armed(java_thread)) {
>>> 71:           short_sleep();
>> 
>> Why not `yield_or_sleep` here?
>
> It can be `yield_or_sleep` here now, I'll rerun a test to verify, I have updated `yield_or_sleep` to reset the counter after `short_sleep`.
> 
> In our test last year, we noticed Java tread may run this loop over 20k times in worse case, the older version `yields` counter won't be reset, so it is possible after safepoint some Java thread will only do `short_sleep`, which may increase allocation latency, I don't want waiting on safepoint poll impact the allocation path after safepoint.

I have update code to use yield_or_sleep here.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962201794

From shade at openjdk.org  Wed Feb 19 20:58:05 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 19 Feb 2025 20:58:05 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
Message-ID: <8ggezlgzCLo_ux2lTJ1UJzVD5VgwSlSo35ze0tQ8XcI=.b4d84743-5e40-4478-82ab-b0de3505e6ab@github.com>

On Wed, 19 Feb 2025 15:58:01 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
> 
> Tip:
> 
> 94069776
> 50993550
> 49321667
> 33903446
> 32291313
> 30587810
> 27759958
> 25080997
> 24657404
> 23874338
> 
> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
> 
> 58428998
> 44410618
> 30788370
> 20636942
> 15986465
> 15307468
> 9686426
> 9432094
> 7473938
> 6854014
> 
> Note: command line for the test:
> 
> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
> 
> 
> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
> 
> 
> 1890706
> 1222180
> 1042758
> 853157
> 792057
> 785697
> 780627
> 757817
> 740607
> 736646
> 725727
> 725596
> 724106
> 
> 
> ### Other test
> - [x] `make test TEST=hotspot_gc_shenandoah`
> - [x] Tier 2

All right, assuming performance results are good. Consider a minor nit:

src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 71:

> 69:         while (SafepointSynchronize::is_synchronizing() &&
> 70:                !SafepointMechanism::local_poll_armed(java_thread)) {
> 71:           short_sleep();

Why not `yield_or_sleep` here?

src/hotspot/share/gc/shenandoah/shenandoahLock.hpp line 48:

> 46:   void yield_or_sleep(int &yields) {
> 47:     if (yields < 5) {
> 48:       os::naked_yield();

Need `#include "runtime/os.hpp"` for this. There is likely a transitive dependency now, but it is cleaner to depend explicitly. Or, maybe move this definition to `shenandoahLock.cpp`, it would be even cleaner then, I think.

src/hotspot/share/gc/shenandoah/shenandoahLock.hpp line 61:

> 59: #else
> 60:     os::naked_short_nanosleep(100000);
> 61: #endif

Any context where this is coming from? This looks like from `SpinYield`? If so, should we target `SpinYield::default_sleep_ns=1000`?

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23701#pullrequestreview-2627707591
PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962048735
PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962211656
PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962050750

From xpeng at openjdk.org  Wed Feb 19 20:58:06 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Feb 2025 20:58:06 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <psAsv6BFypCAa2aQ9O8upmH-enjRsshTC3dhNNypf2Q=.ae182a37-b656-464b-856c-a061b22b6561@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <8ggezlgzCLo_ux2lTJ1UJzVD5VgwSlSo35ze0tQ8XcI=.b4d84743-5e40-4478-82ab-b0de3505e6ab@github.com>
 <5au_4m8XLah7rypwO90JKB5C41b7meh_IVRXYuOYveY=.daf14a11-252e-4dbf-846f-acccae09af18@github.com>
 <psAsv6BFypCAa2aQ9O8upmH-enjRsshTC3dhNNypf2Q=.ae182a37-b656-464b-856c-a061b22b6561@github.com>
Message-ID: <mAyOd8l2wQrk63RRKrkhjF_fL1iLsJMV76j20gLZgDc=.9ed7e649-4cd2-4965-ba59-e3ac8bd92a36@github.com>

On Wed, 19 Feb 2025 18:05:50 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> It is just a magic number we tested, not from `SpinYield`. I chose 100us because Shenandoah GC pause is usually less then 1ms, I also tested 10us but 100us was better in the test.
>
> I looked around HS sources, and I think the closest primitive we have is `HandshakeSpinYield`, which does 10us sleeps. How much worse is 10us in comparison to 100us in your tests? I would prefer to do 10us for all platforms, if performance tests allow us. This would also allow you to inline `short_sleep()`.

It is like more than 2x worse, but still much better(10x) than tip and revert of JDK-8331411, here are the top 10 at-safepoint time for comparison:
10 us:

7982953
5043319
5008139
4597156
4580556
4429245
4403175
4047891
3677389
3582308


100 us:

4553716
1703093
1046248
1038148
780447
778786
778436
778276
728716
721856


I can change it to same seep time for all platforms, but Windows doesn't really support nanosecond resolution sleep, JVM use combination of yield and spin pause to approximate nanosecond resolution sleep for Windows, it should be still fine since it won't be worse then only `yield` as before. In short, I am not expecting any improvement for Windows.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962180175

From xpeng at openjdk.org  Wed Feb 19 20:58:06 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Feb 2025 20:58:06 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <mAyOd8l2wQrk63RRKrkhjF_fL1iLsJMV76j20gLZgDc=.9ed7e649-4cd2-4965-ba59-e3ac8bd92a36@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <8ggezlgzCLo_ux2lTJ1UJzVD5VgwSlSo35ze0tQ8XcI=.b4d84743-5e40-4478-82ab-b0de3505e6ab@github.com>
 <5au_4m8XLah7rypwO90JKB5C41b7meh_IVRXYuOYveY=.daf14a11-252e-4dbf-846f-acccae09af18@github.com>
 <psAsv6BFypCAa2aQ9O8upmH-enjRsshTC3dhNNypf2Q=.ae182a37-b656-464b-856c-a061b22b6561@github.com>
 <mAyOd8l2wQrk63RRKrkhjF_fL1iLsJMV76j20gLZgDc=.9ed7e649-4cd2-4965-ba59-e3ac8bd92a36@github.com>
Message-ID: <DmN_Jirz_OdGaqTgAulloT_qu1Ow7Nf-YY1pTf7aHsw=.48347961-65ef-4172-ac81-da38f97739a5@github.com>

On Wed, 19 Feb 2025 18:29:04 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> I looked around HS sources, and I think the closest primitive we have is `HandshakeSpinYield`, which does 10us sleeps. How much worse is 10us in comparison to 100us in your tests? I would prefer to do 10us for all platforms, if performance tests allow us. This would also allow you to inline `short_sleep()`.
>
> It is like more than 2x worse, but still much better(10x) than tip and revert of JDK-8331411, here are the top 10 at-safepoint time for comparison:
> 10 us:
> 
> 7982953
> 5043319
> 5008139
> 4597156
> 4580556
> 4429245
> 4403175
> 4047891
> 3677389
> 3582308
> 
> 
> 100 us:
> 
> 4553716
> 1703093
> 1046248
> 1038148
> 780447
> 778786
> 778436
> 778276
> 728716
> 721856
> 
> 
> I can change it to same seep time for all platforms, but Windows doesn't really support nanosecond resolution sleep, JVM use combination of yield and spin pause to approximate nanosecond resolution sleep for Windows, it should be still fine since it won't be worse then only `yield` as before. In short, I am not expecting any improvement for Windows.

I have changed to be same for all platforms, but still keep 100us sleep duration given that it did perform better.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962203617

From shade at openjdk.org  Wed Feb 19 20:58:06 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 19 Feb 2025 20:58:06 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <5au_4m8XLah7rypwO90JKB5C41b7meh_IVRXYuOYveY=.daf14a11-252e-4dbf-846f-acccae09af18@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <8ggezlgzCLo_ux2lTJ1UJzVD5VgwSlSo35ze0tQ8XcI=.b4d84743-5e40-4478-82ab-b0de3505e6ab@github.com>
 <5au_4m8XLah7rypwO90JKB5C41b7meh_IVRXYuOYveY=.daf14a11-252e-4dbf-846f-acccae09af18@github.com>
Message-ID: <psAsv6BFypCAa2aQ9O8upmH-enjRsshTC3dhNNypf2Q=.ae182a37-b656-464b-856c-a061b22b6561@github.com>

On Wed, 19 Feb 2025 17:27:15 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahLock.hpp line 61:
>> 
>>> 59: #else
>>> 60:     os::naked_short_nanosleep(100000);
>>> 61: #endif
>> 
>> Any context where this is coming from? This looks like from `SpinYield`? If so, should we target `SpinYield::default_sleep_ns=1000`?
>
> It is just a magic number we tested, not from `SpinYield`. I chose 100us because Shenandoah GC pause is usually less then 1ms, I also tested 10us but 100us was better in the test.

I looked around HS sources, and I think the closest primitive we have is `HandshakeSpinYield`, which does 10us sleeps. How much worse is 10us in comparison to 100us in your tests? I would prefer to do 10us for all platforms, if performance tests allow us. This would also allow you to inline `short_sleep()`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962150403

From xpeng at openjdk.org  Wed Feb 19 20:58:04 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Feb 2025 20:58:04 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
Message-ID: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>

We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:

Tip:

94069776
50993550
49321667
33903446
32291313
30587810
27759958
25080997
24657404
23874338

Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)

58428998
44410618
30788370
20636942
15986465
15307468
9686426
9432094
7473938
6854014

Note: command line for the test:

java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr


With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:


1890706
1222180
1042758
853157
792057
785697
780627
757817
740607
736646
725727
725596
724106


### Other test
- [x] `make test TEST=hotspot_gc_shenandoah`
- [x] Tier 2

-------------

Commit messages:
 - Move impl of yield_or_sleep to cpp file
 - Address review comments
 - Reset yields count to 0 after short sleep
 - Merge branch 'openjdk:master' into shenandoah-lock-fix
 - Put thread to sleep after it yield up to 5 times to contend for ShenandoahLock w/o luck

Changes: https://git.openjdk.org/jdk/pull/23701/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23701&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8350285
  Stats: 16 lines in 2 files changed: 13 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/23701.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23701/head:pull/23701

PR: https://git.openjdk.org/jdk/pull/23701

From xpeng at openjdk.org  Wed Feb 19 20:58:05 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Feb 2025 20:58:05 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <8ggezlgzCLo_ux2lTJ1UJzVD5VgwSlSo35ze0tQ8XcI=.b4d84743-5e40-4478-82ab-b0de3505e6ab@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <8ggezlgzCLo_ux2lTJ1UJzVD5VgwSlSo35ze0tQ8XcI=.b4d84743-5e40-4478-82ab-b0de3505e6ab@github.com>
Message-ID: <5au_4m8XLah7rypwO90JKB5C41b7meh_IVRXYuOYveY=.daf14a11-252e-4dbf-846f-acccae09af18@github.com>

On Wed, 19 Feb 2025 16:56:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
>> 
>> Tip:
>> 
>> 94069776
>> 50993550
>> 49321667
>> 33903446
>> 32291313
>> 30587810
>> 27759958
>> 25080997
>> 24657404
>> 23874338
>> 
>> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
>> 
>> 58428998
>> 44410618
>> 30788370
>> 20636942
>> 15986465
>> 15307468
>> 9686426
>> 9432094
>> 7473938
>> 6854014
>> 
>> Note: command line for the test:
>> 
>> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
>> 
>> 
>> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
>> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
>> 
>> 
>> 1890706
>> 1222180
>> 1042758
>> 853157
>> 792057
>> 785697
>> 780627
>> 757817
>> 740607
>> 736646
>> 725727
>> 725596
>> 724106
>> 
>> 
>> ### Other test
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>> - [x] Tier 2
>
> src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 71:
> 
>> 69:         while (SafepointSynchronize::is_synchronizing() &&
>> 70:                !SafepointMechanism::local_poll_armed(java_thread)) {
>> 71:           short_sleep();
> 
> Why not `yield_or_sleep` here?

It can be `yield_or_sleep` here now, I'll rerun a test to verify, I have updated `yield_or_sleep` to reset the counter after `short_sleep`.

In our test last year, we noticed Java tread may run this loop over 20k times in worse case, the older version `yields` counter won't be reset, so it is possible after safepoint some Java thread will only do `short_sleep`, which may increase allocation latency, I don't want waiting on safepoint poll impact the allocation path after safepoint.

> src/hotspot/share/gc/shenandoah/shenandoahLock.hpp line 48:
> 
>> 46:   void yield_or_sleep(int &yields) {
>> 47:     if (yields < 5) {
>> 48:       os::naked_yield();
> 
> Need `#include "runtime/os.hpp"` for this. There is likely a transitive dependency now, but it is cleaner to depend explicitly. Or, maybe move this definition to `shenandoahLock.cpp`, it would be even cleaner then, I think.

Thanks, just moved the definition to `shenandoahLock.cpp`, it can be static as well.

> src/hotspot/share/gc/shenandoah/shenandoahLock.hpp line 61:
> 
>> 59: #else
>> 60:     os::naked_short_nanosleep(100000);
>> 61: #endif
> 
> Any context where this is coming from? This looks like from `SpinYield`? If so, should we target `SpinYield::default_sleep_ns=1000`?

It is just a magic number we tested, not from `SpinYield`. I chose 100us because Shenandoah GC pause is usually less then 1ms, I also tested 10us but 100us was better in the test.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962088642
PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962243559
PR Review Comment: https://git.openjdk.org/jdk/pull/23701#discussion_r1962094983

From kdnilsen at openjdk.org  Wed Feb 19 21:23:54 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Feb 2025 21:23:54 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
Message-ID: <_qWdKYXFMjPhwL_I2udsXUTyRYKUzOpKMZ3Bx3x-hWQ=.e5b6d1fc-cfda-46af-9c6e-2aedca880353@github.com>

On Wed, 19 Feb 2025 15:58:01 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
> 
> Tip:
> 
> 94069776
> 50993550
> 49321667
> 33903446
> 32291313
> 30587810
> 27759958
> 25080997
> 24657404
> 23874338
> 
> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
> 
> 58428998
> 44410618
> 30788370
> 20636942
> 15986465
> 15307468
> 9686426
> 9432094
> 7473938
> 6854014
> 
> Note: command line for the test:
> 
> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
> 
> 
> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call `futex(FUTEX_WAKE_PRIVATE)` to disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
> 
> 
> 1890706
> 1222180
> 1042758
> 853157
> 792057
> 785697
> 780627
> 757817
> 740607
> 736646
> 725727
> 725596
> 724106
> 
> 
> ### Other test
> - [x] `make test TEST=hotspot_gc_shenandoah`
> - [x] Tier 2

Yielding 5x for every 1 nanosleep seems a bit "arbitrary".  I assume you found that the number 5 delivered the "best performance" compared to other numbers you might have chosen.  I wonder if different architectures with different numbers of cores, different operating systems, and/or different test applications that have different numbers of runnable threads would also perform best with this same magic number 5.

Could we at least add a comment explaining how/why we chose 5 here?

-------------

PR Review: https://git.openjdk.org/jdk/pull/23701#pullrequestreview-2628009211

From xpeng at openjdk.org  Wed Feb 19 21:30:52 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Feb 2025 21:30:52 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <_qWdKYXFMjPhwL_I2udsXUTyRYKUzOpKMZ3Bx3x-hWQ=.e5b6d1fc-cfda-46af-9c6e-2aedca880353@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <_qWdKYXFMjPhwL_I2udsXUTyRYKUzOpKMZ3Bx3x-hWQ=.e5b6d1fc-cfda-46af-9c6e-2aedca880353@github.com>
Message-ID: <sfiTyUCzNp9uw76tfZNQJd_sjNrLXa4xkE0BZbWEX2o=.ff5b2ab8-c396-495e-82a6-2f06eb32fc95@github.com>

On Wed, 19 Feb 2025 21:21:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Yielding 5x for every 1 nanosleep seems a bit "arbitrary". I assume you found that the number 5 delivered the "best performance" compared to other numbers you might have chosen. I wonder if different architectures with different numbers of cores, different operating systems, and/or different test applications that have different numbers of runnable threads would also perform best with this same magic number 5.
> 
> Could we at least add a comment explaining how/why we chose 5 here?

5 is from the old implementation, the old implementation was copied from https://github.com/openjdk/jdk/blob/master/src/hotspot/share/runtime/thread.cpp#L577

I can do a bit more test on this, and add some comments to explain why we choose the magic number 5.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23701#issuecomment-2669800369

From xpeng at openjdk.org  Wed Feb 19 22:38:53 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Feb 2025 22:38:53 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention
In-Reply-To: <sfiTyUCzNp9uw76tfZNQJd_sjNrLXa4xkE0BZbWEX2o=.ff5b2ab8-c396-495e-82a6-2f06eb32fc95@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <_qWdKYXFMjPhwL_I2udsXUTyRYKUzOpKMZ3Bx3x-hWQ=.e5b6d1fc-cfda-46af-9c6e-2aedca880353@github.com>
 <sfiTyUCzNp9uw76tfZNQJd_sjNrLXa4xkE0BZbWEX2o=.ff5b2ab8-c396-495e-82a6-2f06eb32fc95@github.com>
Message-ID: <sQ1yk06Dm5XQKjRYTvGBryjG8a3Nkgsl0_3xck_3V8k=.50ce5df2-c9f5-40da-a3fc-01a41d75db30@github.com>

On Wed, 19 Feb 2025 21:28:40 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> > Yielding 5x for every 1 nanosleep seems a bit "arbitrary". I assume you found that the number 5 delivered the "best performance" compared to other numbers you might have chosen. I wonder if different architectures with different numbers of cores, different operating systems, and/or different test applications that have different numbers of runnable threads would also perform best with this same magic number 5.
> > Could we at least add a comment explaining how/why we chose 5 here?
> 
> 5 is from the old implementation, the old implementation was copied from https://github.com/openjdk/jdk/blob/master/src/hotspot/share/runtime/thread.cpp#L577
> 
> I can do a bit more test on this, and add some comments to explain why we choose the magic number 5.

I have tested 3/5/7, safepoint time is very close in the tests with 3 or 5 yields(3 is slightly better), but it is worse with 7, I can change it to 3.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23701#issuecomment-2669916924

From xpeng at openjdk.org  Wed Feb 19 22:49:22 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Feb 2025 22:49:22 GMT
Subject: RFR: 8350285: Regression caused by ShenandoahLock under extreme
 contention [v2]
In-Reply-To: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
Message-ID: <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>

> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
> 
> Tip:
> 
> 94069776
> 50993550
> 49321667
> 33903446
> 32291313
> 30587810
> 27759958
> 25080997
> 24657404
> 23874338
> 
> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
> 
> 58428998
> 44410618
> 30788370
> 20636942
> 15986465
> 15307468
> 9686426
> 9432094
> 7473938
> 6854014
> 
> Note: command line for the test:
> 
> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
> 
> 
> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call `futex(FUTEX_WAKE_PRIVATE)` to disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
> 
> 
> 1890706
> 1222180
> 1042758
> 853157
> 792057
> 785697
> 780627
> 757817
> 740607
> 736646
> 725727
> 725596
> 724106
> 
> 
> ### Other test
> - [x] `make test TEST=hotspot_gc_shenandoah`
> - [x] Tier 2

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Address review comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23701/files
  - new: https://git.openjdk.org/jdk/pull/23701/files/756f7820..68e1b985

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23701&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23701&range=00-01

  Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/23701.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23701/head:pull/23701

PR: https://git.openjdk.org/jdk/pull/23701

From cslucas at openjdk.org  Thu Feb 20 01:34:56 2025
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 20 Feb 2025 01:34:56 GMT
Subject: RFR: 8343468: GenShen: Enable relocation of remembered set card
 tables [v3]
In-Reply-To: <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
References: <e5SIIH1XtQ-EFSNZk2-rUzYP_HFku9C8LK_O-nVMSzI=.9d639980-5a54-41dc-989d-253669d3c392@github.com>
 <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
Message-ID: <H3ia1ZRo8UI8jaLLR2HyknJBnvBeKNqNd2F2crZNQfY=.c2db240b-f6fb-4b22-a34b-544da692c0bc@github.com>

On Thu, 23 Jan 2025 05:45:43 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> In the current Generational Shenandoah implementation, the pointers to the read and write card tables are established at JVM launch time and fixed during the whole of the application execution. Because they are considered constants, they are embedded as such in JIT-compiled code. 
>> 
>> The cleaning of dirty cards in the read card table is performed during the `init-mark` pause, and our experiments show that it represents a sizable portion of that phase's duration. This pull request makes the addresses of the read and write card tables dynamic, with the end goal of reducing the duration of the `init-mark` pause by moving the cleaning of the dirty cards in the read card table to the `reset` concurrent phase.
>> 
>> The idea is quite simple. Instead of using distinct read and write card tables for the entire duration of the JVM execution, we alternate which card table serves as the read/write table during each GC cycle. In the `reset` phase we concurrently clean the cards in the the current _read_ table so that when the cycle reaches the next `init-mark` phase we have a version of the card table totally clear. In the next `init-mark` pause we swap the pointers to the base of the read and write tables. When the `init-mark` finishes the mutator threads will operate on the table just cleaned in the `reset` phase; the GC will operate on the table that just turned the new _read_ table.
>> 
>> Most of the changes in the patch account for the fact that the write card table is no longer at a fixed address.
>> 
>> The primary benefit of this change is that it eliminates the need to copy and zero the remembered set during the init-mark Safepoint. A secondary benefit is that it allows us to replace the init-mark Safepoint with an `init-mark` handshake?something we plan to work on after this PR is merged.
>> 
>> Our internal performance testing showed a significant reduction in the duration of `init-mark` pauses and no statistically significant regression due to the dynamic loading of the card table address in JIT-compiled code.
>> 
>> Functional testing was performed on Linux, macOS, Windows running on x64, AArch64, and their respective 32-bit versions. I?d appreciate it if someone with access to RISC-V (@luhenry  ?) and PowerPC (@TheRealMDoerr ?)  platforms could review and test the changes for those platforms, as I have limited access to running tests on them.
>
> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
> 
>  - Merge master
>  - Addressing PR comments: some refactorings, ppc fix, off-by-one fix.
>  - Relocation of Card Tables

src/hotspot/share/gc/shared/cardTable.hpp line 205:

> 203:   virtual CardValue* byte_map_base() const { return _byte_map_base; }
> 204: 
> 205:   virtual CardValue* byte_map() const { return _byte_map; }

@shipilev - can you please confirm that this is the part that you didn't like?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23170#discussion_r1962702320

From kdnilsen at openjdk.org  Thu Feb 20 14:22:56 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Feb 2025 14:22:56 GMT
Subject: RFR: 8350285: Shenandoah: Regression caused by ShenandoahLock
 under extreme contention [v2]
In-Reply-To: <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
Message-ID: <3X_Qwm8n-a9jDEWsqcXPQF2piMQ8Pbf7OtjXCYeZB6A=.9ec22dd1-e75f-4696-b3da-48eb4f75bd9a@github.com>

On Wed, 19 Feb 2025 22:49:22 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
>> 
>> Tip:
>> 
>> 94069776
>> 50993550
>> 49321667
>> 33903446
>> 32291313
>> 30587810
>> 27759958
>> 25080997
>> 24657404
>> 23874338
>> 
>> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
>> 
>> 58428998
>> 44410618
>> 30788370
>> 20636942
>> 15986465
>> 15307468
>> 9686426
>> 9432094
>> 7473938
>> 6854014
>> 
>> Note: command line for the test:
>> 
>> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
>> 
>> 
>> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call `futex(FUTEX_WAKE_PRIVATE)` to disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
>> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
>> 
>> 
>> 1890706
>> 1222180
>> 1042758
>> 853157
>> 792057
>> 785697
>> 780627
>> 757817
>> 740607
>> 736646
>> 725727
>> 725596
>> 724106
>> 
>> 
>> ### Other test
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>> - [x] Tier 2
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address review comments

Thanks for followup.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/jdk/pull/23701#pullrequestreview-2630011438

From wkemper at openjdk.org  Thu Feb 20 14:25:33 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 20 Feb 2025 14:25:33 GMT
Subject: RFR: Merge openjdk/jdk21u:master
Message-ID: <hWtHxeea7l4yR3FMxYQ2MK_BY_Q6dfkDGu4mtWCteu4=.71994d2f-f983-413d-a7df-3d87295ff799@github.com>

Merges tag jdk-21.0.7+3

-------------

Commit messages:
 - 8338303: Linux ppc64le with toolchain clang - detection failure in early JVM startup
 - 8346880: [aix] java/lang/ProcessHandle/InfoTest.java still fails: "reported cputime less than expected"
 - 8347576: Error output in libjsound has non matching format strings
 - 8346881: [ubsan] logSelection.cpp:154:24  / logSelectionList.cpp:72:94 : runtime error: applying non-zero offset 1 to null pointer
 - 8345569: [ubsan] adjustments to filemap.cpp and virtualspace.cpp for macOS aarch64
 - 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk
 - 8347171: (dc) java/nio/channels/DatagramChannel/InterruptibleOrNot.java fails with virtual thread factory
 - 8343882: BasicAnnoTests doesn't handle multiple annotations at the same position
 - 8347740: java/io/File/createTempFile/SpecialTempFile.java failing
 - 8302111: Serialization considerations
 - ... and 5 more: https://git.openjdk.org/shenandoah-jdk21u/compare/3e556491...b7d92cd0

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/156/files
  Stats: 1629 lines in 62 files changed: 1037 ins; 330 del; 262 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/156.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/156/head:pull/156

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/156

From shade at openjdk.org  Thu Feb 20 15:35:57 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 20 Feb 2025 15:35:57 GMT
Subject: RFR: 8343468: GenShen: Enable relocation of remembered set card
 tables [v3]
In-Reply-To: <H3ia1ZRo8UI8jaLLR2HyknJBnvBeKNqNd2F2crZNQfY=.c2db240b-f6fb-4b22-a34b-544da692c0bc@github.com>
References: <e5SIIH1XtQ-EFSNZk2-rUzYP_HFku9C8LK_O-nVMSzI=.9d639980-5a54-41dc-989d-253669d3c392@github.com>
 <6_AoWQhldJttOIEOL1T7HSapPzE4Qn2j4WN7E-bI3rM=.2685d3d8-e47c-42a6-845b-b68f50cc568e@github.com>
 <H3ia1ZRo8UI8jaLLR2HyknJBnvBeKNqNd2F2crZNQfY=.c2db240b-f6fb-4b22-a34b-544da692c0bc@github.com>
Message-ID: <q04G8LEjiKzqJZ5Kvwtm6cm96CdtCDmvtfThmd63C0I=.3f9c1325-a708-4c93-a26e-60bf9188b24a@github.com>

On Thu, 20 Feb 2025 01:32:42 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> Cesar Soares Lucas has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
>> 
>>  - Merge master
>>  - Addressing PR comments: some refactorings, ppc fix, off-by-one fix.
>>  - Relocation of Card Tables
>
> src/hotspot/share/gc/shared/cardTable.hpp line 205:
> 
>> 203:   virtual CardValue* byte_map_base() const { return _byte_map_base; }
>> 204: 
>> 205:   virtual CardValue* byte_map() const { return _byte_map; }
> 
> @shipilev - can you please confirm that this is the part that you didn't like?

Yes, I am not fond of extending `CardTable` with virtual members, especially if they can be used on high-performance paths. Not sure if the following idea is viable. 

ShenandoahBarrierSet knows where to get card table base: from Shenandoah thread local data. Now it looks like we need to deal with two problems:
  1. Protect ourselves from accidentally calling `CardTable` methods that may reference "incorrect" `_byte_map_(base)`. To do that, it looks it is enough to initialize `CardTable::_byte_map_(base)` to non-sensical values (`nullptr`-s?), and let the testing crash.
  2. Allow calls to `CardTable` utility methods with our base. For that, I think we can drill a few new (non-virtual) methods in `CardTable`, and enter from Shenandoah through them. So for example `byte_for_index(const size_t card_index)` becomes:
  ```
  CardValue* byte_for_index(const CardValue* base, const size_t card_index) const {
    return base + card_index;
  }
  CardValue* byte_for_index(const size_t card_index) const {
    return byte_for_index(_byte_map, card_index);
  }
  ```

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23170#discussion_r1963810697

From dfenacci at openjdk.org  Thu Feb 20 16:32:18 2025
From: dfenacci at openjdk.org (Damon Fenacci)
Date: Thu, 20 Feb 2025 16:32:18 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure properly
 during initialization (RuntimeStub::new_runtime_stub fatal crash)
Message-ID: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>

# Issue
The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.

# Causes
There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`

# Solution
Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.

Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.

# Testing
The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.

Tests: Tier 1-4 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; release and debug mode)

-------------

Commit messages:
 - JDK-8347406: reduce number of tests again
 - JDK-8347406: update copyright year
 - Merge branch 'master' into JDK-8347406
 - Merge branch 'master' into JDK-8347406
 - JDK-8347406: reduce number of test processes
 - JDK-8347406: set the C2 uncommon and exception trap blobs in OptoRuntime::generate
 - JDK-8347406: fix c2 runtime init return condition
 - JDK-8347406: reduce number of processes in test
 - JDK-8347406: make startup processes run in parallel
 - JDK-8347406: reduce number of startup test attempts
 - ... and 8 more: https://git.openjdk.org/jdk/compare/efbad00c...e930df47

Changes: https://git.openjdk.org/jdk/pull/23630/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23630&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8347406
  Stats: 114 lines in 27 files changed: 38 ins; 3 del; 73 mod
  Patch: https://git.openjdk.org/jdk/pull/23630.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23630/head:pull/23630

PR: https://git.openjdk.org/jdk/pull/23630

From dfenacci at openjdk.org  Thu Feb 20 16:46:52 2025
From: dfenacci at openjdk.org (Damon Fenacci)
Date: Thu, 20 Feb 2025 16:46:52 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
In-Reply-To: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
Message-ID: <cZn8A5ydpUOZC1eokLnF04jlNit6XF_LJ0Ur81daMWs=.b5a4d242-7017-476f-b023-97eb4f2ecbc6@github.com>

On Fri, 14 Feb 2025 11:04:20 GMT, Damon Fenacci <dfenacci at openjdk.org> wrote:

> # Issue
> The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.
> 
> # Causes
> There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
> 1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`
> 
> # Solution
> Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.
> 
> Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.
> 
> # Testing
> The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.
> 
> Tests: Tier 1-4 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; release and debug mode)

@adinn I noticed I touched quite a few runtime files that you recently refactored. I guess it might make sense if you had a look at them. Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23630#issuecomment-2672059853

From dlong at openjdk.org  Thu Feb 20 20:55:52 2025
From: dlong at openjdk.org (Dean Long)
Date: Thu, 20 Feb 2025 20:55:52 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
In-Reply-To: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
Message-ID: <s8--SN34E0d36PkNjweZZ0OaLDWZkzDsk9kUDDghpGg=.51bca4f4-7098-4192-9e4c-71900492132b@github.com>

On Fri, 14 Feb 2025 11:04:20 GMT, Damon Fenacci <dfenacci at openjdk.org> wrote:

> # Issue
> The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.
> 
> # Causes
> There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
> 1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`
> 
> # Solution
> Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.
> 
> Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.
> 
> # Testing
> The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.
> 
> Tests: Tier 1-4 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; release and debug mode)

I don't understand why JDK-8326615 didn't work.  If the minimum codecache size was too small, can't we just increase it?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23630#issuecomment-2672655838

From dlong at openjdk.org  Thu Feb 20 21:27:52 2025
From: dlong at openjdk.org (Dean Long)
Date: Thu, 20 Feb 2025 21:27:52 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
In-Reply-To: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
Message-ID: <97uZIfQ245vs73ViJss3Yg6lsCQBbj-bYtSyG8_GMF8=.c7e59a36-166a-4e1e-95ad-5820cb1042fc@github.com>

On Fri, 14 Feb 2025 11:04:20 GMT, Damon Fenacci <dfenacci at openjdk.org> wrote:

> # Issue
> The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.
> 
> # Causes
> There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
> 1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`
> 
> # Solution
> Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.
> 
> Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.
> 
> # Testing
> The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.
> 
> Tests: Tier 1-4 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; release and debug mode)

src/hotspot/share/c1/c1_Runtime1.cpp line 233:

> 231:                                                  oop_maps,
> 232:                                                  must_gc_arguments,
> 233:                                                  false);

Suggestion:

                                                 false /* alloc_fail_is_fatal */ );

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23630#discussion_r1964359747

From dlong at openjdk.org  Thu Feb 20 21:38:54 2025
From: dlong at openjdk.org (Dean Long)
Date: Thu, 20 Feb 2025 21:38:54 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
In-Reply-To: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
Message-ID: <hDbvpph4D6KJc_tMrde62yCVY5DVdWncy0wjBz67fBk=.3ac9a977-857a-45df-a5e8-284f9e14af60@github.com>

On Fri, 14 Feb 2025 11:04:20 GMT, Damon Fenacci <dfenacci at openjdk.org> wrote:

> # Issue
> The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.
> 
> # Causes
> There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
> 1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`
> 
> # Solution
> Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.
> 
> Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.
> 
> # Testing
> The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.
> 
> Tests: Tier 1-4 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; release and debug mode)

src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp line 306:

> 304:                                 _load_reference_barrier_phantom_rt_code_blob != nullptr;
> 305:   }
> 306:   return _pre_barrier_c1_runtime_code_blob != nullptr && reference_barrier_success;

Wouldn't it be better to return false immediately after each failure, rather than continuing?

src/hotspot/share/gc/z/c1/zBarrierSetC1.cpp line 543:

> 541:   _store_barrier_on_oop_field_without_healing =
> 542:     generate_c1_store_runtime_stub(blob, false /* self_healing */, "store_barrier_on_oop_field_without_healing");
> 543:   return _load_barrier_on_oop_field_preloaded_runtime_stub != nullptr &&

Again, why not return false immediately on first failure?

src/hotspot/share/opto/output.cpp line 3487:

> 3485:         C->record_failure("CodeCache is full");
> 3486:       } else {
> 3487:         C->set_stub_entry_point(rs->entry_point());

Is the deleted rs->is_runtime_stub() assert still useful here?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23630#discussion_r1964370135
PR Review Comment: https://git.openjdk.org/jdk/pull/23630#discussion_r1964371192
PR Review Comment: https://git.openjdk.org/jdk/pull/23630#discussion_r1964372958

From xpeng at openjdk.org  Fri Feb 21 06:50:53 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 21 Feb 2025 06:50:53 GMT
Subject: RFR: 8350285: Shenandoah: Regression caused by ShenandoahLock
 under extreme contention [v2]
In-Reply-To: <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
Message-ID: <3TP8q0-9_fuI5p0CjFlHYzr7UPeLhLsV7hSv0Np3s7I=.f506318a-3243-4efd-b838-f46f9573492e@github.com>

On Wed, 19 Feb 2025 22:49:22 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
>> 
>> Tip:
>> 
>> 94069776
>> 50993550
>> 49321667
>> 33903446
>> 32291313
>> 30587810
>> 27759958
>> 25080997
>> 24657404
>> 23874338
>> 
>> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
>> 
>> 58428998
>> 44410618
>> 30788370
>> 20636942
>> 15986465
>> 15307468
>> 9686426
>> 9432094
>> 7473938
>> 6854014
>> 
>> Note: command line for the test:
>> 
>> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
>> 
>> 
>> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call `futex(FUTEX_WAKE_PRIVATE)` to disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
>> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
>> 
>> 
>> 1890706
>> 1222180
>> 1042758
>> 853157
>> 792057
>> 785697
>> 780627
>> 757817
>> 740607
>> 736646
>> 725727
>> 725596
>> 724106
>> 
>> 
>> ### Other test
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>> - [x] Tier 2
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address review comments

Thanks for the reviews!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23701#issuecomment-2673624414

From xpeng at openjdk.org  Fri Feb 21 06:54:53 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 21 Feb 2025 06:54:53 GMT
Subject: RFR: 8350285: Shenandoah: Regression caused by ShenandoahLock
 under extreme contention [v2]
In-Reply-To: <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
Message-ID: <LD36GgbZbDIbEkD9dUQAjZ90v7Btj1Iw4EXWPfaU9Z4=.3e79e8e7-4aaf-41c3-b1dd-ab5ced0f7fdb@github.com>

On Wed, 19 Feb 2025 22:49:22 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
>> 
>> Tip:
>> 
>> 94069776
>> 50993550
>> 49321667
>> 33903446
>> 32291313
>> 30587810
>> 27759958
>> 25080997
>> 24657404
>> 23874338
>> 
>> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
>> 
>> 58428998
>> 44410618
>> 30788370
>> 20636942
>> 15986465
>> 15307468
>> 9686426
>> 9432094
>> 7473938
>> 6854014
>> 
>> Note: command line for the test:
>> 
>> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
>> 
>> 
>> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call `futex(FUTEX_WAKE_PRIVATE)` to disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
>> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
>> 
>> 
>> 1890706
>> 1222180
>> 1042758
>> 853157
>> 792057
>> 785697
>> 780627
>> 757817
>> 740607
>> 736646
>> 725727
>> 725596
>> 724106
>> 
>> 
>> ### Other test
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>> - [x] Tier 2
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address review comments

> 
> @pengxiaolong This pull request has not yet been marked as ready for integration.
> 

Sorry I misread it

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23701#issuecomment-2673660240

From shade at openjdk.org  Fri Feb 21 08:49:55 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 21 Feb 2025 08:49:55 GMT
Subject: RFR: 8350285: Shenandoah: Regression caused by ShenandoahLock
 under extreme contention [v2]
In-Reply-To: <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
Message-ID: <TuxVAJ5y8H3AbmJ7Jx2gC5uWZNK9IiaYicuTKCdszvU=.87d6a4ca-b478-415e-b533-d1fea66a76e7@github.com>

On Wed, 19 Feb 2025 22:49:22 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
>> 
>> Tip:
>> 
>> 94069776
>> 50993550
>> 49321667
>> 33903446
>> 32291313
>> 30587810
>> 27759958
>> 25080997
>> 24657404
>> 23874338
>> 
>> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
>> 
>> 58428998
>> 44410618
>> 30788370
>> 20636942
>> 15986465
>> 15307468
>> 9686426
>> 9432094
>> 7473938
>> 6854014
>> 
>> Note: command line for the test:
>> 
>> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
>> 
>> 
>> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call `futex(FUTEX_WAKE_PRIVATE)` to disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
>> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
>> 
>> 
>> 1890706
>> 1222180
>> 1042758
>> 853157
>> 792057
>> 785697
>> 780627
>> 757817
>> 740607
>> 736646
>> 725727
>> 725596
>> 724106
>> 
>> 
>> ### Other test
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>> - [x] Tier 2
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address review comments

Marked as reviewed by shade (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23701#pullrequestreview-2632396522

From johannes.bechberger at sap.com  Fri Feb 21 12:43:06 2025
From: johannes.bechberger at sap.com (Bechberger, Johannes)
Date: Fri, 21 Feb 2025 12:43:06 +0000
Subject: Shenandoah and JFR
Message-ID: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>

Hi,

Shenandoah seems to have problems with JFR. I wrote a simple test case that causes assertions errors (and core dumps in release) when run together with JFR.
You can find more at https://gist.github.com/parttimenerd/d090497e84483202d39821e66f753eb0

Regards
Johannes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/shenandoah-dev/attachments/20250221/f7836cd5/attachment-0001.htm>

From duke at openjdk.org  Fri Feb 21 16:34:54 2025
From: duke at openjdk.org (duke)
Date: Fri, 21 Feb 2025 16:34:54 GMT
Subject: RFR: 8350285: Shenandoah: Regression caused by ShenandoahLock
 under extreme contention [v2]
In-Reply-To: <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
 <mSLCRVaNbkmR6HW5EWjkkc9k-njoGIR--fkQ4LAV04A=.2e609fa2-5900-448a-a86c-34a9e1ab7acf@github.com>
Message-ID: <8pPDE34XMhzcayRu3NZhW4JUmANZblcUq9-QZZOFe2g=.95047351-a914-44bf-ac66-b35985b73b18@github.com>

On Wed, 19 Feb 2025 22:49:22 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
>> 
>> Tip:
>> 
>> 94069776
>> 50993550
>> 49321667
>> 33903446
>> 32291313
>> 30587810
>> 27759958
>> 25080997
>> 24657404
>> 23874338
>> 
>> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
>> 
>> 58428998
>> 44410618
>> 30788370
>> 20636942
>> 15986465
>> 15307468
>> 9686426
>> 9432094
>> 7473938
>> 6854014
>> 
>> Note: command line for the test:
>> 
>> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
>> 
>> 
>> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call `futex(FUTEX_WAKE_PRIVATE)` to disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
>> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
>> 
>> 
>> 1890706
>> 1222180
>> 1042758
>> 853157
>> 792057
>> 785697
>> 780627
>> 757817
>> 740607
>> 736646
>> 725727
>> 725596
>> 724106
>> 
>> 
>> ### Other test
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>> - [x] Tier 2
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address review comments

@pengxiaolong 
Your change (at version 68e1b985b939dba0f4dc12a71901bb063769c1f1) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23701#issuecomment-2675015842

From xpeng at openjdk.org  Fri Feb 21 16:41:58 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 21 Feb 2025 16:41:58 GMT
Subject: Integrated: 8350285: Shenandoah: Regression caused by ShenandoahLock
 under extreme contention
In-Reply-To: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
References: <2ZCZUKE71ToGyKRHVR2TNpmgoubol7j2MVENy3p4kdo=.3e005c72-39ca-4fbe-852a-ac90bbfeb63a@github.com>
Message-ID: <t-L0klX4QUeqAXW2KYqiSprj0aClCzqLr5qqt2jlqns=.86fa9e73-1314-4a3a-aa07-abc56ca0cd1a@github.com>

On Wed, 19 Feb 2025 15:58:01 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> We have noticed there is significant regression in at-safepoint time with recent changes made to ShenandoahLock, more specifically this [PR](https://github.com/openjdk/jdk/pull/19570), a local reproducer was written to reproduce the issue, here is the top N at-safepoint time in `ns` comparison:
> 
> Tip:
> 
> 94069776
> 50993550
> 49321667
> 33903446
> 32291313
> 30587810
> 27759958
> 25080997
> 24657404
> 23874338
> 
> Tip + reverting [PR](https://github.com/openjdk/jdk/pull/19570)
> 
> 58428998
> 44410618
> 30788370
> 20636942
> 15986465
> 15307468
> 9686426
> 9432094
> 7473938
> 6854014
> 
> Note: command line for the test:
> 
> java -Xms256m -Xmx256m -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahPacing  -XX:-UseTLAB -Xlog:gc -Xlog:safepoint ~/Alloc.java | grep -Po "At safepoint: \d+ ns" | grep -Po "\d+" | sort -nr
> 
> 
> With further digging, we found the real problem is more runnable threads after the [PR](https://github.com/openjdk/jdk/pull/19570) causes longer time for VM_Thread to call `futex(FUTEX_WAKE_PRIVATE)` to disarm wait barrier when leaving safepoint. Fixing in the issue in VM_Thread benefits other GCs as well but it is more complicated(see the details here https://bugs.openjdk.org/browse/JDK-8350324). 
> With some tweaks in ShenandoahLock, we could mitigate the regression caused by [PR](https://github.com/openjdk/jdk/pull/19570), also improve the long tails of at-saftpoint time by more than 10x, here is the result from the same test with this changes of this PR:
> 
> 
> 1890706
> 1222180
> 1042758
> 853157
> 792057
> 785697
> 780627
> 757817
> 740607
> 736646
> 725727
> 725596
> 724106
> 
> 
> ### Other test
> - [x] `make test TEST=hotspot_gc_shenandoah`
> - [x] Tier 2

This pull request has now been integrated.

Changeset: bd8ad309
Author:    Xiaolong Peng <xpeng at openjdk.org>
Committer: Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/bd8ad309b59bceb3073a8d6411cca74e73508885
Stats:     18 lines in 2 files changed: 15 ins; 0 del; 3 mod

8350285: Shenandoah: Regression caused by ShenandoahLock under extreme contention

Reviewed-by: shade, kdnilsen

-------------

PR: https://git.openjdk.org/jdk/pull/23701

From shipilev at amazon.de  Mon Feb 24 14:19:30 2025
From: shipilev at amazon.de (Aleksey Shipilev)
Date: Mon, 24 Feb 2025 15:19:30 +0100
Subject: Shenandoah and JFR
In-Reply-To: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>
References: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>
Message-ID: <4ff2582f-b8ba-440a-93cc-b94bd6de65da@amazon.de>


On 21.02.25 13:43, Bechberger, Johannes wrote:
> Shenandoah seems to have problems with JFR. I wrote a simple test case that causes assertions errors 
> (and core dumps in release) when run together with JFR.
> 
> You can find more at https://gist.github.com/parttimenerd/d090497e84483202d39821e66f753eb0 <https:// 
> gist.github.com/parttimenerd/d090497e84483202d39821e66f753eb0>

Thank you for reproducer!

Filed:
   https://bugs.openjdk.org/browse/JDK-8350580

-Aleksey

From wkemper at openjdk.org  Mon Feb 24 17:39:54 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 17:39:54 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <xD19CslZfDxxY-cEpYzt4rQJQfNfXtQZiD2ZUHryfvI=.38b1bb1e-98ed-4ba2-a44f-53ccad92f532@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <k8CL9OJ6zsVTkWIN5RsYfb-p1vEMoF4GxEsSJe7MUfk=.ee841060-bee0-4184-8169-55bc7884f63e@github.com>
 <xD19CslZfDxxY-cEpYzt4rQJQfNfXtQZiD2ZUHryfvI=.38b1bb1e-98ed-4ba2-a44f-53ccad92f532@github.com>
Message-ID: <lrm7uZH4_i9PDCXDGm83_s3WwM_db-VK6G00ALTVy44=.bb97ba58-d596-4509-b00d-5d413ea192ba@github.com>

On Sat, 15 Feb 2025 01:41:48 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 98:
>> 
>>> 96:   }
>>> 97: 
>>> 98:   // In case any threads are waiting for a cycle to happen, let them know it isn't.
>> 
>> maybe "it isn't happening", or "it won't happen".
>
> This is interesting. If GC is stopping prior to shutting down the VM, is there any point in notifying these waiting threads. Why not let them wait, and quietly shut things down? Are there JCK or other tests that would fail in that case?

The waiting threads will remain waiting and prevent the JVM from shutting down.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1968113830

From wkemper at openjdk.org  Mon Feb 24 17:53:01 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 17:53:01 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
Message-ID: <jmjBtpeXYGIWT0rGEH0SaqQebqY1_z4q5x1ou4Rublw=.d4124b39-00bb-4953-a6bf-419084c5d749@github.com>

On Fri, 14 Feb 2025 22:56:20 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:
>> 
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - Fix shutdown livelock error
>>  - Fix includes
>>  - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 64:
> 
>> 62: private:
>> 63:   // This lock is used to coordinate setting the _requested_gc_cause, _requested generation
>> 64:   // and _gc_mode. It is important that these be changed together and have a consistent view.
> 
> In that case, for ease of maintenance, I'd move the declaration of all of the 3 data members that this lock protects next to this lock, either immediately preceding or immediately succeeding its declaration in the body of this class.
> 
> Are these data members always both read and written under this lock? If so, then `_gc_mode` below doesn't need to be defined `volatile`.

The `_gc_mode` is read without the lock by the regulator thread. However, the regulator thread does take the lock and reads `_gc_mode` again under the lock before making any state changes.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1968132168

From wkemper at openjdk.org  Mon Feb 24 17:55:57 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 17:55:57 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
Message-ID: <y-aWyOpa5bP7JtRnuNDPYOkQGTv4-UBVchiramT_Yzk=.fd4fc2cf-aea5-4743-8e1c-10b1b41018a0@github.com>

On Sat, 15 Feb 2025 00:05:46 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:
>> 
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - Fix shutdown livelock error
>>  - Fix includes
>>  - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda
>
> src/hotspot/share/gc/shenandoah/shenandoahController.hpp line 66:
> 
>> 64: 
>> 65:   // This cancels the collection cycle and has an option to block
>> 66:   // until another cycle runs and clears the alloc failure gc flag.
> 
> But "the alloc failure gc flag" is gone above. The comment should be updated as well. A public API's description should avoid talking about its internal implementation details here. It's OK to talk about implementation details in the implementation of the method, not in the header spec here.

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1968135382

From wkemper at openjdk.org  Mon Feb 24 18:02:55 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 18:02:55 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
Message-ID: <Rsuv47MWiDT_BG2dySxn-gIoNkvS2BwxaO-oO4fI4pc=.07ac1fbf-fff5-490c-92e9-77bd08d6e913@github.com>

On Sat, 15 Feb 2025 00:59:37 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:
>> 
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - Fix shutdown livelock error
>>  - Fix includes
>>  - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.hpp line 188:
> 
>> 186: 
>> 187:   bool should_start_gc() override;
>> 188:   bool resume_old_cycle();
> 
> Documentation comment please, especially explaining the return value.
> For things that may return `false` and not do anything, it's better to use `try_` prefix. In fact, the method doesn't actually resume the cycle, but checks if we are in a state such that we should resume it. So, I'd name it `should_resume_old_cycle()`, consistent with the name `should_start_gc()` for the previous method.

That makes sense. Will do.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1968146250

From wkemper at openjdk.org  Mon Feb 24 18:23:56 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 18:23:56 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <LmXhohuv__2HFhtSt89EWx63pHiPn1nSdI2OFpzjV38=.30fcd0b9-4c0b-4849-97d6-06a79065c9cf@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
 <LmXhohuv__2HFhtSt89EWx63pHiPn1nSdI2OFpzjV38=.30fcd0b9-4c0b-4849-97d6-06a79065c9cf@github.com>
Message-ID: <lZNb5KAVoSaf94pGHrdc_uoguggjm1L6b2uJct6QS4Q=.fe2092a6-eb65-4d16-9fef-0d1eb6199aa8@github.com>

On Sat, 15 Feb 2025 01:14:03 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 101:
>> 
>>> 99:         || cause == GCCause::_shenandoah_allocation_failure_evac
>>> 100:         || cause == GCCause::_shenandoah_humongous_allocation_failure;
>>> 101:   }
>> 
>> Would it make sense to move this implementation also to the .cpp file like the other static `is_...` methods below?
>
> Or is this guaranteeing inlining into the caller's body, which you might prefer for the callers?

I moved the implementation.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1968193533

From wkemper at openjdk.org  Mon Feb 24 18:37:08 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 18:37:08 GMT
Subject: RFR: Merge openjdk/jdk21u:master [v2]
In-Reply-To: <hWtHxeea7l4yR3FMxYQ2MK_BY_Q6dfkDGu4mtWCteu4=.71994d2f-f983-413d-a7df-3d87295ff799@github.com>
References: <hWtHxeea7l4yR3FMxYQ2MK_BY_Q6dfkDGu4mtWCteu4=.71994d2f-f983-413d-a7df-3d87295ff799@github.com>
Message-ID: <_mTO8Vh8WfMTFIn1rxErd7C_FzY7Rp1xwO9vKIPqUMU=.0183a2b9-c6f5-4da2-9dd7-b3993fdb1389@github.com>

> Merges tag jdk-21.0.7+3

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/156/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/156/files/b7d92cd0..b7d92cd0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=156&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=156&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/156.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/156/head:pull/156

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/156

From wkemper at openjdk.org  Mon Feb 24 18:37:10 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 18:37:10 GMT
Subject: Integrated: Merge openjdk/jdk21u:master
In-Reply-To: <hWtHxeea7l4yR3FMxYQ2MK_BY_Q6dfkDGu4mtWCteu4=.71994d2f-f983-413d-a7df-3d87295ff799@github.com>
References: <hWtHxeea7l4yR3FMxYQ2MK_BY_Q6dfkDGu4mtWCteu4=.71994d2f-f983-413d-a7df-3d87295ff799@github.com>
Message-ID: <lRvhqk85TMBSwAZRFB2-u6cUpeAcOEl_QCtcIdXYrF0=.73ad8574-1c20-4dc0-93d1-8c47971931e0@github.com>

On Thu, 20 Feb 2025 14:20:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.7+3

This pull request has now been integrated.

Changeset: 4e69061b
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/4e69061bb264dcc67f7136d727d4bfce1aa3b4d9
Stats:     1629 lines in 62 files changed: 1037 ins; 330 del; 262 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/156

From wkemper at openjdk.org  Mon Feb 24 18:38:58 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 18:38:58 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc [v5]
In-Reply-To: <SwSImg7B1mdgxMu6tFrrYBYS6194MSObe_4DmZhx4pU=.1139c830-28fe-4e1b-9811-d228f9b24345@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
 <SwSImg7B1mdgxMu6tFrrYBYS6194MSObe_4DmZhx4pU=.1139c830-28fe-4e1b-9811-d228f9b24345@github.com>
Message-ID: <QJOOMJ4XK6Z_RO1XVWHS_fMlOJ7WicI5YT7hC21wfS0=.1e3f2a50-4883-487d-b07a-87684ca3ad8b@github.com>

On Tue, 18 Feb 2025 19:28:28 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:
> 
>  - Merge branch 'master' of https://git.openjdk.org/jdk into defer-generational-full-gc
>  - Merge master
>  - Fix typo in merge conflict resolution
>  - 8348595: GenShen: Fix generational free-memory no-progress check
>    
>    Reviewed-by: phh, xpeng
>  - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
>    
>    Reviewed-by: shade
>  - Merge tag 'jdk-25+10' into defer-generational-full-gc
>    
>    Added tag jdk-25+10 for changeset a637ccf2
>  - Be less eager to upgrade degen to full gc

Marked as reviewed by wkemper (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23552#pullrequestreview-2638092681

From johannes.bechberger at sap.com  Mon Feb 24 19:56:54 2025
From: johannes.bechberger at sap.com (Bechberger, Johannes)
Date: Mon, 24 Feb 2025 19:56:54 +0000
Subject: Shenandoah and JFR
In-Reply-To: <4ff2582f-b8ba-440a-93cc-b94bd6de65da@amazon.de>
References: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <4ff2582f-b8ba-440a-93cc-b94bd6de65da@amazon.de>
Message-ID: <DB5PR02MB102633F63EBE6730559EE1A45EAC02@DB5PR02MB10263.eurprd02.prod.outlook.com>

Thank you. The issue only occurs on Shenandoah, not with any other GC.

Regards
Johannes

From: Aleksey Shipilev <shipilev at amazon.de>
Date: Monday, 24. February 2025 at 09:19
To: Bechberger, Johannes <johannes.bechberger at sap.com>, shenandoah-dev at openjdk.org <shenandoah-dev at openjdk.org>
Subject: Re: Shenandoah and JFR

On 21.02.25 13:43, Bechberger, Johannes wrote:
> Shenandoah seems to have problems with JFR. I wrote a simple test case that causes assertions errors
> (and core dumps in release) when run together with JFR.
>
> You can find more at https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2Fparttimenerd%2Fd090497e84483202d39821e66f753eb0&data=05%7C02%7Cjohannes.bechberger%40sap.com%7C932f6905b2d243d6376608dd54de4cee%7C42f7676cf455423c82f6dc2d99791af7%7C0%7C0%7C638760035947052602%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=PkORXn4kXeP9Kkm30fCSnc8O0n1rOBRrF6E30Pc9Vyg%3D&reserved=0<https://gist.github.com/parttimenerd/d090497e84483202d39821e66f753eb0> <https://
> gist.github.com/parttimenerd/d090497e84483202d39821e66f753eb0>

Thank you for reproducer!

Filed:
   https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.org%2Fbrowse%2FJDK-8350580&data=05%7C02%7Cjohannes.bechberger%40sap.com%7C932f6905b2d243d6376608dd54de4cee%7C42f7676cf455423c82f6dc2d99791af7%7C0%7C0%7C638760035947075602%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=4DVfd9%2FzEFXXNBGq9hphC46cb0rsodNIs2lpnRW1mMM%3D&reserved=0<https://bugs.openjdk.org/browse/JDK-8350580>

-Aleksey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/shenandoah-dev/attachments/20250224/af5fb0c5/attachment-0001.htm>

From wkemper at openjdk.org  Mon Feb 24 20:54:37 2025
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 24 Feb 2025 20:54:37 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v13]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <7e3ebblQE8huN6LKQWeMxBHqJwaN-H6PangQDk57k4g=.1c9d19d0-22e4-4748-9484-87f52055491a@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
> 
> ## Testing
> * jtreg hotspot_gc_shenandoah
> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Address review feedback (better comments, better names)

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/915ffbda..1d887fcb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=12
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=11-12

  Stats: 56 lines in 9 files changed: 24 ins; 17 del; 15 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Tue Feb 25 01:43:15 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Feb 2025 01:43:15 GMT
Subject: RFR: 8350605: assert(!heap->is_uncommit_in_progress()) failed: Cannot
 uncommit bitmaps while resetting them
Message-ID: <s5DmlLfT-EWw85EQFM_PH0wgGWeWLUMz6C9QiXQAR_4=.c2495042-f478-4d14-8c3c-1eab60ca68c2@github.com>

The protocol which is meant to prevent regions from being uncommitted while their bitmaps are being reset may fail. This happens when the control thread attempts to wait for the uncommit thread to finish, but the uncommit thread has not yet indicated that it has started.

-------------

Commit messages:
 - Use lock to protect in progress flag, remove unnecessary stop lock and flag

Changes: https://git.openjdk.org/jdk/pull/23760/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23760&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8350605
  Stats: 74 lines in 2 files changed: 36 ins; 24 del; 14 mod
  Patch: https://git.openjdk.org/jdk/pull/23760.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23760/head:pull/23760

PR: https://git.openjdk.org/jdk/pull/23760

From azafari at openjdk.org  Tue Feb 25 09:57:26 2025
From: azafari at openjdk.org (Afshin Zafari)
Date: Tue, 25 Feb 2025 09:57:26 GMT
Subject: RFR: 8350566: NMT: add size parameter to
 MemTracker::record_virtual_memory_tag
Message-ID: <LnJ-TGOGBbi8qHrNCBXJQvjCTVBJVhZbsWPCqQvfZj8=.33dc5218-4df8-4255-b35b-b5862759f184@github.com>

With the `size` parameter there will be no need to traverse/go through the nodes between the base and end of the region.
Tests:
linux-x64-debug, gtest:NMT* and runtime/NMT*

-------------

Commit messages:
 - applied also to VMT.
 - 8350566: NMT: add size parameter to MemTracker::record_virtual_memory_tag

Changes: https://git.openjdk.org/jdk/pull/23770/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23770&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8350566
  Stats: 23 lines in 14 files changed: 1 ins; 0 del; 22 mod
  Patch: https://git.openjdk.org/jdk/pull/23770.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23770/head:pull/23770

PR: https://git.openjdk.org/jdk/pull/23770

From shipilev at amazon.de  Tue Feb 25 11:49:00 2025
From: shipilev at amazon.de (Aleksey Shipilev)
Date: Tue, 25 Feb 2025 12:49:00 +0100
Subject: Shenandoah and JFR
In-Reply-To: <DB5PR02MB102633F63EBE6730559EE1A45EAC02@DB5PR02MB10263.eurprd02.prod.outlook.com>
References: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <4ff2582f-b8ba-440a-93cc-b94bd6de65da@amazon.de>
 <DB5PR02MB102633F63EBE6730559EE1A45EAC02@DB5PR02MB10263.eurprd02.prod.outlook.com>
Message-ID: <accd1013-8c9c-488c-bb67-ed0d47f3caf6@amazon.de>

Hi Johannes,

On 24.02.25 20:56, Bechberger, Johannes wrote:
> Thank you. The issue only occurs on Shenandoah, not with any other GC.

Your original Gist says: "This works for the current JDK head, JDK 23 and JDK 21. So I would assume 
that it affects all JDKs that include the Shenandoah GC at the moment of writing."

Could you double-check the issue reproduces on JDK 21 for you? I found a bug that affects JDK 25:
  https://bugs.openjdk.org/browse/JDK-8350580?focusedId=14756096#comment-14756096

...and my prospective fix resolves JDK 25 crash. But the reproducer runs fine on current jdk21u-dev.

Thanks,
-Aleksey


From johannes.bechberger at sap.com  Tue Feb 25 11:53:01 2025
From: johannes.bechberger at sap.com (Bechberger, Johannes)
Date: Tue, 25 Feb 2025 11:53:01 +0000
Subject: Shenandoah and JFR
In-Reply-To: <accd1013-8c9c-488c-bb67-ed0d47f3caf6@amazon.de>
References: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <4ff2582f-b8ba-440a-93cc-b94bd6de65da@amazon.de>
 <DB5PR02MB102633F63EBE6730559EE1A45EAC02@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <accd1013-8c9c-488c-bb67-ed0d47f3caf6@amazon.de>
Message-ID: <DB5PR02MB10263878267FE2D4A82E6280AEAC32@DB5PR02MB10263.eurprd02.prod.outlook.com>

Hi Aleksey,

I didn?t check JDK 23?s and JDK 21?s head, just the latest released version.
I?m happy to check this later in the day.

Regards
Johannes

From: Aleksey Shipilev <shipilev at amazon.de>
Date: Tuesday, 25. February 2025 at 06:49
To: Bechberger, Johannes <johannes.bechberger at sap.com>, shenandoah-dev at openjdk.org <shenandoah-dev at openjdk.org>
Subject: Re: Shenandoah and JFR
Hi Johannes,

On 24.02.25 20:56, Bechberger, Johannes wrote:
> Thank you. The issue only occurs on Shenandoah, not with any other GC.

Your original Gist says: "This works for the current JDK head, JDK 23 and JDK 21. So I would assume
that it affects all JDKs that include the Shenandoah GC at the moment of writing."

Could you double-check the issue reproduces on JDK 21 for you? I found a bug that affects JDK 25:
  https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.org%2Fbrowse%2FJDK-8350580%3FfocusedId%3D14756096%23comment-14756096&data=05%7C02%7Cjohannes.bechberger%40sap.com%7Ce55d97d56615412d279308dd55926b9e%7C42f7676cf455423c82f6dc2d99791af7%7C0%7C0%7C638760809550577176%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6hSBngvEa%2F0EdqLVwcFZPG5g9zkxMKnFbP0FdOk90%2F4%3D&reserved=0<https://bugs.openjdk.org/browse/JDK-8350580?focusedId=14756096#comment-14756096>

...and my prospective fix resolves JDK 25 crash. But the reproducer runs fine on current jdk21u-dev.

Thanks,
-Aleksey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/shenandoah-dev/attachments/20250225/23e3a72a/attachment.htm>

From johannes.bechberger at sap.com  Tue Feb 25 12:33:56 2025
From: johannes.bechberger at sap.com (Bechberger, Johannes)
Date: Tue, 25 Feb 2025 12:33:56 +0000
Subject: Shenandoah and JFR
In-Reply-To: <DB5PR02MB10263878267FE2D4A82E6280AEAC32@DB5PR02MB10263.eurprd02.prod.outlook.com>
References: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <4ff2582f-b8ba-440a-93cc-b94bd6de65da@amazon.de>
 <DB5PR02MB102633F63EBE6730559EE1A45EAC02@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <accd1013-8c9c-488c-bb67-ed0d47f3caf6@amazon.de>
 <DB5PR02MB10263878267FE2D4A82E6280AEAC32@DB5PR02MB10263.eurprd02.prod.outlook.com>
Message-ID: <DB5PR02MB1026372CE228593AE3D8EE186EAC32@DB5PR02MB10263.eurprd02.prod.outlook.com>

Hi Aleksey,

I only ever tested it on a 128 core machine (64 CPUs with hyperthreading). It doesn?t seem to fail on my smaller machine, which I currently only have access to, maybe because of the lacking parallelism. I?ll be running it for a while now.

Regards
Johannes

From: shenandoah-dev <shenandoah-dev-retn at openjdk.org> on behalf of Bechberger, Johannes <johannes.bechberger at sap.com>
Date: Tuesday, 25. February 2025 at 06:53
To: Aleksey Shipilev <shipilev at amazon.de>, shenandoah-dev at openjdk.org <shenandoah-dev at openjdk.org>
Subject: Re: Shenandoah and JFR
Hi Aleksey,

I didn?t check JDK 23?s and JDK 21?s head, just the latest released version.
I?m happy to check this later in the day.

Regards
Johannes

From: Aleksey Shipilev <shipilev at amazon.de>
Date: Tuesday, 25. February 2025 at 06:49
To: Bechberger, Johannes <johannes.bechberger at sap.com>, shenandoah-dev at openjdk.org <shenandoah-dev at openjdk.org>
Subject: Re: Shenandoah and JFR
Hi Johannes,

On 24.02.25 20:56, Bechberger, Johannes wrote:
> Thank you. The issue only occurs on Shenandoah, not with any other GC.

Your original Gist says: "This works for the current JDK head, JDK 23 and JDK 21. So I would assume
that it affects all JDKs that include the Shenandoah GC at the moment of writing."

Could you double-check the issue reproduces on JDK 21 for you? I found a bug that affects JDK 25:
  https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.org%2Fbrowse%2FJDK-8350580%3FfocusedId%3D14756096%23comment-14756096&data=05%7C02%7Cjohannes.bechberger%40sap.com%7Ce55d97d56615412d279308dd55926b9e%7C42f7676cf455423c82f6dc2d99791af7%7C0%7C0%7C638760809550577176%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6hSBngvEa%2F0EdqLVwcFZPG5g9zkxMKnFbP0FdOk90%2F4%3D&reserved=0<https://bugs.openjdk.org/browse/JDK-8350580?focusedId=14756096#comment-14756096>

...and my prospective fix resolves JDK 25 crash. But the reproducer runs fine on current jdk21u-dev.

Thanks,
-Aleksey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/shenandoah-dev/attachments/20250225/90c0fc6c/attachment-0001.htm>

From shipilev at amazon.de  Tue Feb 25 12:56:14 2025
From: shipilev at amazon.de (Aleksey Shipilev)
Date: Tue, 25 Feb 2025 13:56:14 +0100
Subject: Shenandoah and JFR
In-Reply-To: <DB5PR02MB10263878267FE2D4A82E6280AEAC32@DB5PR02MB10263.eurprd02.prod.outlook.com>
References: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <4ff2582f-b8ba-440a-93cc-b94bd6de65da@amazon.de>
 <DB5PR02MB102633F63EBE6730559EE1A45EAC02@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <accd1013-8c9c-488c-bb67-ed0d47f3caf6@amazon.de>
 <DB5PR02MB10263878267FE2D4A82E6280AEAC32@DB5PR02MB10263.eurprd02.prod.outlook.com>
Message-ID: <924d2b6d-b383-4f49-9548-9ad373a37cec@amazon.de>


On 25.02.25 12:53, Bechberger, Johannes wrote:
> I didn?t check JDK 23?s and JDK 21?s head, just the latest released version.
> I?m happy to check this later in the day.

Ah. I now tested with {21.0.5-ga, 21.0.6-ga} x {fastdebug, release} from jdk21u-dev, and reproducer 
still does not crash. I still suspect there is a low-frequency bug there, so it would give us a 
signal if it still crashes for you.

Thanks,
-Aleksey

From johannes.bechberger at sap.com  Tue Feb 25 20:30:38 2025
From: johannes.bechberger at sap.com (Bechberger, Johannes)
Date: Tue, 25 Feb 2025 20:30:38 +0000
Subject: Shenandoah and JFR
In-Reply-To: <924d2b6d-b383-4f49-9548-9ad373a37cec@amazon.de>
References: <DB5PR02MB102631A8BD2B978E387CA49B9EAC72@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <4ff2582f-b8ba-440a-93cc-b94bd6de65da@amazon.de>
 <DB5PR02MB102633F63EBE6730559EE1A45EAC02@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <accd1013-8c9c-488c-bb67-ed0d47f3caf6@amazon.de>
 <DB5PR02MB10263878267FE2D4A82E6280AEAC32@DB5PR02MB10263.eurprd02.prod.outlook.com>
 <924d2b6d-b383-4f49-9548-9ad373a37cec@amazon.de>
Message-ID: <DB5PR02MB10263C39567B2B22F3E0353C2EAC32@DB5PR02MB10263.eurprd02.prod.outlook.com>

It didn?t fail for half a day on my 16 core machine. So it requires the large amount of cores.

Regards
Johannes

From: Aleksey Shipilev <shipilev at amazon.de>
Date: Tuesday, 25. February 2025 at 07:56
To: Bechberger, Johannes <johannes.bechberger at sap.com>, shenandoah-dev at openjdk.org <shenandoah-dev at openjdk.org>
Subject: Re: Shenandoah and JFR

On 25.02.25 12:53, Bechberger, Johannes wrote:
> I didn?t check JDK 23?s and JDK 21?s head, just the latest released version.
> I?m happy to check this later in the day.

Ah. I now tested with {21.0.5-ga, 21.0.6-ga} x {fastdebug, release} from jdk21u-dev, and reproducer
still does not crash. I still suspect there is a low-frequency bug there, so it would give us a
signal if it still crashes for you.

Thanks,
-Aleksey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/shenandoah-dev/attachments/20250225/14822a29/attachment.htm>

From wkemper at openjdk.org  Tue Feb 25 22:06:19 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Feb 2025 22:06:19 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v14]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <XaiJPiRLQkihIwVZe2tTxxKHoDQeyOgdw8nZvQEPoh8=.e85b90c5-ecef-490b-94e4-05c22cb28012@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
> 
> ## Testing
> * jtreg hotspot_gc_shenandoah
> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits:

 - Merge tag 'jdk-25+11' into fix-control-regulator-threads
   
   Added tag jdk-25+11 for changeset 0131c1bf
 - Address review feedback (better comments, better names)
 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - Old gen bootstrap cycle must make it to init mark
 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - Improve message for assertion
 - Make shutdown safer for threads requesting (or expecting) gc
 - Do not accept requests if control thread is terminating
 - Notify waiters when control thread terminates
 - Add event for control thread state changes
 - ... and 22 more: https://git.openjdk.org/jdk/compare/0131c1bf...d7858deb

-------------

Changes: https://git.openjdk.org/jdk/pull/23475/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=13
  Stats: 927 lines in 18 files changed: 302 ins; 291 del; 334 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From shade at openjdk.org  Tue Feb 25 22:47:16 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 25 Feb 2025 22:47:16 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <fHTqIoD6NUaf1X47jvCf3sibEr6tYnV9_AhacSs3_JU=.d895b8a9-86cd-49e2-8e57-ac01bd78a004@github.com>

On Tue, 25 Feb 2025 01:20:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> The change is to improve the observability of  Shenandoah GC, basically there are two changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (us):  15,   1,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Weak Roots               413 us
> [11.057s][info][gc,stats  ...

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 308:

> 306: 
> 307:   op_init_mark();
> 308:   ShenandoahHeap::heap()->propagate_gc_state_to_all_threads();

I would say move it downwards into `op_init_mark` and wrap with its own subtimer, e.g.  with `ShenandoahGCPhase phase(ShenandoahPhaseTimings::init_propagate_gcstate);`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1969328009

From xpeng at openjdk.org  Tue Feb 25 22:47:16 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Feb 2025 22:47:16 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings
Message-ID: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>

The change is to improve the observability of  Shenandoah GC, basically there are two changes for  Shenandoah GC timings in this PR:

1. Net GC pause timings include the time to propagate GC state to Java threads
2. Add new timing "Propagate GC state" in Shenandoah GC timing logs

With the change, the new GC timing log will be like:

[11.056s][info][gc,stats    ] Concurrent Reset                     89 us
[11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
[11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
[11.056s][info][gc,stats    ]   Update Region States                3 us
[11.056s][info][gc,stats    ]   Propagate GC state                  1 us
[11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
[11.056s][info][gc,stats    ]   CMR: <total>                      456 us
[11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
[11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
[11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
[11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
[11.057s][info][gc,stats    ]   CM: <total>                      3043 us
[11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
[11.057s][info][gc,stats    ]   Flush SATB                        204 us
[11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
[11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
[11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
[11.057s][info][gc,stats    ]   Propagate GC state                  2 us
[11.057s][info][gc,stats    ]   Update Region States               12 us
[11.057s][info][gc,stats    ]   Choose Collection Set              25 us
[11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
[11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
[11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
[11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (us):  15,   1,   0, ---, ---, ---, 
[11.057s][info][gc,stats    ] Concurrent Weak Roots               413 us
[11.057s][info][gc,stats    ]   Roots                             203 us, parallelism: 1.95x
[11.057s][info][gc,stats    ]     CWR: <total>                    396 us
[11.057s][info][gc,stats    ]     CWR: Code Cache Roots           295 us, workers (us):  90,  96, 109, ---, ---, ---, 
[11.057s][info][gc,stats    ]     CWR: VM Weak Roots              100 us, workers (us):  48,  37,  14, ---, ---, ---, 
[11.057s][info][gc,stats    ]     CWR: CLDG Roots                   2 us, workers (us): ---, ---,   2, ---, ---, ---, 
[11.058s][info][gc,stats    ]   Rendezvous                        197 us
[11.058s][info][gc,stats    ] Concurrent Cleanup                   35 us
[11.058s][info][gc,stats    ] Concurrent Class Unloading          486 us
[11.058s][info][gc,stats    ]   Unlink Stale                      398 us
[11.058s][info][gc,stats    ]     System Dictionary                 5 us
[11.058s][info][gc,stats    ]     Weak Class Links                  0 us
[11.058s][info][gc,stats    ]     Code Roots                      391 us
[11.058s][info][gc,stats    ]   Rendezvous                         69 us
[11.058s][info][gc,stats    ]   Purge Unlinked                      4 us
[11.058s][info][gc,stats    ]     Code Roots                        0 us
[11.058s][info][gc,stats    ]     CLDG                              3 us
[11.058s][info][gc,stats    ] Pause Final Roots (G)               272 us
[11.058s][info][gc,stats    ] Pause Final Roots (N)                18 us
[11.058s][info][gc,stats    ]   Propagate GC state                  3 us



### Test
- [x] make test TEST=hotspot_gc_shenandoah

-------------

Commit messages:
 - Log time to propagate GC state
 - 8350314: Shenandoah: Capture thread state sync times in GC timings

Changes: https://git.openjdk.org/jdk/pull/23759/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8350314
  Stats: 47 lines in 5 files changed: 40 ins; 7 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23759.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23759/head:pull/23759

PR: https://git.openjdk.org/jdk/pull/23759

From wkemper at openjdk.org  Tue Feb 25 23:03:53 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Feb 2025 23:03:53 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <P1Cve-O-H8xvnkErjXPPXKz7MISeiaHg4URyTCgHGrw=.fc891fc1-fcde-4d6d-af63-a0d298729f73@github.com>

On Tue, 25 Feb 2025 01:20:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> The change is to improve the observability of  Shenandoah GC, basically there are two changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (us):  15,   1,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Weak Roots               413 us
> [11.057s][info][gc,stats  ...

Did we really see `propagate_gc_state_to_all_threads` taking a long time? Or was it exiting the safepoint (i.e., after the state had been propagated) that took a long time?

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 694:

> 692:   {
> 693:     ShenandoahGCPhase phase(ShenandoahPhaseTimings::init_propagate_gc_state);
> 694:     ShenandoahHeap::heap()->propagate_gc_state_to_all_threads();

A nit, but can we use the `heap` variable that is already in scope for these `propagate_gc_state_to_all_threads` calls?

-------------

Changes requested by wkemper (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23759#pullrequestreview-2642654296
PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970660730

From xpeng at openjdk.org  Tue Feb 25 23:22:56 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Feb 2025 23:22:56 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings
In-Reply-To: <P1Cve-O-H8xvnkErjXPPXKz7MISeiaHg4URyTCgHGrw=.fc891fc1-fcde-4d6d-af63-a0d298729f73@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <P1Cve-O-H8xvnkErjXPPXKz7MISeiaHg4URyTCgHGrw=.fc891fc1-fcde-4d6d-af63-a0d298729f73@github.com>
Message-ID: <lV7NgrOmayYeneEdNFi6yju_dqrtw7_7nXQSl3tPYao=.39130214-9be9-4bc6-97d0-251b24a07fb6@github.com>

On Tue, 25 Feb 2025 23:01:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Did we really see `propagate_gc_state_to_all_threads` taking a long time? Or was it exiting the safepoint (i.e., after the state had been propagated) that took a long time?

No, we discussed it last week in Slack channel, `propagate_gc_state_to_all_threads` usually takes less than 10 ns for ~1k threads in our test, it not a problem. 

it was the `futex` call when exit the safepoint took long time, we have a [fix](https://bugs.openjdk.org/browse/JDK-8350285) in ShenandoahLock for mitigation since the change we made last year affects scheduler, meanwhile Aleksey is working on https://bugs.openjdk.org/browse/JDK-8350324 which should improve the time to leave safepoint which has much broader impact.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23759#issuecomment-2683510642

From xpeng at openjdk.org  Tue Feb 25 23:27:22 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Feb 2025 23:27:22 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v2]
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <rBC3NQmTWNRvveicXt6frMMwV_uzn4q5otzUI30T068=.7696570e-1e5e-447d-bce8-ba32c38ff4f5@github.com>

> The change is to improve the observability of  Shenandoah GC, basically there are two changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (us):  15,   1,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Weak Roots               413 us
> [11.057s][info][gc,stats  ...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Address review comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23759/files
  - new: https://git.openjdk.org/jdk/pull/23759/files/4940e451..cad4a434

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=00-01

  Stats: 7 lines in 3 files changed: 0 ins; 0 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/23759.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23759/head:pull/23759

PR: https://git.openjdk.org/jdk/pull/23759

From xpeng at openjdk.org  Tue Feb 25 23:27:22 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Feb 2025 23:27:22 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v2]
In-Reply-To: <P1Cve-O-H8xvnkErjXPPXKz7MISeiaHg4URyTCgHGrw=.fc891fc1-fcde-4d6d-af63-a0d298729f73@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <P1Cve-O-H8xvnkErjXPPXKz7MISeiaHg4URyTCgHGrw=.fc891fc1-fcde-4d6d-af63-a0d298729f73@github.com>
Message-ID: <UR0AIf9RTo16h85E4N3oOoDMc2Sgjn-Bo7nAsLBGqwc=.74048011-d7ea-43a2-b4a7-288be7205af7@github.com>

On Tue, 25 Feb 2025 23:00:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Address review comments
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 694:
> 
>> 692:   {
>> 693:     ShenandoahGCPhase phase(ShenandoahPhaseTimings::init_propagate_gc_state);
>> 694:     ShenandoahHeap::heap()->propagate_gc_state_to_all_threads();
> 
> A nit, but can we use the `heap` variable that is already in scope for these `propagate_gc_state_to_all_threads` calls?

Thanks, I have updated the PR to use `heap` variable in scope.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970679549

From wkemper at openjdk.org  Tue Feb 25 23:49:54 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Feb 2025 23:49:54 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v2]
In-Reply-To: <XAVHiEDQWPMilVhMpFeb87wvajX1dto6ItE2DU-NwdU=.d0c183f8-21aa-4b66-93d3-c3e4a82cfb5f@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <rBC3NQmTWNRvveicXt6frMMwV_uzn4q5otzUI30T068=.7696570e-1e5e-447d-bce8-ba32c38ff4f5@github.com>
 <XAVHiEDQWPMilVhMpFeb87wvajX1dto6ItE2DU-NwdU=.d0c183f8-21aa-4b66-93d3-c3e4a82cfb5f@github.com>
Message-ID: <e1XVM1dbioXUmw5tCkgKeVQAFwshIJqlc-YuKLPj7WM=.8b32fbbf-cabd-4204-a723-f29793723767@github.com>

On Tue, 25 Feb 2025 23:45:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Address review comments
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 1081:
> 
>> 1079:     heap->pacer()->setup_for_update_refs();
>> 1080:   }
>> 1081:   {
> 
> This one isn't necessary. This safepoint only happens when the pacer or the verifier is enabled. We moved init update ref gc state propagation into a handshake (see `ShenandoahHeap::concurrent_prepare_for_update_refs`). Sorry, I should have caught this in my first review.

I'm also currently working on removing the `final_roots` safepoint and fixing up the phase timings for the newish concurrent init update refs phase.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970696214

From wkemper at openjdk.org  Tue Feb 25 23:49:53 2025
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Feb 2025 23:49:53 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v2]
In-Reply-To: <rBC3NQmTWNRvveicXt6frMMwV_uzn4q5otzUI30T068=.7696570e-1e5e-447d-bce8-ba32c38ff4f5@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <rBC3NQmTWNRvveicXt6frMMwV_uzn4q5otzUI30T068=.7696570e-1e5e-447d-bce8-ba32c38ff4f5@github.com>
Message-ID: <XAVHiEDQWPMilVhMpFeb87wvajX1dto6ItE2DU-NwdU=.d0c183f8-21aa-4b66-93d3-c3e4a82cfb5f@github.com>

On Tue, 25 Feb 2025 23:27:22 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are two changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (us):  15,   1,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concu...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address review comments

Propagating gc state for init update refs is vestigial.

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 1081:

> 1079:     heap->pacer()->setup_for_update_refs();
> 1080:   }
> 1081:   {

This one isn't necessary. This safepoint only happens when the pacer or the verifier is enabled. We moved init update ref gc state propagation into a handshake (see `ShenandoahHeap::concurrent_prepare_for_update_refs`). Sorry, I should have caught this in my first review.

-------------

Changes requested by wkemper (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23759#pullrequestreview-2642707574
PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970695230

From xpeng at openjdk.org  Wed Feb 26 00:05:03 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 00:05:03 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v2]
In-Reply-To: <e1XVM1dbioXUmw5tCkgKeVQAFwshIJqlc-YuKLPj7WM=.8b32fbbf-cabd-4204-a723-f29793723767@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <rBC3NQmTWNRvveicXt6frMMwV_uzn4q5otzUI30T068=.7696570e-1e5e-447d-bce8-ba32c38ff4f5@github.com>
 <XAVHiEDQWPMilVhMpFeb87wvajX1dto6ItE2DU-NwdU=.d0c183f8-21aa-4b66-93d3-c3e4a82cfb5f@github.com>
 <e1XVM1dbioXUmw5tCkgKeVQAFwshIJqlc-YuKLPj7WM=.8b32fbbf-cabd-4204-a723-f29793723767@github.com>
Message-ID: <zCzBjl7lGnt7Pf5vgJYbONdC1EWelP0sL25m2Ldg9lo=.123a0b82-4daa-48dc-84ab-91bae525e85d@github.com>

On Tue, 25 Feb 2025 23:46:49 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 1081:
>> 
>>> 1079:     heap->pacer()->setup_for_update_refs();
>>> 1080:   }
>>> 1081:   {
>> 
>> This one isn't necessary. This safepoint only happens when the pacer or the verifier is enabled. We moved init update ref gc state propagation into a handshake (see `ShenandoahHeap::concurrent_prepare_for_update_refs`). Sorry, I should have caught this in my first review.
>
> I'm also currently working on removing the `final_roots` safepoint and fixing up the phase timings for the newish concurrent init update refs phase.

Sorry I should have read the code in `op_init_update_refs`, it doesn't change GC state, we can remove the the `propagate_gc_state_to_all_threads` to clean up code a little bit after you replaced the init_update_refs pause with handshake.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970706937

From xpeng at openjdk.org  Wed Feb 26 00:08:57 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 00:08:57 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v2]
In-Reply-To: <zCzBjl7lGnt7Pf5vgJYbONdC1EWelP0sL25m2Ldg9lo=.123a0b82-4daa-48dc-84ab-91bae525e85d@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <rBC3NQmTWNRvveicXt6frMMwV_uzn4q5otzUI30T068=.7696570e-1e5e-447d-bce8-ba32c38ff4f5@github.com>
 <XAVHiEDQWPMilVhMpFeb87wvajX1dto6ItE2DU-NwdU=.d0c183f8-21aa-4b66-93d3-c3e4a82cfb5f@github.com>
 <e1XVM1dbioXUmw5tCkgKeVQAFwshIJqlc-YuKLPj7WM=.8b32fbbf-cabd-4204-a723-f29793723767@github.com>
 <zCzBjl7lGnt7Pf5vgJYbONdC1EWelP0sL25m2Ldg9lo=.123a0b82-4daa-48dc-84ab-91bae525e85d@github.com>
Message-ID: <bmEHZYxjD1AzpQUJLFz_n1CMtcq7mMbohd9wCF9j_FI=.6ed2d101-71c0-4ff1-873c-618ded13af51@github.com>

On Wed, 26 Feb 2025 00:02:45 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> I'm also currently working on removing the `final_roots` safepoint and fixing up the phase timings for the newish concurrent init update refs phase.
>
> Sorry I should have read the code in `op_init_update_refs`, it doesn't change GC state, we can remove the the call of `propagate_gc_state_to_all_threads` to clean up code a little bit, it was missed when you replaced the init_update_refs pause with handshake.

For `final_roots`, I think I should leave it as it is in this PR, later you will remove the timings and gc state propagation into a handshake anyway.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970708588

From wkemper at openjdk.org  Wed Feb 26 00:34:51 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Feb 2025 00:34:51 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v3]
In-Reply-To: <cYv8kf7drUHAMJiLBb-oquj_nUIkgDBPcWwwewUNb5o=.feefcad5-0ddf-42c9-aee5-cc0724afc4cd@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <cYv8kf7drUHAMJiLBb-oquj_nUIkgDBPcWwwewUNb5o=.feefcad5-0ddf-42c9-aee5-cc0724afc4cd@github.com>
Message-ID: <0kZCj6U-om8Gz_iMeXiQf1jSitVjhTvy2PYwLjllvGM=.5321e4f5-7e39-4d5f-adb4-4b41cf23dce8@github.com>

On Wed, 26 Feb 2025 00:01:12 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are two changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (us):  15,   1,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concu...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove propagate_gc_state_to_all_threads call from op_init_update_refs

Thank you for the changes.

-------------

Marked as reviewed by wkemper (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23759#pullrequestreview-2642754796

From wkemper at openjdk.org  Wed Feb 26 00:34:52 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Feb 2025 00:34:52 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v2]
In-Reply-To: <bmEHZYxjD1AzpQUJLFz_n1CMtcq7mMbohd9wCF9j_FI=.6ed2d101-71c0-4ff1-873c-618ded13af51@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <rBC3NQmTWNRvveicXt6frMMwV_uzn4q5otzUI30T068=.7696570e-1e5e-447d-bce8-ba32c38ff4f5@github.com>
 <XAVHiEDQWPMilVhMpFeb87wvajX1dto6ItE2DU-NwdU=.d0c183f8-21aa-4b66-93d3-c3e4a82cfb5f@github.com>
 <e1XVM1dbioXUmw5tCkgKeVQAFwshIJqlc-YuKLPj7WM=.8b32fbbf-cabd-4204-a723-f29793723767@github.com>
 <zCzBjl7lGnt7Pf5vgJYbONdC1EWelP0sL25m2Ldg9lo=.123a0b82-4daa-48dc-84ab-91bae525e85d@github.com>
 <bmEHZYxjD1AzpQUJLFz_n1CMtcq7mMbohd9wCF9j_FI=.6ed2d101-71c0-4ff1-873c-618ded13af51@github.com>
Message-ID: <UzmCKYQVhsGf0qxcapOggE_QzNOgsKlV2j9fufaEY08=.f473ad5d-acd7-45f5-9795-2328ccc7218c@github.com>

On Wed, 26 Feb 2025 00:05:12 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Sorry I should have read the code in `op_init_update_refs`, it doesn't change GC state, we can remove the the call of `propagate_gc_state_to_all_threads` to clean up code a little bit, it was missed when you replaced the init_update_refs pause with handshake.
>
> For `final_roots`, I think I should leave it as it is in this PR, later you will remove the timings and gc state propagation into a handshake anyway.

Yep, I agree. Thank you!

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970726624

From ysr at openjdk.org  Wed Feb 26 01:32:03 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Feb 2025 01:32:03 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v14]
In-Reply-To: <XaiJPiRLQkihIwVZe2tTxxKHoDQeyOgdw8nZvQEPoh8=.e85b90c5-ecef-490b-94e4-05c22cb28012@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <XaiJPiRLQkihIwVZe2tTxxKHoDQeyOgdw8nZvQEPoh8=.e85b90c5-ecef-490b-94e4-05c22cb28012@github.com>
Message-ID: <RRJ21SxAIgKaPbCPiXQ3C49SdUPomhplllsORFubgkc=.12ad60c2-1071-42b3-93fd-90a467b1fd45@github.com>

On Tue, 25 Feb 2025 22:06:19 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are several changes to the operation of Shenandoah's control threads here.
>> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>>   *  The cancellation handling is driven entirely by the cancellation cause
>>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>>   * The shutdown sequence is simpler
>> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
>> 
>> ## Testing
>> * jtreg hotspot_gc_shenandoah
>> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits:
> 
>  - Merge tag 'jdk-25+11' into fix-control-regulator-threads
>    
>    Added tag jdk-25+11 for changeset 0131c1bf
>  - Address review feedback (better comments, better names)
>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>  - Old gen bootstrap cycle must make it to init mark
>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>  - Improve message for assertion
>  - Make shutdown safer for threads requesting (or expecting) gc
>  - Do not accept requests if control thread is terminating
>  - Notify waiters when control thread terminates
>  - Add event for control thread state changes
>  - ... and 22 more: https://git.openjdk.org/jdk/compare/0131c1bf...d7858deb

A few random comments, mostly on documentation, and a few assertion suggestions.

Rest looks good. Thanks for tightening up the code via your changes in this PR.

I don't think I should need to re-review any changes you make stemming from my (rally quite minor) comments.

Reviewed; ?

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 114:

> 112: }
> 113: 
> 114: void ShenandoahGenerationalControlThread::check_for_request(ShenandoahGCRequest& request) {

Comment:

// Fill in the cause, generation requested, and set gc_mode, all under the lock.
// Make any relevant changes to the control state of heuristics or policy objects.


Just as an aside, these internal work methods hve cascading effects on a bunch of states, all covered by the control lock. It would be interesting to see which threads contend for this lock and whether there are circumstances under which the lock might become hot of contended.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 125:

> 123:     if (request.cause == GCCause::_shenandoah_concurrent_gc) {
> 124:       request.generation = _heap->young_generation();
> 125:       _heap->clear_cancelled_gc(false);

label name of parameter to the call to help the reader.  `/* clear_oom_handler */`

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 717:

> 715: 
> 716: void ShenandoahGenerationalControlThread::notify_control_thread(MonitorLocker& ml, GCCause::Cause cause, ShenandoahGeneration* generation) {
> 717:   assert(_control_lock.is_locked(), "Request lock must be held here");

Somewhat paranoid suggestion: I'd use the stronger, if slightly more expensive, `owned_by_self()` rather than the weaker `is_locked()`. You can alternatively use `assert_lock_strong(...)`.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 720:

> 718:   log_debug(gc, thread)("Notify control (%s): %s, %s", gc_mode_name(gc_mode()), GCCause::to_string(cause), generation->name());
> 719:   _requested_gc_cause = cause;
> 720:   _requested_generation = generation;

This _may_ be a good spot to add any potential invariant (sanity) checks for the consistency of `cause` and `generation` to catch any potential issues.

For example, I might check I am not overwriting legit previous requests (?), and that any new request I am creating are sensible.

Such assertion/invariant checks may be relegated into a work method to avoid cluttering this method.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 730:

> 728: 
> 729: void ShenandoahGenerationalControlThread::notify_cancellation(MonitorLocker& ml, GCCause::Cause cause) {
> 730:   assert(_heap->cancelled_gc(), "GC should already be cancelled");

Not sure about this, but if cancellations can happen only when `request*` fields are clear (or may be not?), then this would be a good place to do such invariant checks (much as the suggestion above  in `notify_control_thread()`.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 795:

> 793:   if (_mode != new_mode) {
> 794:     log_debug(gc, thread)("Transition from: %s to: %s", gc_mode_name(_mode), gc_mode_name(new_mode));
> 795:     EventMark event("Control thread transition from: %s, to %s", gc_mode_name(_mode), gc_mode_name(new_mode));

As in my suggestions above for `notify_control_thread` and `notify_cancellation`, this might be an opportune time to do any sanity/consistency checks for `gc_mode` updates (wrt the other two `_request*` fields.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 75:

> 73: 
> 74:   // The mode is read frequently by requesting threads and only ever written by the control thread.
> 75:   volatile GCMode _mode;

A bit of a nit: Any reason not to just call it `_gc_mode` which seems now to be its _de facto_ name due to its accessor method name and how it's referred to in comments?

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 132:

> 130: 
> 131:   void set_gc_mode(GCMode new_mode);
> 132:   void set_gc_mode(MonitorLocker& ml, GCMode new_mode);

// Set gc mode under lock and post a notification. The second variant is called from
// contexts where the lock is already held.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 135:

> 133:   static const char* gc_mode_name(GCMode mode);
> 134: 
> 135:   // Takes the request lock and updates the requested cause and generation, then notifies the control thread.

// The second variant is used from contexts where the lock is already held.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 139:

> 137:   void notify_control_thread(MonitorLocker& ml, GCCause::Cause cause, ShenandoahGeneration* generation);
> 138: 
> 139:   // Notifies the control thread, but does not update the requested cause or generation.

```// The second variant ...```

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 143:

> 141:   void notify_cancellation(MonitorLocker& ml, GCCause::Cause cause);
> 142: 
> 143:   void maybe_set_aging_cycle();

1 line documentation comment for the private APIs.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 150:

> 148:   GCMode prepare_for_explicit_gc_request(ShenandoahGCRequest &request);
> 149: 
> 150:   GCMode prepare_for_concurrent_gc_request(ShenandoahGCRequest &request);

Documentation of private APIs.

Bit of a nit: These all seem to take a request type, fill in some yet unfilled fields, and return a GC mode. So they look to me like `prepare_request_for_<action>` rather than `pepare_for_<action>_request`.

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 443:

> 441: 
> 442: public:
> 443:   // Returns true if and only if cancellation request was successfully communicated.

This comment should probably go at line 455, and here you could say ```// Has GC been cancelled?```

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 687:

> 685:   HeapWord* allocate_from_gclab_slow(Thread* thread, size_t size);
> 686:   HeapWord* allocate_new_gclab(size_t min_size, size_t word_size, size_t* actual_size);
> 687:   bool retry_allocation(size_t original_full_gc_count) const;

It feels like this should be called `should_retry_allocation()`. Also a 1-line documentation comment: ``` // We want to retry an unsuccessful attempt at allocation until at least a full gc. ```

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23475#pullrequestreview-2642645029
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970692572
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970689728
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970730527
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970747110
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970750529
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970752575
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970669392
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970673227
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970754002
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970754196
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970674398
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970678568
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970758140
PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970761571

From ysr at openjdk.org  Wed Feb 26 01:32:04 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Feb 2025 01:32:04 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <jmjBtpeXYGIWT0rGEH0SaqQebqY1_z4q5x1ou4Rublw=.d4124b39-00bb-4953-a6bf-419084c5d749@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
 <jmjBtpeXYGIWT0rGEH0SaqQebqY1_z4q5x1ou4Rublw=.d4124b39-00bb-4953-a6bf-419084c5d749@github.com>
Message-ID: <X0n96lN_PrBh871WowoE0nfQJag_lGvaKcv_siYM1bs=.4dc6df74-99df-44e5-9a50-2def2d959e63@github.com>

On Mon, 24 Feb 2025 17:50:38 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 64:
>> 
>>> 62: private:
>>> 63:   // This lock is used to coordinate setting the _requested_gc_cause, _requested generation
>>> 64:   // and _gc_mode. It is important that these be changed together and have a consistent view.
>> 
>> In that case, for ease of maintenance, I'd move the declaration of all of the 3 data members that this lock protects next to this lock, either immediately preceding or immediately succeeding its declaration in the body of this class.
>> 
>> Are these data members always both read and written under this lock? If so, then `_gc_mode` below doesn't need to be defined `volatile`.
>
> The `_gc_mode` is read without the lock by the regulator thread. However, the regulator thread does take the lock and reads `_gc_mode` again under the lock before making any state changes.

Make sense -- so the regulator uses a double-checked locking idiom. May be note that somewhere where you talk about this lock and the volatile _gc_mode.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970654476

From ysr at openjdk.org  Wed Feb 26 01:32:05 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Feb 2025 01:32:05 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v12]
In-Reply-To: <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <rN13UmcgwnyWyTNOsDcU7tJnW13wzBZ9OzZyIYoZ1F8=.7102a26a-27f5-46c8-ae0b-cd60802e60f8@github.com>
 <XcHnktNvgsV-8y89jJwGEOS6TCwysDt_dC4Wxyze6bQ=.0434ecbe-394a-479b-8ff1-aa7cc4a4f8c2@github.com>
Message-ID: <zEnd5fEo-kKKFC3b7JzBrGfDf8nEbdyC8C_TSCi_XpU=.5ef1d1e3-db6d-48cb-8a33-63fd402fa3a0@github.com>

On Sat, 15 Feb 2025 00:34:59 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits:
>> 
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - Fix shutdown livelock error
>>  - Fix includes
>>  - ... and 20 more: https://git.openjdk.org/jdk/compare/ba6c9659...915ffbda
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 450:
> 
>> 448: 
>> 449:   void cancel_concurrent_mark();
>> 450:   bool cancel_gc(GCCause::Cause cause);
> 
> // Returns true if and only if cancellation request was successfully communicated.

See new comment at line 443 above.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970759318

From ysr at openjdk.org  Wed Feb 26 01:32:06 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Feb 2025 01:32:06 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v14]
In-Reply-To: <RRJ21SxAIgKaPbCPiXQ3C49SdUPomhplllsORFubgkc=.12ad60c2-1071-42b3-93fd-90a467b1fd45@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <XaiJPiRLQkihIwVZe2tTxxKHoDQeyOgdw8nZvQEPoh8=.e85b90c5-ecef-490b-94e4-05c22cb28012@github.com>
 <RRJ21SxAIgKaPbCPiXQ3C49SdUPomhplllsORFubgkc=.12ad60c2-1071-42b3-93fd-90a467b1fd45@github.com>
Message-ID: <yeK-w9H_oVdyotGN36Diupk069SnNucSWEn0XPlNsZ4=.52353398-87a3-4227-8a28-d5a6a2786955@github.com>

On Wed, 26 Feb 2025 01:22:05 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits:
>> 
>>  - Merge tag 'jdk-25+11' into fix-control-regulator-threads
>>    
>>    Added tag jdk-25+11 for changeset 0131c1bf
>>  - Address review feedback (better comments, better names)
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - ... and 22 more: https://git.openjdk.org/jdk/compare/0131c1bf...d7858deb
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 687:
> 
>> 685:   HeapWord* allocate_from_gclab_slow(Thread* thread, size_t size);
>> 686:   HeapWord* allocate_new_gclab(size_t min_size, size_t word_size, size_t* actual_size);
>> 687:   bool retry_allocation(size_t original_full_gc_count) const;
> 
> It feels like this should be called `should_retry_allocation()`. Also a 1-line documentation comment: ``` // We want to retry an unsuccessful attempt at allocation until at least a full gc. ```

PS: do we know that a full gc will reclaim all soft refs? I suppose that is Shenandoah policy?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1970762980

From xpeng at openjdk.org  Wed Feb 26 02:17:36 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 02:17:36 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v4]
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <Cw4sj0dXDVz0QY0BYqo3EhF4Nwu6wigcxMSpPyMO77s=.ba1c25ba-81f3-41f4-8029-42ff2970e110@github.com>

> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Set _gc_state_changed to false at the end of concurrent_prepare_for_update_refs

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23759/files
  - new: https://git.openjdk.org/jdk/pull/23759/files/2e7b2cea..54ae1c0e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=02-03

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23759.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23759/head:pull/23759

PR: https://git.openjdk.org/jdk/pull/23759

From ysr at openjdk.org  Wed Feb 26 02:17:36 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Feb 2025 02:17:36 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v4]
In-Reply-To: <Cw4sj0dXDVz0QY0BYqo3EhF4Nwu6wigcxMSpPyMO77s=.ba1c25ba-81f3-41f4-8029-42ff2970e110@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <Cw4sj0dXDVz0QY0BYqo3EhF4Nwu6wigcxMSpPyMO77s=.ba1c25ba-81f3-41f4-8029-42ff2970e110@github.com>
Message-ID: <TlDKQFTY2JQ_FCg1zonk4XVcoLHbZ9B1b5OJCHKEeYc=.e9562651-ccab-46fb-8494-dd0d7f0b0c89@github.com>

On Wed, 26 Feb 2025 02:14:22 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Set _gc_state_changed to false at the end of concurrent_prepare_for_update_refs

LGTM

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23759#pullrequestreview-2642858719

From ysr at openjdk.org  Wed Feb 26 02:17:36 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Feb 2025 02:17:36 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v3]
In-Reply-To: <cYv8kf7drUHAMJiLBb-oquj_nUIkgDBPcWwwewUNb5o=.feefcad5-0ddf-42c9-aee5-cc0724afc4cd@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <cYv8kf7drUHAMJiLBb-oquj_nUIkgDBPcWwwewUNb5o=.feefcad5-0ddf-42c9-aee5-cc0724afc4cd@github.com>
Message-ID: <7HdJx1s4PkFH9L1AdYjiQMjR4nfl0RM4FDHtdlLDge4=.cf536d80-1679-44f8-b36c-9aab738f7cd8@github.com>

On Wed, 26 Feb 2025 00:01:12 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove propagate_gc_state_to_all_threads call from op_init_update_refs

Changes are fine.

This jumped out in yr sample output:


...
[11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
...
[11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x


which seemed kinda interesting. I assume this is just a consequence of the very little work (and extremely brief time in this phase) here, and can be ignored in this sample output from likely a toy GC, or one where you may have artificially boosted the number of worker threads. Still I thought I'd ask in case you've seen this with bigger timings or more work in any of these phases with low fractional speed-ups.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23759#issuecomment-2683721058

From xpeng at openjdk.org  Wed Feb 26 02:17:36 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 02:17:36 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v2]
In-Reply-To: <UzmCKYQVhsGf0qxcapOggE_QzNOgsKlV2j9fufaEY08=.f473ad5d-acd7-45f5-9795-2328ccc7218c@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <rBC3NQmTWNRvveicXt6frMMwV_uzn4q5otzUI30T068=.7696570e-1e5e-447d-bce8-ba32c38ff4f5@github.com>
 <XAVHiEDQWPMilVhMpFeb87wvajX1dto6ItE2DU-NwdU=.d0c183f8-21aa-4b66-93d3-c3e4a82cfb5f@github.com>
 <e1XVM1dbioXUmw5tCkgKeVQAFwshIJqlc-YuKLPj7WM=.8b32fbbf-cabd-4204-a723-f29793723767@github.com>
 <zCzBjl7lGnt7Pf5vgJYbONdC1EWelP0sL25m2Ldg9lo=.123a0b82-4daa-48dc-84ab-91bae525e85d@github.com>
 <bmEHZYxjD1AzpQUJLFz_n1CMtcq7mMbohd9wCF9j_FI=.6ed2d101-71c0-4ff1-873c-618ded13af51@github.com>
 <UzmCKYQVhsGf0qxcapOggE_QzNOgsKlV2j9fufaEY08=.f473ad5d-acd7-45f5-9795-2328ccc7218c@github.com>
Message-ID: <mJk9KaRr5BPAo6836Zdl0IA7uVlORDmJyY-rSrVTmxA=.80ef1113-6a75-4426-8cc3-3fd86348107a@github.com>

On Wed, 26 Feb 2025 00:31:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> For `final_roots`, I think I should leave it as it is in this PR, later you will remove the timings and gc state propagation into a handshake anyway.
>
> Yep, I agree. Thank you!

To remove propagate_gc_state_to_all_threads, `_gc_state_changed` has to be set to false in `ShenandoahHeap::concurrent_prepare_for_update_refs`, I have updated PR to fix it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970795124

From ysr at openjdk.org  Wed Feb 26 02:25:52 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Feb 2025 02:25:52 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v4]
In-Reply-To: <Cw4sj0dXDVz0QY0BYqo3EhF4Nwu6wigcxMSpPyMO77s=.ba1c25ba-81f3-41f4-8029-42ff2970e110@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <Cw4sj0dXDVz0QY0BYqo3EhF4Nwu6wigcxMSpPyMO77s=.ba1c25ba-81f3-41f4-8029-42ff2970e110@github.com>
Message-ID: <a6xz_UsFoVY6abpPM3nSndATzb2rdgfr7ynDmxMUCfw=.b91f50a4-1367-4028-b210-7aa8d66148f1@github.com>

On Wed, 26 Feb 2025 02:17:36 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Set _gc_state_changed to false at the end of concurrent_prepare_for_update_refs

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1288:

> 1286: 
> 1287:   _update_refs_iterator.reset();
> 1288:   _gc_state_changed = false;

Sorry, what is this and why do we need it. When was it set?

In `set_update_refs_in_progress()` ?

May be we need a better abstraction to toggle the `state_changed` boolean. This seems error prone.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1970802802

From xpeng at openjdk.org  Wed Feb 26 07:53:08 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 07:53:08 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v5]
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <SORiYx_NUA4nDBFMo_bmaDNaP2uRrFHa8xjVYjcbtNA=.c5ce4903-aced-4053-a4eb-9dc845cfe945@github.com>

> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Fix test failure

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23759/files
  - new: https://git.openjdk.org/jdk/pull/23759/files/54ae1c0e..30647a9c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=03-04

  Stats: 6 lines in 2 files changed: 5 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23759.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23759/head:pull/23759

PR: https://git.openjdk.org/jdk/pull/23759

From xpeng at openjdk.org  Wed Feb 26 07:53:09 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 07:53:09 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v4]
In-Reply-To: <a6xz_UsFoVY6abpPM3nSndATzb2rdgfr7ynDmxMUCfw=.b91f50a4-1367-4028-b210-7aa8d66148f1@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <Cw4sj0dXDVz0QY0BYqo3EhF4Nwu6wigcxMSpPyMO77s=.ba1c25ba-81f3-41f4-8029-42ff2970e110@github.com>
 <a6xz_UsFoVY6abpPM3nSndATzb2rdgfr7ynDmxMUCfw=.b91f50a4-1367-4028-b210-7aa8d66148f1@github.com>
Message-ID: <PLGr2ddUm_PGF0t4J6FIyuEvvuoobvlR-p9FW8MLBo8=.a2596190-818b-43e3-a9f3-35e34aac30ba@github.com>

On Wed, 26 Feb 2025 02:23:15 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Set _gc_state_changed to false at the end of concurrent_prepare_for_update_refs
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1288:
> 
>> 1286: 
>> 1287:   _update_refs_iterator.reset();
>> 1288:   _gc_state_changed = false;
> 
> Sorry, what is this and why do we need it. When was it set?
> 
> In `set_update_refs_in_progress()` ?
> 
> May be we need a better abstraction to toggle the `state_changed` boolean. This seems error prone.

You are right, we don't need to set _gc_state_changed to false here, because `concurrent_prepare_for_update_refs` calls `ShenandoahHeap::set_gc_state_concurrent` to set gc state, `ShenandoahHeap::set_gc_state_concurrent` doesn't change  `_gc_state_changed` to true, hence there is no need to reset it false here.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1971094948

From xpeng at openjdk.org  Wed Feb 26 08:12:31 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 08:12:31 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v6]
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <igIVgC6vqKhdIytKeC7vMhFBEgs0rkwd9zQvwaN4Y20=.063ea934-34df-48af-a3f1-ee41856cc848@github.com>

> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Remove trailing whitespace

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23759/files
  - new: https://git.openjdk.org/jdk/pull/23759/files/30647a9c..891dd11d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=04-05

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/23759.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23759/head:pull/23759

PR: https://git.openjdk.org/jdk/pull/23759

From xpeng at openjdk.org  Wed Feb 26 18:58:54 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 18:58:54 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v3]
In-Reply-To: <7HdJx1s4PkFH9L1AdYjiQMjR4nfl0RM4FDHtdlLDge4=.cf536d80-1679-44f8-b36c-9aab738f7cd8@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <cYv8kf7drUHAMJiLBb-oquj_nUIkgDBPcWwwewUNb5o=.feefcad5-0ddf-42c9-aee5-cc0724afc4cd@github.com>
 <7HdJx1s4PkFH9L1AdYjiQMjR4nfl0RM4FDHtdlLDge4=.cf536d80-1679-44f8-b36c-9aab738f7cd8@github.com>
Message-ID: <tod-U0OZwqmzM0BWGOP7qr5YjngYVIpS5S7toYXBvmI=.9ea466b2-4164-4474-bcd8-6c6cb57ef0a4@github.com>

On Wed, 26 Feb 2025 02:13:07 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Changes are fine.
> 
> This jumped out in yr sample output:
> 
> ```
> ...
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> ...
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> ```
> 
> which seemed kinda interesting. I assume this is just a consequence of the very little work (and extremely brief time in this phase) here, and can be ignored in this sample output from likely a toy GC, or one where you may have artificially boosted the number of worker threads. Still I thought I'd ask in case you've seen this with bigger timings or more work in any of these phases with low fractional speed-ups.

I'm not sure how parallelism is calculated, but I think it is caused by the test I was running, the test is very simple and there are only small number of live objects after mark.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23759#issuecomment-2685925735

From shade at openjdk.org  Wed Feb 26 19:44:55 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 26 Feb 2025 19:44:55 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v6]
In-Reply-To: <igIVgC6vqKhdIytKeC7vMhFBEgs0rkwd9zQvwaN4Y20=.063ea934-34df-48af-a3f1-ee41856cc848@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <igIVgC6vqKhdIytKeC7vMhFBEgs0rkwd9zQvwaN4Y20=.063ea934-34df-48af-a3f1-ee41856cc848@github.com>
Message-ID: <ePtatupC9m_bb1gxjf4TPTDoPjGFl91w9mSz1J1y5NY=.22cb946e-63c0-4325-bd78-c10163117fcb@github.com>

On Wed, 26 Feb 2025 08:12:31 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove trailing whitespace

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 693:

> 691: 
> 692:   {
> 693:     ShenandoahGCPhase phase(ShenandoahPhaseTimings::init_propagate_gc_state);

Here and later, I messed up my suggestion. I think these should be `ShenandoahTimingsTracker`, not `ShenandoahGCPhase`. `ShenandoahGCPhase` does more stuff we don't need. `ShenandoahTimingsTracker` does only the basic stuff.

src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 86:

> 84:   op_degenerated();
> 85:   heap->set_degenerated_gc_in_progress(false);
> 86:   {

A bit sad we need to do this due to `op_degenerated` early returns, but fine.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1972273776
PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1972269107

From wkemper at openjdk.org  Wed Feb 26 19:50:59 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Feb 2025 19:50:59 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v14]
In-Reply-To: <RRJ21SxAIgKaPbCPiXQ3C49SdUPomhplllsORFubgkc=.12ad60c2-1071-42b3-93fd-90a467b1fd45@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <XaiJPiRLQkihIwVZe2tTxxKHoDQeyOgdw8nZvQEPoh8=.e85b90c5-ecef-490b-94e4-05c22cb28012@github.com>
 <RRJ21SxAIgKaPbCPiXQ3C49SdUPomhplllsORFubgkc=.12ad60c2-1071-42b3-93fd-90a467b1fd45@github.com>
Message-ID: <qT8jJAx6kdJZw9DiSHBtCwPyeXcXYw753Yq72Q6UISA=.e2ece3da-9aed-41c4-8e05-5d3185f8f0df@github.com>

On Tue, 25 Feb 2025 23:10:45 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits:
>> 
>>  - Merge tag 'jdk-25+11' into fix-control-regulator-threads
>>    
>>    Added tag jdk-25+11 for changeset 0131c1bf
>>  - Address review feedback (better comments, better names)
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - ... and 22 more: https://git.openjdk.org/jdk/compare/0131c1bf...d7858deb
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 75:
> 
>> 73: 
>> 74:   // The mode is read frequently by requesting threads and only ever written by the control thread.
>> 75:   volatile GCMode _mode;
> 
> A bit of a nit: Any reason not to just call it `_gc_mode` which seems now to be its _de facto_ name due to its accessor method name and how it's referred to in comments?

Renamed it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1972289732

From wkemper at openjdk.org  Wed Feb 26 20:03:56 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Feb 2025 20:03:56 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v14]
In-Reply-To: <RRJ21SxAIgKaPbCPiXQ3C49SdUPomhplllsORFubgkc=.12ad60c2-1071-42b3-93fd-90a467b1fd45@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <XaiJPiRLQkihIwVZe2tTxxKHoDQeyOgdw8nZvQEPoh8=.e85b90c5-ecef-490b-94e4-05c22cb28012@github.com>
 <RRJ21SxAIgKaPbCPiXQ3C49SdUPomhplllsORFubgkc=.12ad60c2-1071-42b3-93fd-90a467b1fd45@github.com>
Message-ID: <-UXwakwdCebRDEfV0p5VLeRJqByHrZENp_NwYt30Af8=.4bc68e66-0de3-4b43-867a-9b6e6ba7a045@github.com>

On Tue, 25 Feb 2025 23:22:48 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits:
>> 
>>  - Merge tag 'jdk-25+11' into fix-control-regulator-threads
>>    
>>    Added tag jdk-25+11 for changeset 0131c1bf
>>  - Address review feedback (better comments, better names)
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Old gen bootstrap cycle must make it to init mark
>>  - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
>>  - Improve message for assertion
>>  - Make shutdown safer for threads requesting (or expecting) gc
>>  - Do not accept requests if control thread is terminating
>>  - Notify waiters when control thread terminates
>>  - Add event for control thread state changes
>>  - ... and 22 more: https://git.openjdk.org/jdk/compare/0131c1bf...d7858deb
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp line 150:
> 
>> 148:   GCMode prepare_for_explicit_gc_request(ShenandoahGCRequest &request);
>> 149: 
>> 150:   GCMode prepare_for_concurrent_gc_request(ShenandoahGCRequest &request);
> 
> Documentation of private APIs.
> 
> Bit of a nit: These all seem to take a request type, fill in some yet unfilled fields, and return a GC mode. So they look to me like `prepare_request_for_<action>` rather than `pepare_for_<action>_request`.

Hmm, they do modify the request, so I see your point. But they also touch `ShenandoahHeap` , `ShenandoahPolicy` and `ShenandoahHeuristics`. How about we just drop `_request` from their name?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1972319562

From wkemper at openjdk.org  Wed Feb 26 20:19:59 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Feb 2025 20:19:59 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v14]
In-Reply-To: <yeK-w9H_oVdyotGN36Diupk069SnNucSWEn0XPlNsZ4=.52353398-87a3-4227-8a28-d5a6a2786955@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
 <XaiJPiRLQkihIwVZe2tTxxKHoDQeyOgdw8nZvQEPoh8=.e85b90c5-ecef-490b-94e4-05c22cb28012@github.com>
 <RRJ21SxAIgKaPbCPiXQ3C49SdUPomhplllsORFubgkc=.12ad60c2-1071-42b3-93fd-90a467b1fd45@github.com>
 <yeK-w9H_oVdyotGN36Diupk069SnNucSWEn0XPlNsZ4=.52353398-87a3-4227-8a28-d5a6a2786955@github.com>
Message-ID: <STqufi6hOmlueE3IACZc2rpTII5VCSMKmbnL0VKxVzo=.cf261c4c-e365-4ab3-aab7-80c3f458828b@github.com>

On Wed, 26 Feb 2025 01:24:18 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 687:
>> 
>>> 685:   HeapWord* allocate_from_gclab_slow(Thread* thread, size_t size);
>>> 686:   HeapWord* allocate_new_gclab(size_t min_size, size_t word_size, size_t* actual_size);
>>> 687:   bool retry_allocation(size_t original_full_gc_count) const;
>> 
>> It feels like this should be called `should_retry_allocation()`. Also a 1-line documentation comment: ``` // We want to retry an unsuccessful attempt at allocation until at least a full gc. ```
>
> PS: do we know that a full gc will reclaim all soft refs? I suppose that is Shenandoah policy?

Yes, I like that name better. And yes, full gc will reclaim all soft refs.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23475#discussion_r1972342529

From xpeng at openjdk.org  Wed Feb 26 20:24:06 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 20:24:06 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v6]
In-Reply-To: <ePtatupC9m_bb1gxjf4TPTDoPjGFl91w9mSz1J1y5NY=.22cb946e-63c0-4325-bd78-c10163117fcb@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <igIVgC6vqKhdIytKeC7vMhFBEgs0rkwd9zQvwaN4Y20=.063ea934-34df-48af-a3f1-ee41856cc848@github.com>
 <ePtatupC9m_bb1gxjf4TPTDoPjGFl91w9mSz1J1y5NY=.22cb946e-63c0-4325-bd78-c10163117fcb@github.com>
Message-ID: <EIECnpQxzYaZ-w4cQBNjSwxgH-BkZuOBqrv4ESe4lUs=.9fdde38b-0419-4622-819c-99e5114794ac@github.com>

On Wed, 26 Feb 2025 19:42:10 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove trailing whitespace
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 693:
> 
>> 691: 
>> 692:   {
>> 693:     ShenandoahGCPhase phase(ShenandoahPhaseTimings::init_propagate_gc_state);
> 
> Here and later, I messed up my suggestion. I think these should be `ShenandoahTimingsTracker`, not `ShenandoahGCPhase`. `ShenandoahGCPhase` does more stuff we don't need. `ShenandoahTimingsTracker` does only the basic stuff.

Thanks for pointing out, I'll update it and move them to `ShenandoahTimingsTracker` and test it again.

> src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 86:
> 
>> 84:   op_degenerated();
>> 85:   heap->set_degenerated_gc_in_progress(false);
>> 86:   {
> 
> A bit sad we need to do this due to `op_degenerated` early returns, but fine.

Yeah, this part is not very clean due to op_degenerated early returns.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1972349321
PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1972347797

From wkemper at openjdk.org  Wed Feb 26 21:20:20 2025
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Feb 2025 21:20:20 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v15]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <nm6lRMyla9kXjJ6XF-yg2sI7sIu_g8Av2AhWlAYRZ0E=.b7835e40-bbbe-4f5a-aa36-d00492187e2d@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
> 
> ## Testing
> * jtreg hotspot_gc_shenandoah
> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Improve names and comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/d7858deb..fb7819d0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=14
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=13-14

  Stats: 57 lines in 4 files changed: 24 ins; 2 del; 31 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From xpeng at openjdk.org  Wed Feb 26 21:54:32 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 21:54:32 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v7]
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <XP4zFc2O2V1O7O-GmhmiaT4tN2QZo2PSVehMKMxxeE0=.56ec5983-5de9-4232-8423-d21be82595e4@github.com>

> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Use ShenandoahTimingsTracker instead of ShenandoahGCPhase to track the timings of gc state propagation

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23759/files
  - new: https://git.openjdk.org/jdk/pull/23759/files/891dd11d..858ebc39

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=06
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=05-06

  Stats: 7 lines in 4 files changed: 0 ins; 0 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/23759.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23759/head:pull/23759

PR: https://git.openjdk.org/jdk/pull/23759

From shade at openjdk.org  Wed Feb 26 22:17:59 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 26 Feb 2025 22:17:59 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v7]
In-Reply-To: <XP4zFc2O2V1O7O-GmhmiaT4tN2QZo2PSVehMKMxxeE0=.56ec5983-5de9-4232-8423-d21be82595e4@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <XP4zFc2O2V1O7O-GmhmiaT4tN2QZo2PSVehMKMxxeE0=.56ec5983-5de9-4232-8423-d21be82595e4@github.com>
Message-ID: <U30Jx83gPlAjJr98vGYT_c6CPoq0j68gPxVOjc2sBnY=.a55bbda9-3dd8-4cc7-bf66-ec9e656f72aa@github.com>

On Wed, 26 Feb 2025 21:54:32 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use ShenandoahTimingsTracker instead of ShenandoahGCPhase to track the timings of gc state propagation

I am good with this, thanks. Only one minor nit remains.

src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp line 60:

> 58:   f(init_transfer_satb,                             "  Transfer Old From SATB")        \
> 59:   f(init_update_region_states,                      "  Update Region States")          \
> 60:   f(init_propagate_gc_state,                        "  Propagate GC state")            \

Sorry, one last thing. Note how all our counters are Capitalized, so all these should be `Propagate GC State`.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23759#pullrequestreview-2646073039
PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1972497261

From xpeng at openjdk.org  Wed Feb 26 22:23:24 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 22:23:24 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v7]
In-Reply-To: <U30Jx83gPlAjJr98vGYT_c6CPoq0j68gPxVOjc2sBnY=.a55bbda9-3dd8-4cc7-bf66-ec9e656f72aa@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <XP4zFc2O2V1O7O-GmhmiaT4tN2QZo2PSVehMKMxxeE0=.56ec5983-5de9-4232-8423-d21be82595e4@github.com>
 <U30Jx83gPlAjJr98vGYT_c6CPoq0j68gPxVOjc2sBnY=.a55bbda9-3dd8-4cc7-bf66-ec9e656f72aa@github.com>
Message-ID: <QWRPw44l72VxaFT42IKZgKaPnffL4s-M5VhdQedX0bU=.582082e2-54c9-4301-be6f-0b6fd360b924@github.com>

On Wed, 26 Feb 2025 22:14:44 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Use ShenandoahTimingsTracker instead of ShenandoahGCPhase to track the timings of gc state propagation
>
> src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp line 60:
> 
>> 58:   f(init_transfer_satb,                             "  Transfer Old From SATB")        \
>> 59:   f(init_update_region_states,                      "  Update Region States")          \
>> 60:   f(init_propagate_gc_state,                        "  Propagate GC state")            \
> 
> Sorry, one last thing. Note how all our counters are Capitalized, so all these should be `Propagate GC State`.

Fixed, thanks! sorry I should have noticed the naming convention here.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23759#discussion_r1972502610

From xpeng at openjdk.org  Wed Feb 26 22:23:24 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Feb 2025 22:23:24 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v8]
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>

> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Fix name of counter

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23759/files
  - new: https://git.openjdk.org/jdk/pull/23759/files/858ebc39..caf9d4c3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23759&range=06-07

  Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/23759.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23759/head:pull/23759

PR: https://git.openjdk.org/jdk/pull/23759

From shade at openjdk.org  Wed Feb 26 22:25:59 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 26 Feb 2025 22:25:59 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v8]
In-Reply-To: <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>
Message-ID: <blRBdxumNd9Q8mdgoOlYKIxf7vpOUr_k9qRDOwyK9cI=.f38cb8b2-9975-42ff-beba-56d9ac7973dd@github.com>

On Wed, 26 Feb 2025 22:23:24 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix name of counter

Marked as reviewed by shade (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23759#pullrequestreview-2646089074

From ysr at openjdk.org  Thu Feb 27 00:03:54 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Feb 2025 00:03:54 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v8]
In-Reply-To: <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>
Message-ID: <6KQyDTODPGMJk75Fyygvo36xQp1Z_WdnRuHUx1Wt9Uw=.84f67082-639c-45a0-898d-496b54aa2bae@github.com>

On Wed, 26 Feb 2025 22:23:24 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix name of counter

Marked as reviewed by ysr (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23759#pullrequestreview-2646224512

From xpeng at openjdk.org  Thu Feb 27 00:34:02 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Feb 2025 00:34:02 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v8]
In-Reply-To: <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>
Message-ID: <3HzYErJUBCxonwTuiqe8GJRNv_U1PVaOGedzk1Wsn8s=.430da6d6-1054-4495-8877-0c343347c394@github.com>

On Wed, 26 Feb 2025 22:23:24 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix name of counter

Thanks all for the reviews!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23759#issuecomment-2686513145

From duke at openjdk.org  Thu Feb 27 00:34:03 2025
From: duke at openjdk.org (duke)
Date: Thu, 27 Feb 2025 00:34:03 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v8]
In-Reply-To: <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <VZfs6MXhbX04TnVfhsevZU1KH7aj7y_lXZNe88guSWw=.ae2ccd57-180d-468b-a037-a7349636f183@github.com>
Message-ID: <NXBQOUy-cUQqX-uVM5krHGzBa4M4HiqMHqOkBpm1tBo=.a6b2dfcb-1543-48a6-9177-81518026deed@github.com>

On Wed, 26 Feb 2025 22:23:24 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
>> 
>> 1. Net GC pause timings include the time to propagate GC state to Java threads
>> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
>> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
>> 
>> With the change, the new GC timing log will be like:
>> 
>> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
>> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
>> [11.056s][info][gc,stats    ]   Update Region States                3 us
>> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
>> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
>> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
>> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
>> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
>> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
>> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
>> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
>> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
>> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
>> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
>> [11.057s][info][gc,stats    ]   Update Region States               12 us
>> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
>> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
>> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
>> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
>> [11.057s][info][gc,...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix name of counter

@pengxiaolong 
Your change (at version caf9d4c35a5712b4fa4c234aedb092abbdcc826e) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23759#issuecomment-2686513700

From wkemper at openjdk.org  Thu Feb 27 01:14:29 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 01:14:29 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v16]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <MUd3gJn2UT-RuzufPfU1-UvaWDqi6NKQtUkCMuvif14=.ab389963-175d-4c23-8461-7a4bea68e513@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
> 
> ## Testing
> * jtreg hotspot_gc_shenandoah
> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Add assertions about old gen state when resuming old cycles
 - Remove duplicated field pointer for old generation

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/fb7819d0..d2e90dde

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=15
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=14-15

  Stats: 9 lines in 2 files changed: 2 ins; 2 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From ysr at openjdk.org  Thu Feb 27 02:42:09 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Feb 2025 02:42:09 GMT
Subject: RFR: 8350314: Shenandoah: Capture thread state sync times in GC
 timings [v3]
In-Reply-To: <tod-U0OZwqmzM0BWGOP7qr5YjngYVIpS5S7toYXBvmI=.9ea466b2-4164-4474-bcd8-6c6cb57ef0a4@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
 <cYv8kf7drUHAMJiLBb-oquj_nUIkgDBPcWwwewUNb5o=.feefcad5-0ddf-42c9-aee5-cc0724afc4cd@github.com>
 <7HdJx1s4PkFH9L1AdYjiQMjR4nfl0RM4FDHtdlLDge4=.cf536d80-1679-44f8-b36c-9aab738f7cd8@github.com>
 <tod-U0OZwqmzM0BWGOP7qr5YjngYVIpS5S7toYXBvmI=.9ea466b2-4164-4474-bcd8-6c6cb57ef0a4@github.com>
Message-ID: <j49PifY7iP-zfOyWHWAA-FeMbvxfu5-bWZG5uPUhG6Q=.2e58692d-c8a5-4cef-96fd-53c715cba78d@github.com>

On Wed, 26 Feb 2025 18:56:06 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> I'm not sure how parallelism is calculated, but I think it is caused by the test I was running, the test is very simple and there are only small number of live objects after mark.

Yes that explains it, thanks.

(PS: Parallelism (or really parallel speed-up) is calculated as the ratio of total thread virtual time to the wall-clock (elapsed) time, IIRC. As you noted work in this specific workload is too small for the parallel task overhead, as indicated by the micro-seconds worth of time.)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23759#issuecomment-2686707985

From dfenacci at openjdk.org  Thu Feb 27 08:34:45 2025
From: dfenacci at openjdk.org (Damon Fenacci)
Date: Thu, 27 Feb 2025 08:34:45 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
 [v2]
In-Reply-To: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
Message-ID: <L8H5mzzutgCplBHpqX26fo0t5o3uT3DsRW_w3bpwYyQ=.e15ca771-1341-44c9-8878-31666c966a03@github.com>

> # Issue
> The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.
> 
> # Causes
> There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
> 1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`
> 
> # Solution
> Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.
> 
> Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.
> 
> # Testing
> The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.
> 
> Tests: Tier 1-4 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; release and debug mode)

Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:

  Update src/hotspot/share/c1/c1_Runtime1.cpp
  
  Co-authored-by: Dean Long <17332032+dean-long at users.noreply.github.com>

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23630/files
  - new: https://git.openjdk.org/jdk/pull/23630/files/e930df47..85eb1022

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23630&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23630&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/23630.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23630/head:pull/23630

PR: https://git.openjdk.org/jdk/pull/23630

From dfenacci at openjdk.org  Thu Feb 27 08:34:45 2025
From: dfenacci at openjdk.org (Damon Fenacci)
Date: Thu, 27 Feb 2025 08:34:45 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
In-Reply-To: <s8--SN34E0d36PkNjweZZ0OaLDWZkzDsk9kUDDghpGg=.51bca4f4-7098-4192-9e4c-71900492132b@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
 <s8--SN34E0d36PkNjweZZ0OaLDWZkzDsk9kUDDghpGg=.51bca4f4-7098-4192-9e4c-71900492132b@github.com>
Message-ID: <JmRNqJUJJ9Ql5QiA-ZBnXqvG65RULYv9BDLp0MHXOZ0=.ae2846a1-2b62-4126-9ada-91e0e1357d43@github.com>

On Thu, 20 Feb 2025 20:53:30 GMT, Dean Long <dlong at openjdk.org> wrote:

> I don't understand why JDK-8326615 didn't work. If the minimum codecache size was too small, can't we just increase it?

It seems that the small minimum codecache size wasn?t the core of the problem. In JDK-8326615 I?ve actually tried increasing the minimum code cache size (even calculating the minimum C1 and C2 sizes needed) but I kept hitting the same problem seemingly because what triggers the crash is the code cache exceeding its limits **exactly** during one of the allocations in the 4 call paths listed in the description (any allocation at another point simply turns off C1 or C2 (or both)). Increasing the minimum was possibly making it a bit less probable (i.e. when running something small that would use very little code cache, e.g. `java -version`) but as soon as running something a bit bigger (requiring more code cache), we could still end up calling the problematic allocations right when the code cache was full.
BTW even if the crash was very dependent on C1/C2 thread scheduling etc., I was able to reproduce it with different minimum code cache sizes relatively frequently.

Even so, it might be a good idea to additionally increase the minimum code cache anyway. @dean-long do you think it would make sense to file an RFE for that?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23630#issuecomment-2687239402

From xpeng at openjdk.org  Thu Feb 27 09:52:22 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Feb 2025 09:52:22 GMT
Subject: Integrated: 8350314: Shenandoah: Capture thread state sync times in GC
 timings
In-Reply-To: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
References: <jCkkwO7ELKqaq_8lZbLBbpizKhV5M1oZzKMnplI6J3Q=.1fdbbb6d-79e6-4bbd-8e01-aa30fa145b81@github.com>
Message-ID: <b9sUP5MtWERo7EZQT9zJ2msb_uMm139hp9ytlj9ZgqM=.bd95849e-2b29-4528-b0c5-22224988f381@github.com>

On Tue, 25 Feb 2025 01:20:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> The change is to improve the observability of  Shenandoah GC, basically there are three changes for  Shenandoah GC timings in this PR:
> 
> 1. Net GC pause timings include the time to propagate GC state to Java threads
> 2. Add new timing "Propagate GC state" in Shenandoah GC timing logs
> 3. Removal of the call of `propagate_gc_state_to_all_threads` from "init_update_refs", which handles gc state in handshake already. 
> 
> With the change, the new GC timing log will be like:
> 
> [11.056s][info][gc,stats    ] Concurrent Reset                     89 us
> [11.056s][info][gc,stats    ] Pause Init Mark (G)                 257 us
> [11.056s][info][gc,stats    ] Pause Init Mark (N)                  17 us
> [11.056s][info][gc,stats    ]   Update Region States                3 us
> [11.056s][info][gc,stats    ]   Propagate GC state                  1 us
> [11.056s][info][gc,stats    ] Concurrent Mark Roots               232 us, parallelism: 1.96x
> [11.056s][info][gc,stats    ]   CMR: <total>                      456 us
> [11.056s][info][gc,stats    ]   CMR: Thread Roots                 429 us, workers (us): 139, 148, 142, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: VM Strong Roots               11 us, workers (us):   8,   3,   0, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   CMR: CLDG Roots                    16 us, workers (us):  16, ---, ---, ---, ---, ---, 
> [11.057s][info][gc,stats    ] Concurrent Marking                 1304 us, parallelism: 2.33x
> [11.057s][info][gc,stats    ]   CM: <total>                      3043 us
> [11.057s][info][gc,stats    ]   CM: Parallel Mark                3043 us, workers (us): 1023, 1017, 1003, ---, ---, ---, 
> [11.057s][info][gc,stats    ]   Flush SATB                        204 us
> [11.057s][info][gc,stats    ] Pause Final Mark (G)                865 us
> [11.057s][info][gc,stats    ] Pause Final Mark (N)                234 us
> [11.057s][info][gc,stats    ]   Finish Mark                       129 us, parallelism: 0.01x
> [11.057s][info][gc,stats    ]   Propagate GC state                  2 us
> [11.057s][info][gc,stats    ]   Update Region States               12 us
> [11.057s][info][gc,stats    ]   Choose Collection Set              25 us
> [11.057s][info][gc,stats    ]   Rebuild Free Set                   29 us
> [11.057s][info][gc,stats    ] Concurrent Weak References           67 us, parallelism: 0.25x
> [11.057s][info][gc,stats    ]   CWRF: <total>                      17 us
> [11.057s][info][gc,stats    ]   CWRF: Weak References              17 us, workers (...

This pull request has now been integrated.

Changeset: 01bd7e41
Author:    Xiaolong Peng <xpeng at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/01bd7e417ee3d39067370e616660b7f5c723dc26
Stats:     47 lines in 6 files changed: 40 ins; 7 del; 0 mod

8350314: Shenandoah: Capture thread state sync times in GC timings

Reviewed-by: ysr, shade, wkemper

-------------

PR: https://git.openjdk.org/jdk/pull/23759

From dfenacci at openjdk.org  Thu Feb 27 11:27:25 2025
From: dfenacci at openjdk.org (Damon Fenacci)
Date: Thu, 27 Feb 2025 11:27:25 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
 [v3]
In-Reply-To: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
Message-ID: <iXVgBaLzUa1jQNXTF5if_P7TCRz7BqEqLi2DUBBI8Bs=.48fddaec-0c90-4d25-8a9d-e6e5d6a40dcc@github.com>

> # Issue
> The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.
> 
> # Causes
> There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
> 1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`
> 
> # Solution
> Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.
> 
> Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.
> 
> # Testing
> The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.
> 
> Tests: Tier 1-4 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; release and debug mode)

Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:

  JDK-8347406: return immediately if stub generation fails

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23630/files
  - new: https://git.openjdk.org/jdk/pull/23630/files/85eb1022..26a6747d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23630&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23630&range=01-02

  Stats: 41 lines in 2 files changed: 21 ins; 7 del; 13 mod
  Patch: https://git.openjdk.org/jdk/pull/23630.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23630/head:pull/23630

PR: https://git.openjdk.org/jdk/pull/23630

From dfenacci at openjdk.org  Thu Feb 27 11:35:09 2025
From: dfenacci at openjdk.org (Damon Fenacci)
Date: Thu, 27 Feb 2025 11:35:09 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
 [v3]
In-Reply-To: <hDbvpph4D6KJc_tMrde62yCVY5DVdWncy0wjBz67fBk=.3ac9a977-857a-45df-a5e8-284f9e14af60@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
 <hDbvpph4D6KJc_tMrde62yCVY5DVdWncy0wjBz67fBk=.3ac9a977-857a-45df-a5e8-284f9e14af60@github.com>
Message-ID: <eZI2N3cKKiyU6ec_U9wur4r6Z7MxB31lgqqE23NtHrc=.c6192d0a-bc37-4c99-9627-9f96ff1cb003@github.com>

On Thu, 20 Feb 2025 21:34:16 GMT, Dean Long <dlong at openjdk.org> wrote:

>> Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   JDK-8347406: return immediately if stub generation fails
>
> src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp line 306:
> 
>> 304:                                 _load_reference_barrier_phantom_rt_code_blob != nullptr;
>> 305:   }
>> 306:   return _pre_barrier_c1_runtime_code_blob != nullptr && reference_barrier_success;
> 
> Wouldn't it be better to return false immediately after each failure, rather than continuing?

Yes, totally. Fixed.

> src/hotspot/share/gc/z/c1/zBarrierSetC1.cpp line 543:
> 
>> 541:   _store_barrier_on_oop_field_without_healing =
>> 542:     generate_c1_store_runtime_stub(blob, false /* self_healing */, "store_barrier_on_oop_field_without_healing");
>> 543:   return _load_barrier_on_oop_field_preloaded_runtime_stub != nullptr &&
> 
> Again, why not return false immediately on first failure?

Fixed too.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23630#discussion_r1973402325
PR Review Comment: https://git.openjdk.org/jdk/pull/23630#discussion_r1973402472

From dfenacci at openjdk.org  Thu Feb 27 12:02:44 2025
From: dfenacci at openjdk.org (Damon Fenacci)
Date: Thu, 27 Feb 2025 12:02:44 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
 [v4]
In-Reply-To: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
Message-ID: <JpkNlc2S7obqyCkSQpKH4LmjQTYp89jTbhCAo_V9SzM=.7d7c09b4-9c13-485f-9236-ac96fbe5b57f@github.com>

> # Issue
> The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.
> 
> # Causes
> There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
> 1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`
> 
> # Solution
> Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.
> 
> Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.
> 
> # Testing
> The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.
> 
> Tests: Tier 1-4 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; release and debug mode)

Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:

  JDK-8347406: re-add modified assert

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23630/files
  - new: https://git.openjdk.org/jdk/pull/23630/files/26a6747d..906cd756

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23630&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23630&range=02-03

  Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23630.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23630/head:pull/23630

PR: https://git.openjdk.org/jdk/pull/23630

From dfenacci at openjdk.org  Thu Feb 27 12:06:02 2025
From: dfenacci at openjdk.org (Damon Fenacci)
Date: Thu, 27 Feb 2025 12:06:02 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
 [v4]
In-Reply-To: <hDbvpph4D6KJc_tMrde62yCVY5DVdWncy0wjBz67fBk=.3ac9a977-857a-45df-a5e8-284f9e14af60@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
 <hDbvpph4D6KJc_tMrde62yCVY5DVdWncy0wjBz67fBk=.3ac9a977-857a-45df-a5e8-284f9e14af60@github.com>
Message-ID: <RPdo7Z3j1KD18svu1bCQDuf8S9jTt4KmIonFpme6ffQ=.82298243-1f97-492e-acc3-9524eca5054b@github.com>

On Thu, 20 Feb 2025 21:36:44 GMT, Dean Long <dlong at openjdk.org> wrote:

>> Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   JDK-8347406: re-add modified assert
>
> src/hotspot/share/opto/output.cpp line 3487:
> 
>> 3485:         C->record_failure("CodeCache is full");
>> 3486:       } else {
>> 3487:         C->set_stub_entry_point(rs->entry_point());
> 
> Is the deleted rs->is_runtime_stub() assert still useful here?

A slightly modified one surely is. Inserted it again.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23630#discussion_r1973446189

From wkemper at openjdk.org  Thu Feb 27 14:24:39 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 14:24:39 GMT
Subject: RFR: Merge openjdk/jdk21u:master
Message-ID: <ycuKTbf-AFV9SPyaC2m_hFpQV9IutQrYnSsiXxJ01Ds=.46575025-fcba-416d-867d-5d956bc69d83@github.com>

Merges tag jdk-21.0.7+4

-------------

Commit messages:
 - 8317808: HTTP/2 stream cancelImpl may leave subscriber registered
 - 8344581: [TESTBUG] java/awt/Robot/ScreenCaptureRobotTest.java failing on macOS
 - 8339356: Test javax/net/ssl/SSLSocket/Tls13PacketSize.java failed with java.net.SocketException: An established connection was aborted by the software in your host machine
 - 8342635: javax/swing/JFileChooser/FileSystemView/WindowsDefaultIconSizeTest.java creates tmp file in src dir
 - 8349039: Adjust exception No type named <ThreadType> in database
 - 8327476: Upgrade JLine to 3.26.1
 - 8349729: [21u] AIX jtreg tests fail to compile with qvisibility=hidden
 - 8348675: TrayIcon tests fail in Ubuntu 24.10 Wayland
 - 8346828: javax/swing/JScrollBar/4865918/bug4865918.java still fails in CI
 - 8346324: javax/swing/JScrollBar/4865918/bug4865918.java fails in CI
 - ... and 10 more: https://git.openjdk.org/shenandoah-jdk21u/compare/b7d92cd0...58069061

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/157/files
  Stats: 8524 lines in 155 files changed: 3751 ins; 1651 del; 3122 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/157.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/157/head:pull/157

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/157

From kdnilsen at openjdk.org  Thu Feb 27 17:10:43 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 17:10:43 GMT
Subject: RFR: 8350889: GenShen: Break out of infinite loop of old GC cycles
Message-ID: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>

Add log message when heuristic trigger because previous trigger is pending.  Cancel the pending-trigger condition for old-generation at the start of old-generation GC.

-------------

Commit messages:
 - Fix white space
 - Cancel pending GC trigger at start of old gc
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - ... and 20 more: https://git.openjdk.org/jdk/compare/3c9d64eb...85319914

Changes: https://git.openjdk.org/jdk/pull/23827/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23827&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8350889
  Stats: 2 lines in 2 files changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23827.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23827/head:pull/23827

PR: https://git.openjdk.org/jdk/pull/23827

From wkemper at openjdk.org  Thu Feb 27 17:25:05 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 17:25:05 GMT
Subject: RFR: Merge openjdk/jdk21u:master [v2]
In-Reply-To: <ycuKTbf-AFV9SPyaC2m_hFpQV9IutQrYnSsiXxJ01Ds=.46575025-fcba-416d-867d-5d956bc69d83@github.com>
References: <ycuKTbf-AFV9SPyaC2m_hFpQV9IutQrYnSsiXxJ01Ds=.46575025-fcba-416d-867d-5d956bc69d83@github.com>
Message-ID: <4XNn74PqkBLy6zjzpe2R_313rnjCm_VlLnxrCHfDRAA=.41ab1016-dbc9-46e0-b064-b2db20c049f7@github.com>

> Merges tag jdk-21.0.7+4

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/157/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/157/files/58069061..58069061

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=157&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=157&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/157.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/157/head:pull/157

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/157

From wkemper at openjdk.org  Thu Feb 27 17:25:07 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 17:25:07 GMT
Subject: Integrated: Merge openjdk/jdk21u:master
In-Reply-To: <ycuKTbf-AFV9SPyaC2m_hFpQV9IutQrYnSsiXxJ01Ds=.46575025-fcba-416d-867d-5d956bc69d83@github.com>
References: <ycuKTbf-AFV9SPyaC2m_hFpQV9IutQrYnSsiXxJ01Ds=.46575025-fcba-416d-867d-5d956bc69d83@github.com>
Message-ID: <Y5Udw_Y4kZE9aLcBN4IREOrQB69N0gWaQPTxvxLkZ5M=.06562893-35d5-435a-bf1b-1916a4a0c2db@github.com>

On Thu, 27 Feb 2025 14:20:15 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.7+4

This pull request has now been integrated.

Changeset: 2ec89c90
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/2ec89c90d3379be904c84850a591a4af59815afe
Stats:     8524 lines in 155 files changed: 3751 ins; 1651 del; 3122 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/157

From wkemper at openjdk.org  Thu Feb 27 17:35:01 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 17:35:01 GMT
Subject: RFR: 8350889: GenShen: Break out of infinite loop of old GC cycles
In-Reply-To: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
References: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
Message-ID: <Ad-wBIGCY02UsLrvyHjKOG7jVcauuE29RD-NYhnZVSM=.01dc684a-569b-420e-8d01-e45a1c1f71b6@github.com>

On Thu, 27 Feb 2025 16:58:49 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Add log message when heuristic trigger because previous trigger is pending.  Cancel the pending-trigger condition for old-generation at the start of old-generation GC.

LGTM

-------------

Marked as reviewed by wkemper (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23827#pullrequestreview-2648558352

From ysr at openjdk.org  Thu Feb 27 17:35:02 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Feb 2025 17:35:02 GMT
Subject: RFR: 8350889: GenShen: Break out of infinite loop of old GC cycles
In-Reply-To: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
References: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
Message-ID: <FJ2wp7e2OkrJClyjDei3VpbFErwhJYj323W266--EyM=.31646835-01c9-4860-8bba-987e3e5ed768@github.com>

On Thu, 27 Feb 2025 16:58:49 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Add log message when heuristic trigger because previous trigger is pending.  Cancel the pending-trigger condition for old-generation at the start of old-generation GC.

Changes look good. A somewhat tangential comment (possibly a nit) on old (existing) code.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 246:

> 244:     log_trigger("GC start is already pending");
> 245:     return true;
> 246:   }

Should this happen lower down? Otherwise you miss the sampling of allocation rate at this time, down at line 249. Not sure if that can be an issue. Perhaps not and you just interpolate an average over the duration since the last sample albeit missing the occasional spike or dip, with perhaps the resulting low-pass filtering here ok? It would be OK if there is not an expectation of roughly syncrhonous sampling. Depends on how the code at line 286 works, I guess.

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23827#pullrequestreview-2648522862
PR Review Comment: https://git.openjdk.org/jdk/pull/23827#discussion_r1974023918

From ysr at openjdk.org  Thu Feb 27 17:35:03 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Feb 2025 17:35:03 GMT
Subject: RFR: 8350889: GenShen: Break out of infinite loop of old GC cycles
In-Reply-To: <7YIfJj5yEf_pjdRuHh2IXyiF88Un6h1MQ-TPbQzrkIY=.5ac56e7f-faa7-479f-b16d-4ec3ff552bbc@github.com>
References: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
 <FJ2wp7e2OkrJClyjDei3VpbFErwhJYj323W266--EyM=.31646835-01c9-4860-8bba-987e3e5ed768@github.com>
 <7YIfJj5yEf_pjdRuHh2IXyiF88Un6h1MQ-TPbQzrkIY=.5ac56e7f-faa7-479f-b16d-4ec3ff552bbc@github.com>
Message-ID: <Eppzwml82F0wG6Z62HBmDNasPwsGbCwHZiAAiMvYdlw=.193839cb-b072-4830-b32a-eee3ad3dc72c@github.com>

On Thu, 27 Feb 2025 17:28:07 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 246:
>> 
>>> 244:     log_trigger("GC start is already pending");
>>> 245:     return true;
>>> 246:   }
>> 
>> Should this happen lower down? Otherwise you miss the sampling of allocation rate at this time, down at line 249. Not sure if that can be an issue. Perhaps not and you just interpolate an average over the duration since the last sample albeit missing the occasional spike or dip, with perhaps the resulting low-pass filtering here ok? It would be OK if there is not an expectation of roughly syncrhonous sampling. Depends on how the code at line 286 works, I guess.
>
> PS: the lvalue rate at line 249 isn't used anywhere. Does the compiler complain if you `sample` without assigning the value?

Looked at the code for `ShenandoahAllocationRate`, I guess this may have the effect of making the sampling a bit more asynchronous, but the resulting smoothing over the interval could underestimate the volatility of the rate which is used in an `upper_bound` calculation below. May be that is harmless?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23827#discussion_r1974048690

From ysr at openjdk.org  Thu Feb 27 17:35:03 2025
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Feb 2025 17:35:03 GMT
Subject: RFR: 8350889: GenShen: Break out of infinite loop of old GC cycles
In-Reply-To: <FJ2wp7e2OkrJClyjDei3VpbFErwhJYj323W266--EyM=.31646835-01c9-4860-8bba-987e3e5ed768@github.com>
References: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
 <FJ2wp7e2OkrJClyjDei3VpbFErwhJYj323W266--EyM=.31646835-01c9-4860-8bba-987e3e5ed768@github.com>
Message-ID: <7YIfJj5yEf_pjdRuHh2IXyiF88Un6h1MQ-TPbQzrkIY=.5ac56e7f-faa7-479f-b16d-4ec3ff552bbc@github.com>

On Thu, 27 Feb 2025 17:18:28 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Add log message when heuristic trigger because previous trigger is pending.  Cancel the pending-trigger condition for old-generation at the start of old-generation GC.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 246:
> 
>> 244:     log_trigger("GC start is already pending");
>> 245:     return true;
>> 246:   }
> 
> Should this happen lower down? Otherwise you miss the sampling of allocation rate at this time, down at line 249. Not sure if that can be an issue. Perhaps not and you just interpolate an average over the duration since the last sample albeit missing the occasional spike or dip, with perhaps the resulting low-pass filtering here ok? It would be OK if there is not an expectation of roughly syncrhonous sampling. Depends on how the code at line 286 works, I guess.

PS: the lvalue rate at line 249 isn't used anywhere. Does the compiler complain if you `sample` without assigning the value?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23827#discussion_r1974045814

From wkemper at openjdk.org  Thu Feb 27 18:27:00 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 18:27:00 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap
In-Reply-To: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
Message-ID: <nFs7nHQXgYTdknbVHRKj1yLdxSSMna8kOnptANParFI=.9ac8dce0-76f5-4476-b7b6-21e6f7e46dd5@github.com>

On Thu, 30 Jan 2025 18:55:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Original implementation was not robust to overriding of CardSizeInBytes, especially to smaller values.  This fixes that issue.

Minor nit.

src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp line 192:

> 190: 
> 191:   if (GCCardSizeInBytes < ShenandoahMinCardSizeInBytes) {
> 192:     char buf[512];

It looks like using `err_msg` here is more idiomatic than `os:snprintf` for errors during initialization.

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 950:

> 948:   ShenandoahHeap* const heap = ShenandoahHeap::heap();
> 949:   assert(heap->is_concurrent_weak_root_in_progress(), "Only during this phase");
> 950:   {

This looks like it came from https://github.com/openjdk/jdk/pull/23604. Did you cherry-pick that into this branch? Not sure why it shows in the diff here.

-------------

Changes requested by wkemper (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/23373#pullrequestreview-2648607661
PR Review Comment: https://git.openjdk.org/jdk/pull/23373#discussion_r1974075963
PR Review Comment: https://git.openjdk.org/jdk/pull/23373#discussion_r1974082900

From kdnilsen at openjdk.org  Thu Feb 27 18:35:57 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 18:35:57 GMT
Subject: RFR: 8350889: GenShen: Break out of infinite loop of old GC cycles
In-Reply-To: <Eppzwml82F0wG6Z62HBmDNasPwsGbCwHZiAAiMvYdlw=.193839cb-b072-4830-b32a-eee3ad3dc72c@github.com>
References: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
 <FJ2wp7e2OkrJClyjDei3VpbFErwhJYj323W266--EyM=.31646835-01c9-4860-8bba-987e3e5ed768@github.com>
 <7YIfJj5yEf_pjdRuHh2IXyiF88Un6h1MQ-TPbQzrkIY=.5ac56e7f-faa7-479f-b16d-4ec3ff552bbc@github.com>
 <Eppzwml82F0wG6Z62HBmDNasPwsGbCwHZiAAiMvYdlw=.193839cb-b072-4830-b32a-eee3ad3dc72c@github.com>
Message-ID: <4lsZ9HWnjBflQG5tz0R5gaQf3eHsn21jzvbVTBUGv2c=.a412baa0-2c8d-486e-8cda-880e2c6cc52b@github.com>

On Thu, 27 Feb 2025 17:30:08 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> PS: the lvalue rate at line 249 isn't used anywhere. Does the compiler complain if you `sample` without assigning the value?
>
> Looked at the code for `ShenandoahAllocationRate`, I guess this may have the effect of making the sampling a bit more asynchronous, but the resulting smoothing over the interval could underestimate the volatility of the rate which is used in an `upper_bound` calculation below. May be that is harmless?

Yes.  If we've already decided to trigger, there is no urgency to re-evaluate the allocation rate.  We won't be checking for should_start_gc() again until the end of the current gc cycle.  At that time, we will update the allocation rate based on how many allocation were realized during the GC cycle.  The initial allocation rate estimate will be refined as should_start_gc() is again sampled every ms or so.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23827#discussion_r1974136474

From wkemper at openjdk.org  Thu Feb 27 18:39:59 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 18:39:59 GMT
Subject: RFR: 8349766: GenShen: Bad progress after degen does not always
 need full gc [v5]
In-Reply-To: <SwSImg7B1mdgxMu6tFrrYBYS6194MSObe_4DmZhx4pU=.1139c830-28fe-4e1b-9811-d228f9b24345@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
 <SwSImg7B1mdgxMu6tFrrYBYS6194MSObe_4DmZhx4pU=.1139c830-28fe-4e1b-9811-d228f9b24345@github.com>
Message-ID: <TBcAwk1Vdl5e_4GnXT3fB54Q1fAFeM1FYuyii3V3CMM=.7ee1d917-7fa4-47db-a327-e585a2e161ca@github.com>

On Tue, 18 Feb 2025 19:28:28 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:
> 
>  - Merge branch 'master' of https://git.openjdk.org/jdk into defer-generational-full-gc
>  - Merge master
>  - Fix typo in merge conflict resolution
>  - 8348595: GenShen: Fix generational free-memory no-progress check
>    
>    Reviewed-by: phh, xpeng
>  - 8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)
>    
>    Reviewed-by: shade
>  - Merge tag 'jdk-25+10' into defer-generational-full-gc
>    
>    Added tag jdk-25+10 for changeset a637ccf2
>  - Be less eager to upgrade degen to full gc

Marked as reviewed by wkemper (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23552#pullrequestreview-2648733128

From kdnilsen at openjdk.org  Thu Feb 27 18:43:02 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 18:43:02 GMT
Subject: Integrated: 8349766: GenShen: Bad progress after degen does not always
 need full gc
In-Reply-To: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
References: <c6ZTwYM1ggdMIjTUTnwqYJKDcTa5xt-44Zlb1f_iQd8=.1e3b193e-dfc9-47d8-9a60-6ddee482f4ba@github.com>
Message-ID: <0sDKADu6y5hHS3qy_R2vjlKRi0ttb2Q4XAKqdtnm23k=.872c061e-7409-424d-acd1-9436ed187866@github.com>

On Tue, 11 Feb 2025 03:31:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> In generational mode, only upgrade to full GC from degenerated GC if we've done two degenerated cycles in a row and both indicated bad progress.  Otherwise, start another concurrent GC, which will most likely degenerate also.  But this degenerated cycle will reclaim floating garbage within the young generation much more quickly than a full GC would have done.

This pull request has now been integrated.

Changeset: 3ae80bfb
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/3ae80bfb6085e1a6bcb551c7b0be8f27b6f9fde9
Stats:     20 lines in 2 files changed: 17 ins; 0 del; 3 mod

8349766: GenShen: Bad progress after degen does not always need full gc

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/jdk/pull/23552

From wkemper at openjdk.org  Thu Feb 27 18:47:24 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 18:47:24 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v17]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <ewY0Rioj7w8hMpCU5dpIeL9yib8bhHUoYsWXAN1XBiY=.a751ad53-f62f-41d8-898e-d036addfba0e@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
> 
> ## Testing
> * jtreg hotspot_gc_shenandoah
> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Don't check for shutdown in control thread loop condition
  
  It may cause the thread to exit before it is requested to stop

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23475/files
  - new: https://git.openjdk.org/jdk/pull/23475/files/d2e90dde..150cb798

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=16
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=15-16

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From kdnilsen at openjdk.org  Thu Feb 27 18:56:06 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 18:56:06 GMT
Subject: RFR: 8350889: GenShen: Break out of infinite loop of old GC cycles
In-Reply-To: <4lsZ9HWnjBflQG5tz0R5gaQf3eHsn21jzvbVTBUGv2c=.a412baa0-2c8d-486e-8cda-880e2c6cc52b@github.com>
References: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
 <FJ2wp7e2OkrJClyjDei3VpbFErwhJYj323W266--EyM=.31646835-01c9-4860-8bba-987e3e5ed768@github.com>
 <7YIfJj5yEf_pjdRuHh2IXyiF88Un6h1MQ-TPbQzrkIY=.5ac56e7f-faa7-479f-b16d-4ec3ff552bbc@github.com>
 <Eppzwml82F0wG6Z62HBmDNasPwsGbCwHZiAAiMvYdlw=.193839cb-b072-4830-b32a-eee3ad3dc72c@github.com>
 <4lsZ9HWnjBflQG5tz0R5gaQf3eHsn21jzvbVTBUGv2c=.a412baa0-2c8d-486e-8cda-880e2c6cc52b@github.com>
Message-ID: <iwRXTzoFDbCFc2qa6ELzmo2iiE8PrasP0C3LsUsQlMI=.121fd246-0c82-46d7-a8fc-607cfb6a8ece@github.com>

On Thu, 27 Feb 2025 18:32:55 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Looked at the code for `ShenandoahAllocationRate`, I guess this may have the effect of making the sampling a bit more asynchronous, but the resulting smoothing over the interval could underestimate the volatility of the rate which is used in an `upper_bound` calculation below. May be that is harmless?
>
> Yes.  If we've already decided to trigger, there is no urgency to re-evaluate the allocation rate.  We won't be checking for should_start_gc() again until the end of the current gc cycle.  At that time, we will update the allocation rate based on how many allocation were realized during the GC cycle.  The initial allocation rate estimate will be refined as should_start_gc() is again sampled every ms or so.

I think rate is used below as an argument to a log_trigger() invocation.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23827#discussion_r1974166806

From kdnilsen at openjdk.org  Thu Feb 27 19:21:32 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 19:21:32 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap [v2]
In-Reply-To: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
Message-ID: <cZpothkS7Fbn3IzFWeqzcpRtzL0sye0YWSF_pL8X-U0=.ddb5efce-9a7c-48e7-ad37-e3e5516b7893@github.com>

> Original implementation was not robust to overriding of CardSizeInBytes, especially to smaller values.  This fixes that issue.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Respond to reviewer feedback

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23373/files
  - new: https://git.openjdk.org/jdk/pull/23373/files/7120cdf3..cf06bf0d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23373&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23373&range=00-01

  Stats: 4 lines in 1 file changed: 0 ins; 2 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/23373.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23373/head:pull/23373

PR: https://git.openjdk.org/jdk/pull/23373

From kdnilsen at openjdk.org  Thu Feb 27 19:26:56 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 19:26:56 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap [v2]
In-Reply-To: <nFs7nHQXgYTdknbVHRKj1yLdxSSMna8kOnptANParFI=.9ac8dce0-76f5-4476-b7b6-21e6f7e46dd5@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
 <nFs7nHQXgYTdknbVHRKj1yLdxSSMna8kOnptANParFI=.9ac8dce0-76f5-4476-b7b6-21e6f7e46dd5@github.com>
Message-ID: <YjRGyNwA2QuuBmsGNOD7sawe7zAgvvYEsLQW9AIkua4=.7e335a51-820e-4536-a8b1-dc8e50d7e96e@github.com>

On Thu, 27 Feb 2025 17:49:16 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Respond to reviewer feedback
>
> src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp line 192:
> 
>> 190: 
>> 191:   if (GCCardSizeInBytes < ShenandoahMinCardSizeInBytes) {
>> 192:     char buf[512];
> 
> It looks like using `err_msg` here is more idiomatic than `os:snprintf` for errors during initialization.

Thanks.  I've made this change.

> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 950:
> 
>> 948:   ShenandoahHeap* const heap = ShenandoahHeap::heap();
>> 949:   assert(heap->is_concurrent_weak_root_in_progress(), "Only during this phase");
>> 950:   {
> 
> This looks like it came from https://github.com/openjdk/jdk/pull/23604. Did you cherry-pick that into this branch? Not sure why it shows in the diff here.

Looks like I accidentally did cherry-pick into my gitfarm variant of this branch, and then merged into the github version of the branch.  The merge history must have gotten confused.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23373#discussion_r1974210655
PR Review Comment: https://git.openjdk.org/jdk/pull/23373#discussion_r1974206670

From kdnilsen at openjdk.org  Thu Feb 27 19:26:56 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 19:26:56 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap [v2]
In-Reply-To: <YjRGyNwA2QuuBmsGNOD7sawe7zAgvvYEsLQW9AIkua4=.7e335a51-820e-4536-a8b1-dc8e50d7e96e@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
 <nFs7nHQXgYTdknbVHRKj1yLdxSSMna8kOnptANParFI=.9ac8dce0-76f5-4476-b7b6-21e6f7e46dd5@github.com>
 <YjRGyNwA2QuuBmsGNOD7sawe7zAgvvYEsLQW9AIkua4=.7e335a51-820e-4536-a8b1-dc8e50d7e96e@github.com>
Message-ID: <ON95b3lace_YSgF4ztkjk3aOgun0HQ0SLO861Kp6Uj0=.967715ee-90b8-4a38-a473-9b9c342f52e2@github.com>

On Thu, 27 Feb 2025 19:22:31 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 950:
>> 
>>> 948:   ShenandoahHeap* const heap = ShenandoahHeap::heap();
>>> 949:   assert(heap->is_concurrent_weak_root_in_progress(), "Only during this phase");
>>> 950:   {
>> 
>> This looks like it came from https://github.com/openjdk/jdk/pull/23604. Did you cherry-pick that into this branch? Not sure why it shows in the diff here.
>
> Looks like I accidentally did cherry-pick into my gitfarm variant of this branch, and then merged into the github version of the branch.  The merge history must have gotten confused.

would it help if I cherry-pick the same commit directly into github?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23373#discussion_r1974209324

From wkemper at openjdk.org  Thu Feb 27 19:30:52 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 19:30:52 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap [v2]
In-Reply-To: <ON95b3lace_YSgF4ztkjk3aOgun0HQ0SLO861Kp6Uj0=.967715ee-90b8-4a38-a473-9b9c342f52e2@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
 <nFs7nHQXgYTdknbVHRKj1yLdxSSMna8kOnptANParFI=.9ac8dce0-76f5-4476-b7b6-21e6f7e46dd5@github.com>
 <YjRGyNwA2QuuBmsGNOD7sawe7zAgvvYEsLQW9AIkua4=.7e335a51-820e-4536-a8b1-dc8e50d7e96e@github.com>
 <ON95b3lace_YSgF4ztkjk3aOgun0HQ0SLO861Kp6Uj0=.967715ee-90b8-4a38-a473-9b9c342f52e2@github.com>
Message-ID: <KKwxzSoCguqezrYzjWIih7zOVEJxnOBtUF9BH4Ohyo8=.0a0df694-79c6-408c-b614-1ecce8760fab@github.com>

On Thu, 27 Feb 2025 19:23:49 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Looks like I accidentally did cherry-pick into my gitfarm variant of this branch, and then merged into the github version of the branch.  The merge history must have gotten confused.
>
> would it help if I cherry-pick the same commit directly into github?

I think you could try this on your `fix-small-card-size` branch for this PR:

% git revert 7120cdf36

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23373#discussion_r1974216887

From kdnilsen at openjdk.org  Thu Feb 27 19:43:47 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 19:43:47 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap [v3]
In-Reply-To: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
Message-ID: <T0l6w55XmLmxDjxxrDm29UBH4VYZeuazPJOPY-qdmAU=.492bbb6f-a378-452c-8c6f-5fed25b57625@github.com>

> Original implementation was not robust to overriding of CardSizeInBytes, especially to smaller values.  This fixes that issue.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Revert "8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)"
  
  This reverts commit 7120cdf36c1657a250fd3e60136e7b615fc7b538.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23373/files
  - new: https://git.openjdk.org/jdk/pull/23373/files/cf06bf0d..1fcdc869

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23373&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23373&range=01-02

  Stats: 19 lines in 1 file changed: 0 ins; 14 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/23373.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23373/head:pull/23373

PR: https://git.openjdk.org/jdk/pull/23373

From kdnilsen at openjdk.org  Thu Feb 27 19:43:48 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 19:43:48 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap [v3]
In-Reply-To: <KKwxzSoCguqezrYzjWIih7zOVEJxnOBtUF9BH4Ohyo8=.0a0df694-79c6-408c-b614-1ecce8760fab@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
 <nFs7nHQXgYTdknbVHRKj1yLdxSSMna8kOnptANParFI=.9ac8dce0-76f5-4476-b7b6-21e6f7e46dd5@github.com>
 <YjRGyNwA2QuuBmsGNOD7sawe7zAgvvYEsLQW9AIkua4=.7e335a51-820e-4536-a8b1-dc8e50d7e96e@github.com>
 <ON95b3lace_YSgF4ztkjk3aOgun0HQ0SLO861Kp6Uj0=.967715ee-90b8-4a38-a473-9b9c342f52e2@github.com>
 <KKwxzSoCguqezrYzjWIih7zOVEJxnOBtUF9BH4Ohyo8=.0a0df694-79c6-408c-b614-1ecce8760fab@github.com>
Message-ID: <H389kfpwPSH-cUTT5VXLYyrCZh_ax-TEnXUafdYvzmo=.3aecab15-6308-460f-a50a-40dfc39886cb@github.com>

On Thu, 27 Feb 2025 19:28:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> would it help if I cherry-pick the same commit directly into github?
>
> I think you could try this on your `fix-small-card-size` branch for this PR:
> 
> % git revert 7120cdf36

Thanks.  Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23373#discussion_r1974233880

From wkemper at openjdk.org  Thu Feb 27 19:56:17 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 19:56:17 GMT
Subject: RFR: 8350898: Shenandoah: Eliminate final roots safepoint
Message-ID: <aPh3XMgWPAaOOiGRM2fTFSBxyx_vuazdtOwhDqdcI2k=.28f41b44-de80-4e82-b6b9-1754e7023568@github.com>

This PR converts the final roots safepoint operation into a handshake. The safepoint operation still exists, but is only executed when `ShenandoahVerify` is enabled. In addition to this change, this PR also improves the logging for the concurrent preparation for update references from [PR 22688](https://github.com/openjdk/jdk/pull/22688).

-------------

Commit messages:
 - Fix comments
 - Add whitespace at end of file
 - More detail for init update refs event message
 - Use timing tracker for timing verification
 - Merge remote-tracking branch 'jdk/master' into eliminate-final-roots
 - WIP: Fix up phase timings for newly concurrent final roots and init update refs
 - WIP: Combine satb transfer with state propagation, restore phase timing data
 - WIP: Transfer pointers out of SATB with a handshake
 - WIP: Clear weak roots flag concurrently

Changes: https://git.openjdk.org/jdk/pull/23830/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23830&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8350898
  Stats: 291 lines in 14 files changed: 194 ins; 47 del; 50 mod
  Patch: https://git.openjdk.org/jdk/pull/23830.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23830/head:pull/23830

PR: https://git.openjdk.org/jdk/pull/23830

From wkemper at openjdk.org  Thu Feb 27 21:07:00 2025
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Feb 2025 21:07:00 GMT
Subject: RFR: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap [v3]
In-Reply-To: <T0l6w55XmLmxDjxxrDm29UBH4VYZeuazPJOPY-qdmAU=.492bbb6f-a378-452c-8c6f-5fed25b57625@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
 <T0l6w55XmLmxDjxxrDm29UBH4VYZeuazPJOPY-qdmAU=.492bbb6f-a378-452c-8c6f-5fed25b57625@github.com>
Message-ID: <ZpAIQ9n0TYpFl3PDxVliX7D2FhVONayv1j52nFsrzMo=.58bcfd91-178f-4ac3-b249-3eec228e66be@github.com>

On Thu, 27 Feb 2025 19:43:47 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Original implementation was not robust to overriding of CardSizeInBytes, especially to smaller values.  This fixes that issue.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Revert "8348092: Shenandoah: assert(nk >= _lowest_valid_narrow_klass_id && nk <= _highest_valid_narrow_klass_id) failed: narrowKlass ID out of range (3131947710)"
>   
>   This reverts commit 7120cdf36c1657a250fd3e60136e7b615fc7b538.

Marked as reviewed by wkemper (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/23373#pullrequestreview-2649079470

From kdnilsen at openjdk.org  Thu Feb 27 23:13:07 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 23:13:07 GMT
Subject: Integrated: 8347804: GenShen: Crash with small GCCardSizeInBytes and
 small Java heap
In-Reply-To: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
References: <sRGNBMmcW2QUswlHHWVsxAo3SXmabLyt8xpCrApbnrk=.f8754f91-9b9a-438c-a9e0-c66aa7b9265a@github.com>
Message-ID: <0JEkZIRntf6CSn4JnUeSOLPvbp4GaY_Ci624kY5wRms=.d33ea15d-3e2c-4af8-ba92-a782a5c8ef37@github.com>

On Thu, 30 Jan 2025 18:55:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Original implementation was not robust to overriding of CardSizeInBytes, especially to smaller values.  This fixes that issue.

This pull request has now been integrated.

Changeset: 0a4c5a8a
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/0a4c5a8a483b23ec8c534054187c44f986d137bb
Stats:     86 lines in 5 files changed: 63 ins; 4 del; 19 mod

8347804: GenShen: Crash with small GCCardSizeInBytes and small Java heap

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/jdk/pull/23373

From kdnilsen at openjdk.org  Thu Feb 27 23:26:57 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Feb 2025 23:26:57 GMT
Subject: Integrated: 8350889: GenShen: Break out of infinite loop of old GC
 cycles
In-Reply-To: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
References: <BKAiObLYjj1OqE_oW2S0WyVbZvIvv1rsFBFLT1Ti-yk=.3d82b503-3de0-49fc-ab93-cd27af9e7036@github.com>
Message-ID: <ic4OrDFJJkyQlGBgCpoHYQk-AZ9-310ivf1qGUZvC_8=.bda8ebc3-f4f5-4483-9be4-158b3125ccdf@github.com>

On Thu, 27 Feb 2025 16:58:49 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Add log message when heuristic trigger because previous trigger is pending.  Cancel the pending-trigger condition for old-generation at the start of old-generation GC.

This pull request has now been integrated.

Changeset: ab4b0ef9
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/ab4b0ef9242a4cd964fbcf2d1f3d370234c09408
Stats:     2 lines in 2 files changed: 2 ins; 0 del; 0 mod

8350889: GenShen: Break out of infinite loop of old GC cycles

Reviewed-by: wkemper, ysr

-------------

PR: https://git.openjdk.org/jdk/pull/23827

From xpeng at openjdk.org  Fri Feb 28 00:08:22 2025
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 28 Feb 2025 00:08:22 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v9]
In-Reply-To: <6duTgo8vKHyCUnasOsrHp341B2krxcK8jNogKjX09gs=.af63669e-9c8d-4f17-b055-bf3a03a9618e@github.com>
References: <6duTgo8vKHyCUnasOsrHp341B2krxcK8jNogKjX09gs=.af63669e-9c8d-4f17-b055-bf3a03a9618e@github.com>
Message-ID: <Dg0a2ZAOycpiNFziiVJzJvDLPS0nrWMHbzYgO2Woqng=.28db769a-217a-494c-9ed8-20be71638668@github.com>

> Reset marking bitmaps after collection cycle; for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with Shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly since in most case bitmap for young gen should have been reset after pervious concurrent cycle finishes if there is no need to preserve bitmap states.
> 
> GenShen:
> Before:
> 
> [33.342s][info][gc,stats    ] Concurrent Reset               =    0.023 s (a =     1921 us) (n =    12) (lvls, us =      133,      385,     1191,     1836,     8878)
> 
> 
> After:
> 
> [33.597s][info][gc,stats    ] Concurrent Reset               =    0.004 s (a =      317 us) (n =    13) (lvls, us =       58,      119,      217,      410,      670)
> [33.597s][info][gc,stats    ] Concurrent Reset After Collect =    0.018 s (a =     1365 us) (n =    13) (lvls, us =       91,      186,      818,     1836,     3872)
> 
> 
> Shenandoah:
> Before:
> 
> [33.144s][info][gc,stats    ] Concurrent Reset               =    0.014 s (a =     1067 us) (n =    13) (lvls, us =      139,      277,      898,     1328,     2118)
> 
> After:
> 
> [33.128s][info][gc,stats    ] Concurrent Reset               =    0.003 s (a =      225 us) (n =    13) (lvls, us =       32,       92,      137,      295,      542)
> [33.128s][info][gc,stats    ] Concurrent Reset After Collect =    0.009 s (a =      661 us) (n =    13) (lvls, us =       92,      160,      594,      896,     1661)
> 
> 
> Additional changes:
> * Remove `ShenandoahResetBitmapClosure` and  `ShenandoahPrepareForMarkClosure`, merge the code with `ShenandoahResetBitmapClosure`, saving one iteration over all the regions. 
> * Use API `ShenandoahGeneration::parallel_heap_region_iterate_free` to iterate the regions, two benefits from this:
>   -  Underneath it calls `ShenandoahHeap::parallel_heap_region_iterate`, which is faster for very light tasks, see https://bugs.openjdk.org/browse/JDK-8337154
>   -  `ShenandoahGeneration::parallel_heap_region_iterate_free` decorate the closure with `ShenandoahExcludeRegionClosure`, which simplifies the code in closure.
> * When `_do_old_gc_bootstrap is true`, instead of reset mark bitmap for old gen separately, simply reset the global generations, so we don't need walk the all regions twice. 
> * Clean up FullGC code, remove duplicate code.
> 
> Additional tests:
> - [x] CONF=macosx-aarch64-server-fastdebug make test T...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 25 additional commits since the last revision:

 - Merge branch 'openjdk:master' into reset-bitmap
 - Merge branch 'openjdk:master' into reset-bitmap
 - Merge branch 'openjdk:master' into reset-bitmap
 - Merge branch 'openjdk:master' into reset-bitmap
 - Merge branch 'openjdk:master' into reset-bitmap
 - Adding condition "!_do_old_gc_bootstrap && !heap->is_concurrent_old_mark_in_progress()" back and address some PR comments
 - Remove entry_reset_after_collect from ShenandoahOldGC
 - Remove condition check !_do_old_gc_bootstrap && !heap->is_concurrent_old_mark_in_progress() from op_reset_after_collect
 - Merge branch 'openjdk:master' into reset-bitmap
 - Address review comments
 - ... and 15 more: https://git.openjdk.org/jdk/compare/c65af1fb...7eea9556

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22778/files
  - new: https://git.openjdk.org/jdk/pull/22778/files/c7e9bff3..7eea9556

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22778&range=08
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22778&range=07-08

  Stats: 20235 lines in 598 files changed: 11195 ins; 6911 del; 2129 mod
  Patch: https://git.openjdk.org/jdk/pull/22778.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22778/head:pull/22778

PR: https://git.openjdk.org/jdk/pull/22778

From kdnilsen at openjdk.org  Fri Feb 28 15:58:00 2025
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 28 Feb 2025 15:58:00 GMT
Subject: RFR: 8350605: assert(!heap->is_uncommit_in_progress()) failed:
 Cannot uncommit bitmaps while resetting them
In-Reply-To: <s5DmlLfT-EWw85EQFM_PH0wgGWeWLUMz6C9QiXQAR_4=.c2495042-f478-4d14-8c3c-1eab60ca68c2@github.com>
References: <s5DmlLfT-EWw85EQFM_PH0wgGWeWLUMz6C9QiXQAR_4=.c2495042-f478-4d14-8c3c-1eab60ca68c2@github.com>
Message-ID: <BNbdDRfS0JoWl4v1mlUAM2tkbex74RuvSfcR-XFmvbU=.20a5a8a6-d23e-4654-8f86-618a9e77282d@github.com>

On Tue, 25 Feb 2025 01:38:14 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The protocol which is meant to prevent regions from being uncommitted while their bitmaps are being reset may fail. This happens when the control thread attempts to wait for the uncommit thread to finish, but the uncommit thread has not yet indicated that it has started.
> 
> ## Testing
> GHA, Dacapo, Extremem, Heapothesys, Diluvian, SpecJBB2015, SpecJVM2008 (with and without stress flags, asserts). Also have run the JTREG test that failed this assertion over 10K times (and counting).

Looks good.  Most of our pipeline tests are running with fixed size heap and pretouch.  Do we have good test coverage of  scattered uncommitted regions?

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 146:

> 144:   log_info(gc, start)("%s", msg);
> 145: 
> 146:   const size_t uncommitted_region_count = do_uncommit_work(shrink_before, shrink_until);

a comment here might also be helpful.  i think this is newly uncommitted by the current call.  there may be other preexisting uncommitted regions?

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.hpp line 63:

> 61:   bool is_uncommit_allowed() const;
> 62: 
> 63:   size_t do_uncommit_work(double shrink_before, size_t shrink_until) const;

A comment describing arguments and return value would be nice.  Mention premature termination if gc is requested.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/jdk/pull/23760#pullrequestreview-2651140350
PR Review Comment: https://git.openjdk.org/jdk/pull/23760#discussion_r1975639431
PR Review Comment: https://git.openjdk.org/jdk/pull/23760#discussion_r1975635358

From wkemper at openjdk.org  Fri Feb 28 17:17:17 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Feb 2025 17:17:17 GMT
Subject: RFR: 8349094: GenShen: Race between control and regulator threads
 may violate assertions [v18]
In-Reply-To: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
References: <iYHgN7gLOt2ylFuBwLVcNp2Nbc1RcT18NpPsZbiUq30=.52bbb3ae-b36f-46b4-afe1-7040057c9f97@github.com>
Message-ID: <bkVP6S0m_c6_L2PO0wzcui7WFUjhOxq8MVFAB7rooWg=.d9517fda-d338-439e-aff8-eb89c5bd924a@github.com>

> There are several changes to the operation of Shenandoah's control threads here.
> * The reason for cancellation is now recorded in `ShenandoahHeap::_cancelled_gc` as a `GCCause`, instead of various member variables in the control thread.
>   *  The cancellation handling is driven entirely by the cancellation cause
>   * The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
>   * The shutdown sequence is simpler
> * The generational control thread uses a lock to coordinate updates to the requested cause and generation
>   * APIs have been simplified to avoid converting between the generation `type` and the actual generation instance
>   * The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
>   *  The control thread doesn't loop on its own (unless the pacer is enabled).
> 
> ## Testing
> * jtreg hotspot_gc_shenandoah
> * dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 37 commits:

 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - Don't check for shutdown in control thread loop condition
   
   It may cause the thread to exit before it is requested to stop
 - Add assertions about old gen state when resuming old cycles
 - Remove duplicated field pointer for old generation
 - Improve names and comments
 - Merge tag 'jdk-25+11' into fix-control-regulator-threads
   
   Added tag jdk-25+11 for changeset 0131c1bf
 - Address review feedback (better comments, better names)
 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - Old gen bootstrap cycle must make it to init mark
 - Merge remote-tracking branch 'jdk/master' into fix-control-regulator-threads
 - ... and 27 more: https://git.openjdk.org/jdk/compare/e98df71d...37e445d6

-------------

Changes: https://git.openjdk.org/jdk/pull/23475/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23475&range=17
  Stats: 963 lines in 18 files changed: 327 ins; 294 del; 342 mod
  Patch: https://git.openjdk.org/jdk/pull/23475.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475

PR: https://git.openjdk.org/jdk/pull/23475

From wkemper at openjdk.org  Fri Feb 28 17:37:16 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Feb 2025 17:37:16 GMT
Subject: RFR: 8350605: assert(!heap->is_uncommit_in_progress()) failed:
 Cannot uncommit bitmaps while resetting them [v2]
In-Reply-To: <s5DmlLfT-EWw85EQFM_PH0wgGWeWLUMz6C9QiXQAR_4=.c2495042-f478-4d14-8c3c-1eab60ca68c2@github.com>
References: <s5DmlLfT-EWw85EQFM_PH0wgGWeWLUMz6C9QiXQAR_4=.c2495042-f478-4d14-8c3c-1eab60ca68c2@github.com>
Message-ID: <q6-Ar0u3I85_fltEvCDJl2P0W_h0Bv2xMuBu97QLnrQ=.0d50e72a-1052-4a60-96ac-0387873f0840@github.com>

> The protocol which is meant to prevent regions from being uncommitted while their bitmaps are being reset may fail. This happens when the control thread attempts to wait for the uncommit thread to finish, but the uncommit thread has not yet indicated that it has started.
> 
> ## Testing
> GHA, Dacapo, Extremem, Heapothesys, Diluvian, SpecJBB2015, SpecJVM2008 (with and without stress flags, asserts). Also have run the JTREG test that failed this assertion over 10K times (and counting).

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Improve comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23760/files
  - new: https://git.openjdk.org/jdk/pull/23760/files/ec13274c..b194db8f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23760&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23760&range=00-01

  Stats: 2 lines in 2 files changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23760.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23760/head:pull/23760

PR: https://git.openjdk.org/jdk/pull/23760

From wkemper at openjdk.org  Fri Feb 28 17:44:36 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Feb 2025 17:44:36 GMT
Subject: RFR: 8350605: assert(!heap->is_uncommit_in_progress()) failed:
 Cannot uncommit bitmaps while resetting them [v3]
In-Reply-To: <s5DmlLfT-EWw85EQFM_PH0wgGWeWLUMz6C9QiXQAR_4=.c2495042-f478-4d14-8c3c-1eab60ca68c2@github.com>
References: <s5DmlLfT-EWw85EQFM_PH0wgGWeWLUMz6C9QiXQAR_4=.c2495042-f478-4d14-8c3c-1eab60ca68c2@github.com>
Message-ID: <D8mEnT9GkVd6EV5781dzw9jjakhEWNtCOD4XMhEmxa4=.56f043c2-0bca-439c-8950-8b92c67d121b@github.com>

> The protocol which is meant to prevent regions from being uncommitted while their bitmaps are being reset may fail. This happens when the control thread attempts to wait for the uncommit thread to finish, but the uncommit thread has not yet indicated that it has started.
> 
> ## Testing
> GHA, Dacapo, Extremem, Heapothesys, Diluvian, SpecJBB2015, SpecJVM2008 (with and without stress flags, asserts). Also have run the JTREG test that failed this assertion over 10K times (and counting).

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Comment tweak

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/23760/files
  - new: https://git.openjdk.org/jdk/pull/23760/files/b194db8f..1c32c0e3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=23760&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23760&range=01-02

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/23760.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23760/head:pull/23760

PR: https://git.openjdk.org/jdk/pull/23760

From wkemper at openjdk.org  Fri Feb 28 17:47:53 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Feb 2025 17:47:53 GMT
Subject: RFR: 8350605: assert(!heap->is_uncommit_in_progress()) failed:
 Cannot uncommit bitmaps while resetting them [v3]
In-Reply-To: <D8mEnT9GkVd6EV5781dzw9jjakhEWNtCOD4XMhEmxa4=.56f043c2-0bca-439c-8950-8b92c67d121b@github.com>
References: <s5DmlLfT-EWw85EQFM_PH0wgGWeWLUMz6C9QiXQAR_4=.c2495042-f478-4d14-8c3c-1eab60ca68c2@github.com>
 <D8mEnT9GkVd6EV5781dzw9jjakhEWNtCOD4XMhEmxa4=.56f043c2-0bca-439c-8950-8b92c67d121b@github.com>
Message-ID: <WyqV0TZqUmkvbdsXBjDok5dO0_ti9y5llspwTuiNE_A=.0aed4752-fc21-44cc-99ec-2ec2a808ea28@github.com>

On Fri, 28 Feb 2025 17:44:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> The protocol which is meant to prevent regions from being uncommitted while their bitmaps are being reset may fail. This happens when the control thread attempts to wait for the uncommit thread to finish, but the uncommit thread has not yet indicated that it has started.
>> 
>> ## Testing
>> GHA, Dacapo, Extremem, Heapothesys, Diluvian, SpecJBB2015, SpecJVM2008 (with and without stress flags, asserts). Also have run the JTREG test that failed this assertion over 10K times (and counting).
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Comment tweak

That's a good point. I created a branch that enables uncommit for the test pipelines when I made this original change. I'll resurrect that branch and run that configuration again. Thanks.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23760#issuecomment-2691218679

From dlong at openjdk.org  Fri Feb 28 20:38:53 2025
From: dlong at openjdk.org (Dean Long)
Date: Fri, 28 Feb 2025 20:38:53 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
 [v4]
In-Reply-To: <JpkNlc2S7obqyCkSQpKH4LmjQTYp89jTbhCAo_V9SzM=.7d7c09b4-9c13-485f-9236-ac96fbe5b57f@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
 <JpkNlc2S7obqyCkSQpKH4LmjQTYp89jTbhCAo_V9SzM=.7d7c09b4-9c13-485f-9236-ac96fbe5b57f@github.com>
Message-ID: <2jI87up85vKeQq7xy6WoI987MOuqTqA6I8G75VvC74g=.e8ef9f9c-b8b3-496d-9b48-28c83dc1fb64@github.com>

On Thu, 27 Feb 2025 12:02:44 GMT, Damon Fenacci <dfenacci at openjdk.org> wrote:

>> # Issue
>> The test `src/hotspot/share/opto/c2compiler.cpp` fails intermittently due to a crash that happens when trying to allocate code cache space for C1 and C2 in `RuntimeStub::new_runtime_stub` and `SingletonBlob::operator new`.
>> 
>> # Causes
>> There are a few call paths during the initialization of C1 and C2 that can lead to the code cache allocations in `RuntimeStub::new_runtime_stub` (through `RuntimeStub::operator new`) and `SingletonBlob::operator new` triggering a fatal error if there is no more space. The paths in question are:
>> 1. `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`
>> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`
>> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_uncommon_trap_blob` -> `UncommonTrapBlob::create` -> `new UncommonTrapBlob`
>> 1. `C2Compiler::initialize` -> `C2Compiler::init_c2_runtime` -> `OptoRuntime::generate` -> `OptoRuntime::generate_exception_blob` -> `ExceptionBlob::create` -> `new ExceptionBlob`
>> 
>> # Solution
>> Instead of fatally crashing the we can use the `alloc_fail_is_fatal` flag of `RuntimeStub::new_runtime_stub` to avoid crashing in cases 1 and 2 and add a similar flag to `SingletonBlob::operator new` for cases 3 and 4. In the latter case we need to adjust all calls accordingly.
>> 
>> Note: In [JDK-8326615](https://bugs.openjdk.org/browse/JDK-8326615) it was argued that increasing the minimum code cache size would solve the issue but that wasn't entirely accurate: doing so possibly decreases the chances of a failed allocation in these 4 places but doesn't totally avoid it.
>> 
>> # Testing
>> The original failing regression test in `test/hotspot/jtreg/compiler/startup/StartupOutput.java` has been modified to run multiple times with randomized values (within the original failing range) to increase the chances of hitting the fatal assertion.
>> 
>> Tests: Tier 1-4 (windows-x64, linux-x64/aarch64, and macosx-x64/aarch64; release and debug mode)
>
> Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:
> 
>   JDK-8347406: re-add modified assert

Refreshing my memory, isn't the real problem with trying to fix this with a minimum codecache size is that some of these stubs are not allocated during initial single-threaded JVM startup, but later when the first compiler threads start, and that allows other code blobs to fill up the codecache?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23630#issuecomment-2691503254

From dlong at openjdk.org  Fri Feb 28 20:38:54 2025
From: dlong at openjdk.org (Dean Long)
Date: Fri, 28 Feb 2025 20:38:54 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
 [v4]
In-Reply-To: <2jI87up85vKeQq7xy6WoI987MOuqTqA6I8G75VvC74g=.e8ef9f9c-b8b3-496d-9b48-28c83dc1fb64@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
 <JpkNlc2S7obqyCkSQpKH4LmjQTYp89jTbhCAo_V9SzM=.7d7c09b4-9c13-485f-9236-ac96fbe5b57f@github.com>
 <2jI87up85vKeQq7xy6WoI987MOuqTqA6I8G75VvC74g=.e8ef9f9c-b8b3-496d-9b48-28c83dc1fb64@github.com>
Message-ID: <eQMtkd6Y7PmKnLvdIaiHkIyx-KllJcp0_YzDBlJs6oM=.9d9b5b7a-786d-4acd-8dd8-5f4a9ecc990c@github.com>

On Fri, 28 Feb 2025 20:35:58 GMT, Dean Long <dlong at openjdk.org> wrote:

>> Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   JDK-8347406: re-add modified assert
>
> Refreshing my memory, isn't the real problem with trying to fix this with a minimum codecache size is that some of these stubs are not allocated during initial single-threaded JVM startup, but later when the first compiler threads start, and that allows other code blobs to fill up the codecache?

> Even so, it might be a good idea to additionally increase the minimum code cache anyway. @dean-long do you think it would make sense to file an RFE for that?

Sure, if it's still an issue.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23630#issuecomment-2691503902

From dlong at openjdk.org  Fri Feb 28 20:45:59 2025
From: dlong at openjdk.org (Dean Long)
Date: Fri, 28 Feb 2025 20:45:59 GMT
Subject: RFR: 8347406: [REDO] C1/C2 don't handle allocation failure
 properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
 [v4]
In-Reply-To: <RPdo7Z3j1KD18svu1bCQDuf8S9jTt4KmIonFpme6ffQ=.82298243-1f97-492e-acc3-9524eca5054b@github.com>
References: <JcK2u_y08D9uUqcuD20G5uP8shcKIoAfRIJrpprEy1A=.1823cd2c-29e0-42c6-8dec-0ca91afc40bc@github.com>
 <hDbvpph4D6KJc_tMrde62yCVY5DVdWncy0wjBz67fBk=.3ac9a977-857a-45df-a5e8-284f9e14af60@github.com>
 <RPdo7Z3j1KD18svu1bCQDuf8S9jTt4KmIonFpme6ffQ=.82298243-1f97-492e-acc3-9524eca5054b@github.com>
Message-ID: <rCw40mW6ywIVpwLQQTtJvCi3-p9HGPLRoURmljGbOSA=.6b1b8bf0-25ee-4685-a337-e0f9d4847a63@github.com>

On Thu, 27 Feb 2025 12:03:16 GMT, Damon Fenacci <dfenacci at openjdk.org> wrote:

>> src/hotspot/share/opto/output.cpp line 3487:
>> 
>>> 3485:         C->record_failure("CodeCache is full");
>>> 3486:       } else {
>>> 3487:         C->set_stub_entry_point(rs->entry_point());
>> 
>> Is the deleted rs->is_runtime_stub() assert still useful here?
>
> A slightly modified one surely is. Inserted it again.

I was thinking it could be moved into the `else` clause and simplified further.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23630#discussion_r1975989747

From wkemper at openjdk.org  Fri Feb 28 23:59:11 2025
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Feb 2025 23:59:11 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v9]
In-Reply-To: <Dg0a2ZAOycpiNFziiVJzJvDLPS0nrWMHbzYgO2Woqng=.28db769a-217a-494c-9ed8-20be71638668@github.com>
References: <6duTgo8vKHyCUnasOsrHp341B2krxcK8jNogKjX09gs=.af63669e-9c8d-4f17-b055-bf3a03a9618e@github.com>
 <Dg0a2ZAOycpiNFziiVJzJvDLPS0nrWMHbzYgO2Woqng=.28db769a-217a-494c-9ed8-20be71638668@github.com>
Message-ID: <MLUnq99jinAGCXE9CoKwHzetBaI2ZWvaL9ikNRaWCyk=.b27681a7-39d8-4324-b679-ec0d40927341@github.com>

On Fri, 28 Feb 2025 00:08:22 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Reset marking bitmaps after collection cycle; for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
>> 
>> I have run same workload for 30s with Shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly since in most case bitmap for young gen should have been reset after pervious concurrent cycle finishes if there is no need to preserve bitmap states.
>> 
>> GenShen:
>> Before:
>> 
>> [33.342s][info][gc,stats    ] Concurrent Reset               =    0.023 s (a =     1921 us) (n =    12) (lvls, us =      133,      385,     1191,     1836,     8878)
>> 
>> 
>> After:
>> 
>> [33.597s][info][gc,stats    ] Concurrent Reset               =    0.004 s (a =      317 us) (n =    13) (lvls, us =       58,      119,      217,      410,      670)
>> [33.597s][info][gc,stats    ] Concurrent Reset After Collect =    0.018 s (a =     1365 us) (n =    13) (lvls, us =       91,      186,      818,     1836,     3872)
>> 
>> 
>> Shenandoah:
>> Before:
>> 
>> [33.144s][info][gc,stats    ] Concurrent Reset               =    0.014 s (a =     1067 us) (n =    13) (lvls, us =      139,      277,      898,     1328,     2118)
>> 
>> After:
>> 
>> [33.128s][info][gc,stats    ] Concurrent Reset               =    0.003 s (a =      225 us) (n =    13) (lvls, us =       32,       92,      137,      295,      542)
>> [33.128s][info][gc,stats    ] Concurrent Reset After Collect =    0.009 s (a =      661 us) (n =    13) (lvls, us =       92,      160,      594,      896,     1661)
>> 
>> 
>> Additional changes:
>> * Remove `ShenandoahResetBitmapClosure` and  `ShenandoahPrepareForMarkClosure`, merge the code with `ShenandoahResetBitmapClosure`, saving one iteration over all the regions. 
>> * Use API `ShenandoahGeneration::parallel_heap_region_iterate_free` to iterate the regions, two benefits from this:
>>   -  Underneath it calls `ShenandoahHeap::parallel_heap_region_iterate`, which is faster for very light tasks, see https://bugs.openjdk.org/browse/JDK-8337154
>>   -  `ShenandoahGeneration::parallel_heap_region_iterate_free` decorate the closure with `ShenandoahExcludeRegionClosure`, which simplifies the code in closure.
>> * When `_do_old_gc_bootstrap is true`, instead of reset mark bitmap for old gen separately, simply reset the global generations, so we don't need walk the all regions twice. 
>> * Clean up FullGC code, remove duplicate code.
>> 
>> ...
>
> Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 25 additional commits since the last revision:
> 
>  - Merge branch 'openjdk:master' into reset-bitmap
>  - Merge branch 'openjdk:master' into reset-bitmap
>  - Merge branch 'openjdk:master' into reset-bitmap
>  - Merge branch 'openjdk:master' into reset-bitmap
>  - Merge branch 'openjdk:master' into reset-bitmap
>  - Adding condition "!_do_old_gc_bootstrap && !heap->is_concurrent_old_mark_in_progress()" back and address some PR comments
>  - Remove entry_reset_after_collect from ShenandoahOldGC
>  - Remove condition check !_do_old_gc_bootstrap && !heap->is_concurrent_old_mark_in_progress() from op_reset_after_collect
>  - Merge branch 'openjdk:master' into reset-bitmap
>  - Address review comments
>  - ... and 15 more: https://git.openjdk.org/jdk/compare/ad10cf10...7eea9556

Okay, these changes look good. We don't yet understand why we cannot reset young bitmaps during an old cycle, but we will follow up that investigation separately.

-------------

Marked as reviewed by wkemper (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22778#pullrequestreview-2651973439