From shade at openjdk.org  Tue Oct  1 13:32:41 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 1 Oct 2024 13:32:41 GMT
Subject: Integrated: 8341242: Shenandoah: LRB node is not matched as GC barrier
 after JDK-8340183
In-Reply-To: <QIPZMJIfweyc5taFSjRc3R1zPjHd6_xAvtyli0jW1iU=.c8dbf4eb-ca88-4f46-8703-55358599ab3c@github.com>
References: <QIPZMJIfweyc5taFSjRc3R1zPjHd6_xAvtyli0jW1iU=.c8dbf4eb-ca88-4f46-8703-55358599ab3c@github.com>
Message-ID: <joqwH3wTgPB43c-6ECcmtMaS_PUZgZlhIScOO36myB8=.174756ea-8b36-467e-859a-359304cc1fbc@github.com>

On Mon, 30 Sep 2024 14:54:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> [JDK-8340183](https://bugs.openjdk.org/browse/JDK-8340183) introduced a regression: `ShenandoahBarrierSetC2::is_gc_barrier_node` is now answering `false` for the actual `ShenandoahLoadReferenceBarrierNode`. The fix reinstates the check for LRB node.
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>  - [x] Linux x86_64 server fastdebug, `jdk/jfr/api/consumer/ ` (100x) -- used to fail intermittently, now it does not

This pull request has now been integrated.

Changeset: 684d246c
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/684d246ccf497f599ffcd498f2fbe4b1b2357e27
Stats:     2 lines in 1 file changed: 1 ins; 0 del; 1 mod

8341242: Shenandoah: LRB node is not matched as GC barrier after JDK-8340183

Reviewed-by: rkennke, phh

-------------

PR: https://git.openjdk.org/jdk/pull/21266

From shade at openjdk.org  Tue Oct  1 15:19:58 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 1 Oct 2024 15:19:58 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v6]
In-Reply-To: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
Message-ID: <i1tUBBIz9RyNNTRPbueZ8v63O3_kZYLOpzUazNdsSxQ=.aaa88873-a779-41af-b8a1-04ab606b877e@github.com>

> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
> 
> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
> 
> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
> 
> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
> 
> Additional testing:
>  - [x] `jdk/jfr/event/oldobject/` pass by default (100x times)
>  - [x] `jdk/jfr/event/oldobject/` pass with `-XX:+UseShenandoah` (1000x)

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:

 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Just exclude the tests for Shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Tighten up comments prose
 - Roman's review: more precise GC state check, more includes
 - Referencing INCLUDE_SHENANDOAHGC nominally requires macro.hpp
 - Try to make tests faster / more robust
 - Fix
   
   Fix
   
   Make JFR tests more reliable with new behavior with Shenandoah
   
   More precisely, only skip when Shenandoah already has forwarded objects
   
   Revert "More precisely, only skip when Shenandoah already has forwarded objects"
   
   This reverts commit 403824d4b27e091807a838c2c86b3228f75fe056.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20328/files
  - new: https://git.openjdk.org/jdk/pull/20328/files/249956fc..515c1869

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=04-05

  Stats: 260783 lines in 3028 files changed: 213404 ins; 30416 del; 16963 mod
  Patch: https://git.openjdk.org/jdk/pull/20328.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20328/head:pull/20328

PR: https://git.openjdk.org/jdk/pull/20328

From rkennke at openjdk.org  Tue Oct  1 15:48:54 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 1 Oct 2024 15:48:54 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v11]
In-Reply-To: <TBZCkcFNNnEvat7TBnRkqUPo4Y5TEe1L0iUZ9loD7DI=.39c0249c-f6b2-45c8-b7c8-9e2c9c1b7a99@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <DUgk6HT_Zk5KnbmAd_HzrJi4B8D1J-uuoAGzReWyI8s=.6c04f90a-bf1d-49af-a31d-5224973952d4@github.com>
 <YTte6oY9EeAPHxzzOndL8M202wY91SbeaYC4hPva0W0=.caf00270-89c7-44b0-ab88-a970fb8e840d@github.com>
 <kzodWwxj5qdAqPYcP2s4k2QTXqjju2H6Iftu7dojezk=.1f72fc31-94ba-4017-b330-c8688a8d39a4@github.com>
 <hkGaBWSugvpV0PgoR9rnutHCgQzkeXv1ctH7QwV3zK0=.204737d5-fadb-410a-8891-7db59d36830f@github.com>
 <shoDPnL4nQyaYX4xy-WyzQ7CwsoCbAeCE3c755CkE1o=.caf940aa-5fbb-4cb5-a4a2-68a2452ffe1b@github.com>
 <TBZCkcFNNnEvat7TBnRkqUPo4Y5TEe1L0iUZ9loD7DI=.39c0249c-f6b2-45c8-b7c8-9e2c9c1b7a99@github.com>
Message-ID: <toUdNzwkdDAQDltc4_t6wI06bCE-EHzpIs_t-arIwNY=.7d9ef348-0033-4268-9ed1-7c183855521f@github.com>

On Mon, 30 Sep 2024 12:38:03 GMT, Roberto Casta?eda Lozano <rcastanedalo at openjdk.org> wrote:

> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java:

I think I would disable the tests for now. Is there a good way to say 'run this when UCOH is off OR UseSSE>3?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2386370790

From wkemper at openjdk.org  Tue Oct  1 21:44:48 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 1 Oct 2024 21:44:48 GMT
Subject: RFR: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23:
 runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be
 represented in type 'long int' [v4]
In-Reply-To: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
References: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
Message-ID: <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>

> Use a template version of `right_n_bits` to use the same type for minuend and subtrahend.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Inline unnecessary usages of right_n_bits

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21236/files
  - new: https://git.openjdk.org/jdk/pull/21236/files/97d1272b..fdff7d68

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21236&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21236&range=02-03

  Stats: 37 lines in 3 files changed: 7 ins; 4 del; 26 mod
  Patch: https://git.openjdk.org/jdk/pull/21236.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21236/head:pull/21236

PR: https://git.openjdk.org/jdk/pull/21236

From wkemper at openjdk.org  Tue Oct  1 21:55:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 1 Oct 2024 21:55:36 GMT
Subject: RFR: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23:
 runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be
 represented in type 'long int' [v4]
In-Reply-To: <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
References: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
 <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
Message-ID: <KR5hHY_TVNijBf-reA0t01GBMmso9Gv-x7ZqfhF1xV4=.5bf733f1-41d9-45c3-8645-9bd1e92e1d81@github.com>

On Tue, 1 Oct 2024 21:44:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Use a template version of `right_n_bits` to use the same type for minuend and subtrahend.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Inline unnecessary usages of right_n_bits

I'd prefer to keep the scope of these changes limited to this file. This is the only place where this seems to be a problem in the JDK and I think it's just because we abused a macro we shouldn't have.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21236#issuecomment-2387142276

From wkemper at openjdk.org  Tue Oct  1 22:34:07 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 1 Oct 2024 22:34:07 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
 logging
Message-ID: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>

Not clean. Backport to only take lock for logging when log level is enabled.

-------------

Commit messages:
 - Backport c47a0e005e54551e42ee1ae33d7169417a5f86d4

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/113/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=113&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8334147
  Stats: 34 lines in 4 files changed: 13 ins; 11 del; 10 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/113.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/113/head:pull/113

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/113

From wkemper at openjdk.org  Tue Oct  1 23:12:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 1 Oct 2024 23:12:29 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
 logging [v2]
In-Reply-To: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
References: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
Message-ID: <EH5lrPGPrbkMipgsxB_f1lfprjZ6NYOydxL0eocyQvE=.46090626-bc11-4f00-87fb-1797ea62e535@github.com>

> Not clean. Backport to only take lock for logging when log level is enabled.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Update generational mode to use log status under lock

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/113/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/113/files/38ca9a85..d4ee79ea

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=113&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=113&range=00-01

  Stats: 14 lines in 1 file changed: 0 ins; 5 del; 9 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/113.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/113/head:pull/113

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/113

From kdnilsen at openjdk.org  Wed Oct  2 00:54:53 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 2 Oct 2024 00:54:53 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
Message-ID: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>

This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.

Efficiency improvements include:
1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.

Below, each trial runs for 1 hour, processing 28,000 transactions per second.

Without this change, latency for 4 un-named business services is represented by the following chart:
![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)

With this change, latency for the same services is much better:
![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)

A comparison of the two is provided by the following:
![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

-------------

Commit messages:
 - Respond to reviewer feedback
 - Tidy up comments and remove debug instrumentation
 - Recycle multiple regions before checking deadline
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Revert "Make GC logging less verbose"
 - Make GC logging less verbose
 - Merge branch 'openjdk:master' into master
 - ... and 15 more: https://git.openjdk.org/jdk/compare/5d062e24...acf517f5

Changes: https://git.openjdk.org/jdk/pull/21211/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21211&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341379
  Stats: 28 lines in 1 file changed: 22 ins; 3 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/21211.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21211/head:pull/21211

PR: https://git.openjdk.org/jdk/pull/21211

From wkemper at openjdk.org  Wed Oct  2 00:54:53 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 2 Oct 2024 00:54:53 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <_lvg95UW7HqXx5iAkndxKUNGHTmsXCLj7Ee5t-0SRCE=.47af2a9e-80b3-40d6-9348-407ee1518442@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 944:

> 942:       // yields.  Yielding more frequently when there is heavy contention for the heap lock or for CPU cores is considered the
> 943:       // right thing to do.
> 944:       const size_t REGION_STRIDE = 32;

Maybe call this `REGIONS_PER_BATCH`? When I see `stride` I think how far the loop index moves on each iteration.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21211#discussion_r1783520423

From kdnilsen at openjdk.org  Wed Oct  2 00:54:53 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 2 Oct 2024 00:54:53 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <_lvg95UW7HqXx5iAkndxKUNGHTmsXCLj7Ee5t-0SRCE=.47af2a9e-80b3-40d6-9348-407ee1518442@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <_lvg95UW7HqXx5iAkndxKUNGHTmsXCLj7Ee5t-0SRCE=.47af2a9e-80b3-40d6-9348-407ee1518442@github.com>
Message-ID: <ZYFlU1Ar7iARpbKqCXw33_hxezKblCdeRV_vrm5kFxs=.61fbc5d3-02ad-42c0-b824-ad656da94e45@github.com>

On Tue, 1 Oct 2024 20:53:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
>> 
>> Efficiency improvements include:
>> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
>> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
>> 
>> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
>> 
>> Without this change, latency for 4 un-named business services is represented by the following chart:
>> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
>> 
>> With this change, latency for the same services is much better:
>> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
>> 
>> A comparison of the two is provided by the following:
>> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 944:
> 
>> 942:       // yields.  Yielding more frequently when there is heavy contention for the heap lock or for CPU cores is considered the
>> 943:       // right thing to do.
>> 944:       const size_t REGION_STRIDE = 32;
> 
> Maybe call this `REGIONS_PER_BATCH`? When I see `stride` I think how far the loop index moves on each iteration.

Good suggestion.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21211#discussion_r1783668735

From kdnilsen at openjdk.org  Wed Oct  2 00:54:53 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 2 Oct 2024 00:54:53 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <ZYFlU1Ar7iARpbKqCXw33_hxezKblCdeRV_vrm5kFxs=.61fbc5d3-02ad-42c0-b824-ad656da94e45@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <_lvg95UW7HqXx5iAkndxKUNGHTmsXCLj7Ee5t-0SRCE=.47af2a9e-80b3-40d6-9348-407ee1518442@github.com>
 <ZYFlU1Ar7iARpbKqCXw33_hxezKblCdeRV_vrm5kFxs=.61fbc5d3-02ad-42c0-b824-ad656da94e45@github.com>
Message-ID: <L5ylS-C5nLOeaEwpNog3WFTlgLKHJshOfWYgnIdr9yY=.8b61c54a-d3b9-4889-a0fd-ee6552ec1c0f@github.com>

On Wed, 2 Oct 2024 00:24:00 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 944:
>> 
>>> 942:       // yields.  Yielding more frequently when there is heavy contention for the heap lock or for CPU cores is considered the
>>> 943:       // right thing to do.
>>> 944:       const size_t REGION_STRIDE = 32;
>> 
>> Maybe call this `REGIONS_PER_BATCH`? When I see `stride` I think how far the loop index moves on each iteration.
>
> Good suggestion.

Thanks.  I've made this change.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21211#discussion_r1783698490

From rcastanedalo at openjdk.org  Wed Oct  2 08:29:55 2024
From: rcastanedalo at openjdk.org (Roberto =?UTF-8?B?Q2FzdGHDsWVkYQ==?= Lozano)
Date: Wed, 2 Oct 2024 08:29:55 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v11]
In-Reply-To: <toUdNzwkdDAQDltc4_t6wI06bCE-EHzpIs_t-arIwNY=.7d9ef348-0033-4268-9ed1-7c183855521f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <DUgk6HT_Zk5KnbmAd_HzrJi4B8D1J-uuoAGzReWyI8s=.6c04f90a-bf1d-49af-a31d-5224973952d4@github.com>
 <YTte6oY9EeAPHxzzOndL8M202wY91SbeaYC4hPva0W0=.caf00270-89c7-44b0-ab88-a970fb8e840d@github.com>
 <kzodWwxj5qdAqPYcP2s4k2QTXqjju2H6Iftu7dojezk=.1f72fc31-94ba-4017-b330-c8688a8d39a4@github.com>
 <hkGaBWSugvpV0PgoR9rnutHCgQzkeXv1ctH7QwV3zK0=.204737d5-fadb-410a-8891-7db59d36830f@github.com>
 <shoDPnL4nQyaYX4xy-WyzQ7CwsoCbAeCE3c755CkE1o=.caf940aa-5fbb-4cb5-a4a2-68a2452ffe1b@github.com>
 <TBZCkcFNNnEvat7TBnRkqUPo4Y5TEe1L0iUZ9loD7DI=.39c0249c-f6b2-45c8-b7c8-9e2c9c1b7a99@github.com>
 <toUdNzwkdDAQDltc4_t6wI06bCE-EHzpIs_t-arIwNY=.7d9ef348-0033-4268-9ed1-7c183855521f@github.com>
Message-ID: <ES5Lj2Kg_a8L48WTpysLoDGS8TMHLMTbscJd3QVnAXA=.3eff8632-c747-40d1-a956-342cc45baca4@github.com>

On Tue, 1 Oct 2024 15:46:01 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> > test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java:
> 
> I think I would disable the tests for now. Is there a good way to say 'run this when UCOH is off OR UseSSE>3?

I don't think so, due to a [limitation in the IR framework precondition language](https://bugs.openjdk.org/browse/JDK-8294279): `UseCompactObjectHeaders` can only appear within a ["flag precondition"](https://github.com/openjdk/jdk/blob/efe3573b9b4ecec0630fdc1c61c765713a5b68e6/test/hotspot/jtreg/compiler/lib/ir_framework/IR.java#L109) whereas `UseSSE>3` needs to be expressed as a ["CPU feature precondition"](https://github.com/openjdk/jdk/blob/efe3573b9b4ecec0630fdc1c61c765713a5b68e6/test/hotspot/jtreg/compiler/lib/ir_framework/IR.java#L137C14-L137C31) for portability (`UseSSE` is not defined for aarch64), and these two cannot be combined with logical operators.

I suggest to disable the IR checks of the failing tests using `applyIf = {"UseCompactObjectHeaders", "false"}` as you did for other similar tests (e.g. `TestMulAddS2I.java`), and document it in [JDK-8340010](https://bugs.openjdk.org/browse/JDK-8340010). Maybe also comment in the tests that the failure happens only with `-XX:UseSSE<=3`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2387906401

From rkennke at openjdk.org  Wed Oct  2 15:37:40 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 2 Oct 2024 15:37:40 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v29]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <migFoRhtd3kovSs2fRSGXzdh7vCpbyazoT5Gwzeu8ks=.4bd5afae-9598-4ab3-a7d8-737281236a4e@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with three additional commits since the last revision:

 - Revert "Disable TestSplitPacks::test4a, failing on aarch64"
   
   This reverts commit 059b1573db26d1d2902ca6dadc8413f445234c2a.
 - Simplify object init code in interpreter
 - Disable some vectorization tests that fail with +UCOH and UseSSE<=3

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/059b1573..aea8f00c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=28
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=27-28

  Stats: 47 lines in 6 files changed: 18 ins; 13 del; 16 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From kdnilsen at openjdk.org  Wed Oct  2 16:47:37 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 2 Oct 2024 16:47:37 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <_V337A0U9_TeBPs26Q6QyOfvmYoaqxOxl3MaRrhI16s=.dc7e86a3-a8ff-4617-995c-1417cac18bb8@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

I've got a bit more information about the differences in behavior between no-batch trial 1 and trial 2:
1. Note that trial2 has much worse p9999 latency than trial1
2. The difference is NOT safepoint behavior.  Trial 1 actually had more safepoints that lasted longer than 1 ms, with the longest lasting 5.658220ms.  The longest safepoint in trial 2 was 3.420009 ms.
3. There is evidence to suggest that the difference stems from concurrent cleanup: trial1 had 1 concurrent cleanup event taking more than 1 ms, with time of 1.142 ms, average cleanup time of 85.1 us; trial 3 had 3 concurrent cleanup events taking more than 1 ms, with the max of 1.377 ms, average cleanup time of 85.8 us.
4. For comparison, the three runs with this fix had an average concurrent cleanup event time of 69.8 us.

Qualitative assessment:

This fix allows concurrent cleanup to happen on average in 18.3% less time.  This means it is less likely to collide with a mutator thread in access to the shared heap lock.

When a collision does occur, it is resolved more quickly, allowing the the mutator to proceed in no more than 8 us plus the time to process one batch of 32 regions rather than having to wait a max of 30 us.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2389135297

From coleenp at openjdk.org  Wed Oct  2 17:37:54 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Wed, 2 Oct 2024 17:37:54 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v29]
In-Reply-To: <migFoRhtd3kovSs2fRSGXzdh7vCpbyazoT5Gwzeu8ks=.4bd5afae-9598-4ab3-a7d8-737281236a4e@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <migFoRhtd3kovSs2fRSGXzdh7vCpbyazoT5Gwzeu8ks=.4bd5afae-9598-4ab3-a7d8-737281236a4e@github.com>
Message-ID: <b0uXxDtd9PwrZvPhuZyn24UTaA8G4icxiQ5pX7qL4Bo=.c6c662fa-cae9-4f4c-885d-62ae9d3fe697@github.com>

On Wed, 2 Oct 2024 15:37:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Revert "Disable TestSplitPacks::test4a, failing on aarch64"
>    
>    This reverts commit 059b1573db26d1d2902ca6dadc8413f445234c2a.
>  - Simplify object init code in interpreter
>  - Disable some vectorization tests that fail with +UCOH and UseSSE<=3

Thanks for making this change.  I've reviewed runtime, oops and metaspace code.  It looks good.

-------------

Marked as reviewed by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2343632318

From wkemper at openjdk.org  Wed Oct  2 18:26:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 2 Oct 2024 18:26:35 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <LFYBkILEH4Hgox_fIxyXWvN-RQO2Lbvcn1N7CENSluw=.17b563cf-c328-46a6-8dff-dfdfae0a3c02@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

Looks good. Appreciate the comprehensive analysis.

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/jdk/pull/21211#pullrequestreview-2343780503

From xpeng at openjdk.org  Wed Oct  2 19:47:34 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 2 Oct 2024 19:47:34 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <zq_j_unmN8FkRruxA_NbfzeHi-XI28-EzWLQRNDNTtw=.f2d78be8-416f-46c7-9585-c7dae80806b4@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

Marked as reviewed by xpeng (Author).

Looks good to me, thanks for the detailed analysis!  

My understanding: the major benefits from this PR:
1. Minimize the cost on system call os::javaTimeNanos()
2. Not to start next batch if next it has hold the heap lock over 8us when current batch finishes.

Maybe we should calculate the next batch size based on the reminding time and speed after a batch, the assumption 200ns/region is unlikely to be true for all hardwares, in the worse case, it may hold the lock for up to 15us~16us,

-------------

PR Review: https://git.openjdk.org/jdk/pull/21211#pullrequestreview-2343932607
PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2389542138

From kdnilsen at openjdk.org  Wed Oct  2 19:55:35 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 2 Oct 2024 19:55:35 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <V37pp-DnpiMtgXG9CboWMuh29cWEUOm5xKj6mMxb2pQ=.fe5f74f0-a452-4ca1-b8af-39fd8509efb8@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

It's hard to "predict" the cost of the next batch.  It depends on how many of the regions within the batch might have already been recycled by on-demand actions of mutator threads.  In general, its seems the cost of a batch is typically less than 4 us, because I gathered some stats with instrumentation (no longer present) that says typical number of batches processed between yield is 3.  The first two batches, including the time to yield and acquire lock, must have completed in less than 8 us, or we would not have allowed ourself to start another batch.

True, a different hardware might take more or less time to process batch, but I don't think the behavior will be too sensitive to this.  If it takes less than 2.66 us, then we might get an average of 4 batches processed between yields.  If it takes longer than 8 us, we'll only get 1 batch processed between yields.  But we'll still be making good progress on recycling the trashed regions while remaining very responsive to mutator needs to access the heap lock.

Thanks for reviews.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2389557309
PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2389559197

From phh at openjdk.org  Wed Oct  2 19:56:36 2024
From: phh at openjdk.org (Paul Hohensee)
Date: Wed, 2 Oct 2024 19:56:36 GMT
Subject: RFR: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23:
 runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be
 represented in type 'long int' [v4]
In-Reply-To: <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
References: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
 <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
Message-ID: <BZ2VMlFF2hi7waSqhiuskV9xFtJokL8HF7tGY0VviA8=.a6fc0f98-a243-4159-b87a-927d79e9aaf9@github.com>

On Tue, 1 Oct 2024 21:44:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Use a template version of `right_n_bits` to use the same type for minuend and subtrahend.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Inline unnecessary usages of right_n_bits

Not using right_n_bits() at works for me.

-------------

Marked as reviewed by phh (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21236#pullrequestreview-2343946776

From kdnilsen at openjdk.org  Wed Oct  2 20:09:38 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 2 Oct 2024 20:09:38 GMT
Subject: RFR: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23:
 runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be
 represented in type 'long int' [v4]
In-Reply-To: <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
References: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
 <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
Message-ID: <55A-Hu86kNPGpQ24KiuIeefJ8ewqt2AJQXt-rYZTxbA=.6e61a005-4511-41fb-ace4-84fec3875cf9@github.com>

On Tue, 1 Oct 2024 21:44:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Use a template version of `right_n_bits` to use the same type for minuend and subtrahend.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Inline unnecessary usages of right_n_bits

Thanks.

-------------

Marked as reviewed by kdnilsen (Author).

PR Review: https://git.openjdk.org/jdk/pull/21236#pullrequestreview-2343969312

From ysr at openjdk.org  Wed Oct  2 21:03:36 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 2 Oct 2024 21:03:36 GMT
Subject: RFR: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23:
 runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be
 represented in type 'long int' [v4]
In-Reply-To: <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
References: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
 <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
Message-ID: <f92cOuiD0vvA52S1PP3JTiy0hn_B1HaFDd_0wtbRx4o=.6c239716-4ebb-4889-ad49-c9820248b0d5@github.com>

On Tue, 1 Oct 2024 21:44:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Use a template version of `right_n_bits` to use the same type for minuend and subtrahend.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Inline unnecessary usages of right_n_bits

src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.inline.hpp line 33:

> 31:   if (bit_number >= BitsPerWord) {
> 32:     return -1;
> 33:   }

When would we call here with `bit_number >= BitsPerWord` ? If never, may be we assert that?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21236#discussion_r1785246043

From wkemper at openjdk.org  Wed Oct  2 21:15:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 2 Oct 2024 21:15:39 GMT
Subject: RFR: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23:
 runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be
 represented in type 'long int' [v4]
In-Reply-To: <f92cOuiD0vvA52S1PP3JTiy0hn_B1HaFDd_0wtbRx4o=.6c239716-4ebb-4889-ad49-c9820248b0d5@github.com>
References: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
 <YpHRzYhZ--JxHSZd8AewEqqPNp_Y1NPW4AYunaPOSYE=.29f4c9d6-1a58-4728-9f1b-1cec4fbf38c8@github.com>
 <f92cOuiD0vvA52S1PP3JTiy0hn_B1HaFDd_0wtbRx4o=.6c239716-4ebb-4889-ad49-c9820248b0d5@github.com>
Message-ID: <pjKaTG08MVdZLGvIc_AmG6rg9arm5-n56B_ArDhGFOI=.b2110700-87e5-4c35-9d73-f25d913e2c00@github.com>

On Wed, 2 Oct 2024 21:01:20 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Inline unnecessary usages of right_n_bits
>
> src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.inline.hpp line 33:
> 
>> 31:   if (bit_number >= BitsPerWord) {
>> 32:     return -1;
>> 33:   }
> 
> When would we call here with `bit_number >= BitsPerWord` ? If never, may be we assert that?

It's called with `bit_number == BitsPerWord` (I tried an assertion first).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21236#discussion_r1785256564

From phh at openjdk.org  Wed Oct  2 21:16:37 2024
From: phh at openjdk.org (Paul Hohensee)
Date: Wed, 2 Oct 2024 21:16:37 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <LWzg5a5i_FvDV2vXbx586FJT53LmkgyE4sszTf1RExU=.5d8f5959-8bdd-4a64-a515-fa6aa5ae06d6@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

I worry that we're hard-coding assumed task durations. I'm ok with this PR as it is, but I suggest we add a facility to GC initialization that does dummy tasks such as this in order to get somewhat-realistic times for use in this kind of situation.

-------------

Marked as reviewed by phh (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21211#pullrequestreview-2344117544

From xpeng at openjdk.org  Wed Oct  2 21:29:35 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 2 Oct 2024 21:29:35 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <HQ4qL0ad8aby-eBgvvW122kEX7jRBskQxdGW2L7sLmg=.707d57f7-80a0-4227-800d-6ce5597f3878@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 935:

> 933:     // Avoid another call to javaTimeNanos() if we already know time at which last batch ended
> 934:     batch_start_time = batch_end_time;
> 935:     const jlong deadline = batch_start_time + deadline_ns;

Nit: Maybe before taking the heap lock?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21211#discussion_r1785267490

From sviswanathan at openjdk.org  Wed Oct  2 21:31:52 2024
From: sviswanathan at openjdk.org (Sandhya Viswanathan)
Date: Wed, 2 Oct 2024 21:31:52 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
Message-ID: <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>

On Mon, 30 Sep 2024 17:48:13 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Wait a second, I've probably not been clear. `UseCompactObjectHeaders` is slated to become *on by default* and then slated to go away. That means that array base offets <= 16 bytes will become the default. The generated code will be something like:
>> 
>> 
>> if (haystack_len <= 8) {
>>   // Copy 8 bytes onto stack
>> } else if (haystack_len <= 16) {
>>   // Copy 16 bytes onto stack
>> } else {
>>   // Copy 32 bytes onto stack
>> }
>> 
>> 
>> So that is 2 branches in this prologue code instead of originally 1.
>> 
>> However, I just noticed that what I proposed is not enough. Consider what happens when haystack_len is 17. This would take the last case and copy 32 bytes. But we only have 17+8=25 bytes that we can guarantee to be available for copying. If this happens to be the array at the very beginning of the heap (very rare/unlikely), this would segfault.
>> 
>> I think I need to mull over it some more to come up with a correct fix.
>
> I changed the header<16 version to be a small loop: https://github.com/rkennke/jdk/commit/bcba264ea5c15581647933db1163ca1dae39b6c5
> 
> The idea is the same as before, except it's made as a small loop with a maximum of 4 iterations (backward-branches), and it copies 8 bytes at a time, such that 1. it may copy up to 7 bytes that precede the array and 2. doesn't run over the end of the array (which would potentially crash).
> 
> I am not sure if using XMM_TMP1 and XMM_TMP2 there is ok, or if it would encode better to use one of the regular registers.?
> 
> Also, this new implementation could simply replace the old one, instead of being an alternative. I am not sure if if would make any difference performance-wise.

@rkennke The small loop looks to me that it will run over the end of the array.
Say the haystack_len is 7, the index below would be 0 after the shrq instruction, and the  movq(XMM_TMP1, Address(haystack, index, Address::times_8)) in the loop will read 8 bytes i.e. one byte past the end of the array:
       // num_words (zero-based) = (haystack_len - 1) / 8;
      __ movq(index, haystack_len);
      __ subq(index, 1);
      __ shrq(index, LogBytesPerWord);

      __ bind(L_loop);
      __ movq(XMM_TMP1, Address(haystack, index, Address::times_8));
      __ movq(Address(rsp, index, Address::times_8), XMM_TMP1);
      __ subq(index, 1);
      __ jcc(Assembler::positive, L_loop);

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1785269849

From wkemper at openjdk.org  Wed Oct  2 21:32:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 2 Oct 2024 21:32:28 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
 logging [v3]
In-Reply-To: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
References: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
Message-ID: <LExivLFuxTuF7QTXAZBfu5pm5AGRJyPpYxnjr57FBVg=.ae8b1544-0ce5-4761-ac5b-49867b37f4b9@github.com>

> Not clean. Backport to only take lock for logging when log level is enabled.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix old gen usage of shFreeSet::log_status

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/113/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/113/files/d4ee79ea..275e76d9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=113&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=113&range=01-02

  Stats: 4 lines in 1 file changed: 0 ins; 3 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/113.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/113/head:pull/113

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/113

From wkemper at openjdk.org  Wed Oct  2 22:56:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 2 Oct 2024 22:56:42 GMT
Subject: Integrated: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23:
 runtime error: signed integer overflow: -9223372036854775808 - 1
 cannot be represented in type 'long int'
In-Reply-To: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
References: <I8sOa0g-PbjrI3R9yuuosgg83kNah0m0wiLbsNN8IFM=.495943c2-9206-44c5-abb1-4f68cd98b0c5@github.com>
Message-ID: <LhxGzmvIeoFbGRegpst5fOTg08cXgU4uKhhxqrbi5jE=.ca1a57b8-d28e-4d92-8d79-8734682e395b@github.com>

On Fri, 27 Sep 2024 21:29:37 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Use a template version of `right_n_bits` to use the same type for minuend and subtrahend.

This pull request has now been integrated.

Changeset: 57c1db58
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/57c1db5843db5f2c864318f3234767f436a836e3
Stats:     34 lines in 3 files changed: 7 ins; 0 del; 27 mod

8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type 'long int'

Reviewed-by: phh, kdnilsen

-------------

PR: https://git.openjdk.org/jdk/pull/21236

From kdnilsen at openjdk.org  Wed Oct  2 23:09:36 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 2 Oct 2024 23:09:36 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <hfnUgczQ2rq-W0oc6P9cFGwy9p65BgAk5HNfQCy8-oc=.fef39e73-22e3-48f7-bc6b-60f9214879c3@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

I'll modify the code to adjust for current localized behavior of the host computer.  (e.g. not hard-code assumptions about task durations.)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2389933286

From kdnilsen at openjdk.org  Wed Oct  2 23:09:37 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 2 Oct 2024 23:09:37 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <HQ4qL0ad8aby-eBgvvW122kEX7jRBskQxdGW2L7sLmg=.707d57f7-80a0-4227-800d-6ce5597f3878@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <HQ4qL0ad8aby-eBgvvW122kEX7jRBskQxdGW2L7sLmg=.707d57f7-80a0-4227-800d-6ce5597f3878@github.com>
Message-ID: <zXUtp8Vdp9LG_POgkoJ5WzdxsdY4m0QQFL6giMg7bKQ=.c3375095-2f6d-45d5-baea-0b20142bbf11@github.com>

On Wed, 2 Oct 2024 21:26:28 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
>> 
>> Efficiency improvements include:
>> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
>> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
>> 
>> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
>> 
>> Without this change, latency for 4 un-named business services is represented by the following chart:
>> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
>> 
>> With this change, latency for the same services is much better:
>> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
>> 
>> A comparison of the two is provided by the following:
>> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 935:
> 
>> 933:     // Avoid another call to javaTimeNanos() if we already know time at which last batch ended
>> 934:     batch_start_time = batch_end_time;
>> 935:     const jlong deadline = batch_start_time + deadline_ns;
> 
> Nit: Maybe before taking the heap lock?

I'll adjust this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21211#discussion_r1785368950

From wkemper at openjdk.org  Thu Oct  3 14:26:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 3 Oct 2024 14:26:06 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <W25bSO8UkAWOtK_ZGGhc0VibmlHRJhcfvhRyxVzwPSg=.e536c55e-2d91-4ccd-b597-1477fa4d5f59@github.com>

Merges tag jdk-21.0.5+9

-------------

Commit messages:
 - 8341059: Change Entrust TLS distrust date to November 12, 2024
 - 8341057: Add 2 SSL.com TLS roots

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/114/files
  Stats: 110 lines in 8 files changed: 93 ins; 1 del; 16 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/114.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/114/head:pull/114

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/114

From kdnilsen at openjdk.org  Thu Oct  3 16:19:53 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 3 Oct 2024 16:19:53 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v2]
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <qnUYv4gHanvcqlgax0xXO9frymnVrgj2eR5oxXbnnf0=.c4648064-4531-4033-bc05-22e19cf538e7@github.com>

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Predict next batch time and enforce predictive deadline

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21211/files
  - new: https://git.openjdk.org/jdk/pull/21211/files/acf517f5..c3f1b080

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21211&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21211&range=00-01

  Stats: 20 lines in 1 file changed: 11 ins; 2 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/21211.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21211/head:pull/21211

PR: https://git.openjdk.org/jdk/pull/21211

From xpeng at openjdk.org  Thu Oct  3 16:51:37 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 3 Oct 2024 16:51:37 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v2]
In-Reply-To: <qnUYv4gHanvcqlgax0xXO9frymnVrgj2eR5oxXbnnf0=.c4648064-4531-4033-bc05-22e19cf538e7@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <qnUYv4gHanvcqlgax0xXO9frymnVrgj2eR5oxXbnnf0=.c4648064-4531-4033-bc05-22e19cf538e7@github.com>
Message-ID: <l8QzgE8cyAEPy7hCGafiQxUiv0bUUIFZyuEixZ4dhW4=.381dbc2e-87b4-44d6-856b-818c63eab31d@github.com>

On Thu, 3 Oct 2024 16:19:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
>> 
>> Efficiency improvements include:
>> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
>> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
>> 
>> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
>> 
>> Without this change, latency for 4 un-named business services is represented by the following chart:
>> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
>> 
>> With this change, latency for the same services is much better:
>> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
>> 
>> A comparison of the two is provided by the following:
>> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Predict next batch time and enforce predictive deadline

Marked as reviewed by xpeng (Author).

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 957:

> 955:       batch_end_time = os::javaTimeNanos();
> 956:       // Estimate includes historic combination of yield times and heap lock acquisition times.
> 957:       batch_process_time_estimate = (batch_end_time - recycle_trash_start_time) / total_batches;;

Nit: double semicolons

-------------

PR Review: https://git.openjdk.org/jdk/pull/21211#pullrequestreview-2346133650
PR Review Comment: https://git.openjdk.org/jdk/pull/21211#discussion_r1786541498

From wkemper at openjdk.org  Thu Oct  3 18:27:10 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 3 Oct 2024 18:27:10 GMT
Subject: RFR: 8341485: GenShen: Make evac tracker a non-product feature and
 confine it to generational mode
Message-ID: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>

During the course of development of the generational mode, we added a feature to keep track of which threads were evacuating objects and how many evacuations lost the race. The instrumentation added unwarranted overhead. This change makes this instrumentation a non-product feature for the generational mode and removes it entirely from the non-generational modes.

-------------

Commit messages:
 - Only instantiate evac tracker in generational mode
 - Only use evac tracker with generational mode
 - Do not retry with smaller lab, disable evac stats
 - Make evacuate_object non-virtual (this breaks genshen)

Changes: https://git.openjdk.org/shenandoah/pull/506/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=506&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341485
  Stats: 88 lines in 10 files changed: 33 ins; 28 del; 27 mod
  Patch: https://git.openjdk.org/shenandoah/pull/506.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/506/head:pull/506

PR: https://git.openjdk.org/shenandoah/pull/506

From kdnilsen at openjdk.org  Thu Oct  3 19:55:05 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 3 Oct 2024 19:55:05 GMT
Subject: RFR: 8341485: GenShen: Make evac tracker a non-product feature and
 confine it to generational mode
In-Reply-To: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>
References: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>
Message-ID: <X74Cr2ugD6eZi3Rtze_mtQnY7NTufaVXAsxIjfdfmVQ=.4d6e246e-24e6-49ef-8349-9b300a70639d@github.com>

On Thu, 3 Oct 2024 18:23:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> During the course of development of the generational mode, we added a feature to keep track of which threads were evacuating objects and how many evacuations lost the race. The instrumentation added unwarranted overhead. This change makes this instrumentation a non-product feature for the generational mode and removes it entirely from the non-generational modes.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/506#pullrequestreview-2346577982

From coleenp at openjdk.org  Thu Oct  3 20:30:56 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Thu, 3 Oct 2024 20:30:56 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v29]
In-Reply-To: <migFoRhtd3kovSs2fRSGXzdh7vCpbyazoT5Gwzeu8ks=.4bd5afae-9598-4ab3-a7d8-737281236a4e@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <migFoRhtd3kovSs2fRSGXzdh7vCpbyazoT5Gwzeu8ks=.4bd5afae-9598-4ab3-a7d8-737281236a4e@github.com>
Message-ID: <GqmziZEDXAdRD3_7_SwIQoRphkc_Vk3jR_16rNnV1_8=.57f3b695-549d-472a-9b04-2bb34e9d22d6@github.com>

On Wed, 2 Oct 2024 15:37:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Revert "Disable TestSplitPacks::test4a, failing on aarch64"
>    
>    This reverts commit 059b1573db26d1d2902ca6dadc8413f445234c2a.
>  - Simplify object init code in interpreter
>  - Disable some vectorization tests that fail with +UCOH and UseSSE<=3

I posted a patch for JDK-8341044 for CDSPluginTest.java that was failing in our testing with the Lilliput patch.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2392273233

From kdnilsen at openjdk.org  Thu Oct  3 21:30:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 3 Oct 2024 21:30:12 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v3]
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  fix typo

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21211/files
  - new: https://git.openjdk.org/jdk/pull/21211/files/c3f1b080..055ad411

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21211&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21211&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21211.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21211/head:pull/21211

PR: https://git.openjdk.org/jdk/pull/21211

From kdnilsen at openjdk.org  Thu Oct  3 21:30:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 3 Oct 2024 21:30:12 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v2]
In-Reply-To: <l8QzgE8cyAEPy7hCGafiQxUiv0bUUIFZyuEixZ4dhW4=.381dbc2e-87b4-44d6-856b-818c63eab31d@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <qnUYv4gHanvcqlgax0xXO9frymnVrgj2eR5oxXbnnf0=.c4648064-4531-4033-bc05-22e19cf538e7@github.com>
 <l8QzgE8cyAEPy7hCGafiQxUiv0bUUIFZyuEixZ4dhW4=.381dbc2e-87b4-44d6-856b-818c63eab31d@github.com>
Message-ID: <WtQ0cTd2b8pgDkCHtMY0h07SSdpXRE5Qv0EtfGcHoQs=.50d55173-0399-4a44-afb7-7eba210a87c5@github.com>

On Thu, 3 Oct 2024 16:48:52 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Predict next batch time and enforce predictive deadline
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 957:
> 
>> 955:       batch_end_time = os::javaTimeNanos();
>> 956:       // Estimate includes historic combination of yield times and heap lock acquisition times.
>> 957:       batch_process_time_estimate = (batch_end_time - recycle_trash_start_time) / total_batches;;
> 
> Nit: double semicolons

Thanks.  Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21211#discussion_r1786862316

From wkemper at openjdk.org  Thu Oct  3 21:40:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 3 Oct 2024 21:40:06 GMT
Subject: RFR: 8341099: GenShen: assert(HAS_FWD ==
 _heap->has_forwarded_objects()) failed: Forwarded object status is sane
Message-ID: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>

Evacuations may occur during young or mixed collection cycles. When it is a young cycle during old marking, this barrier may be called to both evacuate cset objects and again to enqueue the targets of the copy (if they are in the old generation). If it is a mixed collection, then old marking will _not_ be in progress and this barrier will not be called to enqueue anything.

-------------

Commit messages:
 - Relax arraycopy assertion to allow for evacuations during old generation marking call

Changes: https://git.openjdk.org/shenandoah/pull/507/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=507&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341099
  Stats: 10 lines in 1 file changed: 8 ins; 1 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/507.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/507/head:pull/507

PR: https://git.openjdk.org/shenandoah/pull/507

From kdnilsen at openjdk.org  Thu Oct  3 21:46:36 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 3 Oct 2024 21:46:36 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v3]
In-Reply-To: <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
Message-ID: <LOiuJi3b3a7C925tL91pol-4bHN3oIRguVH-W5ctEh4=.6497226d-0c3a-456b-bac7-7319829a9f67@github.com>

On Thu, 3 Oct 2024 21:30:12 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
>> 
>> Efficiency improvements include:
>> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
>> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
>> 
>> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
>> 
>> Without this change, latency for 4 un-named business services is represented by the following chart:
>> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
>> 
>> With this change, latency for the same services is much better:
>> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
>> 
>> A comparison of the two is provided by the following:
>> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix typo

I restructured the mechanism that decides whether to start processing another batch of regions.  In the code as originally structured, we would start another batch whenever the time since previous yield is less than 8 us.  In this new version of the code, we start another batch whenever the time since previous yield plus the predicted time to process another batch is less than 10 us.  Prediction of how long a batch will take to process is based on most recent history.  Note that this is highly dependent on the state of the application.  If the application has lots of mutator threads that are urgently allocating, there will be contention for the global heap lock and there will be contention for CPU cores, and both of these will cause the batch processing time to increase.

As expected, this new version does a better job of constraining the amount of time between yields.  With this new code, we will never start processing another batch unless we have some confidence that the new batch will complete on schedule.  Previously, we might have completed processing a first batch at time 7.5 us.  Then, we would have immediately started to process a second patch, even though a prediction of batch processing time would have allowed us to predict that this second batch would not finish until time 15 us.

The new version of the code will sometimes cause mutators to wait slightly longer for yield and the heap lock when the system is "lightly loaded".  This is shown to effect p50 latencies.  When the system is lightly loaded, the time to process a batch (on current hardware) is approximately 1.5 us.   Suppose we finish processing a batch at time 8.25 us.  In the original implementation, we would have immediately yielded.  However, in this new version of the code, we'll take the next batch, because this is predicted to complete at time 9.75 us, which is less than deadline 10 us.  Making this choice allows the GC thread that is recycling trash regions to have higher throughput.

Here are the results of testing this new version of the code:

![image](https://github.com/user-attachments/assets/5acacae4-0604-400c-a3f7-3899274414a8)

and this is how this new version compares to mainline without this PR:

![image](https://github.com/user-attachments/assets/44df0fc2-04c7-4593-923e-05a50f6767ed)

Most of the p99.99 percentile latencies are considerably improved.  There is a slight degradation of p50 latencies.

Which would be considered the preferred solution?  I'll vote for the new code, but can be persuaded either way.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2392391683

From xpeng at openjdk.org  Thu Oct  3 22:14:36 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 3 Oct 2024 22:14:36 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v3]
In-Reply-To: <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
Message-ID: <zyL6i1VSmINE_BGftXBINUGThKK9n0dazp3fHUqJjZg=.d71e8a33-744f-47fc-a657-0976213a2b43@github.com>

On Thu, 3 Oct 2024 21:30:12 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
>> 
>> Efficiency improvements include:
>> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
>> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
>> 
>> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
>> 
>> Without this change, latency for 4 un-named business services is represented by the following chart:
>> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
>> 
>> With this change, latency for the same services is much better:
>> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
>> 
>> A comparison of the two is provided by the following:
>> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix typo

Marked as reviewed by xpeng (Author).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21211#pullrequestreview-2346777439

From xpeng at openjdk.org  Thu Oct  3 22:14:37 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 3 Oct 2024 22:14:37 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v3]
In-Reply-To: <LOiuJi3b3a7C925tL91pol-4bHN3oIRguVH-W5ctEh4=.6497226d-0c3a-456b-bac7-7319829a9f67@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
 <LOiuJi3b3a7C925tL91pol-4bHN3oIRguVH-W5ctEh4=.6497226d-0c3a-456b-bac7-7319829a9f67@github.com>
Message-ID: <jXV48wXi0T5UwkXO3ABs1shz7xwZm8K4VsvJQHsM0QE=.c960354e-f1ff-4eca-9423-c43693368251@github.com>

On Thu, 3 Oct 2024 21:42:51 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Which would be considered the preferred solution? I'll vote for the new code, but can be persuaded either way.

New code looks good to me, the test result also looks pretty good.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2392428849

From ysr at openjdk.org  Thu Oct  3 23:05:04 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 3 Oct 2024 23:05:04 GMT
Subject: RFR: 8341485: GenShen: Make evac tracker a non-product feature and
 confine it to generational mode
In-Reply-To: <NIN5fr7krT9x73nxiXweUpuwWv8eyVMU8D1HUwltEBA=.39c72b86-c981-4bcc-a54e-628c77cff055@github.com>
References: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>
 <NIN5fr7krT9x73nxiXweUpuwWv8eyVMU8D1HUwltEBA=.39c72b86-c981-4bcc-a54e-628c77cff055@github.com>
Message-ID: <9hF-dHv_pmZCFD3fUwVjxmlyeljBD7HfubR-MAJZr1k=.17d39a52-f8a9-4fac-85bd-0d98983c9927@github.com>

On Thu, 3 Oct 2024 22:56:15 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> During the course of development of the generational mode, we added a feature to keep track of which threads were evacuating objects and how many evacuations lost the race. The instrumentation added unwarranted overhead. This change makes this instrumentation a non-product feature for the generational mode and removes it entirely from the non-generational modes.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1258:
> 
>> 1256:       }
>> 1257:     }
>> 1258: 
> 
> This change in behavior is an intentional part of this change? It may be the right change to make, but seems unrelated to the bug summary and the main thrust of the changes. Should this be broken off into a separate ticket?

Looking at upstream, I see that this was detritus from generational heap that may have been unintentionally been left behind.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/506#discussion_r1786924182

From ysr at openjdk.org  Thu Oct  3 23:05:04 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 3 Oct 2024 23:05:04 GMT
Subject: RFR: 8341485: GenShen: Make evac tracker a non-product feature and
 confine it to generational mode
In-Reply-To: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>
References: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>
Message-ID: <NIN5fr7krT9x73nxiXweUpuwWv8eyVMU8D1HUwltEBA=.39c72b86-c981-4bcc-a54e-628c77cff055@github.com>

On Thu, 3 Oct 2024 18:23:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> During the course of development of the generational mode, we added a feature to keep track of which threads were evacuating objects and how many evacuations lost the race. The instrumentation added unwarranted overhead. This change makes this instrumentation a non-product feature for the generational mode and removes it entirely from the non-generational modes.

Looks good to me.

(Left a comment/question about another change that kinda snuck in; but it's probably OK to sneak it in here rather than the extra overhead of a separate PR, if you intended it...)

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1258:

> 1256:       }
> 1257:     }
> 1258: 

This change in behavior is an intentional part of this change? It may be the right change to make, but seems unrelated to the bug summary and the main thrust of the changes. Should this be broken off into a separate ticket?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/506#pullrequestreview-2346814180
PR Review Comment: https://git.openjdk.org/shenandoah/pull/506#discussion_r1786921819

From kdnilsen at openjdk.org  Thu Oct  3 23:22:01 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 3 Oct 2024 23:22:01 GMT
Subject: RFR: 8341099: GenShen: assert(HAS_FWD ==
 _heap->has_forwarded_objects()) failed: Forwarded object status is sane
In-Reply-To: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>
References: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>
Message-ID: <K_yEq6wouoXjfgKQ1mOBVBdTjbGLt8pqt82F9t0KlMg=.fde07038-abbc-454d-8155-a9f0911e4d02@github.com>

On Thu, 3 Oct 2024 21:35:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Evacuations may occur during young or mixed collection cycles. When it is a young cycle during old marking, this barrier may be called to both evacuate cset objects and again to enqueue the targets of the copy (if they are in the old generation). If it is a mixed collection, then old marking will _not_ be in progress and this barrier will not be called to enqueue anything.

Marked as reviewed by kdnilsen (Committer).

src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 406:

> 404:         shenandoah_assert_forwarded_except(elem_ptr, obj, _heap->cancelled_gc());
> 405:         ShenandoahHeap::atomic_update_oop(fwd, elem_ptr, o);
> 406:       }

I think the reason we are removing this assignment here is because we will never be at this line of code if (ENQUEUE && !ctx->is_marked_strong_or_old(obj)).  In other words, if we are evacuating this object, then this object is not in old or this object is in old and we are not marking old.  I'd feel a little more comfortable with this code change if there were at minimum a comment here to explain this, or perhaps even better an assert.

assert(!ENQUEUE || !is_old(obj), "Do not evacuate old while old marking is active")

something like this...

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/507#pullrequestreview-2346831096
PR Review Comment: https://git.openjdk.org/shenandoah/pull/507#discussion_r1786933650

From wkemper at openjdk.org  Thu Oct  3 23:30:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 3 Oct 2024 23:30:59 GMT
Subject: RFR: 8341099: GenShen: assert(HAS_FWD ==
 _heap->has_forwarded_objects()) failed: Forwarded object status is sane
In-Reply-To: <K_yEq6wouoXjfgKQ1mOBVBdTjbGLt8pqt82F9t0KlMg=.fde07038-abbc-454d-8155-a9f0911e4d02@github.com>
References: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>
 <K_yEq6wouoXjfgKQ1mOBVBdTjbGLt8pqt82F9t0KlMg=.fde07038-abbc-454d-8155-a9f0911e4d02@github.com>
Message-ID: <nn0WQIEoWWpMzBUo-jhSsVbjY2-bvCvjWYj_WeYT7Y8=.805ef8b7-fccc-49ee-92da-ef542d2d8077@github.com>

On Thu, 3 Oct 2024 23:18:41 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Evacuations may occur during young or mixed collection cycles. When it is a young cycle during old marking, this barrier may be called to both evacuate cset objects and again to enqueue the targets of the copy (if they are in the old generation). If it is a mixed collection, then old marking will _not_ be in progress and this barrier will not be called to enqueue anything.
>
> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 406:
> 
>> 404:         shenandoah_assert_forwarded_except(elem_ptr, obj, _heap->cancelled_gc());
>> 405:         ShenandoahHeap::atomic_update_oop(fwd, elem_ptr, o);
>> 406:       }
> 
> I think the reason we are removing this assignment here is because we will never be at this line of code if (ENQUEUE && !ctx->is_marked_strong_or_old(obj)).  In other words, if we are evacuating this object, then this object is not in old or this object is in old and we are not marking old.  I'd feel a little more comfortable with this code change if there were at minimum a comment here to explain this, or perhaps even better an assert.
> 
> assert(!ENQUEUE || !is_old(obj), "Do not evacuate old while old marking is active")
> 
> something like this...

Yes, I added an assert at the top of the function to this effect.

  // This function cannot be called to handle marking and evacuation at the same time (they operate on
  // different sides of the copy).
  assert((HAS_FWD || EVAC) != ENQUEUE, "Cannot evacuate and mark both sides of copy.");

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/507#discussion_r1786938578

From kdnilsen at openjdk.org  Fri Oct  4 00:04:49 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 4 Oct 2024 00:04:49 GMT
Subject: RFR: 8341099: GenShen: assert(HAS_FWD ==
 _heap->has_forwarded_objects()) failed: Forwarded object status is sane
In-Reply-To: <nn0WQIEoWWpMzBUo-jhSsVbjY2-bvCvjWYj_WeYT7Y8=.805ef8b7-fccc-49ee-92da-ef542d2d8077@github.com>
References: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>
 <K_yEq6wouoXjfgKQ1mOBVBdTjbGLt8pqt82F9t0KlMg=.fde07038-abbc-454d-8155-a9f0911e4d02@github.com>
 <nn0WQIEoWWpMzBUo-jhSsVbjY2-bvCvjWYj_WeYT7Y8=.805ef8b7-fccc-49ee-92da-ef542d2d8077@github.com>
Message-ID: <xg8EwheXscRFRhZhZY_htfwxYwE5E39-tnoQboxyhmE=.40057a41-def5-475b-a285-25b1eb6f3f05@github.com>

On Thu, 3 Oct 2024 23:28:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 406:
>> 
>>> 404:         shenandoah_assert_forwarded_except(elem_ptr, obj, _heap->cancelled_gc());
>>> 405:         ShenandoahHeap::atomic_update_oop(fwd, elem_ptr, o);
>>> 406:       }
>> 
>> I think the reason we are removing this assignment here is because we will never be at this line of code if (ENQUEUE && !ctx->is_marked_strong_or_old(obj)).  In other words, if we are evacuating this object, then this object is not in old or this object is in old and we are not marking old.  I'd feel a little more comfortable with this code change if there were at minimum a comment here to explain this, or perhaps even better an assert.
>> 
>> assert(!ENQUEUE || !is_old(obj), "Do not evacuate old while old marking is active")
>> 
>> something like this...
>
> Yes, I added an assert at the top of the function to this effect.
> 
>   // This function cannot be called to handle marking and evacuation at the same time (they operate on
>   // different sides of the copy).
>   assert((HAS_FWD || EVAC) != ENQUEUE, "Cannot evacuate and mark both sides of copy.");

Thanks.  I missed the significance of that.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/507#discussion_r1786955264

From rkennke at openjdk.org  Fri Oct  4 10:44:53 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 4 Oct 2024 10:44:53 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
Message-ID: <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>

On Wed, 2 Oct 2024 21:29:28 GMT, Sandhya Viswanathan <sviswanathan at openjdk.org> wrote:

>> I changed the header<16 version to be a small loop: https://github.com/rkennke/jdk/commit/bcba264ea5c15581647933db1163ca1dae39b6c5
>> 
>> The idea is the same as before, except it's made as a small loop with a maximum of 4 iterations (backward-branches), and it copies 8 bytes at a time, such that 1. it may copy up to 7 bytes that precede the array and 2. doesn't run over the end of the array (which would potentially crash).
>> 
>> I am not sure if using XMM_TMP1 and XMM_TMP2 there is ok, or if it would encode better to use one of the regular registers.?
>> 
>> Also, this new implementation could simply replace the old one, instead of being an alternative. I am not sure if if would make any difference performance-wise.
>
> @rkennke The small loop looks to me that it will run over the end of the array.
> Say the haystack_len is 7, the index below would be 0 after the shrq instruction, and the  movq(XMM_TMP1, Address(haystack, index, Address::times_8)) in the loop will read 8 bytes i.e. one byte past the end of the array:
>        // num_words (zero-based) = (haystack_len - 1) / 8;
>       __ movq(index, haystack_len);
>       __ subq(index, 1);
>       __ shrq(index, LogBytesPerWord);
> 
>       __ bind(L_loop);
>       __ movq(XMM_TMP1, Address(haystack, index, Address::times_8));
>       __ movq(Address(rsp, index, Address::times_8), XMM_TMP1);
>       __ subq(index, 1);
>       __ jcc(Assembler::positive, L_loop);

Yes, and that is intentional.

Say, haystack_len is 7, then the first block computes the adjustment of the haystack, which is 8 - (7 % 8) = 1. We adjust the haystack pointer one byte down, so that when we copy (multiple of) 8 bytes, we land on the last byte. We do copy a few bytes that are preceding the array, which is part of the object header and guaranteed to be >= 8 bytes.

Then we compute the number of words to copy, but make it 0-based. That is '0' is 1 word, '1' is 2 words, etc. It makes the loop nicer. In this example we get 0, which means we copy one word from the adjusted haystack, which is correct.

Then comes the actual loop.

Afterwards we adjust the haystack pointer so that it points to the first array element that we just copied onto the stack, ignoring the few garbage bytes that we also copied.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1787528501

From rkennke at openjdk.org  Fri Oct  4 11:15:37 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 4 Oct 2024 11:15:37 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v30]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <aU7Oq51P0gFTm6hW7hQW_fmAYCdLG6JlqW6lBOkaRKE=.e30fa902-dc09-4944-8983-2c14ad068cf1@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 76 commits:

 - Merge remote-tracking branch 'rkennke/JDK-8305895-v4' into JDK-8305895-v4
 - Revert "Disable TestSplitPacks::test4a, failing on aarch64"
   
   This reverts commit 059b1573db26d1d2902ca6dadc8413f445234c2a.
 - Simplify object init code in interpreter
 - Disable some vectorization tests that fail with +UCOH and UseSSE<=3
 - Fix for CDSPluginTest.java
 - Merge tag 'jdk-24+18' into JDK-8305895-v4
   
   Added tag jdk-24+18 for changeset 19642bd3
 - Disable TestSplitPacks::test4a, failing on aarch64
 - @robcasloz review comments
 - Improve CollectedHeap::is_oop()
 - Allow LM_MONITOR on 32-bit platforms
 - ... and 66 more: https://git.openjdk.org/jdk/compare/19642bd3...8742f3c1

-------------

Changes: https://git.openjdk.org/jdk/pull/20677/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=29
  Stats: 4560 lines in 196 files changed: 3207 ins; 724 del; 629 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From coleenp at openjdk.org  Fri Oct  4 12:53:53 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Fri, 4 Oct 2024 12:53:53 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v30]
In-Reply-To: <aU7Oq51P0gFTm6hW7hQW_fmAYCdLG6JlqW6lBOkaRKE=.e30fa902-dc09-4944-8983-2c14ad068cf1@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <aU7Oq51P0gFTm6hW7hQW_fmAYCdLG6JlqW6lBOkaRKE=.e30fa902-dc09-4944-8983-2c14ad068cf1@github.com>
Message-ID: <lHkEF3XJ_h4c3dOro7V35aYk154SucAB8p7Px-2eLyI=.e4680e9a-1df7-47d1-adbc-203e85861035@github.com>

On Fri, 4 Oct 2024 11:15:37 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 76 commits:
> 
>  - Merge remote-tracking branch 'rkennke/JDK-8305895-v4' into JDK-8305895-v4
>  - Revert "Disable TestSplitPacks::test4a, failing on aarch64"
>    
>    This reverts commit 059b1573db26d1d2902ca6dadc8413f445234c2a.
>  - Simplify object init code in interpreter
>  - Disable some vectorization tests that fail with +UCOH and UseSSE<=3
>  - Fix for CDSPluginTest.java
>  - Merge tag 'jdk-24+18' into JDK-8305895-v4
>    
>    Added tag jdk-24+18 for changeset 19642bd3
>  - Disable TestSplitPacks::test4a, failing on aarch64
>  - @robcasloz review comments
>  - Improve CollectedHeap::is_oop()
>  - Allow LM_MONITOR on 32-bit platforms
>  - ... and 66 more: https://git.openjdk.org/jdk/compare/19642bd3...8742f3c1

There's another test failure that we're seeing that's similar to this bug in mainline when running with -XX:+UseCompactObjectHeaders on aarch64: https://bugs.openjdk.org/browse/JDK-8340212

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2393637283

From wkemper at openjdk.org  Fri Oct  4 14:17:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 14:17:35 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <BHnJa1es3gSFbZmoalcW6kpP-y5d9fjQGZO_Lo8TXC8=.231e356f-4e75-4fd7-b79d-4ddcf514a158@github.com>

Merges tag jdk-24+18

-------------

Commit messages:
 - 8341148: Open source several Choice related tests
 - 8334060: Implementation of Late Barrier Expansion for G1
 - 8341091: CDS: Segmented roots array misses roots
 - 8341239: Open source closed frame tests # 3
 - 8341191: Open source few more AWT FileDialog tests
 - 8341177: Opensource few List and a Window test
 - 8309841: Jarsigner should print a warning if an entry is removed
 - 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type 'long int'
 - 8341336: Fix -Wzero-as-null-pointer-constant warnings in PRODUCT-only code
 - 8341037: Use standard layouts in DefaultFrameIconTest.java and MenuCrash.java
 - ... and 88 more: https://git.openjdk.org/shenandoah/compare/10da2c21...19642bd3

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=508&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/508/files
  Stats: 25502 lines in 448 files changed: 21177 ins; 2556 del; 1769 mod
  Patch: https://git.openjdk.org/shenandoah/pull/508.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/508/head:pull/508

PR: https://git.openjdk.org/shenandoah/pull/508

From phh at openjdk.org  Fri Oct  4 16:39:41 2024
From: phh at openjdk.org (Paul Hohensee)
Date: Fri, 4 Oct 2024 16:39:41 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v3]
In-Reply-To: <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
Message-ID: <CRSXXAVFyg8jhZtpIA2MBbzteRUjhCPYjbLZzPazBkk=.6b1f8e9e-9e7f-41bd-ab47-6d08f83726a2@github.com>

On Thu, 3 Oct 2024 21:30:12 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
>> 
>> Efficiency improvements include:
>> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
>> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
>> 
>> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
>> 
>> Without this change, latency for 4 un-named business services is represented by the following chart:
>> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
>> 
>> With this change, latency for the same services is much better:
>> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
>> 
>> A comparison of the two is provided by the following:
>> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix typo

Marked as reviewed by phh (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21211#pullrequestreview-2348488078

From duke at openjdk.org  Fri Oct  4 17:13:40 2024
From: duke at openjdk.org (duke)
Date: Fri, 4 Oct 2024 17:13:40 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v3]
In-Reply-To: <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
Message-ID: <p8nkwxtDn0dLyJtZ1KfMdyen3y_wr6lewJxnIZpbu4Q=.22b18078-31c1-47fc-bdfe-278e6e722f82@github.com>

On Thu, 3 Oct 2024 21:30:12 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
>> 
>> Efficiency improvements include:
>> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
>> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
>> 
>> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
>> 
>> Without this change, latency for 4 un-named business services is represented by the following chart:
>> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
>> 
>> With this change, latency for the same services is much better:
>> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
>> 
>> A comparison of the two is provided by the following:
>> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix typo

@kdnilsen 
Your change (at version 055ad41109b303ab474c2510cc496a3cf87135b8) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2394174956

From ysr at openjdk.org  Fri Oct  4 17:32:42 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 4 Oct 2024 17:32:42 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
 [v3]
In-Reply-To: <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
 <2QJqfhzQfJk9ZDN1AI3QR2KpLvGA-djz8U-Cv7OeKGo=.708970da-45e6-41db-a3cd-86ca272d8a14@github.com>
Message-ID: <b33_eUu9ko2CcVdR5wDtUYVQ5k2cFD8R3tZtuWXpFI0=.4c8e8abb-9571-4f4d-a83b-ebf1c298b881@github.com>

On Thu, 3 Oct 2024 21:30:12 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
>> 
>> Efficiency improvements include:
>> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
>> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
>> 
>> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
>> 
>> Without this change, latency for 4 un-named business services is represented by the following chart:
>> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
>> 
>> With this change, latency for the same services is much better:
>> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
>> 
>> A comparison of the two is provided by the following:
>> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix typo

Nice!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21211#issuecomment-2394202215

From kdnilsen at openjdk.org  Fri Oct  4 17:32:43 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 4 Oct 2024 17:32:43 GMT
Subject: Integrated: 8341379: Shenandoah: Improve lock contention during
 cleanup
In-Reply-To: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
References: <5GMjFWaMjkSuxhPJ8TYJyw_LYdFfhJWrySSaRUjSNq4=.0ee7f896-35b2-43aa-a243-418b1ffb50b0@github.com>
Message-ID: <fkUWzeN7YdmpBB-J4faKA3RblauD8WghXOIjuWcUjsE=.78cb25df-d82e-4236-9984-9bad9f4b151b@github.com>

On Thu, 26 Sep 2024 21:08:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change improves the efficiency of cleaning up (recycling) regions that have been trashed by GC effort.  The affected code runs at the end of FinalMark to reclaim immediate garbage.  It runs at the end of FinalUpdateRefs to reclaim the regions that comprised the collection set, from which all live objects have now been evacuated.
> 
> Efficiency improvements include:
> 1. Rather than invoking the os (while holding the Heap lock) to obtain the time twice for every region recycled, we invoke the os only once for each batch of 32 regions that are to be processed.
> 2. Rather than enforcing that the loop runs no longer than 30 us, we refrain from starting a second batch of regions if more than 8 us has passed since the preceding batch was processed.
> 
> Below, each trial runs for 1 hour, processing 28,000 transactions per second.
> 
> Without this change, latency for 4 un-named business services is represented by the following chart:
> ![image](https://github.com/user-attachments/assets/0e36025b-7b76-4e7a-ab07-303ea49479c3)
> 
> With this change, latency for the same services is much better:
> ![image](https://github.com/user-attachments/assets/aceaf185-6944-4c91-b98e-06ccd1bc2d64)
> 
> A comparison of the two is provided by the following:
> ![image](https://github.com/user-attachments/assets/7145f7b5-2a65-44b0-a94a-ddbc871f236b)

This pull request has now been integrated.

Changeset: f5f0852f
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/f5f0852f51d3dc1001bf3d68b89f4aab31e05e61
Stats:     40 lines in 1 file changed: 32 ins; 4 del; 4 mod

8341379: Shenandoah: Improve lock contention during cleanup

Reviewed-by: xpeng, phh, wkemper

-------------

PR: https://git.openjdk.org/jdk/pull/21211

From wkemper at openjdk.org  Fri Oct  4 17:38:52 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 17:38:52 GMT
Subject: RFR: 8341485: GenShen: Make evac tracker a non-product feature and
 confine it to generational mode
In-Reply-To: <9hF-dHv_pmZCFD3fUwVjxmlyeljBD7HfubR-MAJZr1k=.17d39a52-f8a9-4fac-85bd-0d98983c9927@github.com>
References: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>
 <NIN5fr7krT9x73nxiXweUpuwWv8eyVMU8D1HUwltEBA=.39c72b86-c981-4bcc-a54e-628c77cff055@github.com>
 <9hF-dHv_pmZCFD3fUwVjxmlyeljBD7HfubR-MAJZr1k=.17d39a52-f8a9-4fac-85bd-0d98983c9927@github.com>
Message-ID: <n8rOnbxXI6V2-LAkxn15uXW3QUZNCWu8j3VqqTWnD_Q=.4c2bd48a-1d78-4a30-ab38-b9cbc3b1340d@github.com>

On Thu, 3 Oct 2024 23:00:21 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1258:
>> 
>>> 1256:       }
>>> 1257:     }
>>> 1258: 
>> 
>> This change in behavior is an intentional part of this change? It may be the right change to make, but seems unrelated to the bug summary and the main thrust of the changes. Should this be broken off into a separate ticket?
>
> Looking at upstream, I see that this was detritus from generational heap that may have been unintentionally been left behind.

Yeah, it's a little unrelated to the reset of the changes, but in the spirit of removing differences from `shenandoahHeap` with upstream.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/506#discussion_r1788067308

From wkemper at openjdk.org  Fri Oct  4 17:43:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 17:43:06 GMT
Subject: Integrated: 8341099: GenShen: assert(HAS_FWD ==
 _heap->has_forwarded_objects()) failed: Forwarded object status is sane
In-Reply-To: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>
References: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>
Message-ID: <MgvRO_mdi8f81MFnxb8kXJedFGUOUIq8xeVkWZM8bss=.6b32d295-196a-4e3f-b693-4b3e0b230463@github.com>

On Thu, 3 Oct 2024 21:35:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Evacuations may occur during young or mixed collection cycles. When it is a young cycle during old marking, this barrier may be called to both evacuate cset objects and again to enqueue the targets of the copy (if they are in the old generation). If it is a mixed collection, then old marking will _not_ be in progress and this barrier will not be called to enqueue anything.

This pull request has now been integrated.

Changeset: cbc0fd0a
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/cbc0fd0a2aae095c7fda3d69a113bf504ffef594
Stats:     10 lines in 1 file changed: 8 ins; 1 del; 1 mod

8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/507

From wkemper at openjdk.org  Fri Oct  4 17:38:52 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 17:38:52 GMT
Subject: Integrated: 8341485: GenShen: Make evac tracker a non-product feature
 and confine it to generational mode
In-Reply-To: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>
References: <NImH_ZkS3tMinCkjEM2o-VEN5ZkEWaBpwExBnDRfTPU=.9367445a-c9d0-4ba2-9ba5-92157ee813c4@github.com>
Message-ID: <02N-P2iO6ZLTEq9Yv7MxlD51V0eHwXDwphjTru18THU=.979b7cce-7bb6-472c-82f4-177ea1cc2624@github.com>

On Thu, 3 Oct 2024 18:23:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> During the course of development of the generational mode, we added a feature to keep track of which threads were evacuating objects and how many evacuations lost the race. The instrumentation added unwarranted overhead. This change makes this instrumentation a non-product feature for the generational mode and removes it entirely from the non-generational modes.

This pull request has now been integrated.

Changeset: bd7d34d8
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/bd7d34d8e4c89a855a2398965f776803ded557ea
Stats:     88 lines in 10 files changed: 33 ins; 28 del; 27 mod

8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/506

From wkemper at openjdk.org  Fri Oct  4 17:43:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 17:43:06 GMT
Subject: RFR: 8341099: GenShen: assert(HAS_FWD ==
 _heap->has_forwarded_objects()) failed: Forwarded object status is sane
In-Reply-To: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>
References: <er0pFAhSUO01HOiap9N7H9qRLY8maRl0Ev1aHfBx2OE=.9e96101f-45b9-4040-af6b-3a23f4abff54@github.com>
Message-ID: <TTgNEYWl99Cat_yhcRbcUVZzVe7R4Y-ld5D2ptCrsPU=.cbd9aec5-6891-4cb4-8c32-79333172197b@github.com>

On Thu, 3 Oct 2024 21:35:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Evacuations may occur during young or mixed collection cycles. When it is a young cycle during old marking, this barrier may be called to both evacuate cset objects and again to enqueue the targets of the copy (if they are in the old generation). If it is a mixed collection, then old marking will _not_ be in progress and this barrier will not be called to enqueue anything.

Test failures look unrelated to this change.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/507#issuecomment-2394254366

From wkemper at openjdk.org  Fri Oct  4 17:56:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 17:56:27 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <BHnJa1es3gSFbZmoalcW6kpP-y5d9fjQGZO_Lo8TXC8=.231e356f-4e75-4fd7-b79d-4ddcf514a158@github.com>
References: <BHnJa1es3gSFbZmoalcW6kpP-y5d9fjQGZO_Lo8TXC8=.231e356f-4e75-4fd7-b79d-4ddcf514a158@github.com>
Message-ID: <yoWW8W6g6Mfe-DdDod6tDoazjSoiiyTf1u3PJoTGuKM=.4cea2bbc-ec84-4d4a-9e6c-cff1326b6c27@github.com>

> Merges tag jdk-24+18

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 99 commits:

 - Merge remote-tracking branch 'shenandoah/master' into merge-jdk-24+18
 - 8341148: Open source several Choice related tests
   
   Reviewed-by: abhiscxk
 - 8334060: Implementation of Late Barrier Expansion for G1
   
   Co-authored-by: Roberto Casta?eda Lozano <rcastanedalo at openjdk.org>
   Co-authored-by: Erik ?sterlund <eosterlund at openjdk.org>
   Co-authored-by: Siyao Liu <siyao.l.liu at oracle.com>
   Co-authored-by: Kim Barrett <kbarrett at openjdk.org>
   Co-authored-by: Amit Kumar <amitkumar at openjdk.org>
   Co-authored-by: Martin Doerr <mdoerr at openjdk.org>
   Co-authored-by: Feilong Jiang <fjiang at openjdk.org>
   Co-authored-by: Sergey Nazarkin <snazarki at openjdk.org>
   Reviewed-by: kvn, tschatzl, fyang, ayang, kbarrett
 - 8341091: CDS: Segmented roots array misses roots
   
   Reviewed-by: adinn, iklam
 - 8341239: Open source closed frame tests # 3
   
   Reviewed-by: prr
 - 8341191: Open source few more AWT FileDialog tests
   
   Reviewed-by: prr, psadhukhan
 - 8341177: Opensource few List and a Window test
   
   Reviewed-by: prr
 - 8309841: Jarsigner should print a warning if an entry is removed
   
   Reviewed-by: mullan, hchao
 - 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type 'long int'
   
   Reviewed-by: phh, kdnilsen
 - 8341336: Fix -Wzero-as-null-pointer-constant warnings in PRODUCT-only code
   
   Reviewed-by: stefank, iwalulya, shade
 - ... and 89 more: https://git.openjdk.org/shenandoah/compare/cbc0fd0a...0039c03e

-------------

Changes: https://git.openjdk.org/shenandoah/pull/508/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=508&range=01
  Stats: 25502 lines in 448 files changed: 21177 ins; 2556 del; 1769 mod
  Patch: https://git.openjdk.org/shenandoah/pull/508.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/508/head:pull/508

PR: https://git.openjdk.org/shenandoah/pull/508

From wkemper at openjdk.org  Fri Oct  4 17:56:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 17:56:29 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <BHnJa1es3gSFbZmoalcW6kpP-y5d9fjQGZO_Lo8TXC8=.231e356f-4e75-4fd7-b79d-4ddcf514a158@github.com>
References: <BHnJa1es3gSFbZmoalcW6kpP-y5d9fjQGZO_Lo8TXC8=.231e356f-4e75-4fd7-b79d-4ddcf514a158@github.com>
Message-ID: <7Sa6wvWogAH3vmMNHyCqmMN6JVpxfsD10IGTHBbUF-A=.3e36064e-d7cd-4e96-8aee-cd8326d2bdc4@github.com>

On Fri, 4 Oct 2024 14:12:57 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-24+18

This pull request has now been integrated.

Changeset: 46582099
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/46582099dd91131aeef13496e3fc2c289d368c3b
Stats:     25502 lines in 448 files changed: 21177 ins; 2556 del; 1769 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/508

From wkemper at openjdk.org  Fri Oct  4 17:59:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 17:59:47 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <W25bSO8UkAWOtK_ZGGhc0VibmlHRJhcfvhRyxVzwPSg=.e536c55e-2d91-4ccd-b597-1477fa4d5f59@github.com>
References: <W25bSO8UkAWOtK_ZGGhc0VibmlHRJhcfvhRyxVzwPSg=.e536c55e-2d91-4ccd-b597-1477fa4d5f59@github.com>
Message-ID: <fhrXnkDAqgOpkC7IuiA5TcCYwLkBXjljIptUekGU8RA=.59f8f57b-2d8d-40be-ace7-765a67b637a4@github.com>

> Merges tag jdk-21.0.5+9

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/114/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/114/files/eced83e1..eced83e1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=114&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=114&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/114.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/114/head:pull/114

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/114

From wkemper at openjdk.org  Fri Oct  4 17:59:48 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 17:59:48 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <W25bSO8UkAWOtK_ZGGhc0VibmlHRJhcfvhRyxVzwPSg=.e536c55e-2d91-4ccd-b597-1477fa4d5f59@github.com>
References: <W25bSO8UkAWOtK_ZGGhc0VibmlHRJhcfvhRyxVzwPSg=.e536c55e-2d91-4ccd-b597-1477fa4d5f59@github.com>
Message-ID: <OA64-ya4yq04SEMgGmthjDZRfD4xmirUXEDzDCtmj4w=.dbbc5f78-431e-4603-986e-b01f5f41b1f2@github.com>

On Thu, 3 Oct 2024 14:19:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.5+9

This pull request has now been integrated.

Changeset: 17327d5d
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/17327d5d1d76c17907c27ee771e9b3bcf73ca7c4
Stats:     110 lines in 8 files changed: 93 ins; 1 del; 16 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/114

From wkemper at openjdk.org  Fri Oct  4 18:05:55 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 18:05:55 GMT
Subject: Integrated: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23:
 runtime error: signed integer overflow: -9223372036854775808 - 1
 cannot be represented in type 'long int'
In-Reply-To: <pWQftheTgl5DJnq7tQr4e2FiwIps3P6ZSE1zBOmcqDQ=.e34883a1-a082-4161-91f5-c7893d077c48@github.com>
References: <pWQftheTgl5DJnq7tQr4e2FiwIps3P6ZSE1zBOmcqDQ=.e34883a1-a082-4161-91f5-c7893d077c48@github.com>
Message-ID: <4MeyHQEQ3uQwIT8azsQcjaGdGjniLEX8-PILemFAWYg=.7e1c789b-099f-421e-a96a-24cf06dd0d7e@github.com>

On Fri, 4 Oct 2024 17:56:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport, fixes ubsan error.

This pull request has now been integrated.

Changeset: e06a43a4
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/e06a43a4dba65f806d5eeda9a483dc66204a78f5
Stats:     34 lines in 3 files changed: 7 ins; 0 del; 27 mod

8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type 'long int'

Backport-of: 57c1db5843db5f2c864318f3234767f436a836e3

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/115

From wkemper at openjdk.org  Fri Oct  4 18:02:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 18:02:13 GMT
Subject: RFR: 8332697: ubsan: shenandoahSimpleBitMap.inline.hpp:68:23: runtime
 error: signed integer overflow: -9223372036854775808 - 1 cannot be
 represented in type 'long int'
Message-ID: <pWQftheTgl5DJnq7tQr4e2FiwIps3P6ZSE1zBOmcqDQ=.e34883a1-a082-4161-91f5-c7893d077c48@github.com>

Clean backport, fixes ubsan error.

-------------

Commit messages:
 - Backport 57c1db5843db5f2c864318f3234767f436a836e3

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/115/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=115&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332697
  Stats: 34 lines in 3 files changed: 7 ins; 0 del; 27 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/115.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/115/head:pull/115

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/115

From kdnilsen at openjdk.org  Fri Oct  4 18:04:03 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 4 Oct 2024 18:04:03 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
 logging [v3]
In-Reply-To: <LExivLFuxTuF7QTXAZBfu5pm5AGRJyPpYxnjr57FBVg=.ae8b1544-0ce5-4761-ac5b-49867b37f4b9@github.com>
References: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
 <LExivLFuxTuF7QTXAZBfu5pm5AGRJyPpYxnjr57FBVg=.ae8b1544-0ce5-4761-ac5b-49867b37f4b9@github.com>
Message-ID: <Fd3GajOzCXmyoG_1D0bt5rMZJnikWW8bSAlvTS5HaH0=.8fac1d68-5768-4f53-82ef-e69048cc3586@github.com>

On Wed, 2 Oct 2024 21:32:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Not clean. Backport to only take lock for logging when log level is enabled.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix old gen usage of shFreeSet::log_status

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 184:

> 182:       // global soft refs policy, and we better report it every time heap
> 183:       // usage goes down.
> 184:       heap->update_capacity_and_used_at_gc();

This looks suspiciously like it needs to be under heap lock.  In the original code, it was.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp line 271:

> 269:       // global soft refs policy, and we better report it every time heap
> 270:       // usage goes down.
> 271:       heap->update_capacity_and_used_at_gc();

same question as above: does this need to be under lock?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah-jdk21u/pull/113#discussion_r1788097467
PR Review Comment: https://git.openjdk.org/shenandoah-jdk21u/pull/113#discussion_r1788100805

From ysr at openjdk.org  Fri Oct  4 18:14:00 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 4 Oct 2024 18:14:00 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
 logging [v3]
In-Reply-To: <LExivLFuxTuF7QTXAZBfu5pm5AGRJyPpYxnjr57FBVg=.ae8b1544-0ce5-4761-ac5b-49867b37f4b9@github.com>
References: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
 <LExivLFuxTuF7QTXAZBfu5pm5AGRJyPpYxnjr57FBVg=.ae8b1544-0ce5-4761-ac5b-49867b37f4b9@github.com>
Message-ID: <WodE4fp9Bhj8PUk2LY7GeMachoU-AZqgHD0JLgNapu8=.8735d4f0-0db1-4ad5-b67d-0410251fe9e6@github.com>

On Wed, 2 Oct 2024 21:32:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Not clean. Backport to only take lock for logging when log level is enabled.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix old gen usage of shFreeSet::log_status

Backport LGTM.

To Kelvin's question: I think we don't need the lock as these fields are updated only by the control thread (for consumption by jmx metrics and such). (I could be wrong and didn't check super-carefully.)

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/113#pullrequestreview-2348821403

From wkemper at openjdk.org  Fri Oct  4 18:21:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 18:21:34 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v2]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <BneyQG9faYqRCvUeqGbfun-y4bno0I4KR15685D0gZ0=.94c5052d-cb02-40d2-b158-f3c9d85ceffc@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 475 additional commits since the last revision:

 - Merge branch 'shenandoah/master' into great-genshen-pr-redux
 - Merge
 - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
   
   Reviewed-by: kdnilsen
 - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
   
   Reviewed-by: kdnilsen, ysr
 - Merge
 - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
   
   Reviewed-by: kdnilsen
 - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
   
   Reviewed-by: kdnilsen, shade, ysr
 - Merge
 - 8339643: Port JEP 404 to RISC-V
   
   Reviewed-by: wkemper, kdnilsen
 - 8340395: GenShen: Remove unnecessary check on card barrier flag
   
   Reviewed-by: ysr
 - ... and 465 more: https://git.openjdk.org/jdk/compare/143b2cd8...ed16e3ec

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21273/files
  - new: https://git.openjdk.org/jdk/pull/21273/files/8b25abe0..ed16e3ec

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=00-01

  Stats: 25330 lines in 433 files changed: 21577 ins; 2314 del; 1439 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From wkemper at openjdk.org  Fri Oct  4 18:41:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 18:41:01 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
 logging [v3]
In-Reply-To: <Fd3GajOzCXmyoG_1D0bt5rMZJnikWW8bSAlvTS5HaH0=.8fac1d68-5768-4f53-82ef-e69048cc3586@github.com>
References: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
 <LExivLFuxTuF7QTXAZBfu5pm5AGRJyPpYxnjr57FBVg=.ae8b1544-0ce5-4761-ac5b-49867b37f4b9@github.com>
 <Fd3GajOzCXmyoG_1D0bt5rMZJnikWW8bSAlvTS5HaH0=.8fac1d68-5768-4f53-82ef-e69048cc3586@github.com>
Message-ID: <_4-8b7o298SrAhQmGu9gYcTvSMRc8CSO3ASKTEqKAV4=.240153eb-815a-42f4-8d54-8b8e84507b72@github.com>

On Fri, 4 Oct 2024 17:58:55 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix old gen usage of shFreeSet::log_status
>
> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 184:
> 
>> 182:       // global soft refs policy, and we better report it every time heap
>> 183:       // usage goes down.
>> 184:       heap->update_capacity_and_used_at_gc();
> 
> This looks suspiciously like it needs to be under heap lock.  In the original code, it was.

Yes, I looked at this. These two fields are defined in `CollectedHeap` and reflect `committed` and `used` for Shenandoah. They are only updated by the control thread and they are only read without the heap lock by the control thread or the vm thread to adjust the soft reference policy. The risk would be that if `used` somehow exceeded `committed`, but we have many assertions through `ShenandoahGeneration` to assert that this doesn't happen.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah-jdk21u/pull/113#discussion_r1788141477

From kdnilsen at openjdk.org  Fri Oct  4 19:25:58 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 4 Oct 2024 19:25:58 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
 logging [v3]
In-Reply-To: <LExivLFuxTuF7QTXAZBfu5pm5AGRJyPpYxnjr57FBVg=.ae8b1544-0ce5-4761-ac5b-49867b37f4b9@github.com>
References: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
 <LExivLFuxTuF7QTXAZBfu5pm5AGRJyPpYxnjr57FBVg=.ae8b1544-0ce5-4761-ac5b-49867b37f4b9@github.com>
Message-ID: <eY9OCxTXLVR_9_gsUOulvOvgKayIbgm0moeKh1MnUY0=.ba45952e-4ebf-4d0e-8b1a-31709f82b47d@github.com>

On Wed, 2 Oct 2024 21:32:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Not clean. Backport to only take lock for logging when log level is enabled.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix old gen usage of shFreeSet::log_status

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/113#pullrequestreview-2348975786

From wkemper at openjdk.org  Fri Oct  4 19:28:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 19:28:08 GMT
Subject: RFR: 8341554: Shenandoah: Missing heap lock when updating usage for
 soft ref policy
Message-ID: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>

A recent change to avoid checking the log level under the lock inadvertently removed the lock from an operation that needs it.

-------------

Commit messages:
 - Restore missing heap lock when updating usage at last gc

Changes: https://git.openjdk.org/jdk/pull/21362/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21362&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341554
  Stats: 7 lines in 1 file changed: 3 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/21362.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21362/head:pull/21362

PR: https://git.openjdk.org/jdk/pull/21362

From kdnilsen at openjdk.org  Fri Oct  4 19:28:08 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 4 Oct 2024 19:28:08 GMT
Subject: RFR: 8341554: Shenandoah: Missing heap lock when updating usage
 for soft ref policy
In-Reply-To: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>
References: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>
Message-ID: <8olxAEXHXd9rybQFElBljiWNI2pfyAk-jnsKptCDwsw=.8963ecb2-8d1c-444f-a7a2-417232a1690e@github.com>

On Fri, 4 Oct 2024 19:18:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

> A recent change to avoid checking the log level under the lock inadvertently removed the lock from an operation that needs it.

Marked as reviewed by kdnilsen (Author).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21362#pullrequestreview-2348975142

From ysr at openjdk.org  Fri Oct  4 20:43:36 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 4 Oct 2024 20:43:36 GMT
Subject: RFR: 8341554: Shenandoah: Missing heap lock when updating usage
 for soft ref policy
In-Reply-To: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>
References: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>
Message-ID: <M2hTMrbAtzVJ-baCCUC1BtW6PJbp8okg4TSbsUhHF5E=.2091bc72-2ca0-4bf2-b604-401552beeb0a@github.com>

On Fri, 4 Oct 2024 19:18:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

> A recent change to avoid checking the log level under the lock inadvertently removed the lock from an operation that needs it.

Good catch by @kdnilsen !

Sorry for missing this in the previous review at https://github.com/openjdk/jdk/pull/19915 for @pengxiaolong.

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 186:

> 184:         ShenandoahHeapLocker locker(heap->lock());
> 185:         heap->update_capacity_and_used_at_gc();
> 186:       }

If free set logging is enabled this does a lock/unlock for logging and then another lock/unlock for updating capacity.

But I guess it's unavoidable, and it likely won't be the case that we have free set logging enabled in performance critical production situations, so not worth wasting too much sleep over.

Reviewed!

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21362#pullrequestreview-2349091755
PR Review Comment: https://git.openjdk.org/jdk/pull/21362#discussion_r1788281685

From ysr at openjdk.org  Fri Oct  4 20:52:37 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 4 Oct 2024 20:52:37 GMT
Subject: RFR: 8341554: Shenandoah: Missing heap lock when updating usage
 for soft ref policy
In-Reply-To: <M2hTMrbAtzVJ-baCCUC1BtW6PJbp8okg4TSbsUhHF5E=.2091bc72-2ca0-4bf2-b604-401552beeb0a@github.com>
References: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>
 <M2hTMrbAtzVJ-baCCUC1BtW6PJbp8okg4TSbsUhHF5E=.2091bc72-2ca0-4bf2-b604-401552beeb0a@github.com>
Message-ID: <KOjMvQy3Nx-aK8pTUaAw_8tB6lCHiZZsZYGQKtWupRY=.225ec9be-91cf-4fc1-af9c-5603a2b08db4@github.com>

On Fri, 4 Oct 2024 20:34:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> A recent change to avoid checking the log level under the lock inadvertently removed the lock from an operation that needs it.
>
> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 186:
> 
>> 184:         ShenandoahHeapLocker locker(heap->lock());
>> 185:         heap->update_capacity_and_used_at_gc();
>> 186:       }
> 
> If free set logging is enabled this does a lock/unlock for logging and then another lock/unlock for updating capacity.
> 
> But I guess it's unavoidable, and it likely won't be the case that we have free set logging enabled in performance critical production situations, so not worth wasting too much sleep over.
> 
> Reviewed!

@earthling-amzn : I'd suggest leaving a comment either in the ticket or in the PR (or here in the code?) stating the race that we are vulnerable to if we don't hold the lock, namely the potential skew between used & capacity, and why it's important not to have that skew for consumers of these fields. I imagine that we can demonstrate the issue with a targeted regression test.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21362#discussion_r1788297486

From wkemper at openjdk.org  Fri Oct  4 21:58:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 21:58:44 GMT
Subject: RFR: 8341554: Shenandoah: Missing heap lock when updating usage
 for soft ref policy
In-Reply-To: <KOjMvQy3Nx-aK8pTUaAw_8tB6lCHiZZsZYGQKtWupRY=.225ec9be-91cf-4fc1-af9c-5603a2b08db4@github.com>
References: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>
 <M2hTMrbAtzVJ-baCCUC1BtW6PJbp8okg4TSbsUhHF5E=.2091bc72-2ca0-4bf2-b604-401552beeb0a@github.com>
 <KOjMvQy3Nx-aK8pTUaAw_8tB6lCHiZZsZYGQKtWupRY=.225ec9be-91cf-4fc1-af9c-5603a2b08db4@github.com>
Message-ID: <aI3SFLLPbbc3Qh5D5-zRI0T7yQ8085E3Yhh4Ry0kaxw=.2e3c6223-d93b-4230-80d1-f44c74cbfd39@github.com>

On Fri, 4 Oct 2024 20:48:35 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 186:
>> 
>>> 184:         ShenandoahHeapLocker locker(heap->lock());
>>> 185:         heap->update_capacity_and_used_at_gc();
>>> 186:       }
>> 
>> If free set logging is enabled this does a lock/unlock for logging and then another lock/unlock for updating capacity.
>> 
>> But I guess it's unavoidable, and it likely won't be the case that we have free set logging enabled in performance critical production situations, so not worth wasting too much sleep over.
>> 
>> Reviewed!
>
> @earthling-amzn : I'd suggest leaving a comment either in the ticket or in the PR (or here in the code?) stating the race that we are vulnerable to if we don't hold the lock, namely the potential skew between used & capacity, and why it's important not to have that skew for consumers of these fields. I imagine that we can demonstrate the issue with a targeted regression test.

Yeah, I think coupling the lock with the logging and `update_capacity_and_used` is what led us into trouble to begin with.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21362#discussion_r1788371332

From wkemper at openjdk.org  Fri Oct  4 21:58:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Oct 2024 21:58:44 GMT
Subject: Integrated: 8341554: Shenandoah: Missing heap lock when updating usage
 for soft ref policy
In-Reply-To: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>
References: <vzett0HDc9woTK5zUQh5ISrxCdpMpixxeeGIv0XqvmU=.305f3e59-f893-46d8-b073-f897ff85da7a@github.com>
Message-ID: <rHHWeuYO_r6yRwlsAJLIuv_Gyu_l6nDV7pGUC2eyZRs=.dcd76a04-6956-49fd-b6bf-13cc1e89b72d@github.com>

On Fri, 4 Oct 2024 19:18:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

> A recent change to avoid checking the log level under the lock inadvertently removed the lock from an operation that needs it.

This pull request has now been integrated.

Changeset: bade041d
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/bade041db82a09cf33d4dbcc849f5784b3851f3d
Stats:     7 lines in 1 file changed: 3 ins; 0 del; 4 mod

8341554: Shenandoah: Missing heap lock when updating usage for soft ref policy

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/jdk/pull/21362

From stefank at openjdk.org  Mon Oct  7 08:55:59 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 7 Oct 2024 08:55:59 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v30]
In-Reply-To: <aU7Oq51P0gFTm6hW7hQW_fmAYCdLG6JlqW6lBOkaRKE=.e30fa902-dc09-4944-8983-2c14ad068cf1@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <aU7Oq51P0gFTm6hW7hQW_fmAYCdLG6JlqW6lBOkaRKE=.e30fa902-dc09-4944-8983-2c14ad068cf1@github.com>
Message-ID: <NtlirWYS-adgUWyhc0RA52WzClmcbtRuk3ykReYfsDg=.db48cc39-e30d-49e2-8a61-df4f40ecf73b@github.com>

On Fri, 4 Oct 2024 11:15:37 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 76 commits:
> 
>  - Merge remote-tracking branch 'rkennke/JDK-8305895-v4' into JDK-8305895-v4
>  - Revert "Disable TestSplitPacks::test4a, failing on aarch64"
>    
>    This reverts commit 059b1573db26d1d2902ca6dadc8413f445234c2a.
>  - Simplify object init code in interpreter
>  - Disable some vectorization tests that fail with +UCOH and UseSSE<=3
>  - Fix for CDSPluginTest.java
>  - Merge tag 'jdk-24+18' into JDK-8305895-v4
>    
>    Added tag jdk-24+18 for changeset 19642bd3
>  - Disable TestSplitPacks::test4a, failing on aarch64
>  - @robcasloz review comments
>  - Improve CollectedHeap::is_oop()
>  - Allow LM_MONITOR on 32-bit platforms
>  - ... and 66 more: https://git.openjdk.org/jdk/compare/19642bd3...8742f3c1

I realize that some of my comments was stuck as drafts and had not been published. I took an extra pass over the gc/ and most of oops/ with the intention to approve those parts. However, I see that the comment about `fill_dense_prefix_end` and `MinObjectAlignment` has not been addressed. I don't know if that change is correct, so it would be good to get that scrutinized.

src/hotspot/share/gc/shared/collectedHeap.cpp line 223:

> 221: }
> 222: 
> 223: bool klass_is_sane(oop object) {

Should at least be static. We might also want to keep reporting errors if a klass pointer is null when we don't have a forwarded object:

static bool klass_is_sane(oop object) {
  if (UseCompactObjectHeaders) {
    // With compact headers, we can't safely access the Klass* when
    // the object has been forwarded, because non-full-GC-forwarding
    // distinction between Full-GC and regular GC forwarding.
    markWord mark = object->mark();
    if (mark.is_forwarded()) {
      // We can't access the Klass*. We optimistically assume that
      // it is ok. This happens very rarely.
      return true;
    }

    return Metaspace::contains(mark.klass_without_asserts());
  }

  return Metaspace::contains(object->klass_without_asserts());
}

src/hotspot/share/oops/compressedKlass.cpp line 28:

> 26: #include "logging/log.hpp"
> 27: #include "memory/metaspace.hpp"
> 28: #include "oops/klass.hpp"

Is this include really needed or could this be reverted klass.hpp? If it is needed is should be moved to after compressedKlass.inline.hpp.

src/hotspot/share/oops/compressedKlass.cpp line 31:

> 29: #include "oops/compressedKlass.inline.hpp"
> 30: #include "runtime/globals.hpp"
> 31: #include "runtime/java.hpp"

Do you remember why this was added?

src/hotspot/share/oops/markWord.cpp line 35:

> 33: STATIC_ASSERT(markWord::klass_shift + markWord::klass_bits == 64);
> 34: // The hash (preceding nKlass) shall be a direct neighbor but not interleave
> 35: STATIC_ASSERT(markWord::klass_shift == markWord::hash_bits + markWord::hash_shift);

The code is not consistent in it usage of the name for the klass bits. Here it says `nKlass` in the comment, but the fields are named `klass`. Maybe just change the comment to says `(preceding klass bits)`.

Note that the term `nklass` is not prevalent in the code base, but with this patch its starting to get a foot hold. It might be good to figure out what we do want to call these in field names and variables to at least a little bit more consistency in the code base. Currently we have `nklass`, `nKlass` `nk`, `narrow_klass`.

 In other places we have functions that are named `set_narrow_klass`, but the field is called `nklass` and other places call it `nk`. It would be good to stay consistent with the naming. FWIW, nklass has very little precedence in the code, so cleaning that away might be easiest.thing is to clean out all usages of nklass, because it isn't a

src/hotspot/share/oops/markWord.inline.hpp line 35:

> 33:   assert(UseCompactObjectHeaders, "only used with compact object headers");
> 34:   const narrowKlass nk = value() >> klass_shift;
> 35:   return narrowKlass(value() >> klass_shift);

This sets up an unused variable.
```suggestion 
  const narrowKlass nk = value() >> klass_shift;
  return narrowKlass(value() >> klass_shift);

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2331450326
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1777180846
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1789757910
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1789759027
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1789787634
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1789790323

From stefank at openjdk.org  Mon Oct  7 08:55:59 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 7 Oct 2024 08:55:59 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <-UEFgAIQjGBginN0JRoyuwwJLmDssUEQGE_tCP-tRkw=.01ef3f37-01fa-4931-b4f3-571d21252bbd@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <-UEFgAIQjGBginN0JRoyuwwJLmDssUEQGE_tCP-tRkw=.01ef3f37-01fa-4931-b4f3-571d21252bbd@github.com>
Message-ID: <jj6GEQDn7YAPHbFlHtE6Hqx4BwCAUh2XEFLTJ1gBmeY=.faa0d50c-9e26-4dec-aeb9-15d3ca11b1b2@github.com>

On Thu, 19 Sep 2024 05:36:41 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> src/hotspot/share/gc/parallel/psParallelCompact.cpp line 787:
>> 
>>> 785:     // The gap is always equal to min-fill-size, so nothing to do.
>>> 786:     return;
>>> 787:   }
>> 
>> Reading the comment, it is not obvious that this is correct if you set MinObjectAlignment to something larger than the default value:
>> 
>> void PSParallelCompact::fill_dense_prefix_end(SpaceId id) {
>>   // Comparing two sizes to decide if filling is required:
>>   //
>>   // The size of the filler (min-obj-size) is 2 heap words with the default
>>   // MinObjAlignment, since both markword and klass take 1 heap word.
>>   //
>>   // The size of the gap (if any) right before dense-prefix-end is
>>   // MinObjAlignment.
>>   //
>>   // Need to fill in the gap only if it's smaller than min-obj-size, and the
>>   // filler obj will extend to next region.
>> 
>>   // Note: If min-fill-size decreases to 1, this whole method becomes redundant.
>>   if (UseCompactObjectHeaders) {
>>     // The gap is always equal to min-fill-size, so nothing to do.
>>     return;
>>   }
>>   assert(CollectedHeap::min_fill_size() >= 2, "inv");
>
> Style note: The added code is inserted between a comment and the code that the comment refers to. It would be nice to tidy this up.

Did you figure out if the code above is correct w.r.t. `MinObjectAlignment=16`?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1789797050

From stefank at openjdk.org  Mon Oct  7 08:55:59 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 7 Oct 2024 08:55:59 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v9]
In-Reply-To: <CKaxMvoxrt7YLGw6Zee_SW8uBb8tg3j94mw-Mcf5u0I=.78240440-cee1-413b-ae49-1b85592d5ba7@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <6Rant6SjxpFIHHWNthWc_plOdnGpWPvqj3rxRe144po=.bcdbad7a-e93a-41a3-b958-6ae602c7e083@github.com>
 <NR6CKOtdUaVarB7hiAUIAsIMe03qiTQr5VcEOWGt8mA=.164efc10-f0ff-4ddf-80b8-c56599c0ba7d@github.com>
 <BNQiyA0NCBO-oFZU4Nz3yvaMxDzICFxfqDKWt4wCL_U=.9405414d-f961-4168-8d26-ef5f58069161@github.com>
 <igBchd3diNsHdnQwZS2sj4tjaKejOq4f4EZovOmpmRQ=.abf79fee-9592-4555-ac01-de0170665ae4@github.com>
 <BkuKSFHleaF8I2tWCDIC-rGlWnfGhbK1zQ6CLlB-rVE=.9c7976e3-b0e9-4356-8d4b-d880aa30bffa@github.com>
 <ubcY5Q-aPRCLmhZV9MjBfzzbs84FegDvNkHx8nwPUd0=.46503fd8-d274-4558-bd0f-cc0c4f199667@github.com>
 <2w9H6VAbxm7BgSGRwKAxbI56bG-k4bE_ZDviGrBF36o=.3d4cb47f-0f84-479a-a809-6d0186dfad2d@github.com>
 <CKaxMvoxrt7YLGw6Zee_SW8uBb8tg3j94mw-Mcf5u0I=.78240440-cee1-413b-ae49-1b85592d5ba7@github.com>
Message-ID: <Hp2zrxh4JHI6OyCJY-KW30h6Tjxj5JeGAinxVNrE8wc=.abaa66fc-cfa8-429a-a179-d6e8075a55ac@github.com>

On Fri, 27 Sep 2024 16:31:55 GMT, Yudi Zheng <yzheng at openjdk.org> wrote:

>> This is my current work-in-progress code:
>> https://github.com/stefank/jdk/compare/pull/20677...stefank:jdk:lilliput_remove_prototype_header_wip_2
>> 
>> I've made some large rewrites and I'm currently running it through functional testing.
>
> If @stefank 's patch does not go in this PR, could you please export `Klass::_prototype_header` to JVMCI? Thanks!
> 
> diff --git a/src/hotspot/share/jvmci/vmStructs_jvmci.cpp b/src/hotspot/share/jvmci/vmStructs_jvmci.cpp
> index 9d1b8a1cb9f..e462025074f 100644
> --- a/src/hotspot/share/jvmci/vmStructs_jvmci.cpp
> +++ b/src/hotspot/share/jvmci/vmStructs_jvmci.cpp
> @@ -278,6 +278,7 @@
>    nonstatic_field(Klass,                       _bitmap,                                       uintx)                                 \
>    nonstatic_field(Klass,                       _hash_slot,                                    uint8_t)                               \
>    nonstatic_field(Klass,                       _misc_flags._flags,                            u1)                                    \
> +  nonstatic_field(Klass,                       _prototype_header,                             markWord)                              \
>                                                                                                                                       \
>    nonstatic_field(LocalVariableTableElement,   start_bci,                                     u2)                                    \
>    nonstatic_field(LocalVariableTableElement,   length,                                        u2)                                    \

My patch will not be included in this PR. After JEP 450 has been delivered we'll reconsider if we want that patch or not.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1778950736

From rkennke at openjdk.org  Mon Oct  7 10:52:53 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 7 Oct 2024 10:52:53 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <jj6GEQDn7YAPHbFlHtE6Hqx4BwCAUh2XEFLTJ1gBmeY=.faa0d50c-9e26-4dec-aeb9-15d3ca11b1b2@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <-UEFgAIQjGBginN0JRoyuwwJLmDssUEQGE_tCP-tRkw=.01ef3f37-01fa-4931-b4f3-571d21252bbd@github.com>
 <jj6GEQDn7YAPHbFlHtE6Hqx4BwCAUh2XEFLTJ1gBmeY=.faa0d50c-9e26-4dec-aeb9-15d3ca11b1b2@github.com>
Message-ID: <twhuRW8mlXjzlobkiklKNV0IUCWxzM0tu48uX-4rVeQ=.d98ae5da-502c-4f20-8bba-b9ff963dc53b@github.com>

On Mon, 7 Oct 2024 08:49:58 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Style note: The added code is inserted between a comment and the code that the comment refers to. It would be nice to tidy this up.
>
> Did you figure out if the code above is correct w.r.t. `MinObjectAlignment=16`?

When MinObjectAlignment=16, then that method does nothing anyway:


  if (MinObjAlignment > 1) {
    return;
  }



I think what it really means to say is

  if (MinObjAlignment >=  CollectedHeap::min_fill_size()) {
    return;
  }



That's also what the comment says: "The size of the gap (if any) right before dense-prefix-end is MinObjAlignment. Need to fill in the gap only if it's smaller than min-obj-size, and the filler obj will extend to next region."

If I interpret that correctly, we need to deal with the situation only when MinObjAlignment < min_fill_size, because the filler object would extend to the next region, and we need to adjust the next region and mark-bitmap for that extra word. @albertnetymk might want to confirm.

I'll move the if (UCOH) block down a little bit to right before if (MinObjAlignment) block.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1789983561

From rkennke at openjdk.org  Mon Oct  7 11:03:55 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 7 Oct 2024 11:03:55 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v30]
In-Reply-To: <NtlirWYS-adgUWyhc0RA52WzClmcbtRuk3ykReYfsDg=.db48cc39-e30d-49e2-8a61-df4f40ecf73b@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <aU7Oq51P0gFTm6hW7hQW_fmAYCdLG6JlqW6lBOkaRKE=.e30fa902-dc09-4944-8983-2c14ad068cf1@github.com>
 <NtlirWYS-adgUWyhc0RA52WzClmcbtRuk3ykReYfsDg=.db48cc39-e30d-49e2-8a61-df4f40ecf73b@github.com>
Message-ID: <wZQvls5Bmb1p4U0LFvXishmbr_J18_Il1tDQVbiAolk=.d26b761e-3201-44b8-8b62-aa0cf4dd32ed@github.com>

On Mon, 7 Oct 2024 08:25:55 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 76 commits:
>> 
>>  - Merge remote-tracking branch 'rkennke/JDK-8305895-v4' into JDK-8305895-v4
>>  - Revert "Disable TestSplitPacks::test4a, failing on aarch64"
>>    
>>    This reverts commit 059b1573db26d1d2902ca6dadc8413f445234c2a.
>>  - Simplify object init code in interpreter
>>  - Disable some vectorization tests that fail with +UCOH and UseSSE<=3
>>  - Fix for CDSPluginTest.java
>>  - Merge tag 'jdk-24+18' into JDK-8305895-v4
>>    
>>    Added tag jdk-24+18 for changeset 19642bd3
>>  - Disable TestSplitPacks::test4a, failing on aarch64
>>  - @robcasloz review comments
>>  - Improve CollectedHeap::is_oop()
>>  - Allow LM_MONITOR on 32-bit platforms
>>  - ... and 66 more: https://git.openjdk.org/jdk/compare/19642bd3...8742f3c1
>
> src/hotspot/share/oops/compressedKlass.cpp line 28:
> 
>> 26: #include "logging/log.hpp"
>> 27: #include "memory/metaspace.hpp"
>> 28: #include "oops/klass.hpp"
> 
> Is this include really needed or could this be reverted klass.hpp? If it is needed is should be moved to after compressedKlass.inline.hpp.

I don't think it's needed. I'll remove it.

> src/hotspot/share/oops/compressedKlass.cpp line 31:
> 
>> 29: #include "oops/compressedKlass.inline.hpp"
>> 30: #include "runtime/globals.hpp"
>> 31: #include "runtime/java.hpp"
> 
> Do you remember why this was added?

Looks like this is for vm_exit_during_initialization().

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1789996985
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1789999402

From stefank at openjdk.org  Mon Oct  7 11:43:00 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 7 Oct 2024 11:43:00 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v30]
In-Reply-To: <wZQvls5Bmb1p4U0LFvXishmbr_J18_Il1tDQVbiAolk=.d26b761e-3201-44b8-8b62-aa0cf4dd32ed@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <aU7Oq51P0gFTm6hW7hQW_fmAYCdLG6JlqW6lBOkaRKE=.e30fa902-dc09-4944-8983-2c14ad068cf1@github.com>
 <NtlirWYS-adgUWyhc0RA52WzClmcbtRuk3ykReYfsDg=.db48cc39-e30d-49e2-8a61-df4f40ecf73b@github.com>
 <wZQvls5Bmb1p4U0LFvXishmbr_J18_Il1tDQVbiAolk=.d26b761e-3201-44b8-8b62-aa0cf4dd32ed@github.com>
Message-ID: <xkj4iPPqe3T_nJxW_IE59EF806ZEA4cbXZNphRCmYLQ=.5e23dced-9aa6-418c-bfd5-2aad3b727a68@github.com>

On Mon, 7 Oct 2024 11:00:54 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> src/hotspot/share/oops/compressedKlass.cpp line 31:
>> 
>>> 29: #include "oops/compressedKlass.inline.hpp"
>>> 30: #include "runtime/globals.hpp"
>>> 31: #include "runtime/java.hpp"
>> 
>> Do you remember why this was added?
>
> Looks like this is for vm_exit_during_initialization().

I see. Thanks. (What a funny header file that is)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1790060417

From stefank at openjdk.org  Mon Oct  7 11:51:54 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 7 Oct 2024 11:51:54 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <twhuRW8mlXjzlobkiklKNV0IUCWxzM0tu48uX-4rVeQ=.d98ae5da-502c-4f20-8bba-b9ff963dc53b@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <-UEFgAIQjGBginN0JRoyuwwJLmDssUEQGE_tCP-tRkw=.01ef3f37-01fa-4931-b4f3-571d21252bbd@github.com>
 <jj6GEQDn7YAPHbFlHtE6Hqx4BwCAUh2XEFLTJ1gBmeY=.faa0d50c-9e26-4dec-aeb9-15d3ca11b1b2@github.com>
 <twhuRW8mlXjzlobkiklKNV0IUCWxzM0tu48uX-4rVeQ=.d98ae5da-502c-4f20-8bba-b9ff963dc53b@github.com>
Message-ID: <z1P1My8MgsUMZ6wFP1Ia0aMFzTkD8oxrGwDI4o4mHrY=.d4c22978-7190-432d-9206-dfbbb1a66a9b@github.com>

On Mon, 7 Oct 2024 10:49:39 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Did you figure out if the code above is correct w.r.t. `MinObjectAlignment=16`?
>
> When MinObjectAlignment=16, then that method does nothing anyway:
> 
> 
>   if (MinObjAlignment > 1) {
>     return;
>   }
> 
> 
> 
> I think what it really means to say is
> 
>   if (MinObjAlignment >=  CollectedHeap::min_fill_size()) {
>     return;
>   }
> 
> 
> 
> That's also what the comment says: "The size of the gap (if any) right before dense-prefix-end is MinObjAlignment. Need to fill in the gap only if it's smaller than min-obj-size, and the filler obj will extend to next region."
> 
> If I interpret that correctly, we need to deal with the situation only when MinObjAlignment < min_fill_size, because the filler object would extend to the next region, and we need to adjust the next region and mark-bitmap for that extra word. @albertnetymk might want to confirm.
> 
> I'll move the if (UCOH) block down a little bit to right before if (MinObjAlignment) block.

After re-reading this again I agree with what you're writing. If you make the change to use:

if (MinObjAlignment >= CollectedHeap::min_fill_size()) {
 return;
 }


do you even have to check for UCOH in this function?

I also wonder if you could tweak the comment now that this is not true when UCOH is on:

// The size of the filler (min-obj-size) is 2 heap words with the default
// MinObjAlignment, since both markword and klass take 1 heap word.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1790074043

From rkennke at openjdk.org  Mon Oct  7 12:48:43 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 7 Oct 2024 12:48:43 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v31]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <zFKBeHSnUUhfCv95M3aVKEFRTEcEn_m5iS69tohiAsE=.b652f8fd-9da1-4428-8488-57c06f275da9@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  @stefank review comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/8742f3c1..572f1ac0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=30
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=29-30

  Stats: 20 lines in 6 files changed: 4 ins; 8 del; 8 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Mon Oct  7 13:24:26 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 7 Oct 2024 13:24:26 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v32]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <XdTIEoeBMhkUPVI42RfZLH-PP6QszZ66_FeCrqDtxs0=.3da42b45-52af-4c75-a01e-6df7c81ab9db@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Remove unused variable

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/572f1ac0..60401086

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=31
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=30-31

  Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Mon Oct  7 13:32:41 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 7 Oct 2024 13:32:41 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v2]
In-Reply-To: <BneyQG9faYqRCvUeqGbfun-y4bno0I4KR15685D0gZ0=.94c5052d-cb02-40d2-b158-f3c9d85ceffc@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <BneyQG9faYqRCvUeqGbfun-y4bno0I4KR15685D0gZ0=.94c5052d-cb02-40d2-b158-f3c9d85ceffc@github.com>
Message-ID: <E7tz6jf5RnfpORFkTiDwjvglAPtoW0zH_dt8lUGvyEs=.3d17aa5f-d51b-44af-8496-48722ad543f8@github.com>

On Fri, 4 Oct 2024 18:21:34 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 475 additional commits since the last revision:
> 
>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>    
>    Reviewed-by: kdnilsen
>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>    
>    Reviewed-by: kdnilsen, ysr
>  - Merge
>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>    
>    Reviewed-by: kdnilsen
>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>    
>    Reviewed-by: kdnilsen, shade, ysr
>  - Merge
>  - 8339643: Port JEP 404 to RISC-V
>    
>    Reviewed-by: wkemper, kdnilsen
>  - 8340395: GenShen: Remove unnecessary check on card barrier flag
>    
>    Reviewed-by: ysr
>  - ... and 465 more: https://git.openjdk.org/jdk/compare/e155dc3e...ed16e3ec

There seems to be something missing:

/home/runner/work/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp: In member function ?oop ShenandoahHeap::try_evacuate_object(oop, Thread*, ShenandoahHeapRegion*, ShenandoahAffiliation)?:
/home/runner/work/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:1276:3: error: ?_evac_tracker? was not declared in this scope; did you mean ?_mmu_tracker??
 1276 |   _evac_tracker->begin_evacuation(thread, size * HeapWordSize);
      |   ^~~~~~~~~~~~~
      |   _mmu_tracker
/home/runner/work/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp: In member function ?virtual void ShenandoahHeap::print_tracing_info() const?:
/home/runner/work/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:1534:5: error: ?evac_tracker? was not declared in this scope; did you mean ?mmu_tracker??
 1534 |     evac_tracker()->print_global_on(&ls);
      |     ^~~~~~~~~~~~
      |     mmu_tracker

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2396941518

From rkennke at openjdk.org  Mon Oct  7 13:55:17 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 7 Oct 2024 13:55:17 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v33]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <VWBPrwk40VzajxZ4gNqUF-w3-XkNWyCK1ayQ7TFYxNs=.1e941ed0-5dad-4321-bcd5-1c87517f3aa6@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Rename nklass/nKlass

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/60401086..1ab20774

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=32
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=31-32

  Stats: 14 lines in 6 files changed: 0 ins; 0 del; 14 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Mon Oct  7 13:55:18 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 7 Oct 2024 13:55:18 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v30]
In-Reply-To: <NtlirWYS-adgUWyhc0RA52WzClmcbtRuk3ykReYfsDg=.db48cc39-e30d-49e2-8a61-df4f40ecf73b@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <aU7Oq51P0gFTm6hW7hQW_fmAYCdLG6JlqW6lBOkaRKE=.e30fa902-dc09-4944-8983-2c14ad068cf1@github.com>
 <NtlirWYS-adgUWyhc0RA52WzClmcbtRuk3ykReYfsDg=.db48cc39-e30d-49e2-8a61-df4f40ecf73b@github.com>
Message-ID: <gW425l3Fk9V3WAGejgKf-PunT7yc2abszhJiU7L1jJw=.6704865f-b06e-4f0d-8637-08d66422c8b9@github.com>

On Mon, 7 Oct 2024 08:44:16 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 76 commits:
>> 
>>  - Merge remote-tracking branch 'rkennke/JDK-8305895-v4' into JDK-8305895-v4
>>  - Revert "Disable TestSplitPacks::test4a, failing on aarch64"
>>    
>>    This reverts commit 059b1573db26d1d2902ca6dadc8413f445234c2a.
>>  - Simplify object init code in interpreter
>>  - Disable some vectorization tests that fail with +UCOH and UseSSE<=3
>>  - Fix for CDSPluginTest.java
>>  - Merge tag 'jdk-24+18' into JDK-8305895-v4
>>    
>>    Added tag jdk-24+18 for changeset 19642bd3
>>  - Disable TestSplitPacks::test4a, failing on aarch64
>>  - @robcasloz review comments
>>  - Improve CollectedHeap::is_oop()
>>  - Allow LM_MONITOR on 32-bit platforms
>>  - ... and 66 more: https://git.openjdk.org/jdk/compare/19642bd3...8742f3c1
>
> src/hotspot/share/oops/markWord.cpp line 35:
> 
>> 33: STATIC_ASSERT(markWord::klass_shift + markWord::klass_bits == 64);
>> 34: // The hash (preceding nKlass) shall be a direct neighbor but not interleave
>> 35: STATIC_ASSERT(markWord::klass_shift == markWord::hash_bits + markWord::hash_shift);
> 
> The code is not consistent in it usage of the name for the klass bits. Here it says `nKlass` in the comment, but the fields are named `klass`. Maybe just change the comment to says `(preceding klass bits)`.
> 
> Note that the term `nklass` is not prevalent in the code base, but with this patch its starting to get a foot hold. It might be good to figure out what we do want to call these in field names and variables to at least a little bit more consistency in the code base. Currently we have `nklass`, `nKlass` `nk`, `narrow_klass`.
> 
>  In other places we have functions that are named `set_narrow_klass`, but the field is called `nklass` and other places call it `nk`. It would be good to stay consistent with the naming. FWIW, nklass has very little precedence in the code, so cleaning that away might be easiest.thing is to clean out all usages of nklass, because it isn't a

I renamed all occurrences of nklass and nKlass in shared code to something more useful. I left load_nklass* stuff in aarch64 and x86 code alone for now. Let me know if that addresses your concern.

https://github.com/openjdk/jdk/pull/20677/commits/1ab207746e4c4baaa6da162d7c1535c75342fa2e

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1790270819

From rkennke at openjdk.org  Mon Oct  7 14:28:40 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 7 Oct 2024 14:28:40 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v34]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <jg_gzL1xUQxh6Isj-q5ceTJf4r9UM7CvHWlyoT3H_Ok=.314a0607-2bbe-42c8-b9c3-b5e8c8efb110@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Some more review comments/cleanups

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/1ab20774..17f8eb54

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=33
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=32-33

  Stats: 8 lines in 5 files changed: 3 ins; 3 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From wkemper at openjdk.org  Mon Oct  7 18:48:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 7 Oct 2024 18:48:45 GMT
Subject: RFR: 8341668: Shenandoah: assert(tail_bits < (idx_t)BitsPerWord)
 failed: precondition
Message-ID: <8pCGlrqneW3yypyIwVkzoET45NOw8jKNJQ1x8-nm2PM=.a95b4c84-0fa7-4f44-9ddb-4af73e16a16b@github.com>

`tail_mask` function should not be in the global scope.

-------------

Commit messages:
 - Fix name collision for tail_mask function

Changes: https://git.openjdk.org/jdk/pull/21395/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21395&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341668
  Stats: 3 lines in 2 files changed: 2 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21395.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21395/head:pull/21395

PR: https://git.openjdk.org/jdk/pull/21395

From wkemper at openjdk.org  Mon Oct  7 19:04:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 7 Oct 2024 19:04:32 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
 logging [v4]
In-Reply-To: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
References: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
Message-ID: <WynTqGr7Pd0-SvzL66g1n5X8-jkn-f4KQyKrzkecQt0=.56128348-16a6-4589-9c40-1928e58e9ce4@github.com>

> Not clean. Backport to only take lock for logging when log level is enabled.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Take lock when updating capacity and used for generational mode
 - 8341554: Shenandoah: Missing heap lock when updating usage for soft ref policy
   
   Reviewed-by: kdnilsen, ysr

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/113/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/113/files/275e76d9..36af3a6f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=113&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=113&range=02-03

  Stats: 14 lines in 2 files changed: 6 ins; 0 del; 8 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/113.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/113/head:pull/113

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/113

From wkemper at openjdk.org  Mon Oct  7 20:18:56 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 7 Oct 2024 20:18:56 GMT
Subject: Integrated: 8334147: Shenandoah: Avoid taking lock for disabled free
 set logging
In-Reply-To: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
References: <Lx5UOvFYZxB9C0lHZWL5IjdWjPKVXPAW53pVRdVaFGQ=.8e2251f5-2601-424d-94e1-889130044c4d@github.com>
Message-ID: <PdCKX5gmJ1RqptBogLkCcTc4d8_8WqUorSa8LbyhLp4=.4aaf425f-ce70-415b-a7cf-1f69bc8c8df3@github.com>

On Tue, 1 Oct 2024 22:28:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Not clean. Backport to only take lock for logging when log level is enabled.

This pull request has now been integrated.

Changeset: 8562abbe
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/8562abbe27c11fd3a5aa5ea874c318e2555e9242
Stats:     54 lines in 6 files changed: 23 ins; 23 del; 8 mod

8334147: Shenandoah: Avoid taking lock for disabled free set logging
8341554: Shenandoah: Missing heap lock when updating usage for soft ref policy

Reviewed-by: ysr, kdnilsen
Backport-of: c47a0e005e54551e42ee1ae33d7169417a5f86d4

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/113

From wkemper at openjdk.org  Mon Oct  7 20:48:04 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 7 Oct 2024 20:48:04 GMT
Subject: RFR: 8333728: ubsan: shenandoahFreeSet.cpp:1347:24: runtime error:
 division by zero
Message-ID: <TWNCN1NQt0BMpQ_q9-H36UMc28W_zDmicdJwKZm9Gz0=.fbf477d2-6a7b-497c-8ad1-809a4faebded@github.com>

Clean backport.

-------------

Commit messages:
 - Backport b5575942027281166676678e2081b024ec572644

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/116/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=116&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8333728
  Stats: 3 lines in 1 file changed: 0 ins; 2 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/116.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/116/head:pull/116

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/116

From ysr at openjdk.org  Mon Oct  7 22:00:57 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 7 Oct 2024 22:00:57 GMT
Subject: RFR: 8341668: Shenandoah: assert(tail_bits < (idx_t)BitsPerWord)
 failed: precondition
In-Reply-To: <8pCGlrqneW3yypyIwVkzoET45NOw8jKNJQ1x8-nm2PM=.a95b4c84-0fa7-4f44-9ddb-4af73e16a16b@github.com>
References: <8pCGlrqneW3yypyIwVkzoET45NOw8jKNJQ1x8-nm2PM=.a95b4c84-0fa7-4f44-9ddb-4af73e16a16b@github.com>
Message-ID: <06i89f-JrIQDdhGoIYetADvIq7k3uoynfBTzKfDJ5kc=.102a685c-3302-4e71-84c5-315f68d1e773@github.com>

On Mon, 7 Oct 2024 18:44:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

> `tail_mask` function should not be in the global scope.

Change looks good to me. Reviewed!

In either this PR or, preferably, in the ticket, please include the test and the stack retrace associated with this assertion. Thanks!

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21395#pullrequestreview-2353007726

From sviswanathan at openjdk.org  Mon Oct  7 22:43:15 2024
From: sviswanathan at openjdk.org (Sandhya Viswanathan)
Date: Mon, 7 Oct 2024 22:43:15 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
Message-ID: <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>

On Fri, 4 Oct 2024 10:41:46 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> @rkennke The small loop looks to me that it will run over the end of the array.
>> Say the haystack_len is 7, the index below would be 0 after the shrq instruction, and the  movq(XMM_TMP1, Address(haystack, index, Address::times_8)) in the loop will read 8 bytes i.e. one byte past the end of the array:
>>        // num_words (zero-based) = (haystack_len - 1) / 8;
>>       __ movq(index, haystack_len);
>>       __ subq(index, 1);
>>       __ shrq(index, LogBytesPerWord);
>> 
>>       __ bind(L_loop);
>>       __ movq(XMM_TMP1, Address(haystack, index, Address::times_8));
>>       __ movq(Address(rsp, index, Address::times_8), XMM_TMP1);
>>       __ subq(index, 1);
>>       __ jcc(Assembler::positive, L_loop);
>
> Yes, and that is intentional.
> 
> Say, haystack_len is 7, then the first block computes the adjustment of the haystack, which is 8 - (7 % 8) = 1. We adjust the haystack pointer one byte down, so that when we copy (multiple of) 8 bytes, we land on the last byte. We do copy a few bytes that are preceding the array, which is part of the object header and guaranteed to be >= 8 bytes.
> 
> Then we compute the number of words to copy, but make it 0-based. That is '0' is 1 word, '1' is 2 words, etc. It makes the loop nicer. In this example we get 0, which means we copy one word from the adjusted haystack, which is correct.
> 
> Then comes the actual loop.
> 
> Afterwards we adjust the haystack pointer so that it points to the first array element that we just copied onto the stack, ignoring the few garbage bytes that we also copied.

@rkennke Thanks for the explanation. I attach here a fix which is an extension of existing way of copying while taking care of the smaller object header. Also there are two instances of this in the intrinsic so I have factored the new code into a method and call it from both the places. 
[indexof_fix.patch](https://github.com/user-attachments/files/17285239/indexof_fix.patch)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1790967275

From wkemper at openjdk.org  Tue Oct  8 01:29:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 01:29:03 GMT
Subject: Integrated: 8341668: Shenandoah: assert(tail_bits <
 (idx_t)BitsPerWord) failed: precondition
In-Reply-To: <8pCGlrqneW3yypyIwVkzoET45NOw8jKNJQ1x8-nm2PM=.a95b4c84-0fa7-4f44-9ddb-4af73e16a16b@github.com>
References: <8pCGlrqneW3yypyIwVkzoET45NOw8jKNJQ1x8-nm2PM=.a95b4c84-0fa7-4f44-9ddb-4af73e16a16b@github.com>
Message-ID: <km83LvZvt2R2ZYyWCp8Jg0cghVwXYQYoKhccDXfU7-4=.5a2c71b5-bf62-489e-aa71-cb825a5cf7ca@github.com>

On Mon, 7 Oct 2024 18:44:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

> `tail_mask` function should not be in the global scope. Stack traces in incremental build shows `shenandoahSimpleBitMap` calling a function with the same name defined in `bitMap.cpp`:
> 
> 
>     #8 0x7f5e08a2c4d4 in tail_mask(unsigned long) src/hotspot/share/utilities/bitMap.cpp:431
>     #9 0x7f5e0a951144 in ShenandoahSimpleBitMap::count_trailing_ones(long) const src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp:75
>     #10 0x7f5e0a951b42 in ShenandoahSimpleBitMap::find_first_consecutive_set_bits(long, long, unsigned long) const src/hotspot/share/gc/shenandoah/shenandoahSimpleBitMap.cpp:215

This pull request has now been integrated.

Changeset: 45a63595
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/45a6359588996d25e5e8dadebdcd8d6a00ef786f
Stats:     3 lines in 2 files changed: 2 ins; 0 del; 1 mod

8341668: Shenandoah: assert(tail_bits < (idx_t)BitsPerWord) failed: precondition

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/jdk/pull/21395

From wkemper at openjdk.org  Tue Oct  8 01:29:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 01:29:38 GMT
Subject: RFR: 8341485: GenShen: Make evac tracker a non-product feature and
 confine it to generational mode
Message-ID: <9Lf8gNm7Bws9ixP2If0xW77adGPeRJL2ffHxqxdc_eQ=.6e78fff5-97b0-4f99-8ee7-2eded5ca201c@github.com>

Not clean.

-------------

Commit messages:
 - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/117/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=117&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341485
  Stats: 95 lines in 10 files changed: 34 ins; 32 del; 29 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/117.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/117/head:pull/117

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/117

From duke at openjdk.org  Tue Oct  8 07:05:37 2024
From: duke at openjdk.org (duke)
Date: Tue, 8 Oct 2024 07:05:37 GMT
Subject: Withdrawn: 8325673: GenShen: Share Reserves between Old and Young
 Collector
In-Reply-To: <WBf_ck4u6wLSZaiQwRzt4yCI0K5CKBBuhlVD6f0KkVk=.d748a279-de3f-492b-b13d-faacd1dad8b5@github.com>
References: <WBf_ck4u6wLSZaiQwRzt4yCI0K5CKBBuhlVD6f0KkVk=.d748a279-de3f-492b-b13d-faacd1dad8b5@github.com>
Message-ID: <fM6XZNKXP4N-EHEIVfj0sjKcbGSiYXN5Bci8xa1zDpQ=.78e933d1-f7bc-4193-95fb-0a8853574d45@github.com>

On Mon, 12 Feb 2024 18:22:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Allow young-gen Collector reserve to share memory with old-gen Collector reserve in order to support prompt processing of mixed evacuations, as constrained by ShenandoahOldEvacRatioPercent.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/395

From rkennke at openjdk.org  Tue Oct  8 07:08:52 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 07:08:52 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v35]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <0ahBmcWCHqMMxr9RxUlOPgEJy4WSs7lQgRnxtEonZaY=.28275c5d-2d5f-490c-bf39-b0bb6817d6be@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:

 - Rename nklass in x86 code
 - Fix header < 16 bytes in indexOf intrinsic, by @sviswa7

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/17f8eb54..4d7228e0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=34
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=33-34

  Stats: 148 lines in 6 files changed: 84 ins; 47 del; 17 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Tue Oct  8 07:21:09 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 07:21:09 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
Message-ID: <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>

On Mon, 7 Oct 2024 22:40:37 GMT, Sandhya Viswanathan <sviswanathan at openjdk.org> wrote:

>> Yes, and that is intentional.
>> 
>> Say, haystack_len is 7, then the first block computes the adjustment of the haystack, which is 8 - (7 % 8) = 1. We adjust the haystack pointer one byte down, so that when we copy (multiple of) 8 bytes, we land on the last byte. We do copy a few bytes that are preceding the array, which is part of the object header and guaranteed to be >= 8 bytes.
>> 
>> Then we compute the number of words to copy, but make it 0-based. That is '0' is 1 word, '1' is 2 words, etc. It makes the loop nicer. In this example we get 0, which means we copy one word from the adjusted haystack, which is correct.
>> 
>> Then comes the actual loop.
>> 
>> Afterwards we adjust the haystack pointer so that it points to the first array element that we just copied onto the stack, ignoring the few garbage bytes that we also copied.
>
> @rkennke Thanks for the explanation. I attach here a fix which is an extension of existing way of copying while taking care of the smaller object header. Also there are two instances of this in the intrinsic so I have factored the new code into a method and call it from both the places. 
> [indexof_fix.patch](https://github.com/user-attachments/files/17285239/indexof_fix.patch)

Thank you, @sviswa7! Yes this fix looks correct. I've intergrated it into this PR and re-enabled the indexOf intrinsic for compact headers.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1791328427

From rkennke at openjdk.org  Tue Oct  8 07:21:09 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 07:21:09 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v36]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <mPNm6oxqHzUbpQMEvipfe8R-BgbFjrLQm3eXsSHxPxc=.32a33c8f-21bf-4226-8282-352234755d8e@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with three additional commits since the last revision:

 - Re-enable indexOf intrinsic for compact headers
 - Rename nklass in aarch64
 - Fix comment

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/4d7228e0..0be2fc40

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=35
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=34-35

  Stats: 15 lines in 8 files changed: 0 ins; 1 del; 14 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From ayang at openjdk.org  Tue Oct  8 07:43:18 2024
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Tue, 8 Oct 2024 07:43:18 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <z1P1My8MgsUMZ6wFP1Ia0aMFzTkD8oxrGwDI4o4mHrY=.d4c22978-7190-432d-9206-dfbbb1a66a9b@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <-UEFgAIQjGBginN0JRoyuwwJLmDssUEQGE_tCP-tRkw=.01ef3f37-01fa-4931-b4f3-571d21252bbd@github.com>
 <jj6GEQDn7YAPHbFlHtE6Hqx4BwCAUh2XEFLTJ1gBmeY=.faa0d50c-9e26-4dec-aeb9-15d3ca11b1b2@github.com>
 <twhuRW8mlXjzlobkiklKNV0IUCWxzM0tu48uX-4rVeQ=.d98ae5da-502c-4f20-8bba-b9ff963dc53b@github.com>
 <z1P1My8MgsUMZ6wFP1Ia0aMFzTkD8oxrGwDI4o4mHrY=.d4c22978-7190-432d-9206-dfbbb1a66a9b@github.com>
Message-ID: <PnedT3gP8uCpKbCKzXB5s6-KCwWVz1Yx7dHk7r42Tgk=.d4e87b94-058c-49f3-be69-9c4271d04117@github.com>

On Mon, 7 Oct 2024 11:49:21 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> When MinObjectAlignment=16, then that method does nothing anyway:
>> 
>> 
>>   if (MinObjAlignment > 1) {
>>     return;
>>   }
>> 
>> 
>> 
>> I think what it really means to say is
>> 
>>   if (MinObjAlignment >=  CollectedHeap::min_fill_size()) {
>>     return;
>>   }
>> 
>> 
>> 
>> That's also what the comment says: "The size of the gap (if any) right before dense-prefix-end is MinObjAlignment. Need to fill in the gap only if it's smaller than min-obj-size, and the filler obj will extend to next region."
>> 
>> If I interpret that correctly, we need to deal with the situation only when MinObjAlignment < min_fill_size, because the filler object would extend to the next region, and we need to adjust the next region and mark-bitmap for that extra word. @albertnetymk might want to confirm.
>> 
>> I'll move the if (UCOH) block down a little bit to right before if (MinObjAlignment) block.
>
> After re-reading this again I agree with what you're writing. If you make the change to use:
> 
> if (MinObjAlignment >= CollectedHeap::min_fill_size()) {
>  return;
>  }
> 
> 
> do you even have to check for UCOH in this function?
> 
> I also wonder if you could tweak the comment now that this is not true when UCOH is on:
> 
> // The size of the filler (min-obj-size) is 2 heap words with the default
> // MinObjAlignment, since both markword and klass take 1 heap word.

I took UCOH into account when this code was written -- the current version of PR would fail the following assert. 


  // Note: If min-fill-size decreases to 1, this whole method becomes redundant.
  assert(CollectedHeap::min_fill_size() >= 2, "inv");


The least intrusive way, IMO, is to put `if (UCOH) { return; }` right before `// Note: ...`, kind of like what Roman originally put it. I believe the advantage of this style is that when UCOH before always-true, it's obvious this whole method essentially becomes `return`and can be removed right away.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1791362310

From stefank at openjdk.org  Tue Oct  8 08:15:18 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 8 Oct 2024 08:15:18 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <PnedT3gP8uCpKbCKzXB5s6-KCwWVz1Yx7dHk7r42Tgk=.d4e87b94-058c-49f3-be69-9c4271d04117@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <-UEFgAIQjGBginN0JRoyuwwJLmDssUEQGE_tCP-tRkw=.01ef3f37-01fa-4931-b4f3-571d21252bbd@github.com>
 <jj6GEQDn7YAPHbFlHtE6Hqx4BwCAUh2XEFLTJ1gBmeY=.faa0d50c-9e26-4dec-aeb9-15d3ca11b1b2@github.com>
 <twhuRW8mlXjzlobkiklKNV0IUCWxzM0tu48uX-4rVeQ=.d98ae5da-502c-4f20-8bba-b9ff963dc53b@github.com>
 <z1P1My8MgsUMZ6wFP1Ia0aMFzTkD8oxrGwDI4o4mHrY=.d4c22978-7190-432d-9206-dfbbb1a66a9b@github.com>
 <PnedT3gP8uCpKbCKzXB5s6-KCwWVz1Yx7dHk7r42Tgk=.d4e87b94-058c-49f3-be69-9c4271d04117@github.com>
Message-ID: <UNtxTRNZWBU2tOkvOOcPnwYUWSPQcniD-O719yjVP58=.8b8b3daf-c71d-4759-9e15-842f21dd7c6d@github.com>

On Tue, 8 Oct 2024 07:40:24 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> After re-reading this again I agree with what you're writing. If you make the change to use:
>> 
>> if (MinObjAlignment >= CollectedHeap::min_fill_size()) {
>>  return;
>>  }
>> 
>> 
>> do you even have to check for UCOH in this function?
>> 
>> I also wonder if you could tweak the comment now that this is not true when UCOH is on:
>> 
>> // The size of the filler (min-obj-size) is 2 heap words with the default
>> // MinObjAlignment, since both markword and klass take 1 heap word.
>
> I took UCOH into account when this code was written -- the current version of PR would fail the following assert. 
> 
> 
>   // Note: If min-fill-size decreases to 1, this whole method becomes redundant.
>   assert(CollectedHeap::min_fill_size() >= 2, "inv");
> 
> 
> The least intrusive way, IMO, is to put `if (UCOH) { return; }` right before `// Note: ...`, kind of like what Roman originally put it. I believe the advantage of this style is that when UCOH before always-true, it's obvious this whole method essentially becomes `return`and can be removed right away.

I was thinking that we should remove the entire:

  // Note: If min-fill-size decreases to 1, this whole method becomes redundant.
  assert(CollectedHeap::min_fill_size() >= 2, "inv");

block, since it is now incorrect, guarded by the proper check, and the comment is misleading since we now can have a min-fill-size that is 1.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1791409345

From rkennke at openjdk.org  Tue Oct  8 10:03:40 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 10:03:40 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v37]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <KrjrTqHnaXCIkD8T2a_EfnjTd0nVgxBD45NN6PeqXQY=.a4249a8f-ef4e-4008-9418-565ecf007c22@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Improve PSParallelCompact::fill_dense_prefix_end() even more

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/0be2fc40..d57dbfc5

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=36
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=35-36

  Stats: 9 lines in 1 file changed: 2 ins; 7 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Tue Oct  8 10:19:19 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 10:19:19 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v8]
In-Reply-To: <6usTXIvS83aO2VzX5xu2EnXlpIJ8YbfrWS6b3EI0MhE=.0e8cc603-0cd3-4bd9-b309-55e4dd0f0cb0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <J0g0PSye768dhN3JoXFsZ6FivocNBLPmHNtRUCOmLTw=.5756c64a-4763-4727-9637-ba3fe7b9b17d@github.com>
 <6usTXIvS83aO2VzX5xu2EnXlpIJ8YbfrWS6b3EI0MhE=.0e8cc603-0cd3-4bd9-b309-55e4dd0f0cb0@github.com>
Message-ID: <hguzL3zLq9Svv4sZA-HwiYsvkSDHf5dPNdOmte1HU8Q=.b0caace6-3855-48eb-b93a-e0c81d257e8a@github.com>

On Mon, 9 Sep 2024 11:53:13 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Try to avoid lea in loadNklass (aarch64)
>>  - Fix release build error
>
> src/hotspot/share/oops/klass.hpp line 169:
> 
>> 167:                                 // contention that may happen when a nearby object is modified.
>> 168:   AccessFlags _access_flags;    // Access flags. The class/interface distinction is stored here.
>> 169:                                 // Some flags created by the JVM, not in the class file itself,
> 
> Suggestion:
> 
>   markWord _prototype_header;   // Used to initialize objects' header with compact headers.
> 
> 
> Maybe some comment why this is an instance member.

@tschatzl I just found your comment here, and I'm not sure what you mean, tbh. The prototype_header is a member of Klass because with compact headers, it encodes that Klass in the prototype header. Note that there is planned follow-up work to remove that field and encode the Klass* on the allocation path. https://bugs.openjdk.org/browse/JDK-8341703
Let me know if you still want me to change anything there, or if I can 'resolve' this request.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1791602989

From rkennke at openjdk.org  Tue Oct  8 10:19:20 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 10:19:20 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v8]
In-Reply-To: <NM8izXWScsGxiKeCqUapYVOoPaUfzdHwJq1GvAH58cs=.28e5b32c-59e0-4b24-8244-9580296d2e84@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <J0g0PSye768dhN3JoXFsZ6FivocNBLPmHNtRUCOmLTw=.5756c64a-4763-4727-9637-ba3fe7b9b17d@github.com>
 <6usTXIvS83aO2VzX5xu2EnXlpIJ8YbfrWS6b3EI0MhE=.0e8cc603-0cd3-4bd9-b309-55e4dd0f0cb0@github.com>
 <rhloCBhjvBZ4Cs8LLpD_UyHIBR08b8NIjXwIEQlirrs=.8a06186e-a315-4977-961c-b172e077bd9a@github.com>
 <W-1ANH5uIdfAoNRB5Mp1wbdS_aswZz8JhdHNNzz6pMw=.b3e0db43-1df0-4b0f-a6fe-ca77aa23d8f1@github.com>
 <k5oJ0MxPfzSI0V4d8SGr8ApvuM9Xpuy25WiJsJ51Tl8=.62dd2a4c-497d-42bc-a513-0151786b19c5@github.com>
 <NM8izXWScsGxiKeCqUapYVOoPaUfzdHwJq1GvAH58cs=.28e5b32c-59e0-4b24-8244-9580296d2e84@github.com>
Message-ID: <BeD13wCpmDGgJ5pAAuIn6OyQuXGZ-yiscMW2wv821Gk=.86f10fb1-ba4e-4d36-8959-d06b4c7e4b89@github.com>

On Tue, 10 Sep 2024 09:28:41 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> With compact headers, this value should only be used in C2, and not really as an actual offset. An earlier version of the change had the value in src/hotspot/share/opto/type.hpp instead, and only an assert(!UCOH) in oopDesc::klass_offset_in_bytes(). I think this would be a better solution overall, because it prevents accidental (and wrong) usage of the klass_offset in the runtime. Back then it has been rejected by somebody (don't remember), because it made the C2 diff a little messier, so I kept it like it is now. I would prefer to reinstate it, though.
>
>> (Fwiw, the method is also used during Universe initialization).
> 
> Yes, but only in the -UCOH branch.

We will deal with it in a follow-up. https://bugs.openjdk.org/browse/JDK-8340453

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1791605009

From ayang at openjdk.org  Tue Oct  8 10:23:20 2024
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Tue, 8 Oct 2024 10:23:20 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <UNtxTRNZWBU2tOkvOOcPnwYUWSPQcniD-O719yjVP58=.8b8b3daf-c71d-4759-9e15-842f21dd7c6d@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <-UEFgAIQjGBginN0JRoyuwwJLmDssUEQGE_tCP-tRkw=.01ef3f37-01fa-4931-b4f3-571d21252bbd@github.com>
 <jj6GEQDn7YAPHbFlHtE6Hqx4BwCAUh2XEFLTJ1gBmeY=.faa0d50c-9e26-4dec-aeb9-15d3ca11b1b2@github.com>
 <twhuRW8mlXjzlobkiklKNV0IUCWxzM0tu48uX-4rVeQ=.d98ae5da-502c-4f20-8bba-b9ff963dc53b@github.com>
 <z1P1My8MgsUMZ6wFP1Ia0aMFzTkD8oxrGwDI4o4mHrY=.d4c22978-7190-432d-9206-dfbbb1a66a9b@github.com>
 <PnedT3gP8uCpKbCKzXB5s6-KCwWVz1Yx7dHk7r42Tgk=.d4e87b94-058c-49f3-be69-9c4271d04117@github.com>
 <UNtxTRNZWBU2tOkvOOcPnwYUWSPQcniD-O719yjVP58=.8b8b3daf-c71d-4759-9e15-842f21dd7c6d@github.com>
Message-ID: <PJi9AUPxZGcmcMyXZ01YX8-UkCQWV3CCPlXsR5zoB1w=.23331be1-6069-4241-b628-005309fcfd1b@github.com>

On Tue, 8 Oct 2024 08:12:31 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I took UCOH into account when this code was written -- the current version of PR would fail the following assert. 
>> 
>> 
>>   // Note: If min-fill-size decreases to 1, this whole method becomes redundant.
>>   assert(CollectedHeap::min_fill_size() >= 2, "inv");
>> 
>> 
>> The least intrusive way, IMO, is to put `if (UCOH) { return; }` right before `// Note: ...`, kind of like what Roman originally put it. I believe the advantage of this style is that when UCOH before always-true, it's obvious this whole method essentially becomes `return`and can be removed right away.
>
> I was thinking that we should remove the entire:
> 
>   // Note: If min-fill-size decreases to 1, this whole method becomes redundant.
>   assert(CollectedHeap::min_fill_size() >= 2, "inv");
> 
> block, since it is now incorrect, guarded by the proper check, and the comment is misleading since we now can have a min-fill-size that is 1.

It's still correct when UCOH is disabled -- therefore, the UCOH check can be placed at the start without changing any existing logic. (The "rest" of this method assumes min-fill-size is 2, `assert(CollectedHeap::min_fill_size() == 2, "inv")`.)

In this PR, since this method doesn't access UCOH, it can be easily forgotten to update this method when the UCOH flag is removed eventually -- it's not obvious to me that `MinObjAlignment >= checked_cast<int>(CollectedHeap::min_fill_size())` is related to (or can be affected by) `UCOH` at first glance.

(I slightly prefer having a `if (UCOH)` inside this method, but considering this method will be nuked in the long run, any short-time decision is fine by me, assuming the failing assert is fixed.)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1791611304

From rkennke at openjdk.org  Tue Oct  8 10:30:22 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 10:30:22 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <PJi9AUPxZGcmcMyXZ01YX8-UkCQWV3CCPlXsR5zoB1w=.23331be1-6069-4241-b628-005309fcfd1b@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <-UEFgAIQjGBginN0JRoyuwwJLmDssUEQGE_tCP-tRkw=.01ef3f37-01fa-4931-b4f3-571d21252bbd@github.com>
 <jj6GEQDn7YAPHbFlHtE6Hqx4BwCAUh2XEFLTJ1gBmeY=.faa0d50c-9e26-4dec-aeb9-15d3ca11b1b2@github.com>
 <twhuRW8mlXjzlobkiklKNV0IUCWxzM0tu48uX-4rVeQ=.d98ae5da-502c-4f20-8bba-b9ff963dc53b@github.com>
 <z1P1My8MgsUMZ6wFP1Ia0aMFzTkD8oxrGwDI4o4mHrY=.d4c22978-7190-432d-9206-dfbbb1a66a9b@github.com>
 <PnedT3gP8uCpKbCKzXB5s6-KCwWVz1Yx7dHk7r42Tgk=.d4e87b94-058c-49f3-be69-9c4271d04117@github.com>
 <UNtxTRNZWBU2tOkvOOcPnwYUWSPQcniD-O719yjVP58=.8b8b3daf-c71d-4759-9e15-842f21dd7c6d@github.com>
 <PJi9AUPxZGcmcMyXZ01YX8-UkCQWV3CCPlXsR5zoB1w=.23331be1-6069-4241-b628-005309fcfd1b@github.com>
Message-ID: <v9D7WHqlPwZ5D_94YhUU18rk_KY3M3rwAAZ-FK60src=.4404ecb2-6cd1-47a5-b5fb-c79b501aa766@github.com>

On Tue, 8 Oct 2024 10:20:45 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> I was thinking that we should remove the entire:
>> 
>>   // Note: If min-fill-size decreases to 1, this whole method becomes redundant.
>>   assert(CollectedHeap::min_fill_size() >= 2, "inv");
>> 
>> block, since it is now incorrect, guarded by the proper check, and the comment is misleading since we now can have a min-fill-size that is 1.
>
> It's still correct when UCOH is disabled -- therefore, the UCOH check can be placed at the start without changing any existing logic. (The "rest" of this method assumes min-fill-size is 2, `assert(CollectedHeap::min_fill_size() == 2, "inv")`.)
> 
> In this PR, since this method doesn't access UCOH, it can be easily forgotten to update this method when the UCOH flag is removed eventually -- it's not obvious to me that `MinObjAlignment >= checked_cast<int>(CollectedHeap::min_fill_size())` is related to (or can be affected by) `UCOH` at first glance.
> 
> (I slightly prefer having a `if (UCOH)` inside this method, but considering this method will be nuked in the long run, any short-time decision is fine by me, assuming the failing assert is fixed.)

I added an assert(!UCOH) in the implementation so that we don't forget it once the UCOH flag becomes obsolete.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1791619505

From rkennke at openjdk.org  Tue Oct  8 11:47:55 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 11:47:55 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v38]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <XD3mmjy0GRP2dcKyaoxofOIWmOCNn4EqDzmdPBDKgmw=.0392fff7-367d-4553-8f56-62a0885cbfe2@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Fix include guards

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/d57dbfc5..4035bb61

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=37
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=36-37

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From stefank at openjdk.org  Tue Oct  8 12:51:19 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 8 Oct 2024 12:51:19 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v38]
In-Reply-To: <XD3mmjy0GRP2dcKyaoxofOIWmOCNn4EqDzmdPBDKgmw=.0392fff7-367d-4553-8f56-62a0885cbfe2@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <XD3mmjy0GRP2dcKyaoxofOIWmOCNn4EqDzmdPBDKgmw=.0392fff7-367d-4553-8f56-62a0885cbfe2@github.com>
Message-ID: <RI9Sw9eEIlbW69JPFcs-TCh8tYLnESkCksxLrSbNZzQ=.b182f3c2-7073-4599-a129-b7198408085f@github.com>

On Tue, 8 Oct 2024 11:47:55 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix include guards

I have looked through and reviewed most of the files many times and I've tried to give comments in parts of the code where I'm typically not a maintainer.

I'm giving an official Review/Approval of the gc/ code and most of the files in oops/. I'm specifically not approving the compressedKlass* files, because I think that others that are more vested in those bits,  address ranges, and style need to fully Review those changes.

My review comes with a caveat that the there's no significant regressions in GC pauses, marking times, and GC cycle durations, when UCOH is turned off.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2354390888

From rcastanedalo at openjdk.org  Tue Oct  8 15:47:16 2024
From: rcastanedalo at openjdk.org (Roberto =?UTF-8?B?Q2FzdGHDsWVkYQ==?= Lozano)
Date: Tue, 8 Oct 2024 15:47:16 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
 <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
Message-ID: <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>

On Tue, 8 Oct 2024 07:16:13 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> @rkennke Thanks for the explanation. I attach here a fix which is an extension of existing way of copying while taking care of the smaller object header. Also there are two instances of this in the intrinsic so I have factored the new code into a method and call it from both the places. 
>> [indexof_fix.patch](https://github.com/user-attachments/files/17285239/indexof_fix.patch)
>
> Thank you, @sviswa7! Yes this fix looks correct. I've intergrated it into this PR and re-enabled the indexOf intrinsic for compact headers.

@rkennke @sviswa7 These changes trigger the following assertion failure:


#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (codeBuffer.cpp:1004), pid=96032, tid=29699
#  guarantee(sect->end() <= tend) failed: sanity


when running the following tests with compact object headers disabled (i.e. default JVM settings):

- `java/lang/StringBuffer/ECoreIndexOf.java`
- `java/lang/String/IndexOf.java`

on our test macosx-x64 machines. These machines are equipped with Intel Core i7-8700B processors with the following characteristics:

CPU: total 12 (initial active 12) (6 cores per cpu, 2 threads per core) family 6 model 158 stepping 10 microcode 0xf4, cx8, cmov, fxsr, ht, mmx, 3dnowpref, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, lzcnt, tsc, tscinvbit, avx, avx2, aes, erms, clmul, bmi1, bmi2, rtm, adx, fma, vzeroupper, clflush, clflushopt, rdtscp, f16c


If you need more details to reproduce the issue, please let me know and I will try to help.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1792121119

From rcastanedalo at openjdk.org  Tue Oct  8 16:01:23 2024
From: rcastanedalo at openjdk.org (Roberto =?UTF-8?B?Q2FzdGHDsWVkYQ==?= Lozano)
Date: Tue, 8 Oct 2024 16:01:23 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
 <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
 <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>
Message-ID: <Z0hfAlQlvNDCIcDK3LTPO4yKYN_dJnVa7CVinbFPgLI=.d1f5d4e7-8ff1-417c-8e8f-c2666a64f569@github.com>

On Tue, 8 Oct 2024 15:44:45 GMT, Roberto Casta?eda Lozano <rcastanedalo at openjdk.org> wrote:

>> Thank you, @sviswa7! Yes this fix looks correct. I've intergrated it into this PR and re-enabled the indexOf intrinsic for compact headers.
>
> @rkennke @sviswa7 These changes trigger the following assertion failure:
> 
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (codeBuffer.cpp:1004), pid=96032, tid=29699
> #  guarantee(sect->end() <= tend) failed: sanity
> 
> 
> when running the following tests with compact object headers disabled (i.e. default JVM settings):
> 
> - `java/lang/StringBuffer/ECoreIndexOf.java`
> - `java/lang/String/IndexOf.java`
> 
> on our test macosx-x64 machines. These machines are equipped with Intel Core i7-8700B processors with the following characteristics:
> 
> CPU: total 12 (initial active 12) (6 cores per cpu, 2 threads per core) family 6 model 158 stepping 10 microcode 0xf4, cx8, cmov, fxsr, ht, mmx, 3dnowpref, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, lzcnt, tsc, tscinvbit, avx, avx2, aes, erms, clmul, bmi1, bmi2, rtm, adx, fma, vzeroupper, clflush, clflushopt, rdtscp, f16c
> 
> 
> If you need more details to reproduce the issue, please let me know and I will try to help.

Turns out I can also reproduce the issue on my linux-x64 machine (Intel Core i7-9850H), simply running:

`make run-test TEST="java/lang/String/IndexOf.java" CONF=linux-x64-debug`

In this case I get:


#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (codeBuffer.hpp:200), pid=51958, tid=51975
#  assert(allocates2(pc)) failed: not in CodeBuffer memory: 0x00007c2778843560 <= 0x00007c27788543b3 <= 0x00007c27788543b0


A few more details of my processor:

family 6 model 158 stepping 13
flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp vnmi md_clear flush_l1d arch_capabilities

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1792141874

From rkennke at openjdk.org  Tue Oct  8 16:30:47 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 16:30:47 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v39]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <q4sTPbMm-cK1O35GhIbRytiakYgyKJYrBhotG_Kwn2o=.02620a61-e96d-4c93-af80-d27a61dd9ee7@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Increase compiler code stubs size for indexOf intrinsic

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/4035bb61..b289ef88

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=38
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=37-38

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Tue Oct  8 16:34:19 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 8 Oct 2024 16:34:19 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <Z0hfAlQlvNDCIcDK3LTPO4yKYN_dJnVa7CVinbFPgLI=.d1f5d4e7-8ff1-417c-8e8f-c2666a64f569@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
 <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
 <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>
 <Z0hfAlQlvNDCIcDK3LTPO4yKYN_dJnVa7CVinbFPgLI=.d1f5d4e7-8ff1-417c-8e8f-c2666a64f569@github.com>
Message-ID: <hIf7108CCy2cYFW9shrUuP_oR7GcrDN5czlDK7Erbt0=.49632b4b-d1f2-40a9-80da-47e89f46b479@github.com>

On Tue, 8 Oct 2024 15:58:33 GMT, Roberto Casta?eda Lozano <rcastanedalo at openjdk.org> wrote:

>> @rkennke @sviswa7 These changes trigger the following assertion failure:
>> 
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  Internal Error (codeBuffer.cpp:1004), pid=96032, tid=29699
>> #  guarantee(sect->end() <= tend) failed: sanity
>> 
>> 
>> when running the following tests with compact object headers disabled (i.e. default JVM settings):
>> 
>> - `java/lang/StringBuffer/ECoreIndexOf.java`
>> - `java/lang/String/IndexOf.java`
>> 
>> on our test macosx-x64 machines. These machines are equipped with Intel Core i7-8700B processors with the following characteristics:
>> 
>> CPU: total 12 (initial active 12) (6 cores per cpu, 2 threads per core) family 6 model 158 stepping 10 microcode 0xf4, cx8, cmov, fxsr, ht, mmx, 3dnowpref, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, lzcnt, tsc, tscinvbit, avx, avx2, aes, erms, clmul, bmi1, bmi2, rtm, adx, fma, vzeroupper, clflush, clflushopt, rdtscp, f16c
>> 
>> 
>> If you need more details to reproduce the issue, please let me know and I will try to help.
>
> Turns out I can also reproduce the issue on my linux-x64 machine (Intel Core i7-9850H), simply running:
> 
> `make run-test TEST="java/lang/String/IndexOf.java" CONF=linux-x64-debug`
> 
> In this case I get:
> 
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (codeBuffer.hpp:200), pid=51958, tid=51975
> #  assert(allocates2(pc)) failed: not in CodeBuffer memory: 0x00007c2778843560 <= 0x00007c27788543b3 <= 0x00007c27788543b0
> 
> 
> A few more details of my processor:
> 
> family 6 model 158 stepping 13
> flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp vnmi md_clear flush_l1d arch_capabilities

Oh! We need to increase the compiler stub size for the indexOf changes. Strange that it blows up like this, I was sure there was a better check for this somewhere. I changed it like this, let me know if you agree that this is the correct fix:
https://github.com/openjdk/jdk/pull/20677/commits/b289ef885816958d9806c76f473b10e34a39e247

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1792186244

From sviswanathan at openjdk.org  Tue Oct  8 16:38:23 2024
From: sviswanathan at openjdk.org (Sandhya Viswanathan)
Date: Tue, 8 Oct 2024 16:38:23 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <hIf7108CCy2cYFW9shrUuP_oR7GcrDN5czlDK7Erbt0=.49632b4b-d1f2-40a9-80da-47e89f46b479@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
 <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
 <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>
 <Z0hfAlQlvNDCIcDK3LTPO4yKYN_dJnVa7CVinbFPgLI=.d1f5d4e7-8ff1-417c-8e8f-c2666a64f569@github.com>
 <hIf7108CCy2cYFW9shrUuP_oR7GcrDN5czlDK7Erbt0=.49632b4b-d1f2-40a9-80da-47e89f46b479@github.com>
Message-ID: <rDH4CrN_DbFWc6fYt6hudqSKv7CG_lcQCt7CB-wsljk=.313b0c44-e0af-47ca-a41e-76d1a18e2cdc@github.com>

On Tue, 8 Oct 2024 16:30:56 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Turns out I can also reproduce the issue on my linux-x64 machine (Intel Core i7-9850H), simply running:
>> 
>> `make run-test TEST="java/lang/String/IndexOf.java" CONF=linux-x64-debug`
>> 
>> In this case I get:
>> 
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  Internal Error (codeBuffer.hpp:200), pid=51958, tid=51975
>> #  assert(allocates2(pc)) failed: not in CodeBuffer memory: 0x00007c2778843560 <= 0x00007c27788543b3 <= 0x00007c27788543b0
>> 
>> 
>> A few more details of my processor:
>> 
>> family 6 model 158 stepping 13
>> flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp vnmi md_clear flush_l1d arch_capabilities
>
> Oh! We need to increase the compiler stub size for the indexOf changes. Strange that it blows up like this, I was sure there was a better check for this somewhere. I changed it like this, let me know if you agree that this is the correct fix:
> https://github.com/openjdk/jdk/pull/20677/commits/b289ef885816958d9806c76f473b10e34a39e247

Yes, the fix looks correct. Thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1792191386

From wkemper at openjdk.org  Tue Oct  8 17:20:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 17:20:31 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:

 - Fix merge error
 - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
 - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
 - Merge branch 'shenandoah/master' into great-genshen-pr-redux
 - Merge
 - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
   
   Reviewed-by: kdnilsen
 - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
   
   Reviewed-by: kdnilsen, ysr
 - Merge
 - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
   
   Reviewed-by: kdnilsen
 - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
   
   Reviewed-by: kdnilsen, shade, ysr
 - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1

-------------

Changes: https://git.openjdk.org/jdk/pull/21273/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=02
  Stats: 22593 lines in 229 files changed: 20952 ins; 810 del; 831 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From wkemper at openjdk.org  Tue Oct  8 17:27:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 17:27:23 GMT
Subject: Integrated: 8333728: ubsan: shenandoahFreeSet.cpp:1347:24: runtime
 error: division by zero
In-Reply-To: <TWNCN1NQt0BMpQ_q9-H36UMc28W_zDmicdJwKZm9Gz0=.fbf477d2-6a7b-497c-8ad1-809a4faebded@github.com>
References: <TWNCN1NQt0BMpQ_q9-H36UMc28W_zDmicdJwKZm9Gz0=.fbf477d2-6a7b-497c-8ad1-809a4faebded@github.com>
Message-ID: <SPV6iwBRAyCc_7WaBMJ6U9S42AZwrcRWw1hPX8X3cNk=.bec14799-3b5d-4326-8679-8e0d3e572323@github.com>

On Mon, 7 Oct 2024 20:43:07 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: c18290f4
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/c18290f44bb4dacec5a2d91f12cb4a2595c0ac63
Stats:     3 lines in 1 file changed: 0 ins; 2 del; 1 mod

8333728: ubsan: shenandoahFreeSet.cpp:1347:24: runtime error: division by zero

Backport-of: b5575942027281166676678e2081b024ec572644

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/116

From wkemper at openjdk.org  Tue Oct  8 21:20:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 21:20:06 GMT
Subject: RFR: 8341099: GenShen: assert(HAS_FWD ==
 _heap->has_forwarded_objects()) failed: Forwarded object status is sane
Message-ID: <bdWMmmzer3ZB8gtfEZEM0FmYLk4dm5HgBm6xZdiIP3I=.1011d517-591f-4a4e-98e0-6589b5bd0810@github.com>

Clean backport

-------------

Commit messages:
 - Backport cbc0fd0a2aae095c7fda3d69a113bf504ffef594

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/118/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=118&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341099
  Stats: 10 lines in 1 file changed: 8 ins; 1 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/118.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/118/head:pull/118

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/118

From wkemper at openjdk.org  Tue Oct  8 21:29:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 21:29:41 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
Message-ID: <ogzIbXlaOycilGGY9_CrsH2CIIjrB57OxucBFxDMC2s=.e6cd345b-1cbb-4aa8-9672-59efef700d50@github.com>

Clean backport.

-------------

Commit messages:
 - Backport f5f0852f51d3dc1001bf3d68b89f4aab31e05e61

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/119/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=119&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341379
  Stats: 39 lines in 1 file changed: 32 ins; 3 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/119.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/119/head:pull/119

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/119

From wkemper at openjdk.org  Tue Oct  8 21:39:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 21:39:26 GMT
Subject: Integrated: 8341099: GenShen: assert(HAS_FWD ==
 _heap->has_forwarded_objects()) failed: Forwarded object status is sane
In-Reply-To: <bdWMmmzer3ZB8gtfEZEM0FmYLk4dm5HgBm6xZdiIP3I=.1011d517-591f-4a4e-98e0-6589b5bd0810@github.com>
References: <bdWMmmzer3ZB8gtfEZEM0FmYLk4dm5HgBm6xZdiIP3I=.1011d517-591f-4a4e-98e0-6589b5bd0810@github.com>
Message-ID: <jhcyJ8bTtvbQxuPa_y-47ppOVGfF_2KoL81qlh9idg4=.650e5fe3-2aad-4af2-b7e7-f6992f3d5bc9@github.com>

On Tue, 8 Oct 2024 21:13:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport

This pull request has now been integrated.

Changeset: 9f0f4d61
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/9f0f4d612f79aee4ea73a89cd9c70c3a7d8741e4
Stats:     10 lines in 1 file changed: 8 ins; 1 del; 1 mod

8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane

Backport-of: cbc0fd0a2aae095c7fda3d69a113bf504ffef594

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/118

From ysr at openjdk.org  Tue Oct  8 21:44:21 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 8 Oct 2024 21:44:21 GMT
Subject: RFR: 8341485: GenShen: Make evac tracker a non-product feature and
 confine it to generational mode
In-Reply-To: <9Lf8gNm7Bws9ixP2If0xW77adGPeRJL2ffHxqxdc_eQ=.6e78fff5-97b0-4f99-8ee7-2eded5ca201c@github.com>
References: <9Lf8gNm7Bws9ixP2If0xW77adGPeRJL2ffHxqxdc_eQ=.6e78fff5-97b0-4f99-8ee7-2eded5ca201c@github.com>
Message-ID: <YgiXwqy9LScjGqE3hvmoIAmfZpkoP3tbks_vtZONUIc=.60be83f7-7f03-4509-a8cd-6480a0786299@github.com>

On Tue, 8 Oct 2024 00:17:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Not clean.

Backport LGTM. Reviewed!

(The windows-x64 test failures look unrelated, but have you seen these before?)

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/117#pullrequestreview-2355595950

From wkemper at openjdk.org  Tue Oct  8 22:20:16 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 22:20:16 GMT
Subject: RFR: 8341485: GenShen: Make evac tracker a non-product feature and
 confine it to generational mode
In-Reply-To: <9Lf8gNm7Bws9ixP2If0xW77adGPeRJL2ffHxqxdc_eQ=.6e78fff5-97b0-4f99-8ee7-2eded5ca201c@github.com>
References: <9Lf8gNm7Bws9ixP2If0xW77adGPeRJL2ffHxqxdc_eQ=.6e78fff5-97b0-4f99-8ee7-2eded5ca201c@github.com>
Message-ID: <9QqbzrTIsLq1hxQEijDL8AvXDXeAE0q0ry6CI8UBQS4=.586af6df-92fa-4769-8d80-9978e66322b2@github.com>

On Tue, 8 Oct 2024 00:17:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Not clean.

I have seen these failures on several branches and PRs now.

-------------

PR Comment: https://git.openjdk.org/shenandoah-jdk21u/pull/117#issuecomment-2400917455

From wkemper at openjdk.org  Tue Oct  8 22:57:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 22:57:12 GMT
Subject: Integrated: 8341485: GenShen: Make evac tracker a non-product feature
 and confine it to generational mode
In-Reply-To: <9Lf8gNm7Bws9ixP2If0xW77adGPeRJL2ffHxqxdc_eQ=.6e78fff5-97b0-4f99-8ee7-2eded5ca201c@github.com>
References: <9Lf8gNm7Bws9ixP2If0xW77adGPeRJL2ffHxqxdc_eQ=.6e78fff5-97b0-4f99-8ee7-2eded5ca201c@github.com>
Message-ID: <w61ZbcGB888lfDPqUf3dxD_vcQlICgnnursTfrK2dO4=.5e0412ab-0114-420f-9a78-8b0f0ed8bd1a@github.com>

On Tue, 8 Oct 2024 00:17:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Not clean.

This pull request has now been integrated.

Changeset: 956a98f7
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/956a98f7363075893fea24b28d9994015a4e0979
Stats:     95 lines in 10 files changed: 34 ins; 32 del; 29 mod

8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode

Reviewed-by: ysr
Backport-of: bd7d34d8e4c89a855a2398965f776803ded557ea

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/117

From wkemper at openjdk.org  Tue Oct  8 23:07:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 23:07:29 GMT
Subject: RFR: 8337213: Shenandoah: Add verification for class mirrors
Message-ID: <-n63Smqz-3DQBz4wznhHQ3jMQ9KmB791Glu5XOuL-io=.3f4b4b9d-1e2f-42a3-938c-10fe093c03e4@github.com>

Clean backport, improves verification.

-------------

Commit messages:
 - Backport 156f0b4332bf076165898417cf6678d2fc32df5c

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/120/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=120&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8337213
  Stats: 34 lines in 2 files changed: 34 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/120.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/120/head:pull/120

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/120

From wkemper at openjdk.org  Tue Oct  8 23:29:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 23:29:45 GMT
Subject: RFR: 8338444: Shenandoah: Remove ShenandoahHumongousThreshold tunable
Message-ID: <mUbMD5_4iw0eEAdErnM0VuxpSk6N3lH23LIkJIfZ0hc=.1217bfdb-57df-42e8-b77e-58b2749e8e78@github.com>

Not clean, but same resolution as `shenandoah/master`.

-------------

Commit messages:
 - 8338444: Shenandoah: Remove ShenandoahHumongousThreshold tunable

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/121/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=121&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8338444
  Stats: 327 lines in 11 files changed: 4 ins; 314 del; 9 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/121.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/121/head:pull/121

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/121

From wkemper at openjdk.org  Tue Oct  8 23:30:52 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 23:30:52 GMT
Subject: RFR: 8341668: Shenandoah: assert(tail_bits < (idx_t)BitsPerWord)
 failed: precondition
Message-ID: <I2lpQI6Eh8IWEVk-IfKDUZacIFB835YmoYjU6Gdan7w=.13db8f26-d69d-41e3-8348-72bdb112c56a@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 45a6359588996d25e5e8dadebdcd8d6a00ef786f

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/122/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=122&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341668
  Stats: 3 lines in 2 files changed: 2 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/122.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/122/head:pull/122

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/122

From wkemper at openjdk.org  Tue Oct  8 23:38:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 8 Oct 2024 23:38:11 GMT
Subject: Integrated: 8341668: Shenandoah: assert(tail_bits <
 (idx_t)BitsPerWord) failed: precondition
In-Reply-To: <I2lpQI6Eh8IWEVk-IfKDUZacIFB835YmoYjU6Gdan7w=.13db8f26-d69d-41e3-8348-72bdb112c56a@github.com>
References: <I2lpQI6Eh8IWEVk-IfKDUZacIFB835YmoYjU6Gdan7w=.13db8f26-d69d-41e3-8348-72bdb112c56a@github.com>
Message-ID: <CY7WkPqlzpkwWDKzwHnAP7OqvM8Z6WTNL2Kji0g7TP0=.dfa8628c-685d-4a70-acf8-af8fcc29a242@github.com>

On Tue, 8 Oct 2024 23:18:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: ae9173c0
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/ae9173c02515603365fa800b86dcb15aa0330f30
Stats:     3 lines in 2 files changed: 2 ins; 0 del; 1 mod

8341668: Shenandoah: assert(tail_bits < (idx_t)BitsPerWord) failed: precondition

Backport-of: 45a6359588996d25e5e8dadebdcd8d6a00ef786f

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/122

From lmao at openjdk.org  Wed Oct  9 03:30:07 2024
From: lmao at openjdk.org (Liang Mao)
Date: Wed, 9 Oct 2024 03:30:07 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
Message-ID: <GWpGtSgoRD5u1L7TwA7URqGqlBPMsdQH1_tYFR6uzCY=.309147f4-194a-47eb-882f-c7bfcfcbaf00@github.com>

On Tue, 8 Oct 2024 17:20:31 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
> 
>  - Fix merge error
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>    
>    Reviewed-by: kdnilsen
>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>    
>    Reviewed-by: kdnilsen, ysr
>  - Merge
>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>    
>    Reviewed-by: kdnilsen
>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>    
>    Reviewed-by: kdnilsen, shade, ysr
>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1

Congratulations! Good to see the great progress. I just built this PR for some testing and found something unexpected. I ran the genshen VS shenandoah(default) with jbb2015 on aarch64 N2 with 8 cores and Xmx8g. The critical-jOPS of genshen(5373) is behind shenandoah(6027). Do I miss something on the options?
```java -Xmx8g -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational -Xlog:gc*  -XX:MetaspaceSize=1g   -jar specjbb2015.jar -m COMPOSITE```

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2401194597

From rcastanedalo at openjdk.org  Wed Oct  9 06:30:18 2024
From: rcastanedalo at openjdk.org (Roberto =?UTF-8?B?Q2FzdGHDsWVkYQ==?= Lozano)
Date: Wed, 9 Oct 2024 06:30:18 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <hIf7108CCy2cYFW9shrUuP_oR7GcrDN5czlDK7Erbt0=.49632b4b-d1f2-40a9-80da-47e89f46b479@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
 <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
 <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>
 <Z0hfAlQlvNDCIcDK3LTPO4yKYN_dJnVa7CVinbFPgLI=.d1f5d4e7-8ff1-417c-8e8f-c2666a64f569@github.com>
 <hIf7108CCy2cYFW9shrUuP_oR7GcrDN5czlDK7Erbt0=.49632b4b-d1f2-40a9-80da-47e89f46b479@github.com>
Message-ID: <ouoemlEpzqpgckNviJxccPlr69RhOeCy2KCxMhRX8rg=.203da6e1-fa45-4377-bebe-2c41eb8e344d@github.com>

On Tue, 8 Oct 2024 16:30:56 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Turns out I can also reproduce the issue on my linux-x64 machine (Intel Core i7-9850H), simply running:
>> 
>> `make run-test TEST="java/lang/String/IndexOf.java" CONF=linux-x64-debug`
>> 
>> In this case I get:
>> 
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  Internal Error (codeBuffer.hpp:200), pid=51958, tid=51975
>> #  assert(allocates2(pc)) failed: not in CodeBuffer memory: 0x00007c2778843560 <= 0x00007c27788543b3 <= 0x00007c27788543b0
>> 
>> 
>> A few more details of my processor:
>> 
>> family 6 model 158 stepping 13
>> flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp vnmi md_clear flush_l1d arch_capabilities
>
> Oh! We need to increase the compiler stub size for the indexOf changes. Strange that it blows up like this, I was sure there was a better check for this somewhere. I changed it like this, let me know if you agree that this is the correct fix:
> https://github.com/openjdk/jdk/pull/20677/commits/b289ef885816958d9806c76f473b10e34a39e247

That seems to work, thanks @rkennke!

Since the [indexOf changes](https://github.com/openjdk/jdk/pull/20677/files#diff-ae1139bb5342494f9761e04389b090c543391bfdd7817af1625e854357c96e63) are complex and affect the default JVM configuration, they should be subject to the same level of scrutiny as if they were a standalone RFE, i.e. approved by at least a second reviewer. @sviswa7 could someone else at Intel have a second look and explicitly approve them?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1792911644

From ysr at openjdk.org  Wed Oct  9 16:09:24 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 9 Oct 2024 16:09:24 GMT
Subject: RFR: 8338444: Shenandoah: Remove ShenandoahHumongousThreshold
 tunable
In-Reply-To: <mUbMD5_4iw0eEAdErnM0VuxpSk6N3lH23LIkJIfZ0hc=.1217bfdb-57df-42e8-b77e-58b2749e8e78@github.com>
References: <mUbMD5_4iw0eEAdErnM0VuxpSk6N3lH23LIkJIfZ0hc=.1217bfdb-57df-42e8-b77e-58b2749e8e78@github.com>
Message-ID: <5_P23foFM666Wai5NSDge-156zGeSayTcWjf3RU9-II=.33d1e11f-2043-450e-934c-60f9c9db1c94@github.com>

On Tue, 8 Oct 2024 23:13:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Not clean, but same resolution as `shenandoah/master`.

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/121#pullrequestreview-2357598402

From sviswanathan at openjdk.org  Wed Oct  9 16:25:23 2024
From: sviswanathan at openjdk.org (Sandhya Viswanathan)
Date: Wed, 9 Oct 2024 16:25:23 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <ouoemlEpzqpgckNviJxccPlr69RhOeCy2KCxMhRX8rg=.203da6e1-fa45-4377-bebe-2c41eb8e344d@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
 <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
 <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>
 <Z0hfAlQlvNDCIcDK3LTPO4yKYN_dJnVa7CVinbFPgLI=.d1f5d4e7-8ff1-417c-8e8f-c2666a64f569@github.com>
 <hIf7108CCy2cYFW9shrUuP_oR7GcrDN5czlDK7Erbt0=.49632b4b-d1f2-40a9-80da-47e89f46b479@github.com>
 <ouoemlEpzqpgckNviJxccPlr69RhOeCy2KCxMhRX8rg=.203da6e1-fa45-4377-bebe-2c41eb8e344d@
 github.com>
Message-ID: <Xl2cSg6Ql6ttcW4mtRAi7uq-lKjPX37xNqMF1zG398w=.90ceecb4-e531-417c-ae96-b03dba0fe313@github.com>

On Wed, 9 Oct 2024 06:25:28 GMT, Roberto Casta?eda Lozano <rcastanedalo at openjdk.org> wrote:

>> Oh! We need to increase the compiler stub size for the indexOf changes. Strange that it blows up like this, I was sure there was a better check for this somewhere. I changed it like this, let me know if you agree that this is the correct fix:
>> https://github.com/openjdk/jdk/pull/20677/commits/b289ef885816958d9806c76f473b10e34a39e247
>
> That seems to work, thanks @rkennke!
> 
> Since the [indexOf changes](https://github.com/openjdk/jdk/pull/20677/files#diff-ae1139bb5342494f9761e04389b090c543391bfdd7817af1625e854357c96e63) are complex and affect the default JVM configuration, they should be subject to the same level of scrutiny as if they were a standalone RFE, i.e. approved by at least a second reviewer. @sviswa7 could someone else at Intel have a second look and explicitly approve them?

Yes, @vpaprotsk could review the changes that we made in src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1793814453

From rcastanedalo at openjdk.org  Wed Oct  9 17:44:24 2024
From: rcastanedalo at openjdk.org (Roberto =?UTF-8?B?Q2FzdGHDsWVkYQ==?= Lozano)
Date: Wed, 9 Oct 2024 17:44:24 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <Xl2cSg6Ql6ttcW4mtRAi7uq-lKjPX37xNqMF1zG398w=.90ceecb4-e531-417c-ae96-b03dba0fe313@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
 <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
 <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>
 <Z0hfAlQlvNDCIcDK3LTPO4yKYN_dJnVa7CVinbFPgLI=.d1f5d4e7-8ff1-417c-8e8f-c2666a64f569@github.com>
 <hIf7108CCy2cYFW9shrUuP_oR7GcrDN5czlDK7Erbt0=.49632b4b-d1f2-40a9-80da-47e89f46b479@github.com>
 <ouoemlEpzqpgckNviJxccPlr69RhOeCy2KCxMhRX8rg=.203da6e1-fa45-4377-bebe-2c41eb8e344d@
 github.com>
 <Xl2cSg6Ql6ttcW4mtRAi7uq-lKjPX37xNqMF1zG398w=.90ceecb4-e531-417c-ae96-b03dba0fe313@github.com>
Message-ID: <ffUwCDUB5JmTDFXVZsEKkLiNfkF2D3YTSRp9qX13orc=.f4e454c3-9bf6-46ad-9491-89a50a8e7810@github.com>

On Wed, 9 Oct 2024 16:21:53 GMT, Sandhya Viswanathan <sviswanathan at openjdk.org> wrote:

>> That seems to work, thanks @rkennke!
>> 
>> Since the [indexOf changes](https://github.com/openjdk/jdk/pull/20677/files#diff-ae1139bb5342494f9761e04389b090c543391bfdd7817af1625e854357c96e63) are complex and affect the default JVM configuration, they should be subject to the same level of scrutiny as if they were a standalone RFE, i.e. approved by at least a second reviewer. @sviswa7 could someone else at Intel have a second look and explicitly approve them?
>
> Yes, @vpaprotsk could review the changes that we made in src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp.

Yes, that would be great. In the meantime, I ran a few thousand times the randomized test `java/lang/StringBuffer/ECoreIndexOf.java` with and without compact object headers, on product and debug builds, on different x64 implementations, and found no failures.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1793917912

From kdnilsen at openjdk.org  Wed Oct  9 19:12:53 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 9 Oct 2024 19:12:53 GMT
Subject: RFR: 8341379: Shenandoah: Improve lock contention during cleanup
In-Reply-To: <ogzIbXlaOycilGGY9_CrsH2CIIjrB57OxucBFxDMC2s=.e6cd345b-1cbb-4aa8-9672-59efef700d50@github.com>
References: <ogzIbXlaOycilGGY9_CrsH2CIIjrB57OxucBFxDMC2s=.e6cd345b-1cbb-4aa8-9672-59efef700d50@github.com>
Message-ID: <1lajtLPBgBUzRI4R5K_jS1CYmsAUhjLKPCA0tr9lRAE=.0b7d5048-768c-47b9-89fe-ec7dde27fedd@github.com>

On Tue, 8 Oct 2024 21:24:08 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/119#pullrequestreview-2357903557

From wkemper at openjdk.org  Wed Oct  9 19:48:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 19:48:37 GMT
Subject: git: openjdk/shenandoah-jdk21u: master: 8338444: Shenandoah: Remove
 ShenandoahHumongousThreshold tunable
Message-ID: <7a3bcf04-a68c-42e5-9a76-93e1181aea4d@openjdk.org>

Changeset: 63b4bc1a
Branch: master
Author:    William Kemper <wkemper at openjdk.org>
Date:      2024-10-09 19:47:34 +0000
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/63b4bc1a074763e793d6eae96cd609f90da6961b

8338444: Shenandoah: Remove ShenandoahHumongousThreshold tunable

Reviewed-by: ysr
Backport-of: ef54af39883e76c80a3e012ed91b90973da51bb4

! src/hotspot/share/gc/shenandoah/shenandoahAsserts.cpp
! src/hotspot/share/gc/shenandoah/shenandoahController.cpp
! src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp
! src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp
! src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp
! src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp
! src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.hpp
! src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp
! src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp
- test/hotspot/jtreg/gc/shenandoah/TestHumongousThreshold.java
- test/hotspot/jtreg/gc/shenandoah/options/TestHumongousThresholdArgs.java


From wkemper at openjdk.org  Wed Oct  9 19:51:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 19:51:41 GMT
Subject: Integrated: 8338444: Shenandoah: Remove ShenandoahHumongousThreshold
 tunable
In-Reply-To: <mUbMD5_4iw0eEAdErnM0VuxpSk6N3lH23LIkJIfZ0hc=.1217bfdb-57df-42e8-b77e-58b2749e8e78@github.com>
References: <mUbMD5_4iw0eEAdErnM0VuxpSk6N3lH23LIkJIfZ0hc=.1217bfdb-57df-42e8-b77e-58b2749e8e78@github.com>
Message-ID: <IbYrXjMiXeoxLesvEnsHy-5rfXfnoeYjwG1jARzbIww=.45a69f2e-28d1-4365-82eb-02d23cac2b5d@github.com>

On Tue, 8 Oct 2024 23:13:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Not clean, but same resolution as `shenandoah/master`.

This pull request has now been integrated.

Changeset: 63b4bc1a
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/63b4bc1a074763e793d6eae96cd609f90da6961b
Stats:     327 lines in 11 files changed: 4 ins; 314 del; 9 mod

8338444: Shenandoah: Remove ShenandoahHumongousThreshold tunable

Reviewed-by: ysr
Backport-of: ef54af39883e76c80a3e012ed91b90973da51bb4

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/121

From wkemper at openjdk.org  Wed Oct  9 19:53:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 19:53:27 GMT
Subject: Integrated: 8337213: Shenandoah: Add verification for class mirrors
In-Reply-To: <-n63Smqz-3DQBz4wznhHQ3jMQ9KmB791Glu5XOuL-io=.3f4b4b9d-1e2f-42a3-938c-10fe093c03e4@github.com>
References: <-n63Smqz-3DQBz4wznhHQ3jMQ9KmB791Glu5XOuL-io=.3f4b4b9d-1e2f-42a3-938c-10fe093c03e4@github.com>
Message-ID: <b_CfnMkTFbaPwZlGuUFum3qjhcBAJR28gJw8bt9_t-g=.02c55de9-78fb-41d8-846f-1ccba6b5e9ca@github.com>

On Tue, 8 Oct 2024 22:15:31 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport, improves verification.

This pull request has now been integrated.

Changeset: b7d1cbb1
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/b7d1cbb1dc035c3ab16f091d69443228fb70a249
Stats:     34 lines in 2 files changed: 34 ins; 0 del; 0 mod

8337213: Shenandoah: Add verification for class mirrors

Backport-of: 156f0b4332bf076165898417cf6678d2fc32df5c

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/120

From wkemper at openjdk.org  Wed Oct  9 19:54:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 19:54:30 GMT
Subject: Integrated: 8341379: Shenandoah: Improve lock contention during
 cleanup
In-Reply-To: <ogzIbXlaOycilGGY9_CrsH2CIIjrB57OxucBFxDMC2s=.e6cd345b-1cbb-4aa8-9672-59efef700d50@github.com>
References: <ogzIbXlaOycilGGY9_CrsH2CIIjrB57OxucBFxDMC2s=.e6cd345b-1cbb-4aa8-9672-59efef700d50@github.com>
Message-ID: <u6z8MfPqqSZt5VyutaZYPSQqUP4xTA_MRhzLLOzTuuI=.c1bbbaca-c4f6-46b2-8567-7283d9baebf5@github.com>

On Tue, 8 Oct 2024 21:24:08 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: e7ff0377
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/e7ff03773dec24eb7cf7de78d7ac2a64a05b3491
Stats:     39 lines in 1 file changed: 32 ins; 3 del; 4 mod

8341379: Shenandoah: Improve lock contention during cleanup

Reviewed-by: kdnilsen
Backport-of: f5f0852f51d3dc1001bf3d68b89f4aab31e05e61

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/119

From wkemper at openjdk.org  Wed Oct  9 19:55:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 19:55:39 GMT
Subject: RFR: 8340408: Shenandoah: Remove redundant task stats printing code in
 ShenandoahTaskQueue
Message-ID: <8q9QIzfijX2YzH0gzmesC_Ep2fWPcMfinOjHmygu1kQ=.ab0a91cc-859b-4e92-bd54-a327552fecc2@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 279086d4ce7e05972e099022e8045f39680dd4e8

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/123/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=123&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8340408
  Stats: 49 lines in 4 files changed: 0 ins; 47 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/123.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/123/head:pull/123

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/123

From wkemper at openjdk.org  Wed Oct  9 19:59:50 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 19:59:50 GMT
Subject: RFR: 8340183: Shenandoah: Incorrect match for clone barrier in
 is_gc_barrier_node
Message-ID: <-w39p5a3Z1uEANEUYIVrkSdJWcy-ALre_vnhTPqxyH8=.d85d3be0-aed0-4715-950e-d03bcced36e0@github.com>

A few commits to backport together in this PR.

-------------

Commit messages:
 - Retain support for IU mode barrrier nodes
 - 8341242: Shenandoah: LRB node is not matched as GC barrier after JDK-8340183
 - 8340183: Shenandoah: Incorrect match for clone barrier in is_gc_barrier_node

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/124/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=124&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8340183
  Stats: 22 lines in 3 files changed: 7 ins; 9 del; 6 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/124.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/124/head:pull/124

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/124

From sviswanathan at openjdk.org  Wed Oct  9 20:28:29 2024
From: sviswanathan at openjdk.org (Sandhya Viswanathan)
Date: Wed, 9 Oct 2024 20:28:29 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v26]
In-Reply-To: <ffUwCDUB5JmTDFXVZsEKkLiNfkF2D3YTSRp9qX13orc=.f4e454c3-9bf6-46ad-9491-89a50a8e7810@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <fEpfi4CzTFr1Pl-IyOAVyHNXWlVz33lHKQyzVY0kBc8=.cc39355b-ee7a-4118-952a-28e4184eec84@github.com>
 <DKuidQyyiiJOsrerLCeziselVyG9Ztpzsyc64ZNjlI4=.722fc4f7-d7cc-49df-9d82-5c1094508f74@github.com>
 <hdttC4LhHE6y6Nbq0jjJwxxF4W0hTfJLK_siODDCBkM=.1667f495-fcd9-4b7f-8d58-24e27665eeb3@github.com>
 <fWSdSqQfhOrGpFoExiaO4L-UDtR90_t0gRxp9YBO3_c=.0173618e-f392-4921-8fae-39d44b24de08@github.com>
 <6PTWMepIDuZDdPfN3xNKV1vqUyO_R4yCSeiSTpYIyyQ=.61a5b462-7114-4385-a6d7-40e5c7b0005d@github.com>
 <C5LwWrsa-VO87kdm_sYuisTGMsHoxv4Eoe0QikdkusE=.63b0993c-86c8-4ef6-a3cf-bc8a67e10815@github.com>
 <WOlTJMNHCr_Kk69sEIUwCVI3ZbiRlckY4TSBRfL7zoA=.1a4c59f1-a000-4d9f-abbc-36b24ed5105e@github.com>
 <UjwpWz9Y3W8WRzYHZmDk9EvoiLSCZG1aai5oASGtDKA=.68cab6dd-e67d-4651-a9c9-75dccf42d85f@github.com>
 <l_7f8gGpIdG9H3Ini5vHWruIa0oDi7fkn2Lz5thDI5c=.969b804f-bae2-48e1-9c44-e631d90037e7@github.com>
 <BtvwMUG5tcy_zU4tRG3Q5VVw08qE_3gSE35
 wCxlDnj4=.954353e5-cab2-48f4-b0f2-5852e586ba20@github.com>
 <zi8MM6qUWhvxPSyb3ewa3baXWhmDIDlut2uYUQmWvcw=.7b32fa42-7a3f-43bb-8f15-fe6b8e66c49b@github.com>
 <TSxYcKKfAdx7SsVcq6iwiySCLlkoVbXQO7a25vsiG4k=.29e59a6c-b8e0-42e9-8bdd-cc8ca23df146@github.com>
 <YpG7vj6W3LnbhvvRhaeLmLeBzc_vlzlzmD74xVu1FQY=.14a57f09-74c1-4b2f-962c-c923961c067c@github.com>
 <YrOurf-ql2uciTYRjGDnvVIb3r9K1pYphx2yRism6SQ=.f5b2510c-d384-4e78-906b-08267a218185@github.com>
 <YqRmzdVypPfgEUFG9TqiQvD846aTd9SE9pU746Ed3HU=.a0349ab7-aabc-471c-9375-ef432a187caa@github.com>
 <6yrLSIp1cwJXxYVoMfSLxhbFA9Qdc9P3ML25QW0sfL4=.aa8bedac-1faa-4148-bcfc-a1434ddc9bac@github.com>
 <lD7f-h93ZqWOHoJtIECJfizoH74j3_iwG2VG_jAYR1o=.db4be907-c20a-4f96-9b9e-e34eefa2e7f3@github.com>
 <Z0hfAlQlvNDCIcDK3LTPO4yKYN_dJnVa7CVinbFPgLI=.d1f5d4e7-8ff1-417c-8e8f-c2666a64f569@github.com>
 <hIf7108CCy2cYFW9shrUuP_oR7GcrDN5czlDK7Erbt0=.49632b4b-d1f2-40a9-80da-47e89f46b479@github.com>
 <ouoemlEpzqpgckNviJxccPlr69RhOeCy2KCxMhRX8rg=.203da6e1-fa45-4377-bebe-2c41eb8e344d@
 github.com>
 <Xl2cSg6Ql6ttcW4mtRAi7uq-lKjPX37xNqMF1zG398w=.90ceecb4-e531-417c-ae96-b03dba0fe313@github.com>
 <ffUwCDUB5JmTDFXVZsEKkLiNfkF2D3YTSRp9qX13orc=.f4e454c3-9bf6-46ad-9491-89a50a8e7810@github.com>
Message-ID: <mcGN2Xuld1ZujJVmIkv3NBCPbspQ7YdjXtnb_0ypftk=.75e66e61-97d0-4ae8-bc98-b6af592a1853@github.com>

On Wed, 9 Oct 2024 17:41:37 GMT, Roberto Casta?eda Lozano <rcastanedalo at openjdk.org> wrote:

>> Yes, @vpaprotsk could review the changes that we made in src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp.
>
> Yes, that would be great. In the meantime, I ran a few thousand times the randomized test `java/lang/StringBuffer/ECoreIndexOf.java` with and without compact object headers, on product and debug builds, on different x64 implementations, and found no failures.

Thanks a lot @robcasloz for doing the testing.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1794181528

From wkemper at openjdk.org  Wed Oct  9 21:04:22 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 21:04:22 GMT
Subject: RFR: 8339651: ShenandoahPacer::setup_for_mark,
 ShenandoahPacer::setup_for_updaterefs and ShenandoahPacer::setup_for_evac
 runtime error: division by zero
Message-ID: <V4_JujuCficbks9qbfsAf4yvkFEXi5clAPa6IR2H01E=.77e12334-3365-4eb7-9d8a-300985822196@github.com>

Avoid divide by zero error when there is no memory available.

-------------

Commit messages:
 - Fix 32bit and windows builds
 - Avoid divide-by-zero error

Changes: https://git.openjdk.org/jdk/pull/21432/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21432&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8339651
  Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/21432.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21432/head:pull/21432

PR: https://git.openjdk.org/jdk/pull/21432

From wkemper at openjdk.org  Wed Oct  9 21:05:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 21:05:34 GMT
Subject: Integrated: 8340408: Shenandoah: Remove redundant task stats printing
 code in ShenandoahTaskQueue
In-Reply-To: <8q9QIzfijX2YzH0gzmesC_Ep2fWPcMfinOjHmygu1kQ=.ab0a91cc-859b-4e92-bd54-a327552fecc2@github.com>
References: <8q9QIzfijX2YzH0gzmesC_Ep2fWPcMfinOjHmygu1kQ=.ab0a91cc-859b-4e92-bd54-a327552fecc2@github.com>
Message-ID: <VLXrveP0UFrI2ATZQg-jcCtoVdhrejflmXvtnCBoNe8=.ef765e4d-73b1-485c-90fb-1dbbde5155c6@github.com>

On Wed, 9 Oct 2024 19:21:20 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: 8b78dbab
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/8b78dbabebdd61a550c4620bb459007f62b9e514
Stats:     49 lines in 4 files changed: 0 ins; 47 del; 2 mod

8340408: Shenandoah: Remove redundant task stats printing code in ShenandoahTaskQueue

Backport-of: 279086d4ce7e05972e099022e8045f39680dd4e8

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/123

From wkemper at openjdk.org  Wed Oct  9 21:11:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 21:11:20 GMT
Subject: Integrated: 8335865: Shenandoah: Improve THP pretouch after
 JDK-8315923
Message-ID: <XeNDZRg7FJULrfKIfgeXUcNLH9fg3MmLBDPZCgZYsSk=.b009b286-6810-4fa4-931b-f2b780a15ccc@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 84c3065e8004122f3455a8c28c8719b2c8111c17

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/125/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=125&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8335865
  Stats: 18 lines in 1 file changed: 0 ins; 17 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/125.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/125/head:pull/125

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/125

From wkemper at openjdk.org  Wed Oct  9 21:11:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 9 Oct 2024 21:11:20 GMT
Subject: Integrated: 8335865: Shenandoah: Improve THP pretouch after
 JDK-8315923
In-Reply-To: <XeNDZRg7FJULrfKIfgeXUcNLH9fg3MmLBDPZCgZYsSk=.b009b286-6810-4fa4-931b-f2b780a15ccc@github.com>
References: <XeNDZRg7FJULrfKIfgeXUcNLH9fg3MmLBDPZCgZYsSk=.b009b286-6810-4fa4-931b-f2b780a15ccc@github.com>
Message-ID: <CecbvVYOq5utkr5tR45mTiTu-jnJqlls3uWq5rdMifA=.1ba40bb3-9785-4717-b37f-bfe198aee6d9@github.com>

On Wed, 9 Oct 2024 21:04:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: 0d9d3e68
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/0d9d3e685cdea8e89d30de19bf0857dff9092c83
Stats:     18 lines in 1 file changed: 0 ins; 17 del; 1 mod

8335865: Shenandoah: Improve THP pretouch after JDK-8315923

Backport-of: 84c3065e8004122f3455a8c28c8719b2c8111c17

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/125

From shade at openjdk.org  Thu Oct 10 05:49:09 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 10 Oct 2024 05:49:09 GMT
Subject: RFR: 8339651: ShenandoahPacer::setup_for_mark,
 ShenandoahPacer::setup_for_updaterefs and ShenandoahPacer::setup_for_evac
 runtime error: division by zero
In-Reply-To: <V4_JujuCficbks9qbfsAf4yvkFEXi5clAPa6IR2H01E=.77e12334-3365-4eb7-9d8a-300985822196@github.com>
References: <V4_JujuCficbks9qbfsAf4yvkFEXi5clAPa6IR2H01E=.77e12334-3365-4eb7-9d8a-300985822196@github.com>
Message-ID: <DwmQclFBfOzLDxhUH1C7HguqevVkf9lT1s4-euixHi4=.cbaffc8c-f03c-42f5-8b95-da10b7edafca@github.com>

On Wed, 9 Oct 2024 20:59:49 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Avoid divide by zero error when there is no memory available.

Looks fine, thanks.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21432#pullrequestreview-2359047911

From mli at openjdk.org  Thu Oct 10 08:03:10 2024
From: mli at openjdk.org (Hamlin Li)
Date: Thu, 10 Oct 2024 08:03:10 GMT
Subject: RFR: 8339651: ShenandoahPacer::setup_for_mark,
 ShenandoahPacer::setup_for_updaterefs and ShenandoahPacer::setup_for_evac
 runtime error: division by zero
In-Reply-To: <V4_JujuCficbks9qbfsAf4yvkFEXi5clAPa6IR2H01E=.77e12334-3365-4eb7-9d8a-300985822196@github.com>
References: <V4_JujuCficbks9qbfsAf4yvkFEXi5clAPa6IR2H01E=.77e12334-3365-4eb7-9d8a-300985822196@github.com>
Message-ID: <Yq0pBhB1bIChO89g9_GAUJVDcEG4dMctS1sgWFRWRx8=.4afea77d-0824-4fc5-83d8-21f9908dbb43@github.com>

On Wed, 9 Oct 2024 20:59:49 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Avoid divide by zero error when there is no memory available.

Marked as reviewed by mli (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21432#pullrequestreview-2359414240

From rcastanedalo at openjdk.org  Thu Oct 10 10:03:33 2024
From: rcastanedalo at openjdk.org (Roberto =?UTF-8?B?Q2FzdGHDsWVkYQ==?= Lozano)
Date: Thu, 10 Oct 2024 10:03:33 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v39]
In-Reply-To: <q4sTPbMm-cK1O35GhIbRytiakYgyKJYrBhotG_Kwn2o=.02620a61-e96d-4c93-af80-d27a61dd9ee7@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <q4sTPbMm-cK1O35GhIbRytiakYgyKJYrBhotG_Kwn2o=.02620a61-e96d-4c93-af80-d27a61dd9ee7@github.com>
Message-ID: <ARBYJMJ_9fg6m5fGG6XyheX64J-YKzfyyBIhcX8Zzqs=.f7a44bfa-ca40-4ee7-b49c-4ba3161b0c0c@github.com>

On Tue, 8 Oct 2024 16:30:47 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Increase compiler code stubs size for indexOf intrinsic

Thanks @rkennke and @tstuefe for patiently addressing my comments. I have reviewed the HotSpot compiler parts of this changeset, except those in `src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp` which should be reviewed by someone more familiar with the `indexOf` intrinsic implementation (@sviswa7 has suggested @vpaprotsk for this task). More specifically, my approval covers the following files/directories:


src/hotspot/cpu/aarch64 (excluding interpreter-only changes)
src/hotspot/cpu/x86 (excluding interpreter-only and c2_stubGenerator_x86_64_string.cpp changes)
src/hotspot/share/opto
src/hotspot/share/ci
src/hotspot/share/gc/{shared,x,z}/c2/{x,z}barrierSetC2.cpp
test/hotspot/jtreg/compiler


As I mentioned earlier, after the integration of this changeset and before compact headers can be considered non-experimental, I think C2's dependency on `klass_offset_in_bytes()` (when using compact headers) should be removed, and a more robust C2 model for klass pointer loading should be developed ([JDK-8340453](https://bugs.openjdk.org/browse/JDK-8340453)).

-------------

Marked as reviewed by rcastanedalo (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2359713290

From rkennke at openjdk.org  Thu Oct 10 13:27:23 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 10 Oct 2024 13:27:23 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
Message-ID: <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>

On Tue, 8 Oct 2024 17:20:31 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
> 
>  - Fix merge error
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>    
>    Reviewed-by: kdnilsen
>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>    
>    Reviewed-by: kdnilsen, ysr
>  - Merge
>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>    
>    Reviewed-by: kdnilsen
>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>    
>    Reviewed-by: kdnilsen, shade, ysr
>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1

That is a big change - great work!
In this first batch I've reviewed:


src/hotspot/cpu
src/hotspot/gc/shared
src/hotspot/gc/shenandoah/c1
src/hotspot/gc/shenandoah/c2
src/hotspot/gc/shenandoah/heuristics
src/hotspot/gc/shenandoah/mode


I only have a few comments so far. Will review the remaining parts in separate batches.

src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp line 75:

> 73: 
> 74:   // Create a mask to test if the marking bit is set.
> 75:   // TODO: can we directly test if bit is set?

That comment here is quite justified, IMO. I'm pretty sure that we could test for the flag in a single instruction, instead of doing the and-cmp sequence and even allocating a new register. Unless of course when C1 LIR can't represent it. Have you tried to implement that and failed, and therefore remove the comment?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 96:

> 94:   size_t capacity    = _space_info->soft_max_capacity();
> 95:   size_t max_cset    = (size_t)((1.0 * capacity / 100 * ShenandoahEvacReserve) / ShenandoahEvacWaste);
> 96:   size_t free_target = (capacity * ShenandoahMinFreeThreshold) / 100 + max_cset;

Does re-arranging the math here risk overflow (e.g. on 32-bit)?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.hpp line 79:

> 77: 
> 78: protected:
> 79:   static const uint Moving_Average_Samples = 10; // Number of samples to store in moving averages

I've never seen that style for constants in HotSpot before. I'd either make it MOVING_AVERAGE_SAMPLES or MovingAverageSamples. I think the latter is more prevalent, but I am not certain.

-------------

Changes requested by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2360208858
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795359413
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795377554
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795401675

From rkennke at openjdk.org  Thu Oct 10 13:27:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 10 Oct 2024 13:27:24 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>
Message-ID: <coMcymMrKDj7JFvuSwRamr6_cSvNWRaXiTSKmauHTeg=.b47f58dd-3e36-4c08-8ba4-4e846502bb73@github.com>

On Thu, 10 Oct 2024 13:13:14 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.hpp line 79:
> 
>> 77: 
>> 78: protected:
>> 79:   static const uint Moving_Average_Samples = 10; // Number of samples to store in moving averages
> 
> I've never seen that style for constants in HotSpot before. I'd either make it MOVING_AVERAGE_SAMPLES or MovingAverageSamples. I think the latter is more prevalent, but I am not certain.

Oh but I see that it's used in this source file. Hmm... So maybe stick with it for now and make a follow-up change?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795404177

From wkemper at openjdk.org  Thu Oct 10 18:48:15 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 10 Oct 2024 18:48:15 GMT
Subject: Integrated: 8339651: ShenandoahPacer::setup_for_mark,
 ShenandoahPacer::setup_for_updaterefs and ShenandoahPacer::setup_for_evac
 runtime error: division by zero
In-Reply-To: <V4_JujuCficbks9qbfsAf4yvkFEXi5clAPa6IR2H01E=.77e12334-3365-4eb7-9d8a-300985822196@github.com>
References: <V4_JujuCficbks9qbfsAf4yvkFEXi5clAPa6IR2H01E=.77e12334-3365-4eb7-9d8a-300985822196@github.com>
Message-ID: <72I2dMGwfRo-b9KcfYis69RhEZA5hXVVGqxwRw2LI-A=.2cbb9737-8342-4593-9241-3a45bd533134@github.com>

On Wed, 9 Oct 2024 20:59:49 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Avoid divide by zero error when there is no memory available.

This pull request has now been integrated.

Changeset: 06f34d7e
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/06f34d7ed2ac77e30b2a891b7a0549055ed40da3
Stats:     3 lines in 1 file changed: 3 ins; 0 del; 0 mod

8339651: ShenandoahPacer::setup_for_mark, ShenandoahPacer::setup_for_updaterefs and ShenandoahPacer::setup_for_evac runtime error: division by zero

Reviewed-by: shade, mli

-------------

PR: https://git.openjdk.org/jdk/pull/21432

From wkemper at openjdk.org  Thu Oct 10 19:09:50 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 10 Oct 2024 19:09:50 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <f5btm_m9NvWYiXpK5buIoPgGSvviMVONffYbIIWfrwY=.e625362e-a828-430a-93ba-8acc6077261f@github.com>

Merges tag jdk-24+19

-------------

Commit messages:
 - 8341722: Fix some warnings as errors when building on Linux with toolchain clang
 - 8341882: [BACKOUT] java/nio/file/attribute/BasicFileAttributeView/CreationTime.java#tmp fails on alinux3
 - 8051591: Test javax/swing/JTabbedPane/8007563/Test8007563.java fails
 - 8340809: Open source few more AWT PopupMenu tests
 - 8338884: java/nio/file/attribute/BasicFileAttributeView/CreationTime.java#tmp fails on alinux3
 - 8341859: Optimize ClassFile Benchmark Write
 - 8341755: Optimize argNames in InnerClassLambdaMetafactory
 - 8340985: Open source some Desktop related tests
 - 8341684: Typo in External Specifications link of java.util.Currency
 - 8341366: Suspicious check in Locale.getDisplayName(Locale inLocale)
 - ... and 103 more: https://git.openjdk.org/shenandoah/compare/19642bd3...e7c5bf45

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/509/files
  Stats: 21572 lines in 379 files changed: 18070 ins; 1765 del; 1737 mod
  Patch: https://git.openjdk.org/shenandoah/pull/509.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/509/head:pull/509

PR: https://git.openjdk.org/shenandoah/pull/509

From rkennke at openjdk.org  Thu Oct 10 19:27:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 10 Oct 2024 19:27:24 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
Message-ID: <BqVG4w78ZKCAL7acQX42dPJwjDuAYv41pZnzwgupv1M=.69f4f8a4-1262-4002-a58f-ed9c5660772c@github.com>

On Tue, 8 Oct 2024 17:20:31 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
> 
>  - Fix merge error
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>    
>    Reviewed-by: kdnilsen
>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>    
>    Reviewed-by: kdnilsen, ysr
>  - Merge
>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>    
>    Reviewed-by: kdnilsen
>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>    
>    Reviewed-by: kdnilsen, shade, ysr
>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1

And here comes the first part of `src/hotspot/share/gc/shenandoah` review.

src/hotspot/share/gc/shenandoah/shenandoahAffiliation.hpp line 44:

> 42:     default:
> 43:       ShouldNotReachHere();
> 44:       return "?";

This return is unreachable code (according to my IDE)

src/hotspot/share/gc/shenandoah/shenandoahAffiliation.hpp line 58:

> 56:     default:
> 57:       ShouldNotReachHere();
> 58:       return "?";

Same.

src/hotspot/share/gc/shenandoah/shenandoahCardStats.hpp line 40:

> 38:   DIRTY_SCAN_OBJS     = 6,
> 39:   ALTERNATIONS        = 7,
> 40:   MAX_CARD_STAT_TYPE  = 8

Are the numerical values relevant or what is the reason to spell them out?

src/hotspot/share/gc/shenandoah/shenandoahCardStats.hpp line 46:

> 44:   CARD_STAT_SCAN_RS       = 0,
> 45:   CARD_STAT_UPDATE_REFS   = 1,
> 46:   MAX_CARD_STAT_LOG_TYPE  = 2

Same here?

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 676:

> 674:     case ShenandoahAllocRequest::_alloc_shared_gc: {
> 675:       // Fast-path: try to allocate in the collector view first
> 676:       idx_t leftmost_collector = _partitions.leftmost(ShenandoahFreeSetPartitionId::Collector);

1. The curly bracing that starts at _alloc_plab and then again at _alloc_shared_gc is really really weird.
2. The block that is enclosed by the curly braces is really huge for a switch-case.
It probably looks better if the code can be factored out into a method and be called from the switch cases? I'm not sure if this is easy to do, because there are some returns and breaks sprinkled in the block, too. But this only makes it worse.

-------------

Changes requested by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2361030955
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795859909
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795860226
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795888252
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795888425
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1795936007

From wkemper at openjdk.org  Thu Oct 10 19:42:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 10 Oct 2024 19:42:54 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <f5btm_m9NvWYiXpK5buIoPgGSvviMVONffYbIIWfrwY=.e625362e-a828-430a-93ba-8acc6077261f@github.com>
References: <f5btm_m9NvWYiXpK5buIoPgGSvviMVONffYbIIWfrwY=.e625362e-a828-430a-93ba-8acc6077261f@github.com>
Message-ID: <TAt4A1RVRqOLoWTso_hwAONR58JcRaSklE2iq0SdnaQ=.6b287290-2498-41e1-958c-35da2330cb8b@github.com>

> Merges tag jdk-24+19

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/509/files
  - new: https://git.openjdk.org/shenandoah/pull/509/files/e7c5bf45..e7c5bf45

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=509&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=509&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/509.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/509/head:pull/509

PR: https://git.openjdk.org/shenandoah/pull/509

From wkemper at openjdk.org  Thu Oct 10 19:42:55 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 10 Oct 2024 19:42:55 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <f5btm_m9NvWYiXpK5buIoPgGSvviMVONffYbIIWfrwY=.e625362e-a828-430a-93ba-8acc6077261f@github.com>
References: <f5btm_m9NvWYiXpK5buIoPgGSvviMVONffYbIIWfrwY=.e625362e-a828-430a-93ba-8acc6077261f@github.com>
Message-ID: <GphkMaTnvJSH9IJTNYzz-mtb4nFjBLi71glraj9m0s8=.a28158f5-da12-4705-8d5f-cbc34318cffd@github.com>

On Thu, 10 Oct 2024 19:04:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-24+19

This pull request has now been integrated.

Changeset: a6ed55de
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/a6ed55de205e544e39da8a557f8581e6ffe076be
Stats:     21572 lines in 379 files changed: 18070 ins; 1765 del; 1737 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/509

From rkennke at openjdk.org  Fri Oct 11 12:28:18 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 11 Oct 2024 12:28:18 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
Message-ID: <uBJ3HKwo99Z9hOLVImVS43JCUFKCzMA-i4OcaSyw34w=.5b551cc9-9919-4e3f-8e8d-227e536dcd3c@github.com>

On Tue, 8 Oct 2024 17:20:31 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
> 
>  - Fix merge error
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>    
>    Reviewed-by: kdnilsen
>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>    
>    Reviewed-by: kdnilsen, ysr
>  - Merge
>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>    
>    Reviewed-by: kdnilsen
>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>    
>    Reviewed-by: kdnilsen, shade, ysr
>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1

I reviewed the rest of `src/hotspot/shared/gc/shenandoah`. I tried to find places where this could affect performance or even correctness of single-gen Shenandoah, but could not find any (altough, some places are a bit shady and hard to figure out, e.g. ShFreeSet).

Overall I only have a couple of comments.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 579:

> 577:   st->print("Status: ");
> 578:   if (has_forwarded_objects())                 st->print("has forwarded objects, ");
> 579:   if (is_concurrent_mark_in_progress())        st->print("marking, ");

What is this printing when not running with generational mode?

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 535:

> 533:   ShenandoahPacer*           pacer()             const { return _pacer;             }
> 534: 
> 535:   ShenandoahPhaseTimings*      phase_timings()   const { return _phase_timings;     }

The indentation is off here.

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 335:

> 333: uint ShenandoahHeap::get_object_age(oop obj) {
> 334:   // This is impossible to do unless we "freeze" ABA-type oscillations
> 335:   // With Lilliput, we can do this more easily.

The comment about Lilliput can be removed. Since we only return the actual age when the mark is not displaced, we already to the correct thing. With lightweight-locking, the mark can never be displaced, and this code should just work.

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 396:

> 394: }
> 395: 
> 396: inline bool ShenandoahHeap::is_old(oop obj) const {

What is the difference between this method and the above is_in_old()? Why does it need to check that active generation is young?

src/hotspot/share/gc/shenandoah/shenandoahMmuTracker.hpp line 28:

> 26: #define SHARE_GC_SHENANDOAH_SHENANDOAHMMUTRACKER_HPP
> 27: 
> 28: #include "runtime/mutex.hpp"

I think you don't use Mutex in this file.

-------------

Changes requested by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2362458572
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1796745606
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1796795684
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1796806016
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1796807913
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1796848150

From rkennke at openjdk.org  Fri Oct 11 15:18:25 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 11 Oct 2024 15:18:25 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
Message-ID: <_1_RPbV4VVoczxDhQYp967iOM37EHh3ZcK5b8dvKrQU=.1b9556a6-81d3-428d-8593-bd4cdced44de@github.com>

On Tue, 8 Oct 2024 17:20:31 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
> 
>  - Fix merge error
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>    
>    Reviewed-by: kdnilsen
>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>    
>    Reviewed-by: kdnilsen, ysr
>  - Merge
>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>    
>    Reviewed-by: kdnilsen
>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>    
>    Reviewed-by: kdnilsen, shade, ysr
>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1

I looked at the remaining changes, it's mostly tests. Only some minor comments/requests on those.

test/hotspot/jtreg/gc/shenandoah/TestAllocIntArrays.java line 100:

> 98: 
> 99: /*
> 100:  * @test id=generational

You are making a test configuration and call it 'generational' but then one of the two run configurations doesn't actually run with generational mode. You probably want to separate the two?

test/hotspot/jtreg/gc/shenandoah/TestAllocIntArrays.java line 163:

> 161:         final int min = 0;
> 162:         final int max = 384 * 1024;
> 163:         // Each allocated int array is assumed to consume 16 bytes for alignment and header, plus

With the upcoming 'compact headers' it's going to be only 12 bytes. If that matters, then the actual number should perhaps be a constant? Preexisting and not relevant for this PR, though.

test/hotspot/jtreg/gc/shenandoah/oom/TestAllocLargeObj.java line 1:

> 1: /*

Why are you removing the test?

test/hotspot/jtreg/gc/shenandoah/oom/TestAllocLargerThanHeap.java line 1:

> 1: /*

And this test?

test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 1:

> 1: /*

Or is this new test subsuming several old tests?

-------------

Changes requested by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2362848068
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1796980759
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1796983485
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797098313
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797098581
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797099580

From ysr at openjdk.org  Fri Oct 11 18:49:33 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 11 Oct 2024 18:49:33 GMT
Subject: RFR: 8340183: Shenandoah: Incorrect match for clone barrier in
 is_gc_barrier_node
In-Reply-To: <-w39p5a3Z1uEANEUYIVrkSdJWcy-ALre_vnhTPqxyH8=.d85d3be0-aed0-4715-950e-d03bcced36e0@github.com>
References: <-w39p5a3Z1uEANEUYIVrkSdJWcy-ALre_vnhTPqxyH8=.d85d3be0-aed0-4715-950e-d03bcced36e0@github.com>
Message-ID: <TOFtSTH5RbLlb3z8RP5kVP7S7Nj_6_sKehgoLaGNhRQ=.f0ad608f-9eb3-4663-9b58-48bfd082e4bc@github.com>

On Wed, 9 Oct 2024 19:24:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

> A few commits to backport together in this PR.

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/124#pullrequestreview-2363392757

From wkemper at openjdk.org  Fri Oct 11 19:06:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 19:06:46 GMT
Subject: RFR: 8336742: Shenandoah: Add more verbose logging/stats for mark
 termination attempts
Message-ID: <2lnMM7o93MzXdYywLcaeRQXMG9F-jBfIG9TXRmjvXjc=.07d72235-1203-4f9e-8661-557cb425a6da@github.com>

Not clean.

-------------

Commit messages:
 - Backport 90527a57848f452be3be089a703cbc2af2d1657a

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/126/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=126&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8336742
  Stats: 23 lines in 5 files changed: 10 ins; 1 del; 12 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/126.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/126/head:pull/126

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/126

From wkemper at openjdk.org  Fri Oct 11 21:04:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 21:04:34 GMT
Subject: Integrated: 8340183: Shenandoah: Incorrect match for clone barrier in
 is_gc_barrier_node
In-Reply-To: <-w39p5a3Z1uEANEUYIVrkSdJWcy-ALre_vnhTPqxyH8=.d85d3be0-aed0-4715-950e-d03bcced36e0@github.com>
References: <-w39p5a3Z1uEANEUYIVrkSdJWcy-ALre_vnhTPqxyH8=.d85d3be0-aed0-4715-950e-d03bcced36e0@github.com>
Message-ID: <pI93g7OkgpjumiNYS-XwqugVHGTXIMMPUk-Re6W8zdU=.5c5f4bea-69c8-4970-b9d7-4afb2bdd33b6@github.com>

On Wed, 9 Oct 2024 19:24:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

> A few commits to backport together in this PR.

This pull request has now been integrated.

Changeset: d144b7b6
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/d144b7b668ecafa6c590295236ccf9f4d78f473e
Stats:     22 lines in 3 files changed: 7 ins; 9 del; 6 mod

8340183: Shenandoah: Incorrect match for clone barrier in is_gc_barrier_node
8341242: Shenandoah: LRB node is not matched as GC barrier after JDK-8340183
8340186: Shenandoah: Missing load_reference_barrier_phantom_narrow match in is_shenandoah_lrb_call

Reviewed-by: ysr
Backport-of: ea8f35b98e618bfa55371e45b3ef61fa5289dd94

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/124

From wkemper at openjdk.org  Fri Oct 11 21:11:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 21:11:25 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <_1_RPbV4VVoczxDhQYp967iOM37EHh3ZcK5b8dvKrQU=.1b9556a6-81d3-428d-8593-bd4cdced44de@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <_1_RPbV4VVoczxDhQYp967iOM37EHh3ZcK5b8dvKrQU=.1b9556a6-81d3-428d-8593-bd4cdced44de@github.com>
Message-ID: <fW88Zw-E1Qn13Bld1tENYqIHhtFK8-kK_9tJUXsbE9I=.8d6bdae9-2252-41fd-92c0-c85cdac2b917@github.com>

On Fri, 11 Oct 2024 15:13:07 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 1:
> 
>> 1: /*
> 
> Or is this new test subsuming several old tests?

Correct, we rolled the `TestAlloc` tests into one file.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797425083

From wkemper at openjdk.org  Fri Oct 11 21:17:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 21:17:28 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <coMcymMrKDj7JFvuSwRamr6_cSvNWRaXiTSKmauHTeg=.b47f58dd-3e36-4c08-8ba4-4e846502bb73@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>
 <coMcymMrKDj7JFvuSwRamr6_cSvNWRaXiTSKmauHTeg=.b47f58dd-3e36-4c08-8ba4-4e846502bb73@github.com>
Message-ID: <BqqLUcQv5c1hYnw-djrhPuiYLcHPW4xehxoL59otYMA=.08a60cc7-b1d6-4bcd-9837-03be7af629ec@github.com>

On Thu, 10 Oct 2024 13:14:15 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.hpp line 79:
>> 
>>> 77: 
>>> 78: protected:
>>> 79:   static const uint Moving_Average_Samples = 10; // Number of samples to store in moving averages
>> 
>> I've never seen that style for constants in HotSpot before. I'd either make it MOVING_AVERAGE_SAMPLES or MovingAverageSamples. I think the latter is more prevalent, but I am not certain.
>
> Oh but I see that it's used in this source file. Hmm... So maybe stick with it for now and make a follow-up change?

Yes. I agree this is idiosyncratic formatting for the rest of HotSpot, but it is consistent with pre-existing constants in this file:

  static const intx Concurrent_Adjust   = -1; // recover from penalties
  static const intx Degenerated_Penalty = 10; // how much to penalize average GC duration history on Degenerated GC
  static const intx Full_Penalty        = 20; // how much to penalize average GC duration history on Full GC

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797428779

From wkemper at openjdk.org  Fri Oct 11 21:20:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 21:20:37 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <uBJ3HKwo99Z9hOLVImVS43JCUFKCzMA-i4OcaSyw34w=.5b551cc9-9919-4e3f-8e8d-227e536dcd3c@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <uBJ3HKwo99Z9hOLVImVS43JCUFKCzMA-i4OcaSyw34w=.5b551cc9-9919-4e3f-8e8d-227e536dcd3c@github.com>
Message-ID: <mbqd_3S_N2z-b7NqIWPnl1U0za1AwWY5lXUZK-KgWoI=.10c8544d-b66a-44c5-add5-2924776e7b29@github.com>

On Fri, 11 Oct 2024 10:14:31 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 579:
> 
>> 577:   st->print("Status: ");
>> 578:   if (has_forwarded_objects())                 st->print("has forwarded objects, ");
>> 579:   if (is_concurrent_mark_in_progress())        st->print("marking, ");
> 
> What is this printing when not running with generational mode?

It will print "young marking". We can change this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797430904

From wkemper at openjdk.org  Fri Oct 11 21:24:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 21:24:27 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <uBJ3HKwo99Z9hOLVImVS43JCUFKCzMA-i4OcaSyw34w=.5b551cc9-9919-4e3f-8e8d-227e536dcd3c@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <uBJ3HKwo99Z9hOLVImVS43JCUFKCzMA-i4OcaSyw34w=.5b551cc9-9919-4e3f-8e8d-227e536dcd3c@github.com>
Message-ID: <KQqx9s1UaQvKGVdbT7DfqovPsSDciqK-Swy5TTqNc1s=.dda14ecb-8c8a-4089-85fa-cad144b3f533@github.com>

On Fri, 11 Oct 2024 11:01:54 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 535:
> 
>> 533:   ShenandoahPacer*           pacer()             const { return _pacer;             }
>> 534: 
>> 535:   ShenandoahPhaseTimings*      phase_timings()   const { return _phase_timings;     }
> 
> The indentation is off here.

I'll fix this, but I secretly don't like this style of formatting because it is slightly tedious to maintain. It also suggests that these methods are somehow grouped and formatted together for a reason beyond aesthetics.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 396:
> 
>> 394: }
>> 395: 
>> 396: inline bool ShenandoahHeap::is_old(oop obj) const {
> 
> What is the difference between this method and the above is_in_old()? Why does it need to check that active generation is young?

This is just a bad, confusing method name. I'll fix this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797432875
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797433441

From wkemper at openjdk.org  Fri Oct 11 21:36:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 21:36:27 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <KQqx9s1UaQvKGVdbT7DfqovPsSDciqK-Swy5TTqNc1s=.dda14ecb-8c8a-4089-85fa-cad144b3f533@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <uBJ3HKwo99Z9hOLVImVS43JCUFKCzMA-i4OcaSyw34w=.5b551cc9-9919-4e3f-8e8d-227e536dcd3c@github.com>
 <KQqx9s1UaQvKGVdbT7DfqovPsSDciqK-Swy5TTqNc1s=.dda14ecb-8c8a-4089-85fa-cad144b3f533@github.com>
Message-ID: <avAI9bydxLsKZJVUU89s3hJCI3e95Y33ypRkEuZGxBE=.cdc38975-4d14-47f4-a201-807070b53e76@github.com>

On Fri, 11 Oct 2024 21:20:55 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 535:
>> 
>>> 533:   ShenandoahPacer*           pacer()             const { return _pacer;             }
>>> 534: 
>>> 535:   ShenandoahPhaseTimings*      phase_timings()   const { return _phase_timings;     }
>> 
>> The indentation is off here.
>
> I'll fix this, but I secretly don't like this style of formatting because it is slightly tedious to maintain. It also suggests that these methods are somehow grouped and formatted together for a reason beyond aesthetics.

https://bugs.openjdk.org/browse/JDK-8341992

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797439929

From wkemper at openjdk.org  Fri Oct 11 21:36:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 21:36:27 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <BqVG4w78ZKCAL7acQX42dPJwjDuAYv41pZnzwgupv1M=.69f4f8a4-1262-4002-a58f-ed9c5660772c@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <BqVG4w78ZKCAL7acQX42dPJwjDuAYv41pZnzwgupv1M=.69f4f8a4-1262-4002-a58f-ed9c5660772c@github.com>
Message-ID: <HMSP0scm_Gfhhy8xaDhTpLmqxj-bMrbfzHj-8x7_Fxk=.ef4ff564-6779-422a-acae-5a4c5162e53b@github.com>

On Thu, 10 Oct 2024 17:44:03 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> src/hotspot/share/gc/shenandoah/shenandoahAffiliation.hpp line 58:
> 
>> 56:     default:
>> 57:       ShouldNotReachHere();
>> 58:       return "?";
> 
> Same.

https://bugs.openjdk.org/browse/JDK-8341992

> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 335:
> 
>> 333: uint ShenandoahHeap::get_object_age(oop obj) {
>> 334:   // This is impossible to do unless we "freeze" ABA-type oscillations
>> 335:   // With Lilliput, we can do this more easily.
> 
> The comment about Lilliput can be removed. Since we only return the actual age when the mark is not displaced, we already to the correct thing. With lightweight-locking, the mark can never be displaced, and this code should just work.

https://bugs.openjdk.org/browse/JDK-8341992

> src/hotspot/share/gc/shenandoah/shenandoahMmuTracker.hpp line 28:
> 
>> 26: #define SHARE_GC_SHENANDOAH_SHENANDOAHMMUTRACKER_HPP
>> 27: 
>> 28: #include "runtime/mutex.hpp"
> 
> I think you don't use Mutex in this file.

https://bugs.openjdk.org/browse/JDK-8341992

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797440111
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797439856
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797439708

From wkemper at openjdk.org  Fri Oct 11 22:03:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 22:03:26 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>
Message-ID: <1tM_l9mquiouiroTulNDRfVposOwRRVmj18YD_WJU6I=.20e1a171-4438-417f-af58-18e2ed57d125@github.com>

On Thu, 10 Oct 2024 12:48:05 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp line 75:
> 
>> 73: 
>> 74:   // Create a mask to test if the marking bit is set.
>> 75:   // TODO: can we directly test if bit is set?
> 
> That comment here is quite justified, IMO. I'm pretty sure that we could test for the flag in a single instruction, instead of doing the and-cmp sequence and even allocating a new register. Unless of course when C1 LIR can't represent it. Have you tried to implement that and failed, and therefore remove the comment?

@shipilev and I looked at this: https://github.com/openjdk/jdk/pull/19180#discussion_r1609666303. The performance change was insignificant.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797456666

From ysr at openjdk.org  Fri Oct 11 22:17:28 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 11 Oct 2024 22:17:28 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <BqVG4w78ZKCAL7acQX42dPJwjDuAYv41pZnzwgupv1M=.69f4f8a4-1262-4002-a58f-ed9c5660772c@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <BqVG4w78ZKCAL7acQX42dPJwjDuAYv41pZnzwgupv1M=.69f4f8a4-1262-4002-a58f-ed9c5660772c@github.com>
Message-ID: <V6vSlviRiPnhKtFAFhl19jMWyTRUCTjTwp9gp42TgyE=.7b5da064-1d5e-4f22-a3fb-fa5c6cd0d9a7@github.com>

On Thu, 10 Oct 2024 18:07:27 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> src/hotspot/share/gc/shenandoah/shenandoahCardStats.hpp line 40:
> 
>> 38:   DIRTY_SCAN_OBJS     = 6,
>> 39:   ALTERNATIONS        = 7,
>> 40:   MAX_CARD_STAT_TYPE  = 8
> 
> Are the numerical values relevant or what is the reason to spell them out?

Not needed; explicit enumeration values can be removed.

> src/hotspot/share/gc/shenandoah/shenandoahCardStats.hpp line 46:
> 
>> 44:   CARD_STAT_SCAN_RS       = 0,
>> 45:   CARD_STAT_UPDATE_REFS   = 1,
>> 46:   MAX_CARD_STAT_LOG_TYPE  = 2
> 
> Same here?

Yes, same; explicit enumeration values can be removed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797462093
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797462886

From wkemper at openjdk.org  Fri Oct 11 23:34:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 23:34:32 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <BqVG4w78ZKCAL7acQX42dPJwjDuAYv41pZnzwgupv1M=.69f4f8a4-1262-4002-a58f-ed9c5660772c@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <BqVG4w78ZKCAL7acQX42dPJwjDuAYv41pZnzwgupv1M=.69f4f8a4-1262-4002-a58f-ed9c5660772c@github.com>
Message-ID: <9VnBxOMfOYEdW6nHhKxSvAtURKeC3a8GfpVPo-g7cr8=.dffb958c-a71d-4fd5-9319-eabff1f6c2f9@github.com>

On Thu, 10 Oct 2024 18:47:38 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 676:
> 
>> 674:     case ShenandoahAllocRequest::_alloc_shared_gc: {
>> 675:       // Fast-path: try to allocate in the collector view first
>> 676:       idx_t leftmost_collector = _partitions.leftmost(ShenandoahFreeSetPartitionId::Collector);
> 
> 1. The curly bracing that starts at _alloc_plab and then again at _alloc_shared_gc is really really weird.
> 2. The block that is enclosed by the curly braces is really huge for a switch-case.
> It probably looks better if the code can be factored out into a method and be called from the switch cases? I'm not sure if this is easy to do, because there are some returns and breaks sprinkled in the block, too. But this only makes it worse.

https://bugs.openjdk.org/browse/JDK-8342001

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1797493799

From wkemper at openjdk.org  Fri Oct 11 23:35:24 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 23:35:24 GMT
Subject: RFR: 8341992: GenShen:  Fix formatting, remove unreachable code,
 unused imports and unnecessary comments
Message-ID: <M3pR1UcitWaOlF8920j8qKDkeB3ZxYF5PZ_2SGu6Uqk=.d0378545-f87f-4a27-9f5a-f8ac29ce3e6b@github.com>

Improve hygiene for PR for JEP-404.

-------------

Commit messages:
 - Remove unreachable code, unused imports, unnecessary comments, unusual formatting

Changes: https://git.openjdk.org/shenandoah/pull/510/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=510&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341992
  Stats: 7 lines in 4 files changed: 0 ins; 5 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/510.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/510/head:pull/510

PR: https://git.openjdk.org/shenandoah/pull/510

From wkemper at openjdk.org  Fri Oct 11 23:39:55 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Oct 2024 23:39:55 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into separate
 methods
Message-ID: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>

Prune switch statement which has grown unwieldy.

-------------

Commit messages:
 - Reorder declarations to match order of definitions
 - Factor allocation type cases into separate methods

Changes: https://git.openjdk.org/shenandoah/pull/511/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=511&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342001
  Stats: 279 lines in 2 files changed: 126 ins; 105 del; 48 mod
  Patch: https://git.openjdk.org/shenandoah/pull/511.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/511/head:pull/511

PR: https://git.openjdk.org/shenandoah/pull/511

From ysr at openjdk.org  Sat Oct 12 00:35:25 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 12 Oct 2024 00:35:25 GMT
Subject: RFR: 8341992: GenShen:  Fix formatting, remove unreachable code, 
 unused imports and unnecessary comments
In-Reply-To: <M3pR1UcitWaOlF8920j8qKDkeB3ZxYF5PZ_2SGu6Uqk=.d0378545-f87f-4a27-9f5a-f8ac29ce3e6b@github.com>
References: <M3pR1UcitWaOlF8920j8qKDkeB3ZxYF5PZ_2SGu6Uqk=.d0378545-f87f-4a27-9f5a-f8ac29ce3e6b@github.com>
Message-ID: <7X6T7eHE3bUyl7JZZCzYEG3gP-DyjJA6liykN-h9y4o=.13e4d696-305d-46a3-b060-e7b06774f405@github.com>

On Fri, 11 Oct 2024 23:29:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Improve hygiene for PR for JEP-404.

Marked as reviewed by ysr (Committer).

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 537:

> 535:   ShenandoahPhaseTimings*    phase_timings()     const { return _phase_timings;     }
> 536: 
> 537:   ShenandoahEvacOOMHandler*  oom_evac_handler() { return &_oom_evac_handler; }

Should left curly braces be aligned too? :-)

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/510#pullrequestreview-2363690157
PR Review Comment: https://git.openjdk.org/shenandoah/pull/510#discussion_r1797508202

From ysr at openjdk.org  Sat Oct 12 01:36:27 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 12 Oct 2024 01:36:27 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods
In-Reply-To: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
Message-ID: <tCJgLs1i50stZf0koybSuANAFImfJYW-BzuSG0E7NMo=.a87c1a31-c488-43fd-9451-5130781afea4@github.com>

On Fri, 11 Oct 2024 23:32:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Prune switch statement which has grown unwieldy.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 331:

> 329:   void flip_to_old_gc(ShenandoahHeapRegion* r);
> 330: 
> 331:   // Handle allocation from mutator

"from" -> "by" (or may be even "for" as in the name of the method).

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 340:

> 338:   HeapWord* allocate_for_collector(ShenandoahAllocRequest& req, bool& in_new_region);
> 339:   bool can_allocate_in_new_region(const ShenandoahAllocRequest& req);
> 340:   HeapWord* try_allocate_from_mutator(ShenandoahAllocRequest& req, bool& in_new_region);

May be a 1-line spec comment for each of these methods?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1797524028
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1797523479

From mbaesken at openjdk.org  Mon Oct 14 11:45:44 2024
From: mbaesken at openjdk.org (Matthias Baesken)
Date: Mon, 14 Oct 2024 11:45:44 GMT
Subject: RFR: 8342044: Increase timeout of
 gc/shenandoah/oom/TestClassLoaderLeak.java
Message-ID: <vacjtHaJHS4QuyVB26411a-8BYXpiRfmC_xjpZJfz7I=.567b306b-1376-4a18-903d-5581f1a2212f@github.com>

Test gc/shenandoah/oom/TestClassLoaderLeak.java often times out on WIndows, we should increase the timeout values to avoid test errors.

-------------

Commit messages:
 - JDK-8342044

Changes: https://git.openjdk.org/jdk/pull/21491/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21491&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342044
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21491.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21491/head:pull/21491

PR: https://git.openjdk.org/jdk/pull/21491

From rkennke at openjdk.org  Mon Oct 14 16:58:27 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 14 Oct 2024 16:58:27 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <1tM_l9mquiouiroTulNDRfVposOwRRVmj18YD_WJU6I=.20e1a171-4438-417f-af58-18e2ed57d125@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>
 <1tM_l9mquiouiroTulNDRfVposOwRRVmj18YD_WJU6I=.20e1a171-4438-417f-af58-18e2ed57d125@github.com>
Message-ID: <uZUQVcDqmjsuXAcUNYJ3zUgqkufbC6CNmhIcXRg6ov4=.5dfd1d85-2c61-434a-924a-e8d50221148d@github.com>

On Fri, 11 Oct 2024 22:00:55 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp line 75:
>> 
>>> 73: 
>>> 74:   // Create a mask to test if the marking bit is set.
>>> 75:   // TODO: can we directly test if bit is set?
>> 
>> That comment here is quite justified, IMO. I'm pretty sure that we could test for the flag in a single instruction, instead of doing the and-cmp sequence and even allocating a new register. Unless of course when C1 LIR can't represent it. Have you tried to implement that and failed, and therefore remove the comment?
>
> @shipilev and I looked at this: https://github.com/openjdk/jdk/pull/19180#discussion_r1609666303. The performance change was insignificant.

ok.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1799832437

From wkemper at openjdk.org  Mon Oct 14 17:15:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Oct 2024 17:15:41 GMT
Subject: RFR: 8341992: GenShen:  Fix formatting, remove unreachable code, 
 unused imports and unnecessary comments [v2]
In-Reply-To: <M3pR1UcitWaOlF8920j8qKDkeB3ZxYF5PZ_2SGu6Uqk=.d0378545-f87f-4a27-9f5a-f8ac29ce3e6b@github.com>
References: <M3pR1UcitWaOlF8920j8qKDkeB3ZxYF5PZ_2SGu6Uqk=.d0378545-f87f-4a27-9f5a-f8ac29ce3e6b@github.com>
Message-ID: <HtdkqyPV0zu3Jtev9Cuf6L-Gnwv1U8b0uHuyKcIvjEU=.86b37339-1e04-4374-a2da-22cb1c8e6e64@github.com>

> Improve hygiene for PR for JEP-404.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Align one line function blocks

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/510/files
  - new: https://git.openjdk.org/shenandoah/pull/510/files/c21d514c..736d9c45

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=510&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=510&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/510.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/510/head:pull/510

PR: https://git.openjdk.org/shenandoah/pull/510

From wkemper at openjdk.org  Mon Oct 14 17:26:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Oct 2024 17:26:33 GMT
Subject: Integrated: 8341992: GenShen:  Fix formatting,
 remove unreachable code, unused imports and unnecessary comments
In-Reply-To: <M3pR1UcitWaOlF8920j8qKDkeB3ZxYF5PZ_2SGu6Uqk=.d0378545-f87f-4a27-9f5a-f8ac29ce3e6b@github.com>
References: <M3pR1UcitWaOlF8920j8qKDkeB3ZxYF5PZ_2SGu6Uqk=.d0378545-f87f-4a27-9f5a-f8ac29ce3e6b@github.com>
Message-ID: <7Km4rQkigOXei8dHwnQjAVMgwHt1i7KoRpO-tudYuMY=.ac9da61b-6753-4939-9fbe-93bd4ab48f63@github.com>

On Fri, 11 Oct 2024 23:29:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Improve hygiene for PR for JEP-404.

This pull request has now been integrated.

Changeset: bb793844
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/bb7938446b3b6d6729df627cd580957f4a3a2116
Stats:     7 lines in 4 files changed: 0 ins; 5 del; 2 mod

8341992: GenShen:  Fix formatting, remove unreachable code, unused imports and unnecessary comments

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/510

From mdoerr at openjdk.org  Mon Oct 14 17:40:38 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Mon, 14 Oct 2024 17:40:38 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v11]
In-Reply-To: <toUdNzwkdDAQDltc4_t6wI06bCE-EHzpIs_t-arIwNY=.7d9ef348-0033-4268-9ed1-7c183855521f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <DUgk6HT_Zk5KnbmAd_HzrJi4B8D1J-uuoAGzReWyI8s=.6c04f90a-bf1d-49af-a31d-5224973952d4@github.com>
 <YTte6oY9EeAPHxzzOndL8M202wY91SbeaYC4hPva0W0=.caf00270-89c7-44b0-ab88-a970fb8e840d@github.com>
 <kzodWwxj5qdAqPYcP2s4k2QTXqjju2H6Iftu7dojezk=.1f72fc31-94ba-4017-b330-c8688a8d39a4@github.com>
 <hkGaBWSugvpV0PgoR9rnutHCgQzkeXv1ctH7QwV3zK0=.204737d5-fadb-410a-8891-7db59d36830f@github.com>
 <shoDPnL4nQyaYX4xy-WyzQ7CwsoCbAeCE3c755CkE1o=.caf940aa-5fbb-4cb5-a4a2-68a2452ffe1b@github.com>
 <TBZCkcFNNnEvat7TBnRkqUPo4Y5TEe1L0iUZ9loD7DI=.39c0249c-f6b2-45c8-b7c8-9e2c9c1b7a99@github.com>
 <toUdNzwkdDAQDltc4_t6wI06bCE-EHzpIs_t-arIwNY=.7d9ef348-0033-4268-9ed1-7c183855521f@github.com>
Message-ID: <9F4j9U7srQ_aLXdOairjcjGy9Rm-gO8MXPCrqz03iec=.00254e94-dbcc-49ce-a0de-3b72cd2e2b4e@github.com>

On Tue, 1 Oct 2024 15:46:01 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> Indeed, I could re-enable all tests in:
>>> 
>>> ```
>>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationMismatchedAccess.java
>>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java
>>> test/hotspot/jtreg/compiler/loopopts/superword/TestIndependentPacksWithCyclicDependency.java
>>> ```
>>> 
>>> but unfortunately not those others:
>>> 
>>> ```
>>> > > > test/hotspot/jtreg/compiler/loopopts/superword/TestAlignVector.java
>>> > > > test/hotspot/jtreg/compiler/loopopts/superword/TestMulAddS2I.java
>>> ```
>>> 
>>> I think the issue with all of them is that vectorization in those scenarios only works when the operations inside the loop start at an array index that addresses an element at 8-byte-aligned offset.
>>> 
>>> I have filed https://bugs.openjdk.org/browse/JDK-8340010 to track it.
>> 
>> @rkennke A test run of the current changeset in our internal CI system revealed that the following tests fail (because of missing vectorization) when using `-XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders -XX:UseSSE=N` with `N <= 3` on an Intel Xeon Platinum 8358 machine:
>> 
>> - test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java
>> - test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationMismatchedAccess.java
>> - test/hotspot/jtreg/compiler/vectorization/runner/LoopCombinedOpTest.java
>> 
>> Here are the failure details:
>> 
>> 
>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java:
>> 
>> 1) Method "public static void compiler.c2.irTests.TestVectorizationNotRun.test(byte[],long[])" - [Failed IR rules: 1]:
>>    * @IR rule 1: "@compiler.lib.ir_framework.IR(phase={DEFAULT}, applyIfPlatformAnd={}, applyIfCPUFeatureOr={}, counts={"_#V#LOAD_VECTOR_L#_", ">=1", "_#STORE_VECTOR#_", ">=1"}, applyIfPlatform={}, applyIfPlatformOr={}, failOn={}, applyIfOr={}, applyIfCPUFeatureAnd={}, applyIf={}, applyIfCPUFeature={}, applyIfAnd={}, applyIfNot={})"
>>      > Phase "PrintIdeal":
>>        - counts: Graph contains wrong number of nodes:
>>          * Constraint 1: "(\\d+(\\s){2}(LoadVector.*)+(\\s){2}===.*vector[A-Za-z]\[2\]:\{long\})"
>>            - Failed comparison: [found] 0 >= 1 [given]
>>            - No nodes matched!
>>          * Constraint 2: "(\\d+(\\s){2}(StoreVector.*)+(\\s){2}===.*)"
>>            - Failed comparison: [found] 0 >= 1 [given]
>>            - No nodes matched!
>> 
>> 
>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationMismatchedAccess.java:
>> 
>> 1) Method "public static void compiler.c2.irTests.TestVectorizati...
>
>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java:
> 
> I think I would disable the tests for now. Is there a good way to say 'run this when UCOH is off OR UseSSE>3?

@rkennke: I have a PPC64 implementation: https://github.com/TheRealMDoerr/jdk/commit/6722f8be9a0940fab6417d4de58ec1538c436702
Do you want to include it? Should we also ask s390 and riscv folks?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2411867295

From shade at openjdk.org  Mon Oct 14 18:52:39 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 14 Oct 2024 18:52:39 GMT
Subject: RFR: 8342079: Shenandoah: Eliminate extra
 ShenandoahInitMarkRootsClosure
Message-ID: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>

`ShenandoahInitMarkRootsClosure` in `ShenandoahSTWMark` is semantically similar to the shared `ShenandoahMarkRefsClosure`. It misses subclassing the metadata-visiting superclass, though, and `ShenandoahMarkRefsClosure` does it right. It is cleaner and safer to use the super-class directly, as concurrent mark does it. 

Additional testing:
 - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

-------------

Commit messages:
 - Fix

Changes: https://git.openjdk.org/jdk/pull/21504/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21504&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342079
  Stats: 30 lines in 1 file changed: 1 ins; 27 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/21504.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21504/head:pull/21504

PR: https://git.openjdk.org/jdk/pull/21504

From wkemper at openjdk.org  Mon Oct 14 18:55:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Oct 2024 18:55:13 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
Message-ID: <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>

> Prune switch statement which has grown unwieldy.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Improve comments for new methods

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/511/files
  - new: https://git.openjdk.org/shenandoah/pull/511/files/0360f2ff..8156aad2

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=511&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=511&range=00-01

  Stats: 12 lines in 1 file changed: 10 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/511.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/511/head:pull/511

PR: https://git.openjdk.org/shenandoah/pull/511

From rkennke at openjdk.org  Mon Oct 14 19:11:30 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 14 Oct 2024 19:11:30 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v11]
In-Reply-To: <toUdNzwkdDAQDltc4_t6wI06bCE-EHzpIs_t-arIwNY=.7d9ef348-0033-4268-9ed1-7c183855521f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <DUgk6HT_Zk5KnbmAd_HzrJi4B8D1J-uuoAGzReWyI8s=.6c04f90a-bf1d-49af-a31d-5224973952d4@github.com>
 <YTte6oY9EeAPHxzzOndL8M202wY91SbeaYC4hPva0W0=.caf00270-89c7-44b0-ab88-a970fb8e840d@github.com>
 <kzodWwxj5qdAqPYcP2s4k2QTXqjju2H6Iftu7dojezk=.1f72fc31-94ba-4017-b330-c8688a8d39a4@github.com>
 <hkGaBWSugvpV0PgoR9rnutHCgQzkeXv1ctH7QwV3zK0=.204737d5-fadb-410a-8891-7db59d36830f@github.com>
 <shoDPnL4nQyaYX4xy-WyzQ7CwsoCbAeCE3c755CkE1o=.caf940aa-5fbb-4cb5-a4a2-68a2452ffe1b@github.com>
 <TBZCkcFNNnEvat7TBnRkqUPo4Y5TEe1L0iUZ9loD7DI=.39c0249c-f6b2-45c8-b7c8-9e2c9c1b7a99@github.com>
 <toUdNzwkdDAQDltc4_t6wI06bCE-EHzpIs_t-arIwNY=.7d9ef348-0033-4268-9ed1-7c183855521f@github.com>
Message-ID: <JCG6RiKoWCBiqIqi-DsgwqdFE1X_9KrJH1Sqnpdl-2E=.72327548-6e1c-44df-ae77-0fcdb3b92c32@github.com>

On Tue, 1 Oct 2024 15:46:01 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> Indeed, I could re-enable all tests in:
>>> 
>>> ```
>>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationMismatchedAccess.java
>>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java
>>> test/hotspot/jtreg/compiler/loopopts/superword/TestIndependentPacksWithCyclicDependency.java
>>> ```
>>> 
>>> but unfortunately not those others:
>>> 
>>> ```
>>> > > > test/hotspot/jtreg/compiler/loopopts/superword/TestAlignVector.java
>>> > > > test/hotspot/jtreg/compiler/loopopts/superword/TestMulAddS2I.java
>>> ```
>>> 
>>> I think the issue with all of them is that vectorization in those scenarios only works when the operations inside the loop start at an array index that addresses an element at 8-byte-aligned offset.
>>> 
>>> I have filed https://bugs.openjdk.org/browse/JDK-8340010 to track it.
>> 
>> @rkennke A test run of the current changeset in our internal CI system revealed that the following tests fail (because of missing vectorization) when using `-XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders -XX:UseSSE=N` with `N <= 3` on an Intel Xeon Platinum 8358 machine:
>> 
>> - test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java
>> - test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationMismatchedAccess.java
>> - test/hotspot/jtreg/compiler/vectorization/runner/LoopCombinedOpTest.java
>> 
>> Here are the failure details:
>> 
>> 
>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java:
>> 
>> 1) Method "public static void compiler.c2.irTests.TestVectorizationNotRun.test(byte[],long[])" - [Failed IR rules: 1]:
>>    * @IR rule 1: "@compiler.lib.ir_framework.IR(phase={DEFAULT}, applyIfPlatformAnd={}, applyIfCPUFeatureOr={}, counts={"_#V#LOAD_VECTOR_L#_", ">=1", "_#STORE_VECTOR#_", ">=1"}, applyIfPlatform={}, applyIfPlatformOr={}, failOn={}, applyIfOr={}, applyIfCPUFeatureAnd={}, applyIf={}, applyIfCPUFeature={}, applyIfAnd={}, applyIfNot={})"
>>      > Phase "PrintIdeal":
>>        - counts: Graph contains wrong number of nodes:
>>          * Constraint 1: "(\\d+(\\s){2}(LoadVector.*)+(\\s){2}===.*vector[A-Za-z]\[2\]:\{long\})"
>>            - Failed comparison: [found] 0 >= 1 [given]
>>            - No nodes matched!
>>          * Constraint 2: "(\\d+(\\s){2}(StoreVector.*)+(\\s){2}===.*)"
>>            - Failed comparison: [found] 0 >= 1 [given]
>>            - No nodes matched!
>> 
>> 
>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationMismatchedAccess.java:
>> 
>> 1) Method "public static void compiler.c2.irTests.TestVectorizati...
>
>> test/hotspot/jtreg/compiler/c2/irTests/TestVectorizationNotRun.java:
> 
> I think I would disable the tests for now. Is there a good way to say 'run this when UCOH is off OR UseSSE>3?

> @rkennke: I have a PPC64 implementation: [TheRealMDoerr at 6722f8b](https://github.com/TheRealMDoerr/jdk/commit/6722f8be9a0940fab6417d4de58ec1538c436702) Do you want to include it? Should we also ask s390 and riscv folks?

AFAIK, @Hamlin-Li is working on the RISCV port. Not sure who would do s390. If it's available before intergration, I'll include it, but I'll not wait for it.

Thanks for the PPC64 port, I'll include it in this PR!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2412025660

From shade at openjdk.org  Mon Oct 14 19:24:43 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 14 Oct 2024 19:24:43 GMT
Subject: RFR: 8342081: Shenandoah: Remove extra
 ShenandoahMarkUpdateRefsSuperClosure
Message-ID: <btbf6MorZ7rEPf0UQBMVMeiUnPsUSpBapEhNHI4SldY=.452c9730-7048-43aa-9253-36797ecc6e09@github.com>

`ShenandoahMarkUpdateRefsSuperClosure` has only one real subclass, which is used in STW mark. We can collapse the class hierarchy a bit here. 

Additional testing:
 - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

-------------

Commit messages:
 - Fix

Changes: https://git.openjdk.org/jdk/pull/21505/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21505&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342081
  Stats: 22 lines in 2 files changed: 2 ins; 11 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/21505.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21505/head:pull/21505

PR: https://git.openjdk.org/jdk/pull/21505

From ysr at openjdk.org  Mon Oct 14 21:25:33 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 14 Oct 2024 21:25:33 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
Message-ID: <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>

On Mon, 14 Oct 2024 18:55:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Prune switch statement which has grown unwieldy.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Improve comments for new methods

Refactoring looks good to me. Left a few comments that may or may not help, but raised a few questions in my mind. These aren't changes you made, but looking at the change raised questions that may have been raised and addressed earlier perhaps. (cc @kdnilsen).

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 762:

> 760: 
> 761: HeapWord* ShenandoahFreeSet::allocate_for_mutator(ShenandoahAllocRequest &req, bool &in_new_region) {
> 762:   maybe_change_allocation_bias();

Why update alloc bias if allocation id doomed to fail at next line? Is that the intention? If not, may be do the bias update only when there is chance for success in the allocation? See suggestions further below.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 774:

> 772:   }
> 773: 
> 774:   return allocate_from_right_to_left(req, in_new_region);

Just a thought: Might make the code easier to read, maintain, and reason about, if instead of a boolean value indicating left bias, if we used a two value enum: LeftBias, RightBias, and used an `_alloc_bias` array for the partitions.

Then code might read:

    .... set_alloc_bias(LeftBias);   // set allocation bias to left

and the above code might look like:

   if (_partitions.is_empy(SFSPI::Mutator)) {
     return nullptr;
   }
   update_alloc_bias();
   switch (_partitions._alloc_bias[ShenandoahPartitionId::Mutator]) {
     case LeftBias:  return allocate_from_left(req, in_new_region);
     case RightBias: return allocate_from_right(req, in_new_region);
     default: ShouldNotReachHere();
   }

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 777:

> 775: }
> 776: 
> 777: void ShenandoahFreeSet::maybe_change_allocation_bias() {

Since this is a counting update, I'd rename it to `update_allocation_bias()`.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 799:

> 797:                                 - _partitions.rightmost_empty(ShenandoahFreeSetPartitionId::Mutator));
> 798:     _partitions.set_bias_from_left_to_right(ShenandoahFreeSetPartitionId::Mutator, (non_empty_on_right < non_empty_on_left));
> 799:     _alloc_bias_weight = _InitialAllocBiasWeight;

The value of 256 for `_InitialAllocBiasWeight` seems a bit of magic. How was it chosen, and is it subject to bit-rot with code evolving? I can imagine that the bias update at a higher frequency is apt to not matter much, and since the leftmost and rightmost empty calculation is not free, so larger values are favored.

Also should we use the more usual `static const ssize_t INITIAL_ALLOC_BIAS_WEIGHT` Hotspot style for constants, rather than the camel case with an initial underscore (which seems not common)?

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 813:

> 811:            "Boundaries or find_last_set_bit failed: " SSIZE_FORMAT, idx);
> 812:     ShenandoahHeapRegion* r = _heap->get_region(idx);
> 813:     // try_allocate_in() increases used if the allocation is successful.

I think the comment about what `try_allocate_in()` does belongs as a spec comment for that method, rather than here. I assume `used` in this comment refers to the used bytes for the (here, mutator) partition?

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 832:

> 830:            "Boundaries or find_last_set_bit failed: " SSIZE_FORMAT, idx);
> 831:     ShenandoahHeapRegion* r = _heap->get_region(idx);
> 832:     // try_allocate_in() increases used if the allocation is successful.

Same comment re "used".

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 899:

> 897: }
> 898: 
> 899: HeapWord* ShenandoahFreeSet::try_allocate_from_mutator(ShenandoahAllocRequest& req, bool& in_new_region) {

Why does this always use right-biased allocation from the mutator partition? (And not update allocation bias state?) Is this intentional? I assume it's because these aren't fresh allocations, but evacuations. Do evacuations with right bias end up closer to allocations for other evacuations? A brief comment explaining this difference from mutator allocation would be useful.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 334:

> 332:   HeapWord* allocate_for_mutator(ShenandoahAllocRequest &req, bool &in_new_region);
> 333: 
> 334:   // Chose whether to change the allocation bias from the left or right side of the heap.

Chose -> Choose

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 335:

> 333: 
> 334:   // Chose whether to change the allocation bias from the left or right side of the heap.
> 335:   void maybe_change_allocation_bias();

Weak suggestion above: `update_allocation_bias()`

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/511#pullrequestreview-2367465853
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800019810
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800016996
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800008130
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800048082
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800074385
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800077212
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800147723
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1799987970
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800112075

From ysr at openjdk.org  Mon Oct 14 21:25:33 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 14 Oct 2024 21:25:33 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
Message-ID: <6_pZ_HkM8cBs7a7XME6bh3GTW5GQTIs06ytGgeHIQu0=.223e4471-f6cc-4176-83b3-807b977e3bb0@github.com>

On Mon, 14 Oct 2024 20:12:24 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Improve comments for new methods
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 774:
> 
>> 772:   }
>> 773: 
>> 774:   return allocate_from_right_to_left(req, in_new_region);
> 
> Just a thought: Might make the code easier to read, maintain, and reason about, if instead of a boolean value indicating left bias, if we used a two value enum: LeftBias, RightBias, and used an `_alloc_bias` array for the partitions.
> 
> Then code might read:
> 
>     .... set_alloc_bias(LeftBias);   // set allocation bias to left
> 
> and the above code might look like:
> 
>    if (_partitions.is_empy(SFSPI::Mutator)) {
>      return nullptr;
>    }
>    update_alloc_bias();
>    switch (_partitions._alloc_bias[ShenandoahPartitionId::Mutator]) {
>      case LeftBias:  return allocate_from_left(req, in_new_region);
>      case RightBias: return allocate_from_right(req, in_new_region);
>      default: ShouldNotReachHere();
>    }

I also wonder if `allocate_from_left` etc., since they seem to involve only mutator allocations, should include `mutator` somewhere in the name of the method; if so, I wonder about why we maintain the bias variable for the other partitions? (I suppose we don't but since each is a partition and the variable is associated with each partition, it's there. If we don't think we'll need  bias for other allocations, it might make sense to move it to the free set, but I'd hold off on that until it's clear that directional bias for allocation isn't going to be useful for collector allocations.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800129152

From mdoerr at openjdk.org  Mon Oct 14 21:49:28 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Mon, 14 Oct 2024 21:49:28 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v39]
In-Reply-To: <q4sTPbMm-cK1O35GhIbRytiakYgyKJYrBhotG_Kwn2o=.02620a61-e96d-4c93-af80-d27a61dd9ee7@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <q4sTPbMm-cK1O35GhIbRytiakYgyKJYrBhotG_Kwn2o=.02620a61-e96d-4c93-af80-d27a61dd9ee7@github.com>
Message-ID: <LjXNg7ygXw6KlgE0O5mMhgJ2IpNSpcZV1VKvF-p3qdQ=.608e20a4-25a8-4b9d-acaa-39eead7c4617@github.com>

On Tue, 8 Oct 2024 16:30:47 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Increase compiler code stubs size for indexOf intrinsic

Thanks!
@offamitkumar: It could still be done after this PR is integrated, but I guess you want to provide an s390 implementation.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2412394373

From wkemper at openjdk.org  Mon Oct 14 22:15:49 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Oct 2024 22:15:49 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
Message-ID: <e0lQC0yv3JKW2xD7SgxcLyfYcd_yaBV8w2gWtyzOYaU=.0b6fe843-9e73-4831-a01a-018e7eae574c@github.com>

On Mon, 14 Oct 2024 20:16:17 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Improve comments for new methods
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 762:
> 
>> 760: 
>> 761: HeapWord* ShenandoahFreeSet::allocate_for_mutator(ShenandoahAllocRequest &req, bool &in_new_region) {
>> 762:   maybe_change_allocation_bias();
> 
> Why update alloc bias if allocation is doomed to fail at next line? Is that the intention? If not, may be do the bias update only when there is chance for success in the allocation? See suggestions further below.

I had this same thought, but ended on the side of preserving existing behavior. @kdnilsen , should we not update the allocation bias if we can detect an early allocation failure?

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 799:
> 
>> 797:                                 - _partitions.rightmost_empty(ShenandoahFreeSetPartitionId::Mutator));
>> 798:     _partitions.set_bias_from_left_to_right(ShenandoahFreeSetPartitionId::Mutator, (non_empty_on_right < non_empty_on_left));
>> 799:     _alloc_bias_weight = _InitialAllocBiasWeight;
> 
> The value of 256 for `_InitialAllocBiasWeight` seems a bit of magic. How was it chosen, and is it subject to bit-rot with code evolving? I can imagine that the bias update at a higher frequency is apt to not matter much, and since the leftmost and rightmost empty calculation is not free, so larger values are favored.
> 
> Also should we use the more usual `static const ssize_t INITIAL_ALLOC_BIAS_WEIGHT` Hotspot style for constants, rather than the camel case with an initial underscore (which seems not common)?

I'm not sure where the initial value comes from. I will change to the idiomatic name convention.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800183845
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800184751

From wkemper at openjdk.org  Mon Oct 14 22:15:49 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Oct 2024 22:15:49 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <6_pZ_HkM8cBs7a7XME6bh3GTW5GQTIs06ytGgeHIQu0=.223e4471-f6cc-4176-83b3-807b977e3bb0@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
 <6_pZ_HkM8cBs7a7XME6bh3GTW5GQTIs06ytGgeHIQu0=.223e4471-f6cc-4176-83b3-807b977e3bb0@github.com>
Message-ID: <Q7MSHEmR-0FFkD8eYKPJr28hUow5odDYMjQTRpnJXkA=.cb5fc819-6de8-452a-977b-8861d866ecc5@github.com>

On Mon, 14 Oct 2024 20:54:56 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 774:
>> 
>>> 772:   }
>>> 773: 
>>> 774:   return allocate_from_right_to_left(req, in_new_region);
>> 
>> Just a thought: Might make the code easier to read, maintain, and reason about, if instead of a boolean value indicating left bias, if we used a two value enum: LeftBias, RightBias, and used an `_alloc_bias` array for the partitions.
>> 
>> Then code might read:
>> 
>>     .... set_alloc_bias(LeftBias);   // set allocation bias to left
>> 
>> and the above code might look like:
>> 
>>    if (_partitions.is_empy(SFSPI::Mutator)) {
>>      return nullptr;
>>    }
>>    update_alloc_bias();
>>    switch (_partitions._alloc_bias[ShenandoahPartitionId::Mutator]) {
>>      case LeftBias:  return allocate_from_left(req, in_new_region);
>>      case RightBias: return allocate_from_right(req, in_new_region);
>>      default: ShouldNotReachHere();
>>    }
>
> I also wonder if `allocate_from_left` etc., since they seem to involve only mutator allocations, should include `mutator` somewhere in the name of the method; if so, I wonder about why we maintain the bias variable for the other partitions? (I suppose we don't but since each is a partition and the variable is associated with each partition, it's there. If we don't think we'll need  bias for other allocations, it might make sense to move it to the free set, but I'd hold off on that until it's clear that directional bias for allocation isn't going to be useful for collector allocations.

We do also use the allocation bias for collector allocations. I have a deeper refactoring related to this allocation bias here: https://github.com/openjdk/shenandoah/pull/512, but I've left it in draft status pending pipeline test results.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800182570

From wkemper at openjdk.org  Mon Oct 14 22:18:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Oct 2024 22:18:36 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
Message-ID: <KWRWwkqC4tTzlpgQWzsiqmom8rZ5ZWY6fZxgaQUmRAQ=.b3a0e79a-1356-480c-b309-76724212c0e4@github.com>

On Mon, 14 Oct 2024 20:36:41 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Improve comments for new methods
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 813:
> 
>> 811:            "Boundaries or find_last_set_bit failed: " SSIZE_FORMAT, idx);
>> 812:     ShenandoahHeapRegion* r = _heap->get_region(idx);
>> 813:     // try_allocate_in() increases used if the allocation is successful.
> 
> I think the comment about what `try_allocate_in()` does belongs as a spec comment for that method, rather than here. I assume `used` in this comment refers to the used bytes for the (here, mutator) partition?

Moved the comment and expanded it some.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800186238

From wkemper at openjdk.org  Mon Oct 14 22:22:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Oct 2024 22:22:41 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
Message-ID: <Uc6blop6Nw4Mjij98YkDhflS-K_R5oVQLTEp2OsL45c=.79c929df-a04e-4618-ada0-82db34ef3598@github.com>

On Mon, 14 Oct 2024 21:21:33 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Improve comments for new methods
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 899:
> 
>> 897: }
>> 898: 
>> 899: HeapWord* ShenandoahFreeSet::try_allocate_from_mutator(ShenandoahAllocRequest& req, bool& in_new_region) {
> 
> Why does this always use right-biased allocation from the mutator partition? (And not update allocation bias state?) Is this intentional? I assume it's because these aren't fresh allocations, but evacuations. Do evacuations with right bias end up closer to allocations for other evacuations? A brief comment explaining this difference from mutator allocation would be useful.

I think it makes sense that the collector would take regions from the mutator according to the allocation bias for the _collector_. @kdnilsen , what do you think?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1800188425

From wkemper at openjdk.org  Mon Oct 14 22:28:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Oct 2024 22:28:42 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v3]
In-Reply-To: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
Message-ID: <-irLV__bWna_S9g_ViKJemxCGM-FNixrIgxawjK08zg=.f9462d04-187c-4038-8fc8-feb0ea1882ec@github.com>

> Prune switch statement which has grown unwieldy.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Review feedback

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/511/files
  - new: https://git.openjdk.org/shenandoah/pull/511/files/8156aad2..86d75325

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=511&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=511&range=01-02

  Stats: 10 lines in 2 files changed: 2 ins; 1 del; 7 mod
  Patch: https://git.openjdk.org/shenandoah/pull/511.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/511/head:pull/511

PR: https://git.openjdk.org/shenandoah/pull/511

From ysr at openjdk.org  Mon Oct 14 22:55:09 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 14 Oct 2024 22:55:09 GMT
Subject: RFR: 8342081: Shenandoah: Remove extra
 ShenandoahMarkUpdateRefsSuperClosure
In-Reply-To: <btbf6MorZ7rEPf0UQBMVMeiUnPsUSpBapEhNHI4SldY=.452c9730-7048-43aa-9253-36797ecc6e09@github.com>
References: <btbf6MorZ7rEPf0UQBMVMeiUnPsUSpBapEhNHI4SldY=.452c9730-7048-43aa-9253-36797ecc6e09@github.com>
Message-ID: <W_FxGYzL5R4mvNEKwRP50G1zv9erkdkdt8jtqArJ-mY=.3fb768a0-fd50-4611-95bf-cf357dfa0b45@github.com>

On Mon, 14 Oct 2024 19:20:08 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `ShenandoahMarkUpdateRefsSuperClosure` has only one real subclass, which is used in STW mark. We can collapse the class hierarchy a bit here. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

lgtm

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21505#pullrequestreview-2367773398

From mli at openjdk.org  Tue Oct 15 08:09:36 2024
From: mli at openjdk.org (Hamlin Li)
Date: Tue, 15 Oct 2024 08:09:36 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v23]
In-Reply-To: <cL1TRIR8QmiIxhVU8LmxcEjUcneCNyLOh7ik4fgmIcM=.abc5dea2-6d92-4b6e-9e3f-01137441d02e@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <0BrAbBTKmpqTGDrc--2znzO8t07yoqabwa6g2K05GHI=.d3c17fd5-4770-4623-8d2f-604816afc033@github.com>
 <cL1TRIR8QmiIxhVU8LmxcEjUcneCNyLOh7ik4fgmIcM=.abc5dea2-6d92-4b6e-9e3f-01137441d02e@github.com>
Message-ID: <G60MN3iR1XqQsIpQ-u-TUb5UK7DmCkOOIZ8_UnTgS64=.d0da6080-2991-4578-af68-453ffc63f2ef@github.com>

On Thu, 19 Sep 2024 15:01:26 GMT, Hamlin Li <mli at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'lilliput/JEP-450-temporary-fix-branch-2' into JDK-8305895-v4
>>  - review feedback
>
> In both aarch64.ad and x86_64.ad, `MachUEPNode::format` might need some change accordingly?

> AFAIK, @Hamlin-Li is working on the RISCV port. Not sure who would do s390. If it's available before intergration, I'll include it, but I'll not wait for it.

Thanks! We're having some internal (riscv specific) discussion & review, should be able to provide the patch soon.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2413178574

From amitkumar at openjdk.org  Tue Oct 15 08:14:30 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Tue, 15 Oct 2024 08:14:30 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v39]
In-Reply-To: <LjXNg7ygXw6KlgE0O5mMhgJ2IpNSpcZV1VKvF-p3qdQ=.608e20a4-25a8-4b9d-acaa-39eead7c4617@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <q4sTPbMm-cK1O35GhIbRytiakYgyKJYrBhotG_Kwn2o=.02620a61-e96d-4c93-af80-d27a61dd9ee7@github.com>
 <LjXNg7ygXw6KlgE0O5mMhgJ2IpNSpcZV1VKvF-p3qdQ=.608e20a4-25a8-4b9d-acaa-39eead7c4617@github.com>
Message-ID: <ay-vYduVhGNK9HQodtCnP9Awa_NXaGRtGYOh55KqgxU=.a4ba42c8-eafa-4517-87a8-a92bcbc398be@github.com>

On Mon, 14 Oct 2024 21:47:00 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:

>@offamitkumar: It could still be done after this PR is integrated, but I guess you want to provide an s390 implementation.

I haven't looked into it yet. I am looking into other issues for now, but I will if I can get time to work on this.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2413190779

From rkennke at openjdk.org  Tue Oct 15 08:46:57 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 15 Oct 2024 08:46:57 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v40]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <LMXcf9mNlOcl32Ii2GjKD3MU85-5ShuLCQ_R_53tKB8=.1f6fd73c-99a2-4c70-a925-55aa52904d9e@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  PPC64 implementation of Compact Object Headers (JEP 450)

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/b289ef88..6722f8be

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=39
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=38-39

  Stats: 161 lines in 9 files changed: 95 ins; 39 del; 27 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From shade at openjdk.org  Tue Oct 15 08:49:10 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 15 Oct 2024 08:49:10 GMT
Subject: RFR: 8342044: Increase timeout of
 gc/shenandoah/oom/TestClassLoaderLeak.java
In-Reply-To: <vacjtHaJHS4QuyVB26411a-8BYXpiRfmC_xjpZJfz7I=.567b306b-1376-4a18-903d-5581f1a2212f@github.com>
References: <vacjtHaJHS4QuyVB26411a-8BYXpiRfmC_xjpZJfz7I=.567b306b-1376-4a18-903d-5581f1a2212f@github.com>
Message-ID: <938a5y4eZ2YjOq3EzYlOZ0Kh9a_GqWR40PEQ43o5UjY=.f46cbd4b-846d-4473-a253-79995fcd9b0c@github.com>

On Mon, 14 Oct 2024 11:40:17 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> Test gc/shenandoah/oom/TestClassLoaderLeak.java often times out on WIndows, we should increase the timeout values to avoid test errors.

Marked as reviewed by shade (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21491#pullrequestreview-2368591927

From rkennke at openjdk.org  Tue Oct 15 09:02:16 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 15 Oct 2024 09:02:16 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v41]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <JBrhBOknUeYekenNA45sQtigIIHUEQaKpCvdBIZVdeA=.73bbcac9-af12-4672-92e0-cbe9b6921490@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 90 commits:

 - Merge tag 'jdk-24+19' into JDK-8305895-v4
   
   Added tag jdk-24+19 for changeset e7c5bf45
 - PPC64 implementation of Compact Object Headers (JEP 450)
 - Increase compiler code stubs size for indexOf intrinsic
 - Fix include guards
 - Improve PSParallelCompact::fill_dense_prefix_end() even more
 - Re-enable indexOf intrinsic for compact headers
 - Rename nklass in aarch64
 - Fix comment
 - Rename nklass in x86 code
 - Fix header < 16 bytes in indexOf intrinsic, by @sviswa7
 - ... and 80 more: https://git.openjdk.org/jdk/compare/e7c5bf45...86f94fee

-------------

Changes: https://git.openjdk.org/jdk/pull/20677/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=40
  Stats: 4865 lines in 205 files changed: 3383 ins; 818 del; 664 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Tue Oct 15 10:47:55 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 15 Oct 2024 10:47:55 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v42]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <5x48SX55xY_BRxqqcTTvGp_ocrKDH7t5VuJY-MDQuTA=.eed6083d-e2dc-4888-a2d5-b6934f098289@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Fix aarch64.ad

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/86f94fee..005498b1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=41
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=40-41

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From tschatzl at openjdk.org  Tue Oct 15 11:28:31 2024
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Tue, 15 Oct 2024 11:28:31 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v8]
In-Reply-To: <6usTXIvS83aO2VzX5xu2EnXlpIJ8YbfrWS6b3EI0MhE=.0e8cc603-0cd3-4bd9-b309-55e4dd0f0cb0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <J0g0PSye768dhN3JoXFsZ6FivocNBLPmHNtRUCOmLTw=.5756c64a-4763-4727-9637-ba3fe7b9b17d@github.com>
 <6usTXIvS83aO2VzX5xu2EnXlpIJ8YbfrWS6b3EI0MhE=.0e8cc603-0cd3-4bd9-b309-55e4dd0f0cb0@github.com>
Message-ID: <N0f3YokOST3MTXR_iRwPjQGBEz15Pb9KB1zffJJwFuI=.d7ae3343-2fdd-43e0-855d-de8975603732@github.com>

On Mon, 9 Sep 2024 11:53:13 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Try to avoid lea in loadNklass (aarch64)
>>  - Fix release build error
>
> src/hotspot/share/oops/klass.hpp line 169:
> 
>> 167:                                 // contention that may happen when a nearby object is modified.
>> 168:   AccessFlags _access_flags;    // Access flags. The class/interface distinction is stored here.
>> 169:                                 // Some flags created by the JVM, not in the class file itself,
> 
> Suggestion:
> 
>   markWord _prototype_header;   // Used to initialize objects' header with compact headers.
> 
> 
> Maybe some comment why this is an instance member.

>@tschatzl I just found your comment here, and I'm not sure what you mean, tbh. The prototype_header is a member of Klass because with compact headers, it encodes that Klass in the prototype header. Note that there is planned follow-up work to remove that field and encode the Klass* on the allocation path. https://bugs.openjdk.org/browse/JDK-8341703

You explained what I had wanted to see here - why do we need a per-klass prototype header, because the markWord contains it ;)
Given that it is going away, I retract this comment and the request can be resolved.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1800983876

From mbaesken at openjdk.org  Tue Oct 15 11:55:15 2024
From: mbaesken at openjdk.org (Matthias Baesken)
Date: Tue, 15 Oct 2024 11:55:15 GMT
Subject: RFR: 8342044: Increase timeout of
 gc/shenandoah/oom/TestClassLoaderLeak.java
In-Reply-To: <vacjtHaJHS4QuyVB26411a-8BYXpiRfmC_xjpZJfz7I=.567b306b-1376-4a18-903d-5581f1a2212f@github.com>
References: <vacjtHaJHS4QuyVB26411a-8BYXpiRfmC_xjpZJfz7I=.567b306b-1376-4a18-903d-5581f1a2212f@github.com>
Message-ID: <ST9FMtBtLyorP8fJyHxdKdWi9gptRsEruvilJV-XFkQ=.662a2a43-4b0e-49eb-ab41-fcf4753e1d44@github.com>

On Mon, 14 Oct 2024 11:40:17 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> Test gc/shenandoah/oom/TestClassLoaderLeak.java often times out on WIndows, we should increase the timeout values to avoid test errors.

Thanks for the review !

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21491#issuecomment-2413700365

From mbaesken at openjdk.org  Tue Oct 15 11:55:16 2024
From: mbaesken at openjdk.org (Matthias Baesken)
Date: Tue, 15 Oct 2024 11:55:16 GMT
Subject: Integrated: 8342044: Increase timeout of
 gc/shenandoah/oom/TestClassLoaderLeak.java
In-Reply-To: <vacjtHaJHS4QuyVB26411a-8BYXpiRfmC_xjpZJfz7I=.567b306b-1376-4a18-903d-5581f1a2212f@github.com>
References: <vacjtHaJHS4QuyVB26411a-8BYXpiRfmC_xjpZJfz7I=.567b306b-1376-4a18-903d-5581f1a2212f@github.com>
Message-ID: <gsqPgg4BHeLmUEFXafmUElIrn032Wlw1RrS82NYG3QI=.893163ec-6c22-41d5-b5c1-d5a8354745f0@github.com>

On Mon, 14 Oct 2024 11:40:17 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> Test gc/shenandoah/oom/TestClassLoaderLeak.java often times out on WIndows, we should increase the timeout values to avoid test errors.

This pull request has now been integrated.

Changeset: 2c0c6535
Author:    Matthias Baesken <mbaesken at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/2c0c65353b2f67bdcd954b4d2c2ae3e9b24d1c22
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8342044: Increase timeout of gc/shenandoah/oom/TestClassLoaderLeak.java

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/jdk/pull/21491

From shade at openjdk.org  Tue Oct 15 13:03:44 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 15 Oct 2024 13:03:44 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
Message-ID: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>

We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 

Additional testing:
 - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
 - [ ] Performance tests (no regressions)

-------------

Commit messages:
 - Fix

Changes: https://git.openjdk.org/jdk/pull/21517/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21517&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342119
  Stats: 37 lines in 5 files changed: 6 ins; 24 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/21517.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21517/head:pull/21517

PR: https://git.openjdk.org/jdk/pull/21517

From kdnilsen at openjdk.org  Tue Oct 15 17:37:40 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 15 Oct 2024 17:37:40 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <Uc6blop6Nw4Mjij98YkDhflS-K_R5oVQLTEp2OsL45c=.79c929df-a04e-4618-ada0-82db34ef3598@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
 <Uc6blop6Nw4Mjij98YkDhflS-K_R5oVQLTEp2OsL45c=.79c929df-a04e-4618-ada0-82db34ef3598@github.com>
Message-ID: <BH5BdFZcK58GScDUPilyv5ouEnMEt8lnzFOVIMrQ_9o=.0b847aaf-e870-4e1a-8757-31157908adbd@github.com>

On Mon, 14 Oct 2024 22:19:37 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 899:
>> 
>>> 897: }
>>> 898: 
>>> 899: HeapWord* ShenandoahFreeSet::try_allocate_from_mutator(ShenandoahAllocRequest& req, bool& in_new_region) {
>> 
>> Why does this always use right-biased allocation from the mutator partition? (And not update allocation bias state?) Is this intentional? I assume it's because these aren't fresh allocations, but evacuations. Do evacuations with right bias end up closer to allocations for other evacuations? A brief comment explaining this difference from mutator allocation would be useful.
>
> I think it makes sense that the collector would take regions from the mutator according to the allocation bias for the _collector_. @kdnilsen , what do you think?

When the collector is trying to allocate from the "mutator partition", it biases towards high end of memory because our general goal is long-lived data (except humongous objects which live at the low-end of memory) lives at high end of memory.  Survivor regions (regions allocated by the collector) are assumed to be longer lived than ephemeral objects (regions allocated by the mutator).

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1801640062

From kdnilsen at openjdk.org  Tue Oct 15 17:43:45 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 15 Oct 2024 17:43:45 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v3]
In-Reply-To: <-irLV__bWna_S9g_ViKJemxCGM-FNixrIgxawjK08zg=.f9462d04-187c-4038-8fc8-feb0ea1882ec@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <-irLV__bWna_S9g_ViKJemxCGM-FNixrIgxawjK08zg=.f9462d04-187c-4038-8fc8-feb0ea1882ec@github.com>
Message-ID: <Q9JSydz-OwpmrPMU-YuKJ5SZA1wUpsci-OR_x_XW9nA=.e8e40eb6-f2b7-49e7-b47f-2df11d57e234@github.com>

On Mon, 14 Oct 2024 22:28:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Prune switch statement which has grown unwieldy.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Review feedback

Thanks.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/511#pullrequestreview-2370133361

From ysr at openjdk.org  Tue Oct 15 18:00:41 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Oct 2024 18:00:41 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v3]
In-Reply-To: <-irLV__bWna_S9g_ViKJemxCGM-FNixrIgxawjK08zg=.f9462d04-187c-4038-8fc8-feb0ea1882ec@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <-irLV__bWna_S9g_ViKJemxCGM-FNixrIgxawjK08zg=.f9462d04-187c-4038-8fc8-feb0ea1882ec@github.com>
Message-ID: <NcZ1iaWM8IDe0k8BYSErBy5HsUDkCEgY8Sa7fFugH8U=.22c11d3a-56c4-4585-9b6e-bbcbb3652139@github.com>

On Mon, 14 Oct 2024 22:28:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Prune switch statement which has grown unwieldy.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Review feedback

Reviwed; ?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/511#pullrequestreview-2370175839

From ysr at openjdk.org  Tue Oct 15 18:00:41 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Oct 2024 18:00:41 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <e0lQC0yv3JKW2xD7SgxcLyfYcd_yaBV8w2gWtyzOYaU=.0b6fe843-9e73-4831-a01a-018e7eae574c@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
 <e0lQC0yv3JKW2xD7SgxcLyfYcd_yaBV8w2gWtyzOYaU=.0b6fe843-9e73-4831-a01a-018e7eae574c@github.com>
Message-ID: <cPvPUJbwX2-I0rP9qTq-m12jcKg2SffT99PcnALZnZI=.ee7ace0d-aa5f-4b59-818e-5f7bd20e3e27@github.com>

On Mon, 14 Oct 2024 22:11:34 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 762:
>> 
>>> 760: 
>>> 761: HeapWord* ShenandoahFreeSet::allocate_for_mutator(ShenandoahAllocRequest &req, bool &in_new_region) {
>>> 762:   maybe_change_allocation_bias();
>> 
>> Why update alloc bias if allocation is doomed to fail at next line? Is that the intention? If not, may be do the bias update only when there is chance for success in the allocation? See suggestions further below.
>
> I had this same thought, but ended on the side of preserving existing behavior. @kdnilsen , should we not update the allocation bias if we can detect an early allocation failure?

ok

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 799:
>> 
>>> 797:                                 - _partitions.rightmost_empty(ShenandoahFreeSetPartitionId::Mutator));
>>> 798:     _partitions.set_bias_from_left_to_right(ShenandoahFreeSetPartitionId::Mutator, (non_empty_on_right < non_empty_on_left));
>>> 799:     _alloc_bias_weight = _InitialAllocBiasWeight;
>> 
>> The value of 256 for `_InitialAllocBiasWeight` seems a bit of magic. How was it chosen, and is it subject to bit-rot with code evolving? I can imagine that the bias update at a higher frequency is apt to not matter much, and since the leftmost and rightmost empty calculation is not free, so larger values are favored.
>> 
>> Also should we use the more usual `static const ssize_t INITIAL_ALLOC_BIAS_WEIGHT` Hotspot style for constants, rather than the camel case with an initial underscore (which seems not common)?
>
> I'm not sure where the initial value comes from. I will change to the idiomatic name convention.

ok

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1801672424
PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1801672577

From ysr at openjdk.org  Tue Oct 15 18:00:42 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Oct 2024 18:00:42 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <Q7MSHEmR-0FFkD8eYKPJr28hUow5odDYMjQTRpnJXkA=.cb5fc819-6de8-452a-977b-8861d866ecc5@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
 <6_pZ_HkM8cBs7a7XME6bh3GTW5GQTIs06ytGgeHIQu0=.223e4471-f6cc-4176-83b3-807b977e3bb0@github.com>
 <Q7MSHEmR-0FFkD8eYKPJr28hUow5odDYMjQTRpnJXkA=.cb5fc819-6de8-452a-977b-8861d866ecc5@github.com>
Message-ID: <8TLkEN-HvqJJb3RtqQLaqIhJbdx9ejq_wojMO_c-b1k=.8c326191-cabe-40f6-b401-77bbcddff392@github.com>

On Mon, 14 Oct 2024 22:09:25 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> I also wonder if `allocate_from_left` etc., since they seem to involve only mutator allocations, should include `mutator` somewhere in the name of the method; if so, I wonder about why we maintain the bias variable for the other partitions? (I suppose we don't but since each is a partition and the variable is associated with each partition, it's there. If we don't think we'll need  bias for other allocations, it might make sense to move it to the free set, but I'd hold off on that until it's clear that directional bias for allocation isn't going to be useful for collector allocations.
>
> We do also use the allocation bias for collector allocations. I have a deeper refactoring related to this allocation bias here: https://github.com/openjdk/shenandoah/pull/512, but I've left it in draft status pending pipeline test results.

ok. Yes, not immediately necessary. Thanks.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1801673018

From ysr at openjdk.org  Tue Oct 15 18:00:42 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Oct 2024 18:00:42 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v2]
In-Reply-To: <BH5BdFZcK58GScDUPilyv5ouEnMEt8lnzFOVIMrQ_9o=.0b847aaf-e870-4e1a-8757-31157908adbd@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
 <c7P37xonREafh_uDgst9BmIJVbusua98Ez9rEgA1xBk=.1ad66df1-2a0b-4db7-8714-69910ec3b730@github.com>
 <aOS-O-3gAOeXcGWWS5He8uuV6s0Vp9GdKWltQfIHMhM=.bda18be8-0787-4cfe-aae8-b1e2b8b82686@github.com>
 <Uc6blop6Nw4Mjij98YkDhflS-K_R5oVQLTEp2OsL45c=.79c929df-a04e-4618-ada0-82db34ef3598@github.com>
 <BH5BdFZcK58GScDUPilyv5ouEnMEt8lnzFOVIMrQ_9o=.0b847aaf-e870-4e1a-8757-31157908adbd@github.com>
Message-ID: <OhZSty-yi3LiTrGTSmUAVlRpRAA1zW9oqf6JBo2kp5s=.ac180482-10b4-4359-9334-e3cdf61a159d@github.com>

On Tue, 15 Oct 2024 17:35:04 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> I think it makes sense that the collector would take regions from the mutator according to the allocation bias for the _collector_. @kdnilsen , what do you think?
>
> When the collector is trying to allocate from the "mutator partition", it biases towards high end of memory because our general goal is long-lived data (except humongous objects which live at the low-end of memory) lives at high end of memory.  Survivor regions (regions allocated by the collector) are assumed to be longer lived than ephemeral objects (regions allocated by the mutator).

Makes sense. A single line comment to that effect would be great.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/511#discussion_r1801671949

From wkemper at openjdk.org  Tue Oct 15 18:08:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 18:08:13 GMT
Subject: RFR: 8342001: GenShen: Factor cases for allocation type into
 separate methods [v4]
In-Reply-To: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
Message-ID: <O9jvOOJDOmjP0lGi5BiuMvuMNIf_IXinOog464WJQGI=.9de4960b-d16f-4f0c-b8ec-9a8c42867b55@github.com>

> Prune switch statement which has grown unwieldy.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Add comment explaining why collector perfers to steal regions from the right

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/511/files
  - new: https://git.openjdk.org/shenandoah/pull/511/files/86d75325..8ed69adc

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=511&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=511&range=02-03

  Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/511.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/511/head:pull/511

PR: https://git.openjdk.org/shenandoah/pull/511

From wkemper at openjdk.org  Tue Oct 15 18:08:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 18:08:14 GMT
Subject: Integrated: 8342001: GenShen: Factor cases for allocation type into
 separate methods
In-Reply-To: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
References: <E3a_qKx7PxPMGH2-VdwkcvljbRADVojnzqa3DSzfNmc=.20bdc121-2ace-425d-9379-1ed02e6a0338@github.com>
Message-ID: <OVdx9-VbKMbYVl2wZlz5Xxdmdo9tyz6R97lUlZMHnME=.1639852e-9f44-4309-952d-56cc5a4bf266@github.com>

On Fri, 11 Oct 2024 23:32:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Prune switch statement which has grown unwieldy.

This pull request has now been integrated.

Changeset: 81b631fb
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/81b631fbc7fc7078786b11c7a85971735ce93b86
Stats:     294 lines in 2 files changed: 140 ins; 106 del; 48 mod

8342001: GenShen: Factor cases for allocation type into separate methods

Reviewed-by: ysr, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/511

From wkemper at openjdk.org  Tue Oct 15 18:31:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 18:31:00 GMT
Subject: RFR: 8342214: GenShen: Reduce code duplication in shFreeSet with
 iterator abstraction
Message-ID: <cW56kShFSmK-x3nahU1gim1F1pu9wXcgNDWWAvBVbPI=.495beaca-8199-4009-9282-48b9ec28edb2@github.com>

Introduce `ShenandoahLeftRightIterator` and `ShenandoahRightLeftIterator` classes to encapsulate allocation bias to one side of the heap or the other.

-------------

Commit messages:
 - Use iterator abstraction to reduce code duplication

Changes: https://git.openjdk.org/shenandoah/pull/512/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=512&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342214
  Stats: 195 lines in 2 files changed: 80 ins; 69 del; 46 mod
  Patch: https://git.openjdk.org/shenandoah/pull/512.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/512/head:pull/512

PR: https://git.openjdk.org/shenandoah/pull/512

From wkemper at openjdk.org  Tue Oct 15 18:43:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 18:43:42 GMT
Subject: Integrated: 8336742: Shenandoah: Add more verbose logging/stats for
 mark termination attempts
In-Reply-To: <2lnMM7o93MzXdYywLcaeRQXMG9F-jBfIG9TXRmjvXjc=.07d72235-1203-4f9e-8661-557cb425a6da@github.com>
References: <2lnMM7o93MzXdYywLcaeRQXMG9F-jBfIG9TXRmjvXjc=.07d72235-1203-4f9e-8661-557cb425a6da@github.com>
Message-ID: <WWvHdTlNBI9aZuSBHVpysWsOlnl9-RllRi07_jzczek=.926a23e8-a6b6-4f72-a1e0-5a5ffc723a46@github.com>

On Fri, 11 Oct 2024 19:01:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Not clean.

This pull request has now been integrated.

Changeset: 520fb4ef
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/520fb4efc7da089b4c9243013a55d22b3bdb5243
Stats:     23 lines in 5 files changed: 10 ins; 1 del; 12 mod

8336742: Shenandoah: Add more verbose logging/stats for mark termination attempts

Backport-of: 90527a57848f452be3be089a703cbc2af2d1657a

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/126

From wkemper at openjdk.org  Tue Oct 15 20:30:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 20:30:46 GMT
Subject: RFR: 8336914: Shenandoah: Missing verification steps after JDK-8255765
Message-ID: <Za7PsfamuZfieUO0M-6L6jBNsZ4FSd8QUDCouHHMIw8=.941b7b08-6d89-4a57-b979-49d15bda2de6@github.com>

Almost clean.

-------------

Commit messages:
 - Backport e51e40c2b9f51d012c01407e0b8dadaab464753e

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/127/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=127&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8336914
  Stats: 3 lines in 1 file changed: 2 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/127.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/127/head:pull/127

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/127

From wkemper at openjdk.org  Tue Oct 15 20:34:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 20:34:47 GMT
Subject: RFR: 8338688: Shenandoah: Avoid calling java_lang_Class accessors in
 asserts/verifier
Message-ID: <ue3bwIs7rBnelqtzVXDatNPsH7SygWjPSemP4Cn7vAQ=.8ae88d37-ed25-4f65-8920-91ebf92d5633@github.com>

Trivial to resolve.

-------------

Commit messages:
 - Backport e297e8817f486e4af850c97fcff859c3e9a9e21c

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/128/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=128&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8338688
  Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/128.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/128/head:pull/128

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/128

From wkemper at openjdk.org  Tue Oct 15 20:49:24 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 20:49:24 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <OrqMuPkkk9F5W2zYe_QYaKzm-pJD9U_Y8knJHqzzN-Y=.3abef30c-03b3-4bc0-a0ca-aed4a54617fc@github.com>
Message-ID: <7Y0f1A161C8p_W13beNZEsIGywt8i5JQz9LS-LQabQ4=.cb780e1a-7519-4a9b-9c63-c65898de6749@github.com>

On Thu, 10 Oct 2024 12:58:56 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 96:
> 
>> 94:   size_t capacity    = _space_info->soft_max_capacity();
>> 95:   size_t max_cset    = (size_t)((1.0 * capacity / 100 * ShenandoahEvacReserve) / ShenandoahEvacWaste);
>> 96:   size_t free_target = (capacity * ShenandoahMinFreeThreshold) / 100 + max_cset;
> 
> Does re-arranging the math here risk overflow (e.g. on 32-bit)?

https://bugs.openjdk.org/browse/JDK-8342239

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1801956645

From wkemper at openjdk.org  Tue Oct 15 20:51:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 20:51:08 GMT
Subject: RFR: 8342239: GenShen: Revert changes in adaptive heuristic to avoid
 overflow on 32 bit
Message-ID: <qpNQOncvdl4bL2icPHX1d56rsiobbfJ2MiOINx5pDjA=.bc6700da-beb4-4afb-bfcb-7fa5da8783bb@github.com>

One of our changes to increase precision comes with the risk of overflow on 32 bit platforms. We're going to revert it for safety's sake.

-------------

Commit messages:
 - Avoid overflow on 32bit platforms

Changes: https://git.openjdk.org/shenandoah/pull/513/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=513&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342239
  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/513.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/513/head:pull/513

PR: https://git.openjdk.org/shenandoah/pull/513

From wkemper at openjdk.org  Tue Oct 15 20:54:18 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 20:54:18 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <V6vSlviRiPnhKtFAFhl19jMWyTRUCTjTwp9gp42TgyE=.7b5da064-1d5e-4f22-a3fb-fa5c6cd0d9a7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <BqVG4w78ZKCAL7acQX42dPJwjDuAYv41pZnzwgupv1M=.69f4f8a4-1262-4002-a58f-ed9c5660772c@github.com>
 <V6vSlviRiPnhKtFAFhl19jMWyTRUCTjTwp9gp42TgyE=.7b5da064-1d5e-4f22-a3fb-fa5c6cd0d9a7@github.com>
Message-ID: <Ii8-wuHr3qir9XW9Nb4Kv18M1Dh4d_fbmo1IGZC0NmA=.5f174cf3-038a-4bd5-84d2-4b5bda54c2ff@github.com>

On Fri, 11 Oct 2024 22:12:25 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahCardStats.hpp line 40:
>> 
>>> 38:   DIRTY_SCAN_OBJS     = 6,
>>> 39:   ALTERNATIONS        = 7,
>>> 40:   MAX_CARD_STAT_TYPE  = 8
>> 
>> Are the numerical values relevant or what is the reason to spell them out?
>
> Not needed; explicit enumeration values can be removed.

https://github.com/openjdk/shenandoah/pull/514

>> src/hotspot/share/gc/shenandoah/shenandoahCardStats.hpp line 46:
>> 
>>> 44:   CARD_STAT_SCAN_RS       = 0,
>>> 45:   CARD_STAT_UPDATE_REFS   = 1,
>>> 46:   MAX_CARD_STAT_LOG_TYPE  = 2
>> 
>> Same here?
>
> Yes, same; explicit enumeration values can be removed.

https://github.com/openjdk/shenandoah/pull/514

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1801975222
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1801975588

From wkemper at openjdk.org  Tue Oct 15 20:56:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 20:56:12 GMT
Subject: RFR: 8342255: GenShen: Remove unnecessary enum initial values
Message-ID: <z1Tw-WheUODYEQplzaaIy-jgbx1JgZScwQphXkvuMww=.02b643c8-479b-4f17-97bc-a1c350ba8023@github.com>

C++ does not require explicit initial values for enumeration members.

-------------

Commit messages:
 - Remove unnecessary values for enumeration

Changes: https://git.openjdk.org/shenandoah/pull/514/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=514&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342255
  Stats: 12 lines in 1 file changed: 0 ins; 0 del; 12 mod
  Patch: https://git.openjdk.org/shenandoah/pull/514.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/514/head:pull/514

PR: https://git.openjdk.org/shenandoah/pull/514

From wkemper at openjdk.org  Tue Oct 15 21:06:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 21:06:54 GMT
Subject: RFR: 8342278: GenShen: Move non-generational mode test out of
 generational test configuration
Message-ID: <joOkZ4SECXheIBJtzuQTUi_Y0AKAGx2poOc6Kcj2leM=.5a771fea-adb0-4df8-b98c-cc8b7a3d51df@github.com>

Test has unintentionally configured a non-generational mode test to run under the generational test section.

-------------

Commit messages:
 - Put non-generational mode test config under a separate test id

Changes: https://git.openjdk.org/shenandoah/pull/515/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=515&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342278
  Stats: 6 lines in 1 file changed: 5 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/515.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/515/head:pull/515

PR: https://git.openjdk.org/shenandoah/pull/515

From wkemper at openjdk.org  Tue Oct 15 22:41:10 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 22:41:10 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
In-Reply-To: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
Message-ID: <HEQ6h3zNDOMoc-POIX6N7tCeea3qNVfa-zALiBt28os=.78e02c03-047f-4e2b-b995-5bc4964d9007@github.com>

On Tue, 15 Oct 2024 08:53:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>  - [ ] Performance tests (no regressions)

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21517#pullrequestreview-2370816667

From duke at openjdk.org  Tue Oct 15 22:44:35 2024
From: duke at openjdk.org (Volodymyr Paprotski)
Date: Tue, 15 Oct 2024 22:44:35 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v42]
In-Reply-To: <5x48SX55xY_BRxqqcTTvGp_ocrKDH7t5VuJY-MDQuTA=.eed6083d-e2dc-4888-a2d5-b6934f098289@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <5x48SX55xY_BRxqqcTTvGp_ocrKDH7t5VuJY-MDQuTA=.eed6083d-e2dc-4888-a2d5-b6934f098289@github.com>
Message-ID: <zsXCn2nynPEIqUg3XFoI_UV_VUoIGyEmvPr2nRwkUiQ=.a3ff867d-cdb3-4a2a-bbcd-69065e3739bb@github.com>

On Tue, 15 Oct 2024 10:47:55 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix aarch64.ad

Finished reviewing `src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp`, line by line and comparing old snippets that got merged into the new function: looks good to me, every (new) case handled 

Only have some minor comments about comments.

src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp line 414:

> 412:   // to the valid haystack bytes on the stack.
> 413:   {
> 414:     const Register haystack = rbx;

Keep `rax` as index for clarity? Although it is really used as a temp..


const Register index = rax;
const Register haystack = rbx;
copy_to_stack(haystack, haystack_len, false, index , XMM_TMP1, _masm);

src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp line 1568:

> 1566:   assert((COPIED_HAYSTACK_STACK_SIZE == 64), "Must be 64!");
> 1567: 
> 1568:   // Copy incoming haystack onto stack

Old comment was slightly more precise. Move here. i.e.
`// Copy incoming haystack onto stack (haystack <= 32 bytes)`

src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp line 1634:

> 1632: 
> 1633: 
> 1634: // Copy the small (< 32 byte) haystack to the stack.  Allows for vector reads without page fault

Just to be pedantic, its `(<=32)` - this function also handles 32bytes case.

-  line 401: 

 __ cmpq(haystack_len, 0x20);
 __ ja(L_bigSwitchTop);

- though line 293 (`highly_optimized_short_cases`) only seems to route16-byte cases here: 
```__ cmpq(haystack_len_p, isU ? 8 : 16);```

src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp line 1659:

> 1657:   Label L_moreThan8, L_moreThan16, L_moreThan24, L_adjustHaystack;
> 1658: 
> 1659:   assert(arrayOopDesc::base_offset_in_bytes(isU ? T_CHAR : T_BYTE) >= 8,

If we had to also optimize for header-size 16, it might be possible to remove one jump here. Looks correct for either size.

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2370735887
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1802041876
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1802044880
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1802088545
PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1802073195

From wkemper at openjdk.org  Tue Oct 15 22:58:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 22:58:45 GMT
Subject: RFR: 8336914: Shenandoah: Missing verification steps after
 JDK-8255765
In-Reply-To: <Za7PsfamuZfieUO0M-6L6jBNsZ4FSd8QUDCouHHMIw8=.941b7b08-6d89-4a57-b979-49d15bda2de6@github.com>
References: <Za7PsfamuZfieUO0M-6L6jBNsZ4FSd8QUDCouHHMIw8=.941b7b08-6d89-4a57-b979-49d15bda2de6@github.com>
Message-ID: <8ex2Don9QoJDLS4D_B7_NDxO_jHNKgP6l4Wz43CVOqY=.0b6c781b-9e15-40a9-8a45-07e95c683f10@github.com>

On Tue, 15 Oct 2024 20:26:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Almost clean.

This change (in a slightly different form) is already here.

-------------

PR Comment: https://git.openjdk.org/shenandoah-jdk21u/pull/127#issuecomment-2415307012

From wkemper at openjdk.org  Tue Oct 15 22:58:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 22:58:46 GMT
Subject: Withdrawn: 8336914: Shenandoah: Missing verification steps after
 JDK-8255765
In-Reply-To: <Za7PsfamuZfieUO0M-6L6jBNsZ4FSd8QUDCouHHMIw8=.941b7b08-6d89-4a57-b979-49d15bda2de6@github.com>
References: <Za7PsfamuZfieUO0M-6L6jBNsZ4FSd8QUDCouHHMIw8=.941b7b08-6d89-4a57-b979-49d15bda2de6@github.com>
Message-ID: <Cem0eJb2o3EYYlpjpa4KdAPH8IUGEU4sam4DXaUNMsw=.63910c12-30ad-450b-a2d6-14250ab88035@github.com>

On Tue, 15 Oct 2024 20:26:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Almost clean.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/127

From wkemper at openjdk.org  Tue Oct 15 23:03:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 23:03:21 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <_1_RPbV4VVoczxDhQYp967iOM37EHh3ZcK5b8dvKrQU=.1b9556a6-81d3-428d-8593-bd4cdced44de@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <_1_RPbV4VVoczxDhQYp967iOM37EHh3ZcK5b8dvKrQU=.1b9556a6-81d3-428d-8593-bd4cdced44de@github.com>
Message-ID: <1FBHl-vUAvyx5tv2VG7jM86Lx5DM1aaIgE9-jTNv-9s=.b68703f8-f9bf-4952-9e1f-9da7a68bb7de@github.com>

On Fri, 11 Oct 2024 13:43:46 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> test/hotspot/jtreg/gc/shenandoah/TestAllocIntArrays.java line 100:
> 
>> 98: 
>> 99: /*
>> 100:  * @test id=generational
> 
> You are making a test configuration and call it 'generational' but then one of the two run configurations doesn't actually run with generational mode. You probably want to separate the two?

https://bugs.openjdk.org/browse/JDK-8342278

> test/hotspot/jtreg/gc/shenandoah/TestAllocIntArrays.java line 163:
> 
>> 161:         final int min = 0;
>> 162:         final int max = 384 * 1024;
>> 163:         // Each allocated int array is assumed to consume 16 bytes for alignment and header, plus
> 
> With the upcoming 'compact headers' it's going to be only 12 bytes. If that matters, then the actual number should perhaps be a constant? Preexisting and not relevant for this PR, though.

Yes, not sure that level of precision is really required for this test either.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1802109516
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1802109841

From wkemper at openjdk.org  Tue Oct 15 23:04:55 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Oct 2024 23:04:55 GMT
Subject: RFR: 8338662: Shenandoah: Remove excessive
 ShenandoahVerifier::verify_during_evacuation
Message-ID: <hom8apBnmd0qQ-8gwPKB655wWKl_RxL77IC39GhTl3E=.5769684c-6d5d-4899-b4c0-063c22580e57@github.com>

Not clean, combined with a related change.

-------------

Commit messages:
 - Fix merge error
 - 8336915: Shenandoah: Remove unused ShenandoahVerifier::verify_after_evacuation
 - 8338662: Shenandoah: Remove excessive ShenandoahVerifier::verify_during_evacuation

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/129/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=129&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8338662
  Stats: 54 lines in 3 files changed: 0 ins; 52 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/129.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/129/head:pull/129

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/129

From shade at openjdk.org  Wed Oct 16 07:16:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 16 Oct 2024 07:16:14 GMT
Subject: RFR: 8342081: Shenandoah: Remove extra
 ShenandoahMarkUpdateRefsSuperClosure
In-Reply-To: <btbf6MorZ7rEPf0UQBMVMeiUnPsUSpBapEhNHI4SldY=.452c9730-7048-43aa-9253-36797ecc6e09@github.com>
References: <btbf6MorZ7rEPf0UQBMVMeiUnPsUSpBapEhNHI4SldY=.452c9730-7048-43aa-9253-36797ecc6e09@github.com>
Message-ID: <_FC-WFqdRik4X5-TqwcpV4-T-F507RyufWE_zB8u_fs=.72fd6dc2-97d9-4860-942d-8641a90abffd@github.com>

On Mon, 14 Oct 2024 19:20:08 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `ShenandoahMarkUpdateRefsSuperClosure` has only one real subclass, which is used in STW mark. We can collapse the class hierarchy a bit here. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21505#issuecomment-2415922052

From shade at openjdk.org  Wed Oct 16 07:16:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 16 Oct 2024 07:16:14 GMT
Subject: Integrated: 8342081: Shenandoah: Remove extra
 ShenandoahMarkUpdateRefsSuperClosure
In-Reply-To: <btbf6MorZ7rEPf0UQBMVMeiUnPsUSpBapEhNHI4SldY=.452c9730-7048-43aa-9253-36797ecc6e09@github.com>
References: <btbf6MorZ7rEPf0UQBMVMeiUnPsUSpBapEhNHI4SldY=.452c9730-7048-43aa-9253-36797ecc6e09@github.com>
Message-ID: <OLjaU7drwUVXwnx_ThPJSN72QzPak4b3Ep0oDHOCepc=.ec4aea67-b22e-478f-9869-f53e0c933293@github.com>

On Mon, 14 Oct 2024 19:20:08 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `ShenandoahMarkUpdateRefsSuperClosure` has only one real subclass, which is used in STW mark. We can collapse the class hierarchy a bit here. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

This pull request has now been integrated.

Changeset: 724de682
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/724de682091623cd9877ee4e5f13123ef1d92ddf
Stats:     22 lines in 2 files changed: 2 ins; 11 del; 9 mod

8342081: Shenandoah: Remove extra ShenandoahMarkUpdateRefsSuperClosure

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/jdk/pull/21505

From rkennke at openjdk.org  Wed Oct 16 09:01:34 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 16 Oct 2024 09:01:34 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v42]
In-Reply-To: <zsXCn2nynPEIqUg3XFoI_UV_VUoIGyEmvPr2nRwkUiQ=.a3ff867d-cdb3-4a2a-bbcd-69065e3739bb@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <5x48SX55xY_BRxqqcTTvGp_ocrKDH7t5VuJY-MDQuTA=.eed6083d-e2dc-4888-a2d5-b6934f098289@github.com>
 <zsXCn2nynPEIqUg3XFoI_UV_VUoIGyEmvPr2nRwkUiQ=.a3ff867d-cdb3-4a2a-bbcd-69065e3739bb@github.com>
Message-ID: <9pMfQtqoAkkOP0pYjYrqozV1umS5A-BYo2a0GsNcihA=.7779b1c5-993c-4c60-9b65-31fb3c57e659@github.com>

On Tue, 15 Oct 2024 21:30:13 GMT, Volodymyr Paprotski <duke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix aarch64.ad
>
> src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp line 414:
> 
>> 412:   // to the valid haystack bytes on the stack.
>> 413:   {
>> 414:     const Register haystack = rbx;
> 
> Keep `rax` as index for clarity? Although it is really used as a temp..
> 
> 
> const Register index = rax;
> const Register haystack = rbx;
> copy_to_stack(haystack, haystack_len, false, index , XMM_TMP1, _masm);

I'll use rax as tmp, then.

const Register tmp = rax;
const Register haystack = rbx;
copy_to_stack(haystack, haystack_len, false, tmp , XMM_TMP1, _masm);

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1802659159

From rkennke at openjdk.org  Wed Oct 16 09:05:32 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 16 Oct 2024 09:05:32 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v42]
In-Reply-To: <zsXCn2nynPEIqUg3XFoI_UV_VUoIGyEmvPr2nRwkUiQ=.a3ff867d-cdb3-4a2a-bbcd-69065e3739bb@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <5x48SX55xY_BRxqqcTTvGp_ocrKDH7t5VuJY-MDQuTA=.eed6083d-e2dc-4888-a2d5-b6934f098289@github.com>
 <zsXCn2nynPEIqUg3XFoI_UV_VUoIGyEmvPr2nRwkUiQ=.a3ff867d-cdb3-4a2a-bbcd-69065e3739bb@github.com>
Message-ID: <vwaTIEUTS7HUI9kyrTATPU4rBNCkfdadecgfLxP6s3k=.f0f5ef80-dd3b-4511-9051-4918ba664f61@github.com>

On Tue, 15 Oct 2024 22:09:54 GMT, Volodymyr Paprotski <duke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix aarch64.ad
>
> src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp line 1659:
> 
>> 1657:   Label L_moreThan8, L_moreThan16, L_moreThan24, L_adjustHaystack;
>> 1658: 
>> 1659:   assert(arrayOopDesc::base_offset_in_bytes(isU ? T_CHAR : T_BYTE) >= 8,
> 
> If we had to also optimize for header-size 16, it might be possible to remove one jump here. Looks correct for either size.

Yeah. The old code optimized for header-size >= 16. But given that compact headers will soon become the default, I don't think it's worth optimizing for the old header layout.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1802665723

From rkennke at openjdk.org  Wed Oct 16 09:16:36 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 16 Oct 2024 09:16:36 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v42]
In-Reply-To: <zsXCn2nynPEIqUg3XFoI_UV_VUoIGyEmvPr2nRwkUiQ=.a3ff867d-cdb3-4a2a-bbcd-69065e3739bb@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <5x48SX55xY_BRxqqcTTvGp_ocrKDH7t5VuJY-MDQuTA=.eed6083d-e2dc-4888-a2d5-b6934f098289@github.com>
 <zsXCn2nynPEIqUg3XFoI_UV_VUoIGyEmvPr2nRwkUiQ=.a3ff867d-cdb3-4a2a-bbcd-69065e3739bb@github.com>
Message-ID: <0wbOnb32bfMQybp2M7vDrJpuTDCIrpKzvUy0KYGHtMU=.ec15027b-8a36-4402-ac33-330383d98e48@github.com>

On Tue, 15 Oct 2024 22:31:27 GMT, Volodymyr Paprotski <duke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix aarch64.ad
>
> src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp line 1634:
> 
>> 1632: 
>> 1633: 
>> 1634: // Copy the small (< 32 byte) haystack to the stack.  Allows for vector reads without page fault
> 
> Just to be pedantic, its `(<=32)` - this function also handles 32bytes case.
> 
> -  line 401: 
> 
>  __ cmpq(haystack_len, 0x20);
>  __ ja(L_bigSwitchTop);
> 
> - though line 293 (`highly_optimized_short_cases`) only seems to route16-byte cases here: 
> ```__ cmpq(haystack_len_p, isU ? 8 : 16);```

I am not sure what you are looking at, but line 293 reads:

__ cmpq(haystack_len_p, isU ? 16 : 32);


for me. IOW, it routes > 32 byte cases to `L_begin`. But the following cmp/ja also routes <= 32 byte cases there, when `needle_len > 6`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1802694667

From rkennke at openjdk.org  Wed Oct 16 09:31:12 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 16 Oct 2024 09:31:12 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v43]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <d8BYPKWeRrC0aK7WeqwXe2pphNUZRgqKf23s15leeNE=.8e8e13a2-5f2a-4ee5-86fb-beef1ea64555@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Address comments by @vpaprotsk

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/005498b1..1fd365df

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=42
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=41-42

  Stats: 4 lines in 1 file changed: 1 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From coleenp at openjdk.org  Wed Oct 16 12:16:34 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Wed, 16 Oct 2024 12:16:34 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v43]
In-Reply-To: <d8BYPKWeRrC0aK7WeqwXe2pphNUZRgqKf23s15leeNE=.8e8e13a2-5f2a-4ee5-86fb-beef1ea64555@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <d8BYPKWeRrC0aK7WeqwXe2pphNUZRgqKf23s15leeNE=.8e8e13a2-5f2a-4ee5-86fb-beef1ea64555@github.com>
Message-ID: <XDTAUAApVTM7iZAGsuosoHPDcEf5NH6rO4JPm1W-YjA=.7cf382e3-f9ca-4d2a-b22e-81f003da88b7@github.com>

On Wed, 16 Oct 2024 09:31:12 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address comments by @vpaprotsk

We're seeing failures in our nightly testing for tests runtime/cds/appcds/SharedBaseAddress.java and runtime/cds/SharedBaseAddress.java which I'm tracking in this bug [JDK-8340212](https://bugs.openjdk.org/browse/JDK-8340212)

This patch should problem list these two tests on aarch64 when UseCompactObjectHeaders is on (if possible to be that specific), or just plain problem list it until I have a fix for it.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2416647209

From rkennke at openjdk.org  Wed Oct 16 13:46:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 16 Oct 2024 13:46:24 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v44]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <logRSsOv-ZbKrosBAddqiXOOAcr-tMODEDmGksY6uJs=.ee83f2c3-2b57-41f5-b9f4-9b36811ced37@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Problem-list SharedBaseAddress tests on aarch64

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/1fd365df..ec42f4d6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=43
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=42-43

  Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Wed Oct 16 13:46:25 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 16 Oct 2024 13:46:25 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v43]
In-Reply-To: <XDTAUAApVTM7iZAGsuosoHPDcEf5NH6rO4JPm1W-YjA=.7cf382e3-f9ca-4d2a-b22e-81f003da88b7@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <d8BYPKWeRrC0aK7WeqwXe2pphNUZRgqKf23s15leeNE=.8e8e13a2-5f2a-4ee5-86fb-beef1ea64555@github.com>
 <XDTAUAApVTM7iZAGsuosoHPDcEf5NH6rO4JPm1W-YjA=.7cf382e3-f9ca-4d2a-b22e-81f003da88b7@github.com>
Message-ID: <-8DXPgoraRNtE7uJw-Pdk5Z3eJAzIbhVRJOX5JH85UY=.358823f0-a30f-4994-b566-cdf064eac8f0@github.com>

On Wed, 16 Oct 2024 12:13:32 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> We're seeing failures in our nightly testing for tests runtime/cds/appcds/SharedBaseAddress.java and runtime/cds/SharedBaseAddress.java which I'm tracking in this bug [JDK-8340212](https://bugs.openjdk.org/browse/JDK-8340212)
> 
> This patch should problem list these two tests on aarch64 when UseCompactObjectHeaders is on (if possible to be that specific), or just plain problem list it until I have a fix for it.

Thanks for pointing this out. I've problem-listed both tests on aarch64.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2416881886

From kdnilsen at openjdk.org  Wed Oct 16 15:26:34 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 16 Oct 2024 15:26:34 GMT
Subject: RFR: 8342214: GenShen: Reduce code duplication in shFreeSet with
 iterator abstraction
In-Reply-To: <cW56kShFSmK-x3nahU1gim1F1pu9wXcgNDWWAvBVbPI=.495beaca-8199-4009-9282-48b9ec28edb2@github.com>
References: <cW56kShFSmK-x3nahU1gim1F1pu9wXcgNDWWAvBVbPI=.495beaca-8199-4009-9282-48b9ec28edb2@github.com>
Message-ID: <SFHYefUb-5rSKVBRDK0geJjny1MjB9BsFPDUcuCQ2ZM=.ad3ec329-8947-4009-96c1-60cc42306c9d@github.com>

On Mon, 14 Oct 2024 22:07:41 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Introduce `ShenandoahLeftRightIterator` and `ShenandoahRightLeftIterator` classes to encapsulate allocation bias to one side of the heap or the other.

Thanks.  Looks like a nice improvement.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/512#pullrequestreview-2372856073

From coleenp at openjdk.org  Wed Oct 16 15:32:42 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Wed, 16 Oct 2024 15:32:42 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v44]
In-Reply-To: <logRSsOv-ZbKrosBAddqiXOOAcr-tMODEDmGksY6uJs=.ee83f2c3-2b57-41f5-b9f4-9b36811ced37@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <logRSsOv-ZbKrosBAddqiXOOAcr-tMODEDmGksY6uJs=.ee83f2c3-2b57-41f5-b9f4-9b36811ced37@github.com>
Message-ID: <jcJX7xDf0w0wDgFA-eoYVaK0rEgIUHTmbpgHiOUZEVw=.1b4528e8-805a-4914-928b-4fd906668204@github.com>

On Wed, 16 Oct 2024 13:46:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Problem-list SharedBaseAddress tests on aarch64

Marked as reviewed by coleenp (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2372873633

From coleenp at openjdk.org  Wed Oct 16 15:45:34 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Wed, 16 Oct 2024 15:45:34 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v44]
In-Reply-To: <logRSsOv-ZbKrosBAddqiXOOAcr-tMODEDmGksY6uJs=.ee83f2c3-2b57-41f5-b9f4-9b36811ced37@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <logRSsOv-ZbKrosBAddqiXOOAcr-tMODEDmGksY6uJs=.ee83f2c3-2b57-41f5-b9f4-9b36811ced37@github.com>
Message-ID: <cQ6B7ZfZdJrEd_dsmQouGG3i45UcX_e_5f1K7fdvTuI=.82c67fe3-671a-4b1e-bfb1-979e65d56b19@github.com>

On Wed, 16 Oct 2024 13:46:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Problem-list SharedBaseAddress tests on aarch64

src/hotspot/share/oops/compressedKlass.cpp line 185:

> 183: #endif
> 184: 
> 185:   DEBUG_ONLY(sanity_check_after_initialization();)

This is here twice.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1803363047

From rkennke at openjdk.org  Wed Oct 16 16:04:21 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 16 Oct 2024 16:04:21 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v45]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <nXvefC2q0Zv4iZH5NdDzBC_sxM3weHdrZpC6vPmUJGM=.c2ddaf31-6b8c-407c-b2a2-ab710486092c@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Remove extra sanity check

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/ec42f4d6..e4c08780

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=44
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=43-44

  Stats: 4 lines in 1 file changed: 0 ins; 4 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From kdnilsen at openjdk.org  Wed Oct 16 16:41:15 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 16 Oct 2024 16:41:15 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
In-Reply-To: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
Message-ID: <i7iI7I2ff2jZFSaBo3y1sglW8PWJR3ushaCrCi5JC9o=.49cfb21d-3aae-4b4d-92fa-3eb8e3695c91@github.com>

On Tue, 15 Oct 2024 08:53:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>  - [ ] Performance tests (no regressions)

Marked as reviewed by kdnilsen (Author).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21517#pullrequestreview-2373084021

From wkemper at openjdk.org  Wed Oct 16 18:29:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Oct 2024 18:29:11 GMT
Subject: RFR: 8342079: Shenandoah: Remove extra
 ShenandoahInitMarkRootsClosure
In-Reply-To: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>
References: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>
Message-ID: <aQWEEkE0DZXOdDGjMB3S9wx5-QKrKb3SJjiJrRfL1Ao=.c7a7afd6-a8c7-4f29-9d5b-061a814b7361@github.com>

On Mon, 14 Oct 2024 18:42:53 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `ShenandoahInitMarkRootsClosure` in `ShenandoahSTWMark` is semantically similar to the shared `ShenandoahMarkRefsClosure`. It misses subclassing the metadata-visiting superclass, though, and `ShenandoahMarkRefsClosure` does it right. It is cleaner and safer to use the super-class directly, as concurrent mark does it. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21504#pullrequestreview-2373331951

From wkemper at openjdk.org  Wed Oct 16 18:40:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Oct 2024 18:40:33 GMT
Subject: RFR: 8338688: Shenandoah: Avoid calling java_lang_Class accessors
 in asserts/verifier
In-Reply-To: <ue3bwIs7rBnelqtzVXDatNPsH7SygWjPSemP4Cn7vAQ=.8ae88d37-ed25-4f65-8920-91ebf92d5633@github.com>
References: <ue3bwIs7rBnelqtzVXDatNPsH7SygWjPSemP4Cn7vAQ=.8ae88d37-ed25-4f65-8920-91ebf92d5633@github.com>
Message-ID: <dOcUh3eEwm0vlmwxzbL42S7ZouyqY2VP8WXjgNirm5U=.1e5a75b6-4372-46ce-b8fb-0d51d05cc2c5@github.com>

On Tue, 15 Oct 2024 20:30:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Trivial to resolve.

MacOS failures look like an infrastructure issue. Windows test failure is a single timeout, which has been seen often of late.

-------------

PR Comment: https://git.openjdk.org/shenandoah-jdk21u/pull/128#issuecomment-2417620404

From wkemper at openjdk.org  Wed Oct 16 19:38:10 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Oct 2024 19:38:10 GMT
Subject: RFR: 8338662: Shenandoah: Remove excessive
 ShenandoahVerifier::verify_during_evacuation [v2]
In-Reply-To: <hom8apBnmd0qQ-8gwPKB655wWKl_RxL77IC39GhTl3E=.5769684c-6d5d-4899-b4c0-063c22580e57@github.com>
References: <hom8apBnmd0qQ-8gwPKB655wWKl_RxL77IC39GhTl3E=.5769684c-6d5d-4899-b4c0-063c22580e57@github.com>
Message-ID: <wkBgvhFal6CRYe58kc8xDn6adUjStD3pgaXiSmqRKZ0=.1918e0b6-42d2-40fa-89c4-a1cd5ed791b5@github.com>

> Not clean, combined with a related change.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Align gc state with `verify_before_updaterefs` expectations

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/129/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/129/files/78873b91..7721cda9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=129&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=129&range=00-01

  Stats: 3 lines in 1 file changed: 2 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/129.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/129/head:pull/129

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/129

From ysr at openjdk.org  Wed Oct 16 20:01:41 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 16 Oct 2024 20:01:41 GMT
Subject: RFR: 8342278: GenShen: Move non-generational mode test out of
 generational test configuration
In-Reply-To: <joOkZ4SECXheIBJtzuQTUi_Y0AKAGx2poOc6Kcj2leM=.5a771fea-adb0-4df8-b98c-cc8b7a3d51df@github.com>
References: <joOkZ4SECXheIBJtzuQTUi_Y0AKAGx2poOc6Kcj2leM=.5a771fea-adb0-4df8-b98c-cc8b7a3d51df@github.com>
Message-ID: <db5WtyRcRSGmpECiwmiO4ON30kCQ8e2NZsbKTLXbMSY=.b75d2544-d2a6-4b65-89d0-62df8886c8ee@github.com>

On Tue, 15 Oct 2024 21:02:04 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Test has unintentionally configured a non-generational mode test to run under the generational test section.

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/515#pullrequestreview-2373549470

From ysr at openjdk.org  Wed Oct 16 20:02:32 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 16 Oct 2024 20:02:32 GMT
Subject: RFR: 8342255: GenShen: Remove unnecessary enum initial values
In-Reply-To: <z1Tw-WheUODYEQplzaaIy-jgbx1JgZScwQphXkvuMww=.02b643c8-479b-4f17-97bc-a1c350ba8023@github.com>
References: <z1Tw-WheUODYEQplzaaIy-jgbx1JgZScwQphXkvuMww=.02b643c8-479b-4f17-97bc-a1c350ba8023@github.com>
Message-ID: <85w_Iu2dCDdSeimTr2m3wAg97WSxTnV74rU7H0kLKgg=.cefa1486-4516-4c5c-8767-75f6e90cff28@github.com>

On Tue, 15 Oct 2024 20:50:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> C++ does not require explicit initial values for enumeration members.

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/514#pullrequestreview-2373552069

From ysr at openjdk.org  Wed Oct 16 20:05:37 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 16 Oct 2024 20:05:37 GMT
Subject: RFR: 8342239: GenShen: Revert changes in adaptive heuristic to
 avoid overflow on 32 bit
In-Reply-To: <qpNQOncvdl4bL2icPHX1d56rsiobbfJ2MiOINx5pDjA=.bc6700da-beb4-4afb-bfcb-7fa5da8783bb@github.com>
References: <qpNQOncvdl4bL2icPHX1d56rsiobbfJ2MiOINx5pDjA=.bc6700da-beb4-4afb-bfcb-7fa5da8783bb@github.com>
Message-ID: <QsQT_5zG-hyPV_MMSW_ZnG51pvGLlK4DUfpj67lPi2o=.795e0196-87c6-4f08-8512-8d313451ef9e@github.com>

On Tue, 15 Oct 2024 20:46:11 GMT, William Kemper <wkemper at openjdk.org> wrote:

> One of our changes to increase precision comes with the risk of overflow on 32 bit platforms. We're going to revert it for safety's sake.

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/513#pullrequestreview-2373556057

From ysr at openjdk.org  Wed Oct 16 20:07:30 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 16 Oct 2024 20:07:30 GMT
Subject: RFR: 8342214: GenShen: Reduce code duplication in shFreeSet with
 iterator abstraction
In-Reply-To: <cW56kShFSmK-x3nahU1gim1F1pu9wXcgNDWWAvBVbPI=.495beaca-8199-4009-9282-48b9ec28edb2@github.com>
References: <cW56kShFSmK-x3nahU1gim1F1pu9wXcgNDWWAvBVbPI=.495beaca-8199-4009-9282-48b9ec28edb2@github.com>
Message-ID: <TtXZfv1oR1P9WDtLpdn2X8KnFDrP9aDBh61onvk1ygs=.459e6892-275d-4414-8668-03f483f349ea@github.com>

On Mon, 14 Oct 2024 22:07:41 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Introduce `ShenandoahLeftRightIterator` and `ShenandoahRightLeftIterator` classes to encapsulate allocation bias to one side of the heap or the other.

LGTM. A few minor comments which, if they should entail changes, don't require a re-review.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 58:

> 56:   idx_t _end;
> 57:   ShenandoahRegionPartitions* _partitions;
> 58:   ShenandoahFreeSetPartitionId _partition;

I'd call this a `_pid`, short for free set partition id. The type mismatch of `_partitions` (plural) and `_partition` (singular) makes the current form of naming a bit awkward, and the single-letter difference rife for confusion when reading code (although the type-system saves one here).

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 61:

> 59: public:
> 60:   explicit ShenandoahLeftRightIterator(ShenandoahRegionPartitions* partitions, ShenandoahFreeSetPartitionId partition, bool use_empty = false)
> 61:     : _idx(0), _end(0), _partitions(partitions), _partition(partition) {

Here and in the right-biased iterator, why do we first do the 0-initialization, which will soon be over-written with the correct value. Why not just do the correct initialization once? That would allow you to make, e.g. `_end` a const. I'd similarly const all other fields other than `_idx` which is manipulated by the iterator.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 751:

> 749: }
> 750: 
> 751: HeapWord* ShenandoahFreeSet::allocate_from_partition_with_affiliation(ShenandoahAffiliation affiliation,

Since the "from_partition" part is decided in the method, based on the request, I'd rename this `allocate_with_affiliation(...)`.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 296:

> 294:   // Return the address of memory allocated, setting in_new_region to true iff the allocation is taken
> 295:   // from a region that was previously empty.  Return nullptr if memory could not be allocated.
> 296:   inline HeapWord* allocate_from_partition_with_affiliation(ShenandoahAffiliation affiliation,

The "from_partition" in the name of the method sounds a bit mysterious since the first argument of the old method is now gone. How about just `allocate_with_affiliation`, and let the method signature distinguish between the method at line 347 -- or if you prefer rename the latter to `allocate_with_affiliation_using_iterator()`?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/512#pullrequestreview-2370469538
PR Review Comment: https://git.openjdk.org/shenandoah/pull/512#discussion_r1801865787
PR Review Comment: https://git.openjdk.org/shenandoah/pull/512#discussion_r1801854655
PR Review Comment: https://git.openjdk.org/shenandoah/pull/512#discussion_r1802113523
PR Review Comment: https://git.openjdk.org/shenandoah/pull/512#discussion_r1802119863

From wkemper at openjdk.org  Wed Oct 16 20:07:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Oct 2024 20:07:35 GMT
Subject: Integrated: 8342255: GenShen: Remove unnecessary enum initial values
In-Reply-To: <z1Tw-WheUODYEQplzaaIy-jgbx1JgZScwQphXkvuMww=.02b643c8-479b-4f17-97bc-a1c350ba8023@github.com>
References: <z1Tw-WheUODYEQplzaaIy-jgbx1JgZScwQphXkvuMww=.02b643c8-479b-4f17-97bc-a1c350ba8023@github.com>
Message-ID: <DqXL2Om7ULSBkG03KRTncHUR3Y41RA4Pc0wGRawEiZM=.80c04148-d926-472a-8c25-92d01eb9deb1@github.com>

On Tue, 15 Oct 2024 20:50:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

> C++ does not require explicit initial values for enumeration members.

This pull request has now been integrated.

Changeset: 441d1c1d
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/441d1c1dfa3ac4d43b5aabcb419b05c055be7f8f
Stats:     12 lines in 1 file changed: 0 ins; 0 del; 12 mod

8342255: GenShen: Remove unnecessary enum initial values

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/514

From wkemper at openjdk.org  Wed Oct 16 20:08:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Oct 2024 20:08:37 GMT
Subject: Integrated: 8342278: GenShen: Move non-generational mode test out of
 generational test configuration
In-Reply-To: <joOkZ4SECXheIBJtzuQTUi_Y0AKAGx2poOc6Kcj2leM=.5a771fea-adb0-4df8-b98c-cc8b7a3d51df@github.com>
References: <joOkZ4SECXheIBJtzuQTUi_Y0AKAGx2poOc6Kcj2leM=.5a771fea-adb0-4df8-b98c-cc8b7a3d51df@github.com>
Message-ID: <XB8GOXq_tEDFb2WOnS_As4DeUj4cRxQgbsHPOnfE0K0=.34f94836-4af7-40a7-9c46-c9354db71691@github.com>

On Tue, 15 Oct 2024 21:02:04 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Test has unintentionally configured a non-generational mode test to run under the generational test section.

This pull request has now been integrated.

Changeset: 00a3ba6c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/00a3ba6c9148c2e18ba7f30f03fd823c6301ea75
Stats:     6 lines in 1 file changed: 5 ins; 0 del; 1 mod

8342278: GenShen: Move non-generational mode test out of generational test configuration

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/515

From ysr at openjdk.org  Wed Oct 16 20:09:32 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 16 Oct 2024 20:09:32 GMT
Subject: RFR: 8338534: GenShen: Handle alloc failure differently when
 immediate garbage is pending
In-Reply-To: <MYG7vU4GT9XxbI0v7ej7H9jKdMA6V1J-gwZUPOWhyGw=.e38d708c-0192-478f-8369-15b3135550d1@github.com>
References: <6H95t8NJ4xvUe0xClp8yZEVtaVqfpg85E1zZ9jnh0nk=.9e68ac2d-342c-4f4c-ac97-250db7592420@github.com>
 <MYG7vU4GT9XxbI0v7ej7H9jKdMA6V1J-gwZUPOWhyGw=.e38d708c-0192-478f-8369-15b3135550d1@github.com>
Message-ID: <8LMd5BIoWGO64mF1b1Mu-g5akwxuDeIzEgr5GGSlViI=.0f2f6a33-0f93-49eb-918d-9b7db3dcd443@github.com>

On Wed, 21 Aug 2024 17:07:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several changes are implemented here:
>> 
>> 1. Re-order the phases that execute immediately after final-mark so that we do concurrent-cleanup quicker (but still after concurrent weak references)
>> 2. After immediate garbage has been reclaimed by concurrent cleanup, notify waiting allocators
>> 3. If an allocation failure occurs while immediate garbage recycling is pending, stall the allocation but do not cancel the concurrent gc.
>
> BTW, I can't figure out what jcheck whitespace is complaining about.  I think it is reporting the wrong line number.  jcheck on my local copy does not report a whitespace problem.

@kdnilsen : Should this become a draft for now?

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/479#issuecomment-2417843032

From wkemper at openjdk.org  Wed Oct 16 20:11:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Oct 2024 20:11:30 GMT
Subject: Integrated: 8342239: GenShen: Revert changes in adaptive heuristic to
 avoid overflow on 32 bit
In-Reply-To: <qpNQOncvdl4bL2icPHX1d56rsiobbfJ2MiOINx5pDjA=.bc6700da-beb4-4afb-bfcb-7fa5da8783bb@github.com>
References: <qpNQOncvdl4bL2icPHX1d56rsiobbfJ2MiOINx5pDjA=.bc6700da-beb4-4afb-bfcb-7fa5da8783bb@github.com>
Message-ID: <1idov1cY2Kp19fodb6RYe-w4P_n5iUVD-DoT13MYGbE=.c78daf04-cefc-4ad1-81fd-045438466507@github.com>

On Tue, 15 Oct 2024 20:46:11 GMT, William Kemper <wkemper at openjdk.org> wrote:

> One of our changes to increase precision comes with the risk of overflow on 32 bit platforms. We're going to revert it for safety's sake.

This pull request has now been integrated.

Changeset: 05d194bf
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/05d194bf76b17a20c0b7a7571f8e2c1fbcbb7701
Stats:     2 lines in 1 file changed: 0 ins; 0 del; 2 mod

8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/513

From ysr at openjdk.org  Wed Oct 16 20:29:32 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 16 Oct 2024 20:29:32 GMT
Subject: RFR: 8338662: Shenandoah: Remove excessive
 ShenandoahVerifier::verify_during_evacuation [v2]
In-Reply-To: <wkBgvhFal6CRYe58kc8xDn6adUjStD3pgaXiSmqRKZ0=.1918e0b6-42d2-40fa-89c4-a1cd5ed791b5@github.com>
References: <hom8apBnmd0qQ-8gwPKB655wWKl_RxL77IC39GhTl3E=.5769684c-6d5d-4899-b4c0-063c22580e57@github.com>
 <wkBgvhFal6CRYe58kc8xDn6adUjStD3pgaXiSmqRKZ0=.1918e0b6-42d2-40fa-89c4-a1cd5ed791b5@github.com>
Message-ID: <6_lj6eBIR_tLHqUnBexUdPw_rv1BAUQQLTYBTWSREso=.27aa1569-1e3a-490a-9feb-60d9cd27c25a@github.com>

On Wed, 16 Oct 2024 19:38:10 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Not clean, combined with a related change.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Align gc state with `verify_before_updaterefs` expectations

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/129#pullrequestreview-2373607915

From ysr at openjdk.org  Wed Oct 16 20:38:25 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 16 Oct 2024 20:38:25 GMT
Subject: RFR: 8338688: Shenandoah: Avoid calling java_lang_Class accessors
 in asserts/verifier
In-Reply-To: <ue3bwIs7rBnelqtzVXDatNPsH7SygWjPSemP4Cn7vAQ=.8ae88d37-ed25-4f65-8920-91ebf92d5633@github.com>
References: <ue3bwIs7rBnelqtzVXDatNPsH7SygWjPSemP4Cn7vAQ=.8ae88d37-ed25-4f65-8920-91ebf92d5633@github.com>
Message-ID: <3NQWCS-QnLzPo15i5-Ik82ngaS2wuzEgp5GzSXSV5aI=.4eaaed68-2b39-40e0-8482-4236e91320ce@github.com>

On Tue, 15 Oct 2024 20:30:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Trivial to resolve.

Backport looks good to me.

Seems like an infra issue or something that's causing whole swathes of pre-submit tests to fail. May be rerun those tests, unless it's a wider infra issue?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/128#pullrequestreview-2373627645

From duke at openjdk.org  Wed Oct 16 20:42:31 2024
From: duke at openjdk.org (Volodymyr Paprotski)
Date: Wed, 16 Oct 2024 20:42:31 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v45]
In-Reply-To: <nXvefC2q0Zv4iZH5NdDzBC_sxM3weHdrZpC6vPmUJGM=.c2ddaf31-6b8c-407c-b2a2-ab710486092c@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <nXvefC2q0Zv4iZH5NdDzBC_sxM3weHdrZpC6vPmUJGM=.c2ddaf31-6b8c-407c-b2a2-ab710486092c@github.com>
Message-ID: <Q7F4DYDHbZ5PFV9p0aI7CJRgiWRUIB58IEBiPkVlQ3Y=.60539a7f-7eea-42d9-b453-508e6d9cdf7b@github.com>

On Wed, 16 Oct 2024 16:04:21 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove extra sanity check

Looks good to me (reviewed just `src/hotspot/cpu/x86/c2_stubGenerator_x86_64_string.cpp`)
Thanks!

-------------

Marked as reviewed by vpaprotsk at github.com (no known OpenJDK username).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2373635524

From wkemper at openjdk.org  Wed Oct 16 21:11:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Oct 2024 21:11:33 GMT
Subject: RFR: 8342214: GenShen: Reduce code duplication in shFreeSet with
 iterator abstraction
In-Reply-To: <TtXZfv1oR1P9WDtLpdn2X8KnFDrP9aDBh61onvk1ygs=.459e6892-275d-4414-8668-03f483f349ea@github.com>
References: <cW56kShFSmK-x3nahU1gim1F1pu9wXcgNDWWAvBVbPI=.495beaca-8199-4009-9282-48b9ec28edb2@github.com>
 <TtXZfv1oR1P9WDtLpdn2X8KnFDrP9aDBh61onvk1ygs=.459e6892-275d-4414-8668-03f483f349ea@github.com>
Message-ID: <xkUEuVqs4QG21Om-GfZU2oI09myLo8dVWzYgf4rWK64=.c74a0202-98cf-405b-97b8-4c77785c24d5@github.com>

On Tue, 15 Oct 2024 20:24:27 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Introduce `ShenandoahLeftRightIterator` and `ShenandoahRightLeftIterator` classes to encapsulate allocation bias to one side of the heap or the other.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 58:
> 
>> 56:   idx_t _end;
>> 57:   ShenandoahRegionPartitions* _partitions;
>> 58:   ShenandoahFreeSetPartitionId _partition;
> 
> I'd call this a `_pid`, short for free set partition id. The type mismatch of `_partitions` (plural) and `_partition` (singular) makes the current form of naming a bit awkward, and the single-letter difference rife for confusion when reading code (although the type-system saves one here).

`_pid` makes me think of `pid` (process ID). I'll go with `_partition_id`.

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 296:
> 
>> 294:   // Return the address of memory allocated, setting in_new_region to true iff the allocation is taken
>> 295:   // from a region that was previously empty.  Return nullptr if memory could not be allocated.
>> 296:   inline HeapWord* allocate_from_partition_with_affiliation(ShenandoahAffiliation affiliation,
> 
> The "from_partition" in the name of the method sounds a bit mysterious since the first argument of the old method is now gone. How about just `allocate_with_affiliation`, and let the method signature distinguish between the method at line 347 -- or if you prefer rename the latter to `allocate_with_affiliation_using_iterator()`?

I'll take out `from_partition`.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/512#discussion_r1803800707
PR Review Comment: https://git.openjdk.org/shenandoah/pull/512#discussion_r1803801414

From wkemper at openjdk.org  Wed Oct 16 23:30:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Oct 2024 23:30:32 GMT
Subject: Integrated: 8342214: GenShen: Reduce code duplication in shFreeSet
 with iterator abstraction
In-Reply-To: <cW56kShFSmK-x3nahU1gim1F1pu9wXcgNDWWAvBVbPI=.495beaca-8199-4009-9282-48b9ec28edb2@github.com>
References: <cW56kShFSmK-x3nahU1gim1F1pu9wXcgNDWWAvBVbPI=.495beaca-8199-4009-9282-48b9ec28edb2@github.com>
Message-ID: <2PVpAllaP9Jtcxm5Rwsn2o-hoHnGmjVGqDi6bLBlmQ0=.c594d703-cc26-4d10-8038-ee70f222913d@github.com>

On Mon, 14 Oct 2024 22:07:41 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Introduce `ShenandoahLeftRightIterator` and `ShenandoahRightLeftIterator` classes to encapsulate allocation bias to one side of the heap or the other.

This pull request has now been integrated.

Changeset: 60c6dd9d
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/60c6dd9d274ae5b880406d4242f6fe0f6658d0c4
Stats:     195 lines in 2 files changed: 80 ins; 69 del; 46 mod

8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/512

From mli at openjdk.org  Thu Oct 17 10:07:35 2024
From: mli at openjdk.org (Hamlin Li)
Date: Thu, 17 Oct 2024 10:07:35 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v43]
In-Reply-To: <-8DXPgoraRNtE7uJw-Pdk5Z3eJAzIbhVRJOX5JH85UY=.358823f0-a30f-4994-b566-cdf064eac8f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <d8BYPKWeRrC0aK7WeqwXe2pphNUZRgqKf23s15leeNE=.8e8e13a2-5f2a-4ee5-86fb-beef1ea64555@github.com>
 <XDTAUAApVTM7iZAGsuosoHPDcEf5NH6rO4JPm1W-YjA=.7cf382e3-f9ca-4d2a-b22e-81f003da88b7@github.com>
 <-8DXPgoraRNtE7uJw-Pdk5Z3eJAzIbhVRJOX5JH85UY=.358823f0-a30f-4994-b566-cdf064eac8f0@github.com>
Message-ID: <I5wW9So_ctsg-umi8UqIV8vWKbFUTHej5n_Jep3G_9Y=.17d468f8-1d13-4ed5-9584-bd097216fe4a@github.com>

On Wed, 16 Oct 2024 13:42:42 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We're seeing failures in our nightly testing for tests runtime/cds/appcds/SharedBaseAddress.java and runtime/cds/SharedBaseAddress.java which I'm tracking in this bug [JDK-8340212](https://bugs.openjdk.org/browse/JDK-8340212)
>> 
>> This patch should problem list these two tests on aarch64 when UseCompactObjectHeaders is on (if possible to be that specific), or just plain problem list it until I have a fix for it.
>
>> We're seeing failures in our nightly testing for tests runtime/cds/appcds/SharedBaseAddress.java and runtime/cds/SharedBaseAddress.java which I'm tracking in this bug [JDK-8340212](https://bugs.openjdk.org/browse/JDK-8340212)
>> 
>> This patch should problem list these two tests on aarch64 when UseCompactObjectHeaders is on (if possible to be that specific), or just plain problem list it until I have a fix for it.
> 
> Thanks for pointing this out. I've problem-listed both tests on aarch64.

@rkennke Here is the [riscv implementation](https://github.com/rkennke/jdk/compare/JDK-8305895-v4...rivosinc:jdk-compact-2:compact-header-riscv?expand=1#diff-5808bc502bdf55f1ae7ba30504c8ee6eb92527f0c11670a35d6279d671b52c6bR271), could you help to include it in this pr? Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2419103904

From rkennke at openjdk.org  Thu Oct 17 10:57:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 17 Oct 2024 10:57:24 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Compact header riscv (#3)
  
  Implement compact headers on RISCV
  ---------
  
  Co-authored-by: hamlin <hamlin at rivosinc.com>

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/e4c08780..1b907cc8

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=45
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=44-45

  Stats: 136 lines in 10 files changed: 76 ins; 31 del; 29 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From tschatzl at openjdk.org  Thu Oct 17 12:35:37 2024
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Thu, 17 Oct 2024 12:35:37 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
Message-ID: <f8MC9w61VqSlgiSOJSYoKQy1kStQVVnNHBgY7h4BLO8=.1804d5c9-3cdb-4664-812e-e10239917c23@github.com>

On Thu, 17 Oct 2024 10:57:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Compact header riscv (#3)
>   
>   Implement compact headers on RISCV
>   ---------
>   
>   Co-authored-by: hamlin <hamlin at rivosinc.com>

Mostly only looked at gc files.

-------------

Marked as reviewed by tschatzl (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2375093923

From yzheng at openjdk.org  Thu Oct 17 13:07:32 2024
From: yzheng at openjdk.org (Yudi Zheng)
Date: Thu, 17 Oct 2024 13:07:32 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
Message-ID: <1Bg-C0suC9IfxXU-2Yw5pvXwySVaEFCg6EpiVgsSw70=.f6f0d596-b465-4137-99de-fba8236e8908@github.com>

On Thu, 17 Oct 2024 10:57:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Compact header riscv (#3)
>   
>   Implement compact headers on RISCV
>   ---------
>   
>   Co-authored-by: hamlin <hamlin at rivosinc.com>

JVMCI changes look good!

-------------

Marked as reviewed by yzheng (Committer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2375182126

From rkennke at openjdk.org  Thu Oct 17 13:48:11 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 17 Oct 2024 13:48:11 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
In-Reply-To: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
Message-ID: <4VHK8WGYCR_Hlw9BJgx93bnCkgvO1bcJcbTiHwuQyHQ=.38d16e95-ec74-4462-9696-e40112a147ef@github.com>

On Tue, 15 Oct 2024 08:53:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>  - [ ] Performance tests (no regressions)

Changes look fine. I don't think we would ever do concurrent updates in Thread roots, but yeah let's err on the side of caution. Perf difference should be tiny because there'd be no contention anyway.

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21517#pullrequestreview-2375298089

From shade at openjdk.org  Thu Oct 17 14:16:47 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 17 Oct 2024 14:16:47 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
 [v2]
In-Reply-To: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
Message-ID: <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>

> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>  - [x] Performance tests (no regressions)

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:

 - Merge branch 'master' into JDK-8342119-shenandoah-remove-extra-ur-cl
 - Switch back to non-concurrent closure, rename update methods
 - Fix

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21517/files
  - new: https://git.openjdk.org/jdk/pull/21517/files/f930e5a4..8e0727da

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21517&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21517&range=00-01

  Stats: 18413 lines in 583 files changed: 14207 ins; 1972 del; 2234 mod
  Patch: https://git.openjdk.org/jdk/pull/21517.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21517/head:pull/21517

PR: https://git.openjdk.org/jdk/pull/21517

From shade at openjdk.org  Thu Oct 17 14:16:47 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 17 Oct 2024 14:16:47 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
In-Reply-To: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
Message-ID: <GrRLrfQoukPWzBRrDwQfXxKg-VAz9zutrvYM8qbW3BA=.4966c365-03d5-453f-b0ae-36ff1f8325f3@github.com>

On Tue, 15 Oct 2024 08:53:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>  - [x] Performance tests (no regressions)

Extended performance tests showed ever-so-slight regression in thread root times with concurrent closure. So new revision rolls that back, and uses the similar non-concurrent closure we used before. I also renamed a few things to make it very explicit where we are using non-concurrent closures, even in concurrent mode.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21517#issuecomment-2419666178

From wkemper at openjdk.org  Thu Oct 17 14:24:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 14:24:38 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <Ct8cfDAhqA6Q8O_Glqw3BZ1o4Ki8nJfMACy0uOzFuVE=.ebd1e6be-185a-4b1c-967f-2df30c400df0@github.com>

Merges tag jdk-21.0.5-ga

-------------

Commit messages:
 - 8341989: [21u] Back out JDK-8327501 and JDK-8328366
 - 8341674: [21u] Remove designator DEFAULT_PROMOTED_VERSION_PRE=ea for release 21.0.5
 - 8332644: Improve graph optimizations
 - 8331446: Improve deserialization support
 - 8328726: Better Kerberos support
 - 8335713: Enhance vectorization analysis
 - 8328544: Improve handling of vectorization
 - 8328286: Enhance HTTP client
 - 8311208: Improve CDS Support
 - 8307383: Enhance DTLS connections

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/130/files
  Stats: 2649 lines in 59 files changed: 2213 ins; 67 del; 369 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/130.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/130/head:pull/130

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/130

From rkennke at openjdk.org  Thu Oct 17 14:56:13 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 17 Oct 2024 14:56:13 GMT
Subject: RFR: 8342079: Shenandoah: Remove extra
 ShenandoahInitMarkRootsClosure
In-Reply-To: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>
References: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>
Message-ID: <2aDG2ewinuL51sG05CsichuyCQXL9h1B9Djnsi94A2g=.b6bd152a-82d6-4dfe-bd3e-b243f85593dc@github.com>

On Mon, 14 Oct 2024 18:42:53 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `ShenandoahInitMarkRootsClosure` in `ShenandoahSTWMark` is semantically similar to the shared `ShenandoahMarkRefsClosure`. It misses subclassing the metadata-visiting superclass, though, and `ShenandoahMarkRefsClosure` does it right. It is cleaner and safer to use the super-class directly, as concurrent mark does it. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

Yes, that makes sense. The extra metadata stuff would not be used during init-mark, anyway.

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21504#pullrequestreview-2375499969

From shade at openjdk.org  Thu Oct 17 15:06:13 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 17 Oct 2024 15:06:13 GMT
Subject: Integrated: 8342079: Shenandoah: Remove extra
 ShenandoahInitMarkRootsClosure
In-Reply-To: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>
References: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>
Message-ID: <HeUGUYqL0QNCFOcQ_EzohYfSz-CRFOhxE1YNvnIN1Zo=.b5c2afe4-9c19-4f4d-9b0e-97dd1b32d51a@github.com>

On Mon, 14 Oct 2024 18:42:53 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `ShenandoahInitMarkRootsClosure` in `ShenandoahSTWMark` is semantically similar to the shared `ShenandoahMarkRefsClosure`. It misses subclassing the metadata-visiting superclass, though, and `ShenandoahMarkRefsClosure` does it right. It is cleaner and safer to use the super-class directly, as concurrent mark does it. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

This pull request has now been integrated.

Changeset: 979895d1
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/979895d175797a71c52da12f245d1040a27172cf
Stats:     30 lines in 1 file changed: 1 ins; 27 del; 2 mod

8342079: Shenandoah: Remove extra ShenandoahInitMarkRootsClosure

Reviewed-by: wkemper, rkennke

-------------

PR: https://git.openjdk.org/jdk/pull/21504

From shade at openjdk.org  Thu Oct 17 15:06:13 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 17 Oct 2024 15:06:13 GMT
Subject: RFR: 8342079: Shenandoah: Remove extra
 ShenandoahInitMarkRootsClosure
In-Reply-To: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>
References: <wQe8tUlBQRovFLXCS3VzYbfzEd-cwePnxg8GBTP9cbw=.9a25d35b-ce88-4503-84e1-04e18503b0c8@github.com>
Message-ID: <ZOFKDO8kHVEHm4NjE3f4MEGZz04Zlc2_S_CBitGeVwE=.0d109027-03f0-46ad-a089-80f7a8bf31eb@github.com>

On Mon, 14 Oct 2024 18:42:53 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `ShenandoahInitMarkRootsClosure` in `ShenandoahSTWMark` is semantically similar to the shared `ShenandoahMarkRefsClosure`. It misses subclassing the metadata-visiting superclass, though, and `ShenandoahMarkRefsClosure` does it right. It is cleaner and safer to use the super-class directly, as concurrent mark does it. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21504#issuecomment-2419798118

From rkennke at openjdk.org  Thu Oct 17 15:27:11 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 17 Oct 2024 15:27:11 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
 [v2]
In-Reply-To: <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
 <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>
Message-ID: <ZdDDvVp3Sk94G-kljfQ6DRBjUpq21V7r-KKaN-CmMD4=.5829ca25-f833-4c66-9c43-d34faa824c7d@github.com>

On Thu, 17 Oct 2024 14:16:47 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Performance tests (no regressions)
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8342119-shenandoah-remove-extra-ur-cl
>  - Switch back to non-concurrent closure, rename update methods
>  - Fix

Even better! Thanks!

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21517#pullrequestreview-2375583406

From kdnilsen at openjdk.org  Thu Oct 17 15:58:15 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 17 Oct 2024 15:58:15 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
 [v2]
In-Reply-To: <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
 <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>
Message-ID: <2bMiL9iWiNCU3-irO-Lg_mIhUOEgr8wNVQuSzcKBsHs=.3b35b5fe-45cb-4f6e-b700-dbbd343095d2@github.com>

On Thu, 17 Oct 2024 14:16:47 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Performance tests (no regressions)
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8342119-shenandoah-remove-extra-ur-cl
>  - Switch back to non-concurrent closure, rename update methods
>  - Fix

Marked as reviewed by kdnilsen (Author).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21517#pullrequestreview-2375663864

From wkemper at openjdk.org  Thu Oct 17 16:51:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 16:51:38 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
 [v2]
In-Reply-To: <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
 <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>
Message-ID: <WvHfdukKsH8bFtZBq3WlLqzDOAsZlDbZ94obIw3Au2k=.e33add3b-9a96-4bb7-881b-c2a33de1378b@github.com>

On Thu, 17 Oct 2024 14:16:47 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Performance tests (no regressions)
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8342119-shenandoah-remove-extra-ur-cl
>  - Switch back to non-concurrent closure, rename update methods
>  - Fix

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21517#pullrequestreview-2375768816

From shade at openjdk.org  Thu Oct 17 16:51:38 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 17 Oct 2024 16:51:38 GMT
Subject: RFR: 8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure
 [v2]
In-Reply-To: <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
 <p2Jygn_f__MR9Xkxy1FS6wxz9wg8_ri0k2JcWXBNuuQ=.150a5924-a35d-4a00-b26f-cebf62b0a119@github.com>
Message-ID: <KSfjzU51JCz99Td1DWKH-CEN02Iupxe8-T6gNVNy4ws=.f963504a-8dea-433a-b0f4-467c76e769b3@github.com>

On Thu, 17 Oct 2024 14:16:47 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>>  - [x] Performance tests (no regressions)
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8342119-shenandoah-remove-extra-ur-cl
>  - Switch back to non-concurrent closure, rename update methods
>  - Fix

Thank you!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21517#issuecomment-2420033090

From wkemper at openjdk.org  Thu Oct 17 17:48:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 17:48:35 GMT
Subject: git: openjdk/shenandoah-jdk21u: master: 8338662: Shenandoah: Remove
 excessive ShenandoahVerifier::verify_during_evacuation
Message-ID: <8c505c13-d78f-47f6-8638-aedbf050d26c@openjdk.org>

Changeset: 84edb563
Branch: master
Author:    William Kemper <wkemper at openjdk.org>
Date:      2024-10-17 17:47:40 +0000
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/84edb563c21374d5eceb1a79ea5b361215a080bc

8338662: Shenandoah: Remove excessive ShenandoahVerifier::verify_during_evacuation
8336915: Shenandoah: Remove unused ShenandoahVerifier::verify_after_evacuation
8336914: Shenandoah: Missing verification steps after JDK-8255765

Reviewed-by: ysr
Backport-of: 6cf7f9c4a76b99ed7aa4dc7ee54692331fc73408

! src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp
! src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp
! src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp
! src/hotspot/share/gc/shenandoah/shenandoahVerifier.hpp


From wkemper at openjdk.org  Thu Oct 17 17:51:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 17:51:44 GMT
Subject: git: openjdk/shenandoah-jdk21u: master: 8338688: Shenandoah: Avoid
 calling java_lang_Class accessors in asserts/verifier
Message-ID: <9028d8ce-cb78-4b0e-b829-2194dbad7ffe@openjdk.org>

Changeset: b90ae41b
Branch: master
Author:    William Kemper <wkemper at openjdk.org>
Date:      2024-10-17 17:50:27 +0000
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/b90ae41b42bdd31fd7127d260e95d51ddad62960

8338688: Shenandoah: Avoid calling java_lang_Class accessors in asserts/verifier

Reviewed-by: ysr
Backport-of: e297e8817f486e4af850c97fcff859c3e9a9e21c

! src/hotspot/share/gc/shenandoah/shenandoahAsserts.cpp
! src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp


From wkemper at openjdk.org  Thu Oct 17 17:52:19 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 17:52:19 GMT
Subject: Integrated: 8338662: Shenandoah: Remove excessive
 ShenandoahVerifier::verify_during_evacuation
In-Reply-To: <hom8apBnmd0qQ-8gwPKB655wWKl_RxL77IC39GhTl3E=.5769684c-6d5d-4899-b4c0-063c22580e57@github.com>
References: <hom8apBnmd0qQ-8gwPKB655wWKl_RxL77IC39GhTl3E=.5769684c-6d5d-4899-b4c0-063c22580e57@github.com>
Message-ID: <gYdzcXChEo8BQwbfCP6eGcU7DhmbO9EUDFrHBgGiFTA=.6618d09f-bd67-43b8-89ee-d8447c01e2ff@github.com>

On Tue, 15 Oct 2024 22:59:02 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Not clean, combined with a related change.

This pull request has now been integrated.

Changeset: 84edb563
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/84edb563c21374d5eceb1a79ea5b361215a080bc
Stats:     57 lines in 4 files changed: 2 ins; 53 del; 2 mod

8338662: Shenandoah: Remove excessive ShenandoahVerifier::verify_during_evacuation
8336915: Shenandoah: Remove unused ShenandoahVerifier::verify_after_evacuation
8336914: Shenandoah: Missing verification steps after JDK-8255765

Reviewed-by: ysr
Backport-of: 6cf7f9c4a76b99ed7aa4dc7ee54692331fc73408

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/129

From wkemper at openjdk.org  Thu Oct 17 17:55:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 17:55:30 GMT
Subject: Integrated: 8338688: Shenandoah: Avoid calling java_lang_Class
 accessors in asserts/verifier
In-Reply-To: <ue3bwIs7rBnelqtzVXDatNPsH7SygWjPSemP4Cn7vAQ=.8ae88d37-ed25-4f65-8920-91ebf92d5633@github.com>
References: <ue3bwIs7rBnelqtzVXDatNPsH7SygWjPSemP4Cn7vAQ=.8ae88d37-ed25-4f65-8920-91ebf92d5633@github.com>
Message-ID: <EqannPiih7hHYqrKfXBShycEfK6IvKvoRJPLM8FfJNs=.8702842e-a9a5-4609-9ec6-7bb523205021@github.com>

On Tue, 15 Oct 2024 20:30:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Trivial to resolve.

This pull request has now been integrated.

Changeset: b90ae41b
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/b90ae41b42bdd31fd7127d260e95d51ddad62960
Stats:     2 lines in 2 files changed: 0 ins; 0 del; 2 mod

8338688: Shenandoah: Avoid calling java_lang_Class accessors in asserts/verifier

Reviewed-by: ysr
Backport-of: e297e8817f486e4af850c97fcff859c3e9a9e21c

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/128

From shade at openjdk.org  Thu Oct 17 18:13:15 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 17 Oct 2024 18:13:15 GMT
Subject: Integrated: 8342119: Shenandoah: Remove extra
 ShenandoahUpdateRefsClosure
In-Reply-To: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
References: <8v42ZVUQ6lc58IzwWUJeSd0IjkRNJIquDoEX1qRbOvM=.ea84c8d0-745c-42f7-b989-ecb2be9aaca1@github.com>
Message-ID: <qHm4dFOo_2Awg2XxYeLo9HXYkc1LJosx_YbMecNvdCY=.cd885e97-9a19-46a6-ab7b-f73d56b0f9ba@github.com>

On Tue, 15 Oct 2024 08:53:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> We have a duplicate `ShenandoahUpdateRefsClosure`, which we use in a few places. For bulk GC operations, we have `ShenandoahConcUpdateRefsClosure` and `ShenandoahSTWUpdateRefsClosure` that are semantically similar to that one. We can drop the `ShenandoahUpdateRefsClosure` in favor of these two existing ones. 
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>  - [x] Performance tests (no regressions)

This pull request has now been integrated.

Changeset: b993227e
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/b993227e2f3233c7b34a77eebbeeb1c51d2b7cfe
Stats:     46 lines in 9 files changed: 4 ins; 27 del; 15 mod

8342119: Shenandoah: Remove extra ShenandoahUpdateRefsClosure

Reviewed-by: wkemper, kdnilsen, rkennke

-------------

PR: https://git.openjdk.org/jdk/pull/21517

From xpeng at openjdk.org  Thu Oct 17 20:52:11 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 17 Oct 2024 20:52:11 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
Message-ID: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>

All the shenandoah passed, still waiting for our test farm to verify performance.

Reset marking bitmaps after collection cycle, for GenShen only do this for young generation.

I have run same workload with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 

Classic shenandoah baseline:
`
[30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
[30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
[30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
...
`

Classic shenandoah reset bitmaps after cycle:
`
[30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
[30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
[30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
[30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
...
`

GenShen baseline
`
[31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
[31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
[31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
...
`

GenShen reset bitmaps after cycle

`
[30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
[30.977s][info][gc,stats    ] Concurrent Reset After Collect =    0.130 s (a =      281 us) (n =   462) (lvls, us =      104,      193,      229,      326,     1120)
[30.978s][info][gc,stats    ] Pause Init Mark (G)            =    0.111 s (a =      241 us) (n =   462) (lvls, us =      105,      223,      240,      273,      631)
[30.978s][info][gc,stats    ] Pause Init Mark (N)            =    0.040 s (a =       87 us) (n =   462) (lvls, us =       64,       75,       87,       98,      122)
...
`

Throughput is also improved by ~21% if use the GC counts as proxy metrics.

-------------

Commit messages:
 - Reset bitmap after ShenandoahOldGC
 - Not rest bitmap after cycle if it needs to bootstrap old gc
 - typo in comments
 - Minimize the regions to walk when reset bitmap
 - comments
 - bug fix
 - Fix most of test failures
 - Not to set mark incomplete in reset_mark_bitmap
 - Reset mark bitmaps of global generation after old GC
 - set_need_bitmap_reset in full GC
 - ... and 14 more: https://git.openjdk.org/shenandoah/compare/81b631fb...f984398b

Changes: https://git.openjdk.org/shenandoah/pull/516/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8338737
  Stats: 100 lines in 7 files changed: 90 ins; 2 del; 8 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From wkemper at openjdk.org  Thu Oct 17 21:04:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 21:04:06 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name
Message-ID: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>

Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.

-------------

Commit messages:
 - Fix confusing method name
 - Revert "Test if we still (or ever) needed confusing is_old method"
 - Test if we still (or ever) needed confusing is_old method

Changes: https://git.openjdk.org/shenandoah/pull/517/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=517&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342560
  Stats: 7 lines in 3 files changed: 0 ins; 0 del; 7 mod
  Patch: https://git.openjdk.org/shenandoah/pull/517.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/517/head:pull/517

PR: https://git.openjdk.org/shenandoah/pull/517

From wkemper at openjdk.org  Thu Oct 17 21:29:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 21:29:00 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <ibMBPhNzQ5-w4qDgiDUlPnO0Xmfnolf_ZV9J6542z0E=.50026ab4-83d9-4753-8b2e-9b27d995318e@github.com>

On Thu, 17 Oct 2024 08:29:43 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

I have some concerns about the correctness here. Consider this scenario:

1. At the end of a young _or mixed_ collection, `op_reset_after_collect` will reset young regions and unset the `needs reset` flag for the young generation.
2. An old region (A) from the mixed collection could be repurposed as a young region (it's bitmap will not be reset when it is recycled).
3. The next young/mixed collection will not reset the bitmap for region A because the `needs reset` flag is clear.
4. The collector will see stale mark bits for region A and will probably crash.

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 1206:

> 1204:     if (!_do_old_gc_bootstrap) {
> 1205:       // Only reset for young generation, bitmap for old generation must be retained,
> 1206:       // except there is collection(global/old/degen/full) trigged to collet regions in old gen.

s/collet/collect

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 235:

> 233:   if (need_bitmap_reset()) {
> 234:     if (heap->mode()->is_generational() && is_global() && !heap->young_generation()->need_bitmap_reset()) {
> 235:       //Only need to reset bitmap for old generation.

I'd like to see an assert here that all the bitmaps for all the young regions are already clear.

-------------

Changes requested by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/516#pullrequestreview-2376307456
PR Review Comment: https://git.openjdk.org/shenandoah/pull/516#discussion_r1805440742
PR Review Comment: https://git.openjdk.org/shenandoah/pull/516#discussion_r1805439068

From wkemper at openjdk.org  Thu Oct 17 21:41:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 21:41:45 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <KQqx9s1UaQvKGVdbT7DfqovPsSDciqK-Swy5TTqNc1s=.dda14ecb-8c8a-4089-85fa-cad144b3f533@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <uBJ3HKwo99Z9hOLVImVS43JCUFKCzMA-i4OcaSyw34w=.5b551cc9-9919-4e3f-8e8d-227e536dcd3c@github.com>
 <KQqx9s1UaQvKGVdbT7DfqovPsSDciqK-Swy5TTqNc1s=.dda14ecb-8c8a-4089-85fa-cad144b3f533@github.com>
Message-ID: <gheaI-FO443qkaR77RLSHuQZHjskTN7JCyY923ewW0k=.007ee5a2-1710-4358-9c18-378cacbc2d77@github.com>

On Fri, 11 Oct 2024 21:21:57 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 396:
>> 
>>> 394: }
>>> 395: 
>>> 396: inline bool ShenandoahHeap::is_old(oop obj) const {
>> 
>> What is the difference between this method and the above is_in_old()? Why does it need to check that active generation is young?
>
> This is just a bad, confusing method name. I'll fix this.

https://bugs.openjdk.org/browse/JDK-8342560

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1805475522

From wkemper at openjdk.org  Thu Oct 17 21:43:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 21:43:40 GMT
Subject: RFR: 8342044: Increase timeout of
 gc/shenandoah/oom/TestClassLoaderLeak.java
Message-ID: <qVxl54dbGW_9ZBw9qFSIVZRCy4Jp_WMOJJZsYyxjOvI=.8df5d4de-1a90-4e25-bb5e-47116234be7e@github.com>

Clean backport. Increase test timeout.

-------------

Commit messages:
 - Backport 2c0c65353b2f67bdcd954b4d2c2ae3e9b24d1c22

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/131/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=131&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342044
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/131.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/131/head:pull/131

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/131

From xpeng at openjdk.org  Thu Oct 17 21:44:32 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 17 Oct 2024 21:44:32 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
In-Reply-To: <ibMBPhNzQ5-w4qDgiDUlPnO0Xmfnolf_ZV9J6542z0E=.50026ab4-83d9-4753-8b2e-9b27d995318e@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
 <ibMBPhNzQ5-w4qDgiDUlPnO0Xmfnolf_ZV9J6542z0E=.50026ab4-83d9-4753-8b2e-9b27d995318e@github.com>
Message-ID: <T2hZUvIXrJzK4xKxVdd9yoBawtsb-iEModqjyjFRGvI=.4dbe61da-71d9-474f-8965-97aed654d909@github.com>

On Thu, 17 Oct 2024 21:25:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

> I have some concerns about the correctness here. Consider this scenario:
> 
> 1. At the end of a young _or mixed_ collection, `op_reset_after_collect` will reset young regions and unset the `needs reset` flag for the young generation.
> 2. An old region (A) from the mixed collection could be repurposed as a young region (it's bitmap will not be reset when it is recycled).
> 3. The next young/mixed collection will not reset the bitmap for region A because the `needs reset` flag is clear.
> 4. The collector will see stale mark bits for region A and will probably crash.

My understanding is: op_reset_after_collect is called at the end of a successful collection after the old region (A) is  repurposed as a young region, if so it is not an issue, I'll double confirm.

> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 235:
> 
>> 233:   if (need_bitmap_reset()) {
>> 234:     if (heap->mode()->is_generational() && is_global() && !heap->young_generation()->need_bitmap_reset()) {
>> 235:       //Only need to reset bitmap for old generation.
> 
> I'd like to see an assert here that all the bitmaps for all the young regions are already clear.

I'll add it, thanks.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/516#issuecomment-2420637408
PR Review Comment: https://git.openjdk.org/shenandoah/pull/516#discussion_r1805471619

From xpeng at openjdk.org  Thu Oct 17 21:44:32 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 17 Oct 2024 21:44:32 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
In-Reply-To: <T2hZUvIXrJzK4xKxVdd9yoBawtsb-iEModqjyjFRGvI=.4dbe61da-71d9-474f-8965-97aed654d909@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
 <ibMBPhNzQ5-w4qDgiDUlPnO0Xmfnolf_ZV9J6542z0E=.50026ab4-83d9-4753-8b2e-9b27d995318e@github.com>
 <T2hZUvIXrJzK4xKxVdd9yoBawtsb-iEModqjyjFRGvI=.4dbe61da-71d9-474f-8965-97aed654d909@github.com>
Message-ID: <CD-B4h-HJA3pMhdlxqyi35oXsawdbEQZO_KA2uR12MI=.d2087618-3c10-41dd-bdc1-833e6267cda5@github.com>

On Thu, 17 Oct 2024 21:36:51 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 235:
>> 
>>> 233:   if (need_bitmap_reset()) {
>>> 234:     if (heap->mode()->is_generational() && is_global() && !heap->young_generation()->need_bitmap_reset()) {
>>> 235:       //Only need to reset bitmap for old generation.
>> 
>> I'd like to see an assert here that all the bitmaps for all the young regions are already clear.
>
> I'll add it, thanks.

My understanding is: op_reset_after_collect is called at the end of a successful collection after the old region (A) is repurposed as a young region, if so it is not an issue, I'll double confirm.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/516#discussion_r1805478665

From ysr at openjdk.org  Thu Oct 17 21:49:24 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 17 Oct 2024 21:49:24 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name
In-Reply-To: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
Message-ID: <R33eHVRbSeqO1B__lZiFNf6lftx1jOLc7ICjbwrFZ04=.f094f7f3-fe21-43b4-9851-095e54949b7b@github.com>

On Thu, 17 Oct 2024 20:56:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.

A couple of comments. Will re-review again after your response. Thanks!

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 606:

> 604:   inline bool is_in_young(const void* p) const;
> 605:   inline bool is_in_old(const void* p) const;
> 606:   inline bool is_not_in_active_young_collection(oop obj) const;

Needs a single line documentation comment, given its not very natural/intuitive semantics.

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 394:

> 392: }
> 393: 
> 394: inline bool ShenandoahHeap::is_not_in_active_young_collection(oop obj) const {

This sounds to me more like: `is_in_old_when_active_young()`.

The "not" is the name makes the negative of its semantics suspect. Think of the sense of what one might believe if one identified the negation with `is_in_active_young_collection()` which I'd take to be `active_generation()->is_young() && is_in_young(obj)`, rather than the actual negation of the condition which would be: `!active_generation()->is_young() || !is_in_old(obj)`.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/517#pullrequestreview-2376364492
PR Review Comment: https://git.openjdk.org/shenandoah/pull/517#discussion_r1805481523
PR Review Comment: https://git.openjdk.org/shenandoah/pull/517#discussion_r1805480760

From ysr at openjdk.org  Thu Oct 17 21:49:24 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 17 Oct 2024 21:49:24 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name
In-Reply-To: <R33eHVRbSeqO1B__lZiFNf6lftx1jOLc7ICjbwrFZ04=.f094f7f3-fe21-43b4-9851-095e54949b7b@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
 <R33eHVRbSeqO1B__lZiFNf6lftx1jOLc7ICjbwrFZ04=.f094f7f3-fe21-43b4-9851-095e54949b7b@github.com>
Message-ID: <Os_OeWYvDlrhiuZf2Ou9bl6HU_kgjF7NK69o3MAllDA=.e52553e8-4aea-4a66-8083-cd2adcd0d80e@github.com>

On Thu, 17 Oct 2024 21:43:03 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 394:
> 
>> 392: }
>> 393: 
>> 394: inline bool ShenandoahHeap::is_not_in_active_young_collection(oop obj) const {
> 
> This sounds to me more like: `is_in_old_when_active_young()`.
> 
> The "not" is the name makes the negative of its semantics suspect. Think of the sense of what one might believe if one identified the negation with `is_in_active_young_collection()` which I'd take to be `active_generation()->is_young() && is_in_young(obj)`, rather than the actual negation of the condition which would be: `!active_generation()->is_young() || !is_in_old(obj)`.

Can we verify that the predicate is stable in the context of an arbitrary current caller?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/517#discussion_r1805482200

From wkemper at openjdk.org  Thu Oct 17 21:53:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 21:53:08 GMT
Subject: RFR: 8342564: GenShen: Only reference young/old generation names in
 generational mode
Message-ID: <qL8sfb2IQ1Oyzr367bBXdR_1Y4OhHJ1dj6-KX8cBcgc=.b4ae5fd2-4230-44e9-bfde-04124ad736b9@github.com>

When printing the heap, only reference young/old marking when running in generational mode.

-------------

Commit messages:
 - Only reference young/old marking in generational mode

Changes: https://git.openjdk.org/shenandoah/pull/518/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=518&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342564
  Stats: 6 lines in 1 file changed: 4 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/518.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/518/head:pull/518

PR: https://git.openjdk.org/shenandoah/pull/518

From xpeng at openjdk.org  Thu Oct 17 22:00:53 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 17 Oct 2024 22:00:53 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v2]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <BaqIY2wl8XSkQ_PmUlz1SkeKxTefZ7goFNHJp5dcHns=.561b3262-280c-4045-b896-6931891a6890@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Address review comments

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/516/files
  - new: https://git.openjdk.org/shenandoah/pull/516/files/f984398b..d49ae59c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=00-01

  Stats: 2 lines in 2 files changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From xpeng at openjdk.org  Thu Oct 17 22:28:19 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 17 Oct 2024 22:28:19 GMT
Subject: RFR: 8342564: GenShen: Only reference young/old generation names
 in generational mode
In-Reply-To: <qL8sfb2IQ1Oyzr367bBXdR_1Y4OhHJ1dj6-KX8cBcgc=.b4ae5fd2-4230-44e9-bfde-04124ad736b9@github.com>
References: <qL8sfb2IQ1Oyzr367bBXdR_1Y4OhHJ1dj6-KX8cBcgc=.b4ae5fd2-4230-44e9-bfde-04124ad736b9@github.com>
Message-ID: <DMLnlZEFLRaJFGhpAQ4Fi0k8abLaum1ISu95blaygJo=.8f62ef41-6612-4fbf-98a8-a6387450e810@github.com>

On Thu, 17 Oct 2024 21:49:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When printing the heap, only reference young/old marking when running in generational mode.

Looks good, thanks!

-------------

Marked as reviewed by xpeng (no project role).

PR Review: https://git.openjdk.org/shenandoah/pull/518#pullrequestreview-2376426049

From xpeng at openjdk.org  Thu Oct 17 22:43:32 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 17 Oct 2024 22:43:32 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v2]
In-Reply-To: <T2hZUvIXrJzK4xKxVdd9yoBawtsb-iEModqjyjFRGvI=.4dbe61da-71d9-474f-8965-97aed654d909@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
 <ibMBPhNzQ5-w4qDgiDUlPnO0Xmfnolf_ZV9J6542z0E=.50026ab4-83d9-4753-8b2e-9b27d995318e@github.com>
 <T2hZUvIXrJzK4xKxVdd9yoBawtsb-iEModqjyjFRGvI=.4dbe61da-71d9-474f-8965-97aed654d909@github.com>
Message-ID: <v4EqyIYeYL8QPzZofCoZegrnI1f1cqVqbMbV7K5iyUc=.167fb231-0b83-4b8a-8d27-a5fb1ba10e92@github.com>

On Thu, 17 Oct 2024 21:36:36 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> > I have some concerns about the correctness here. Consider this scenario:
> > 
> > 1. At the end of a young _or mixed_ collection, `op_reset_after_collect` will reset young regions and unset the `needs reset` flag for the young generation.
> > 2. An old region (A) from the mixed collection could be repurposed as a young region (it's bitmap will not be reset when it is recycled).
> > 3. The next young/mixed collection will not reset the bitmap for region A because the `needs reset` flag is clear.
> > 4. The collector will see stale mark bits for region A and will probably crash.
> 
> My understanding is: op_reset_after_collect is called at the end of a successful collection after the old region (A) is repurposed as a young region, if so it is not an issue, I'll double confirm.

op_reset_after_collect also reset the regions are not affiliated, after a mixed/young collection, the old region should have been recycled before resetting bitmaps, so even the recycled old region is repurposed as young region later, it should be ok. 

If the young/mixed_ collection is set to bootstrap old GC, the reset of bitmaps is delayed to old GC to avoid dirty bitmaps

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/516#issuecomment-2420740518

From ysr at openjdk.org  Thu Oct 17 23:05:14 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 17 Oct 2024 23:05:14 GMT
Subject: RFR: 8342564: GenShen: Only reference young/old generation names
 in generational mode
In-Reply-To: <qL8sfb2IQ1Oyzr367bBXdR_1Y4OhHJ1dj6-KX8cBcgc=.b4ae5fd2-4230-44e9-bfde-04124ad736b9@github.com>
References: <qL8sfb2IQ1Oyzr367bBXdR_1Y4OhHJ1dj6-KX8cBcgc=.b4ae5fd2-4230-44e9-bfde-04124ad736b9@github.com>
Message-ID: <Zcvj7N2XqAUdaGVxMm3mrcBR8TmSaaTojFXaCD3RXD0=.1a9ace4a-a121-421d-801f-6113a1bd0380@github.com>

On Thu, 17 Oct 2024 21:49:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When printing the heap, only reference young/old marking when running in generational mode.

Good change!

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/518#pullrequestreview-2376483971

From wkemper at openjdk.org  Thu Oct 17 23:31:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Oct 2024 23:31:21 GMT
Subject: Integrated: 8342044: Increase timeout of
 gc/shenandoah/oom/TestClassLoaderLeak.java
In-Reply-To: <qVxl54dbGW_9ZBw9qFSIVZRCy4Jp_WMOJJZsYyxjOvI=.8df5d4de-1a90-4e25-bb5e-47116234be7e@github.com>
References: <qVxl54dbGW_9ZBw9qFSIVZRCy4Jp_WMOJJZsYyxjOvI=.8df5d4de-1a90-4e25-bb5e-47116234be7e@github.com>
Message-ID: <4CBHcMPLwsWS7NmVPVMXXLvTU8SnGxIatq59JXE7V9g=.de4d920f-0a8b-46d0-bdf1-d1f199fb9f29@github.com>

On Thu, 17 Oct 2024 21:39:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport. Increase test timeout.

This pull request has now been integrated.

Changeset: 74749bb9
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/74749bb94f2f0686a9aa3629198262c88c2bd59c
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8342044: Increase timeout of gc/shenandoah/oom/TestClassLoaderLeak.java

Backport-of: 2c0c65353b2f67bdcd954b4d2c2ae3e9b24d1c22

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/131

From xpeng at openjdk.org  Fri Oct 18 00:08:18 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 18 Oct 2024 00:08:18 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v2]
In-Reply-To: <v4EqyIYeYL8QPzZofCoZegrnI1f1cqVqbMbV7K5iyUc=.167fb231-0b83-4b8a-8d27-a5fb1ba10e92@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
 <ibMBPhNzQ5-w4qDgiDUlPnO0Xmfnolf_ZV9J6542z0E=.50026ab4-83d9-4753-8b2e-9b27d995318e@github.com>
 <T2hZUvIXrJzK4xKxVdd9yoBawtsb-iEModqjyjFRGvI=.4dbe61da-71d9-474f-8965-97aed654d909@github.com>
 <v4EqyIYeYL8QPzZofCoZegrnI1f1cqVqbMbV7K5iyUc=.167fb231-0b83-4b8a-8d27-a5fb1ba10e92@github.com>
Message-ID: <kH6FbabtxfmXMpcEygxeVHUofC2xET1f24vxkk1APdA=.f01457b7-eee9-4eb2-a77e-0034edce68fb@github.com>

On Thu, 17 Oct 2024 22:39:31 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> > > I have some concerns about the correctness here. Consider this scenario:
> > > 
> > > 1. At the end of a young _or mixed_ collection, `op_reset_after_collect` will reset young regions and unset the `needs reset` flag for the young generation.
> > > 2. An old region (A) from the mixed collection could be repurposed as a young region (it's bitmap will not be reset when it is recycled).
> > > 3. The next young/mixed collection will not reset the bitmap for region A because the `needs reset` flag is clear.
> > > 4. The collector will see stale mark bits for region A and will probably crash.
> > 
> > 
> > My understanding is: op_reset_after_collect is called at the end of a successful collection after the old region (A) is repurposed as a young region, if so it is not an issue, I'll double confirm.
> 
> op_reset_after_collect also reset the regions are not affiliated, after a mixed/young collection, the old region should have been recycled before resetting bitmaps, so even the recycled old region is repurposed as young region later, it should be ok.
> 
> If the young/mixed_ collection is set to bootstrap old GC, the reset of bitmaps is delayed to old GC to avoid dirty bitmaps

By repeatedly running test gc/stress/gcold/TestGCOldWithShenandoah.java#generational many times, I could produce a crash, not sure root cause yet, but this might be the root cause.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/516#issuecomment-2420893946

From xpeng at openjdk.org  Fri Oct 18 00:30:57 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 18 Oct 2024 00:30:57 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v2]
In-Reply-To: <kH6FbabtxfmXMpcEygxeVHUofC2xET1f24vxkk1APdA=.f01457b7-eee9-4eb2-a77e-0034edce68fb@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
 <ibMBPhNzQ5-w4qDgiDUlPnO0Xmfnolf_ZV9J6542z0E=.50026ab4-83d9-4753-8b2e-9b27d995318e@github.com>
 <T2hZUvIXrJzK4xKxVdd9yoBawtsb-iEModqjyjFRGvI=.4dbe61da-71d9-474f-8965-97aed654d909@github.com>
 <v4EqyIYeYL8QPzZofCoZegrnI1f1cqVqbMbV7K5iyUc=.167fb231-0b83-4b8a-8d27-a5fb1ba10e92@github.com>
 <kH6FbabtxfmXMpcEygxeVHUofC2xET1f24vxkk1APdA=.f01457b7-eee9-4eb2-a77e-0034edce68fb@github.com>
Message-ID: <8HTvOzBRY0ADrJN4b6pgl-LIexJJBYgMa9SfIpo-vb0=.56a0abf0-b48f-4a9c-9ea6-70afdf4a848b@github.com>

On Fri, 18 Oct 2024 00:04:47 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> By repeatedly running test gc/stress/gcold/TestGCOldWithShenandoah.java#generational many times, I could produce a crash, not sure root cause yet, but this might be the root cause.

It is not a crash, looks like it is not related, I'll run test w/o the change to verify:

Exception in thread "main" java.lang.NullPointerException: Inflater has been closed
        at java.base/java.util.zip.Inflater.ensureOpen(Inflater.java:705)
        at java.base/java.util.zip.Inflater.reset(Inflater.java:676)
        at java.base/java.util.zip.ZipFile$CleanableResource.releaseInflater(ZipFile.java:764)
        at java.base/java.util.zip.ZipFile$InflaterCleanupAction.run(ZipFile.java:440)
        at java.base/jdk.internal.ref.CleanerImpl$PhantomCleanableRef.performCleanup(CleanerImpl.java:178)
        at java.base/jdk.internal.ref.PhantomCleanable.clean(PhantomCleanable.java:133)
        at java.base/java.util.zip.ZipFile$ZipFileInflaterInputStream.close(ZipFile.java:470)
        at java.base/jdk.internal.loader.Resource.getBytes(Resource.java:121)
        at java.base/jdk.internal.loader.URLClassPath$JarLoader$2.getBytes(URLClassPath.java:872)
        at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:859)
        at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:528)
        at com.sun.javatest.regtest.agent.MainWrapper.main(MainWrapper.java:94)

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/516#issuecomment-2420930682

From xpeng at openjdk.org  Fri Oct 18 08:22:56 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 18 Oct 2024 08:22:56 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v3]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <gpbOcbWGZ2VSN-KHIGxjfAvpPUFrj0vygGXlv9frFe4=.93c232c1-1cf0-415d-bc40-89e703c8c5a9@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/516/files
  - new: https://git.openjdk.org/shenandoah/pull/516/files/d49ae59c..d4fddac5

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=01-02

  Stats: 5 lines in 1 file changed: 5 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From shade at openjdk.org  Fri Oct 18 09:19:31 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 18 Oct 2024 09:19:31 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one header
Message-ID: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>

Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.

Additional testing:
 - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

-------------

Commit messages:
 - Fix

Changes: https://git.openjdk.org/jdk/pull/21579/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21579&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342588
  Stats: 448 lines in 16 files changed: 188 ins; 234 del; 26 mod
  Patch: https://git.openjdk.org/jdk/pull/21579.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21579/head:pull/21579

PR: https://git.openjdk.org/jdk/pull/21579

From shade at openjdk.org  Fri Oct 18 10:31:23 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 18 Oct 2024 10:31:23 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one
 header [v2]
In-Reply-To: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
Message-ID: <14GqBlYL2XDV0k6WC1La5SmYLSAyYcf2FlVE4V75T2s=.bd5ddb3f-e520-44ef-85f9-22d498d5151e@github.com>

> Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Fix Zero builds

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21579/files
  - new: https://git.openjdk.org/jdk/pull/21579/files/cbc94ff4..c9af382c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21579&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21579&range=00-01

  Stats: 12 lines in 2 files changed: 7 ins; 3 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/21579.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21579/head:pull/21579

PR: https://git.openjdk.org/jdk/pull/21579

From wkemper at openjdk.org  Fri Oct 18 14:17:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 14:17:39 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <AvYjeULgwDZR2HLH06QJ8W9rf_cjhKBPJLaFOvJeEkI=.87b9391c-e058-4c4f-ba52-0e02521159c8@github.com>

Merges tag jdk-24+20

-------------

Commit messages:
 - 8338851: Hoist os::Posix::realpath() to os::realpath() and implement on Windows
 - 8342338: Remove redundant IIOPURLTest.java
 - 8341715: PPC64: ObjectMonitor::_owner should be reset unconditionally in nmethod unlocking
 - 8342042: PPC64: compiler_fast_unlock_object flags failure instead of success
 - 8340279: Open source several AWT Dialog tests - Batch 2
 - 8341072: Open source several AWT Canvas and Rectangle related tests
 - 8341328: Refactor initial Assertion Predicate creation into separate classes
 - 8340313: Crash due to invalid oop in nmethod after C1 patching
 - 8338751: ConfigureNotify behavior has changed in KWin 6.2
 - 8340140: Open some dialog awt tests 3
 - ... and 86 more: https://git.openjdk.org/shenandoah/compare/e7c5bf45...7a64fbbb

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=519&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/519/files
  Stats: 24434 lines in 541 files changed: 20214 ins; 1645 del; 2575 mod
  Patch: https://git.openjdk.org/shenandoah/pull/519.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/519/head:pull/519

PR: https://git.openjdk.org/shenandoah/pull/519

From xpeng at openjdk.org  Fri Oct 18 15:26:18 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 18 Oct 2024 15:26:18 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v3]
In-Reply-To: <8HTvOzBRY0ADrJN4b6pgl-LIexJJBYgMa9SfIpo-vb0=.56a0abf0-b48f-4a9c-9ea6-70afdf4a848b@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
 <ibMBPhNzQ5-w4qDgiDUlPnO0Xmfnolf_ZV9J6542z0E=.50026ab4-83d9-4753-8b2e-9b27d995318e@github.com>
 <T2hZUvIXrJzK4xKxVdd9yoBawtsb-iEModqjyjFRGvI=.4dbe61da-71d9-474f-8965-97aed654d909@github.com>
 <v4EqyIYeYL8QPzZofCoZegrnI1f1cqVqbMbV7K5iyUc=.167fb231-0b83-4b8a-8d27-a5fb1ba10e92@github.com>
 <kH6FbabtxfmXMpcEygxeVHUofC2xET1f24vxkk1APdA=.f01457b7-eee9-4eb2-a77e-0034edce68fb@github.com>
 <8HTvOzBRY0ADrJN4b6pgl-LIexJJBYgMa9SfIpo-vb0=.56a0abf0-b48f-4a9c-9ea6-70afdf4a848b@github.com>
Message-ID: <SJo2wGPpqUnZp_SDvgczv4bVOuyxkuRP_XtGZNgRBQ8=.79c2602b-9b1b-490a-8882-5b87d21f96f1@github.com>

On Fri, 18 Oct 2024 00:27:09 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> It is not a crash, looks like it is not related, I'll run test w/o the change to verify:

I have run `TestGCOldWithShenandoah.java#generational ` on GenShen tip, looks likes GenShen has bug causing stale mark bits, although it is rare, the test failed after 73 repeats with crash:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:622), pid=26873, tid=26959
#  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
#
# JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x172ac44]  ShenandoahConcurrentGC::op_init_mark()+0x384
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /local/home/xlpeng/repos/jdk-xlpeng/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid26873.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/516#issuecomment-2422722567

From xpeng at openjdk.org  Fri Oct 18 15:31:24 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 18 Oct 2024 15:31:24 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v3]
In-Reply-To: <gpbOcbWGZ2VSN-KHIGxjfAvpPUFrj0vygGXlv9frFe4=.93c232c1-1cf0-415d-bc40-89e703c8c5a9@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
 <gpbOcbWGZ2VSN-KHIGxjfAvpPUFrj0vygGXlv9frFe4=.93c232c1-1cf0-415d-bc40-89e703c8c5a9@github.com>
Message-ID: <3csqXR0qGm8idpeKFxOmRruw9_9_p-MSDJJGnyeUEjI=.aa6a1752-9814-4d7d-87a2-97b427f005fa@github.com>

On Fri, 18 Oct 2024 08:22:56 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> All the shenandoah passed, still waiting for our test farm to verify performance.
>> 
>> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
>> 
>> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
>> 
>> Classic shenandoah baseline:
>> 
>> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
>> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
>> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
>> ...
>> 
>> 
>> Classic shenandoah reset bitmaps after cycle:
>> 
>> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
>> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
>> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
>> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
>> ...
>> 
>> 
>> GenShen baseline
>> 
>> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
>> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
>> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
>> ...
>> 
>> 
>> GenShen reset bitmaps after cycle
>> 
>> 
>> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (l...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION

My change also has stale mark bits issue I believe, gitfarm Genshen test fails sometimes due to crash caused by stale bitmap marks.

Amended one fix in this PR, it should solve the issue: https://github.com/openjdk/shenandoah/pull/516/commits/d4fddac5a1e7321ff3b4441bd5cd096e949cd56a#diff-7b87cde2e13d8769dc21cf6f7ae45c92942776fcb44275339182b7fa5c290ce0

I have run TestGCOldWithShenandoah.java#generational 100 times w/o crash caused by stale bitmap marks last night.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/516#issuecomment-2422731057

From xpeng at openjdk.org  Fri Oct 18 15:36:05 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 18 Oct 2024 15:36:05 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v3]
In-Reply-To: <gpbOcbWGZ2VSN-KHIGxjfAvpPUFrj0vygGXlv9frFe4=.93c232c1-1cf0-415d-bc40-89e703c8c5a9@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
 <gpbOcbWGZ2VSN-KHIGxjfAvpPUFrj0vygGXlv9frFe4=.93c232c1-1cf0-415d-bc40-89e703c8c5a9@github.com>
Message-ID: <REzLBFMg3A5UjbJ6WZ-ubEBikrPpH9u_O18nuYs78DI=.7ebbdcdb-5e8e-4f65-bd82-6e424604ab1d@github.com>

On Fri, 18 Oct 2024 08:22:56 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> All the shenandoah passed, still waiting for our test farm to verify performance.
>> 
>> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
>> 
>> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
>> 
>> Classic shenandoah baseline:
>> 
>> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
>> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
>> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
>> ...
>> 
>> 
>> Classic shenandoah reset bitmaps after cycle:
>> 
>> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
>> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
>> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
>> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
>> ...
>> 
>> 
>> GenShen baseline
>> 
>> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
>> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
>> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
>> ...
>> 
>> 
>> GenShen reset bitmaps after cycle
>> 
>> 
>> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (l...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION

src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 856:

> 854:   if (region_affiliation == OLD_GENERATION) {
> 855:     //Need to reset bitmap in next GC after change affiliation from OLD_GENERATION
> 856:     heap->young_generation()->set_need_bitmap_reset();

Instead of setting the flag, the other option is to reset bitmaps for the region, which should also fix the crash in GenShen TIP.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/516#discussion_r1806691788

From wkemper at openjdk.org  Fri Oct 18 16:02:48 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 16:02:48 GMT
Subject: Integrated: 8342564: GenShen: Only reference young/old generation
 names in generational mode
In-Reply-To: <qL8sfb2IQ1Oyzr367bBXdR_1Y4OhHJ1dj6-KX8cBcgc=.b4ae5fd2-4230-44e9-bfde-04124ad736b9@github.com>
References: <qL8sfb2IQ1Oyzr367bBXdR_1Y4OhHJ1dj6-KX8cBcgc=.b4ae5fd2-4230-44e9-bfde-04124ad736b9@github.com>
Message-ID: <PZvrzsAHtsSegt2TFVNZEiasSveXyV04LJlO3G505jc=.76673763-f714-4249-9c7b-3bef43965ebd@github.com>

On Thu, 17 Oct 2024 21:49:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When printing the heap, only reference young/old marking when running in generational mode.

This pull request has now been integrated.

Changeset: 71def7f8
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/71def7f88ef906a4a50cf48050b34ca2c58bb45d
Stats:     6 lines in 1 file changed: 4 ins; 0 del; 2 mod

8342564: GenShen: Only reference young/old generation names in generational mode

Reviewed-by: xpeng, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/518

From wkemper at openjdk.org  Fri Oct 18 16:06:55 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 16:06:55 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <Ct8cfDAhqA6Q8O_Glqw3BZ1o4Ki8nJfMACy0uOzFuVE=.ebd1e6be-185a-4b1c-967f-2df30c400df0@github.com>
References: <Ct8cfDAhqA6Q8O_Glqw3BZ1o4Ki8nJfMACy0uOzFuVE=.ebd1e6be-185a-4b1c-967f-2df30c400df0@github.com>
Message-ID: <oBJ3FJ4Ht1dLMTCcAwFo97XyPQWYMI7tc8B-QsT3hkU=.0fc3abde-61cd-4782-9270-5c97945e9a0e@github.com>

On Thu, 17 Oct 2024 14:19:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.5-ga

This pull request has now been integrated.

Changeset: 723199d3
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/723199d3218ae04958d460f67bb96964a10aabcf
Stats:     2649 lines in 59 files changed: 2213 ins; 67 del; 369 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/130

From wkemper at openjdk.org  Fri Oct 18 16:06:53 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 16:06:53 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <Ct8cfDAhqA6Q8O_Glqw3BZ1o4Ki8nJfMACy0uOzFuVE=.ebd1e6be-185a-4b1c-967f-2df30c400df0@github.com>
References: <Ct8cfDAhqA6Q8O_Glqw3BZ1o4Ki8nJfMACy0uOzFuVE=.ebd1e6be-185a-4b1c-967f-2df30c400df0@github.com>
Message-ID: <Y3HLyhmW8RHSY0gIrpklaXObnaD-DZ1IXMYOhzllJcM=.70e07f79-d74f-487a-aa29-c8f56ccbb9c9@github.com>

> Merges tag jdk-21.0.5-ga

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/130/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/130/files/dfcd8d2e..dfcd8d2e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=130&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=130&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/130.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/130/head:pull/130

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/130

From wkemper at openjdk.org  Fri Oct 18 16:26:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 16:26:44 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name [v2]
In-Reply-To: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
Message-ID: <9Idd4Yda2huiWbtP_jd4X4R80igmpcYtkyUU8G7T_PI=.caf6e1ac-44f5-42d9-9437-2dae263b5f85@github.com>

> Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.

William Kemper has updated the pull request incrementally with six additional commits since the last revision:

 - Use an even better name, add an explanatory comment
 - 8342564: GenShen: Only reference young/old generation names in generational mode
   
   Reviewed-by: xpeng, ysr
 - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
   
   Reviewed-by: kdnilsen, ysr
 - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
   
   Reviewed-by: ysr
 - 8342278: GenShen: Move non-generational mode test out of generational test configuration
   
   Reviewed-by: ysr
 - 8342255: GenShen: Remove unnecessary enum initial values
   
   Reviewed-by: ysr

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/517/files
  - new: https://git.openjdk.org/shenandoah/pull/517/files/da36b0a2..9bb4dfb7

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=517&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=517&range=00-01

  Stats: 229 lines in 9 files changed: 93 ins; 69 del; 67 mod
  Patch: https://git.openjdk.org/shenandoah/pull/517.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/517/head:pull/517

PR: https://git.openjdk.org/shenandoah/pull/517

From xpeng at openjdk.org  Fri Oct 18 16:29:48 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 18 Oct 2024 16:29:48 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v4]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <T9dRbxCBUIfywgaklWXvJqcaFDsyfvyLxLh4OMFIrm4=.7b2a5973-3119-40ca-a094-4fbb38daef04@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 27 additional commits since the last revision:

 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION
 - Address review comments
 - Reset bitmap after ShenandoahOldGC
 - Not rest bitmap after cycle if it needs to bootstrap old gc
 - typo in comments
 - Minimize the regions to walk when reset bitmap
 - comments
 - bug fix
 - Fix most of test failures
 - ... and 17 more: https://git.openjdk.org/shenandoah/compare/f5e8affa...d2835924

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/516/files
  - new: https://git.openjdk.org/shenandoah/pull/516/files/d4fddac5..d2835924

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=02-03

  Stats: 221 lines in 6 files changed: 89 ins; 69 del; 63 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From wkemper at openjdk.org  Fri Oct 18 16:55:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 16:55:44 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name [v3]
In-Reply-To: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
Message-ID: <ru4_Q_uZ-c3VNwk5G1HuVlS0d5Ka90iCyF5bBJt-Qjs=.76ce93f3-712d-4694-bab6-abdcd2cb715d@github.com>

> Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:

 - Merge remote-tracking branch 'shenandoah/master' into fix-confusing-method
 - Use an even better name, add an explanatory comment
 - 8342564: GenShen: Only reference young/old generation names in generational mode
   
   Reviewed-by: xpeng, ysr
 - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
   
   Reviewed-by: kdnilsen, ysr
 - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
   
   Reviewed-by: ysr
 - 8342278: GenShen: Move non-generational mode test out of generational test configuration
   
   Reviewed-by: ysr
 - 8342255: GenShen: Remove unnecessary enum initial values
   
   Reviewed-by: ysr
 - Fix confusing method name
 - Revert "Test if we still (or ever) needed confusing is_old method"
   
   This reverts commit 5fcaef551d3286bd3e5eae485ac11c5b15ce7ed8.
 - Test if we still (or ever) needed confusing is_old method

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/517/files
  - new: https://git.openjdk.org/shenandoah/pull/517/files/9bb4dfb7..90c81bdb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=517&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=517&range=01-02

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/517.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/517/head:pull/517

PR: https://git.openjdk.org/shenandoah/pull/517

From wkemper at openjdk.org  Fri Oct 18 16:55:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 16:55:45 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name [v2]
In-Reply-To: <9Idd4Yda2huiWbtP_jd4X4R80igmpcYtkyUU8G7T_PI=.caf6e1ac-44f5-42d9-9437-2dae263b5f85@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
 <9Idd4Yda2huiWbtP_jd4X4R80igmpcYtkyUU8G7T_PI=.caf6e1ac-44f5-42d9-9437-2dae263b5f85@github.com>
Message-ID: <a2B8GDy_gXwYQ9wHUF7SiGEBI9lUlp3FNMHUAUdZw2E=.bffaae46-fd8f-46e5-bbbb-b00cf181be75@github.com>

On Fri, 18 Oct 2024 16:26:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.
>
> William Kemper has updated the pull request incrementally with six additional commits since the last revision:
> 
>  - Use an even better name, add an explanatory comment
>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>    
>    Reviewed-by: xpeng, ysr
>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>    
>    Reviewed-by: ysr
>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>    
>    Reviewed-by: ysr
>  - 8342255: GenShen: Remove unnecessary enum initial values
>    
>    Reviewed-by: ysr

This PR is pulling commits from a rebased merge now. I might have to clean this up and force a push.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/517#issuecomment-2422873214

From wkemper at openjdk.org  Fri Oct 18 16:55:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 16:55:45 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name [v3]
In-Reply-To: <R33eHVRbSeqO1B__lZiFNf6lftx1jOLc7ICjbwrFZ04=.f094f7f3-fe21-43b4-9851-095e54949b7b@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
 <R33eHVRbSeqO1B__lZiFNf6lftx1jOLc7ICjbwrFZ04=.f094f7f3-fe21-43b4-9851-095e54949b7b@github.com>
Message-ID: <op28VqU_6NQfNQVLaIbZz3mA8j9lwa7TWiNNHZY3YZM=.dde623c3-66d4-41a4-9dec-38e0d7b0d847@github.com>

On Thu, 17 Oct 2024 21:43:53 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into fix-confusing-method
>>  - Use an even better name, add an explanatory comment
>>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>>    
>>    Reviewed-by: xpeng, ysr
>>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>>    
>>    Reviewed-by: ysr
>>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>>    
>>    Reviewed-by: ysr
>>  - 8342255: GenShen: Remove unnecessary enum initial values
>>    
>>    Reviewed-by: ysr
>>  - Fix confusing method name
>>  - Revert "Test if we still (or ever) needed confusing is_old method"
>>    
>>    This reverts commit 5fcaef551d3286bd3e5eae485ac11c5b15ce7ed8.
>>  - Test if we still (or ever) needed confusing is_old method
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 606:
> 
>> 604:   inline bool is_in_young(const void* p) const;
>> 605:   inline bool is_in_old(const void* p) const;
>> 606:   inline bool is_not_in_active_young_collection(oop obj) const;
> 
> Needs a single line documentation comment, given its not very natural/intuitive semantics.

Added a comment.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/517#discussion_r1806775858

From wkemper at openjdk.org  Fri Oct 18 16:55:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 16:55:45 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name [v3]
In-Reply-To: <Os_OeWYvDlrhiuZf2Ou9bl6HU_kgjF7NK69o3MAllDA=.e52553e8-4aea-4a66-8083-cd2adcd0d80e@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
 <R33eHVRbSeqO1B__lZiFNf6lftx1jOLc7ICjbwrFZ04=.f094f7f3-fe21-43b4-9851-095e54949b7b@github.com>
 <Os_OeWYvDlrhiuZf2Ou9bl6HU_kgjF7NK69o3MAllDA=.e52553e8-4aea-4a66-8083-cd2adcd0d80e@github.com>
Message-ID: <xVupcGcQNbq0wpZ3A6ffQrNQy5vIjIU1t-smkT-ZJyQ=.9a52917c-c976-4a08-afc0-bfbf21822c1c@github.com>

On Thu, 17 Oct 2024 21:44:55 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 394:
>> 
>>> 392: }
>>> 393: 
>>> 394: inline bool ShenandoahHeap::is_not_in_active_young_collection(oop obj) const {
>> 
>> This sounds to me more like: `is_in_old_when_active_young()`.
>> 
>> The "not" is the name makes the negative of its semantics suspect. Think of the sense of what one might believe if one identified the negation with `is_in_active_young_collection()` which I'd take to be `active_generation()->is_young() && is_in_young(obj)`, rather than the actual negation of the condition which would be: `!active_generation()->is_young() || !is_in_old(obj)`.
>
> Can we verify that the predicate is stable in the context of an arbitrary current caller?

Active generation only changes on a safepoint. However, generational membership may change during evacuation.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/517#discussion_r1806775585

From wkemper at openjdk.org  Fri Oct 18 18:21:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 18:21:33 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v4]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <K3hhDTHhqdR_JM1PPMAx9lKjLU35BPoMPHBA0OVpvtw=.096eea95-ccd3-4b4f-b925-0875f76db658@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 487 commits:

 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
   
   Reviewed-by: kdnilsen, ysr
 - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
   
   Reviewed-by: ysr
 - 8342278: GenShen: Move non-generational mode test out of generational test configuration
   
   Reviewed-by: ysr
 - 8342255: GenShen: Remove unnecessary enum initial values
   
   Reviewed-by: ysr
 - 8342001: GenShen: Factor cases for allocation type into separate methods
   
   Reviewed-by: ysr, kdnilsen
 - 8341992: GenShen:  Fix formatting, remove unreachable code, unused imports and unnecessary comments
   
   Reviewed-by: ysr
 - Merge
 - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
 - Fix merge error
 - ... and 477 more: https://git.openjdk.org/jdk/compare/3da68900...b3e4b4ca

-------------

Changes: https://git.openjdk.org/jdk/pull/21273/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=03
  Stats: 22819 lines in 229 files changed: 21077 ins; 890 del; 852 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From ysr at openjdk.org  Fri Oct 18 21:37:38 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 18 Oct 2024 21:37:38 GMT
Subject: RFR: 8342580: GenShen: TestChurnNotifications fails executing in
 unintended test-id modes with ShenandoahGCMode=generational
Message-ID: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>

Generational mode should not be tested with explicit heuristics settings, which are silently ignored. The test failed in these settings because it needs to be told which memory pool to check for notifications, which needs to be specified differently for generational. Ideally, we'd have the test figure this out rather than needing this to be explicitly specified. For now I just tightened the `@requires` clause to run only in the intended modes.

Related cleanups will follow separately (such as emitting a warning when an incompatible heuristics option is specified, code which may have been lost in a previous refactor). See details in the ticket.

**Testing** : tested with no specific collector, with Shenandoah, withGenShen, and with other collectors with test explicitly specified. As intended, in the first and second case 6 test-ids are run, in the thirdd case 1 test-id is run, and in the remaining cases, 0 test-ids are run.

-------------

Commit messages:
 - Merge branch 'master' into test_requires
 - Don't test generational with heuristics it ignores. Separately, we'll
 - TestChurnNotifications fails when jtreg run with GenShen

Changes: https://git.openjdk.org/shenandoah/pull/520/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=520&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342580
  Stats: 5 lines in 1 file changed: 0 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/shenandoah/pull/520.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/520/head:pull/520

PR: https://git.openjdk.org/shenandoah/pull/520

From wkemper at openjdk.org  Fri Oct 18 22:03:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 18 Oct 2024 22:03:37 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <AvYjeULgwDZR2HLH06QJ8W9rf_cjhKBPJLaFOvJeEkI=.87b9391c-e058-4c4f-ba52-0e02521159c8@github.com>
References: <AvYjeULgwDZR2HLH06QJ8W9rf_cjhKBPJLaFOvJeEkI=.87b9391c-e058-4c4f-ba52-0e02521159c8@github.com>
Message-ID: <EnAE_9VGj1aJQlGx_WP83GIyc6R62H-eIJ_mO4-DcpY=.08895f95-859a-4372-9d4d-72009c063987@github.com>

> Merges tag jdk-24+20

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 97 commits:

 - Merge tag 'jdk-24+20' into merge-jdk-24+20
   
   Added tag jdk-24+20 for changeset 7a64fbbb
 - 8338851: Hoist os::Posix::realpath() to os::realpath() and implement on Windows
   
   Reviewed-by: dholmes, stuefe, jwaters
 - 8342338: Remove redundant IIOPURLTest.java
   
   Reviewed-by: cjplummer, amenkov
 - 8341715: PPC64: ObjectMonitor::_owner should be reset unconditionally in nmethod unlocking
   
   Reviewed-by: mdoerr, lucy
 - 8342042: PPC64: compiler_fast_unlock_object flags failure instead of success
   
   Reviewed-by: mdoerr, aboldtch, fbredberg
 - 8340279: Open source several AWT Dialog tests - Batch 2
   
   Reviewed-by: abhiscxk, prr
 - 8341072: Open source several AWT Canvas and Rectangle related tests
   
   Reviewed-by: prr
 - 8341328: Refactor initial Assertion Predicate creation into separate classes
   
   Reviewed-by: thartmann, kvn
 - 8340313: Crash due to invalid oop in nmethod after C1 patching
   
   Reviewed-by: tschatzl, kvn, dlong
 - 8338751: ConfigureNotify behavior has changed in KWin 6.2
   
   Reviewed-by: prr, azvegint, serb
 - ... and 87 more: https://git.openjdk.org/shenandoah/compare/71def7f8...cffd2fa9

-------------

Changes: https://git.openjdk.org/shenandoah/pull/519/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=519&range=01
  Stats: 24434 lines in 541 files changed: 20214 ins; 1645 del; 2575 mod
  Patch: https://git.openjdk.org/shenandoah/pull/519.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/519/head:pull/519

PR: https://git.openjdk.org/shenandoah/pull/519

From ysr at openjdk.org  Fri Oct 18 23:29:05 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 18 Oct 2024 23:29:05 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name [v3]
In-Reply-To: <ru4_Q_uZ-c3VNwk5G1HuVlS0d5Ka90iCyF5bBJt-Qjs=.76ce93f3-712d-4694-bab6-abdcd2cb715d@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
 <ru4_Q_uZ-c3VNwk5G1HuVlS0d5Ka90iCyF5bBJt-Qjs=.76ce93f3-712d-4694-bab6-abdcd2cb715d@github.com>
Message-ID: <MMpQbAeCtbH6lofUxNn75JcW0o4E5dOs0AKfNEJ0x08=.a1d702f9-8268-4017-95df-1c87e2527bf8@github.com>

On Fri, 18 Oct 2024 16:55:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into fix-confusing-method
>  - Use an even better name, add an explanatory comment
>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>    
>    Reviewed-by: xpeng, ysr
>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>    
>    Reviewed-by: ysr
>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>    
>    Reviewed-by: ysr
>  - 8342255: GenShen: Remove unnecessary enum initial values
>    
>    Reviewed-by: ysr
>  - Fix confusing method name
>  - Revert "Test if we still (or ever) needed confusing is_old method"
>    
>    This reverts commit 5fcaef551d3286bd3e5eae485ac11c5b15ce7ed8.
>  - Test if we still (or ever) needed confusing is_old method

LGTM. I like the new name and the clear comment. Thanks!

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/517#pullrequestreview-2379075923

From ysr at openjdk.org  Fri Oct 18 23:37:05 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 18 Oct 2024 23:37:05 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name [v2]
In-Reply-To: <a2B8GDy_gXwYQ9wHUF7SiGEBI9lUlp3FNMHUAUdZw2E=.bffaae46-fd8f-46e5-bbbb-b00cf181be75@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
 <9Idd4Yda2huiWbtP_jd4X4R80igmpcYtkyUU8G7T_PI=.caf6e1ac-44f5-42d9-9437-2dae263b5f85@github.com>
 <a2B8GDy_gXwYQ9wHUF7SiGEBI9lUlp3FNMHUAUdZw2E=.bffaae46-fd8f-46e5-bbbb-b00cf181be75@github.com>
Message-ID: <TNYm4ClyQo9OTvDbclY-cSBFqaFDqaNcGkN15CkBC5Q=.ead828ae-9142-4238-8e69-51ff32aa2150@github.com>

On Fri, 18 Oct 2024 16:50:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This PR is pulling commits from a rebased merge now. I might have to clean this up and force a push.

The failing tests should (hopefully, not sure because I hadn't seen this failure mode before) clear up with the PR I hope to publish later today at https://github.com/openjdk/shenandoah/compare/master...ysramakrishna:shenandoah:test_cleanup once all testing completes: https://bugs.openjdk.org/browse/JDK-8342570

You can probably ignore it for now.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/517#issuecomment-2423371322

From wkemper at openjdk.org  Sat Oct 19 00:02:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Sat, 19 Oct 2024 00:02:37 GMT
Subject: RFR: 8342560: GenShen: Fix confusing method name [v3]
In-Reply-To: <ru4_Q_uZ-c3VNwk5G1HuVlS0d5Ka90iCyF5bBJt-Qjs=.76ce93f3-712d-4694-bab6-abdcd2cb715d@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
 <ru4_Q_uZ-c3VNwk5G1HuVlS0d5Ka90iCyF5bBJt-Qjs=.76ce93f3-712d-4694-bab6-abdcd2cb715d@github.com>
Message-ID: <dRZ6WTPS66dxU-JWr2W-5aDKUuW-dc1scMdNyPuAe1s=.8bc0eb58-1b99-463b-8f10-d2b4e404cd09@github.com>

On Fri, 18 Oct 2024 16:55:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into fix-confusing-method
>  - Use an even better name, add an explanatory comment
>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>    
>    Reviewed-by: xpeng, ysr
>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>    
>    Reviewed-by: ysr
>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>    
>    Reviewed-by: ysr
>  - 8342255: GenShen: Remove unnecessary enum initial values
>    
>    Reviewed-by: ysr
>  - Fix confusing method name
>  - Revert "Test if we still (or ever) needed confusing is_old method"
>    
>    This reverts commit 5fcaef551d3286bd3e5eae485ac11c5b15ce7ed8.
>  - Test if we still (or ever) needed confusing is_old method

I don't think the test failures are related to this change, or the change you're working on. It looks like it has something to do with weak references:

java.lang.RuntimeException: expected testPhantom1 to be cleared
	at gc.shenandoah.TestReferenceRefersToShenandoah.fail(TestReferenceRefersToShenandoah.java:155)
	at gc.shenandoah.TestReferenceRefersToShenandoah.expectCleared(TestReferenceRefersToShenandoah.java:166)

Unfortunately, there is a failure in the reporting script that obscures this. I'll try to reproduce this failure separately.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/517#issuecomment-2423387308

From wkemper at openjdk.org  Sat Oct 19 00:02:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Sat, 19 Oct 2024 00:02:37 GMT
Subject: Integrated: 8342560: GenShen: Fix confusing method name
In-Reply-To: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
References: <fhYIxDKJZu5mqfcVDyr8xtLSR0WaWvmn4g8I9lAc76c=.208afdde-f971-40dd-a464-7a6b26c2c717@github.com>
Message-ID: <Z8WQ7ETjPwJIG-1hrHO2aAtNtKuK46PtFjfY2wJ6J7M=.4f56bd27-3cda-402a-a115-eb6940c4bbb4@github.com>

On Thu, 17 Oct 2024 20:56:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Rename `is_old` to make it more clear that this about pointers that are outside the scope of a young collection.

This pull request has now been integrated.

Changeset: 561c5e0c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/561c5e0c35728fd4411cc346b18a3ee76d33a505
Stats:     11 lines in 3 files changed: 4 ins; 0 del; 7 mod

8342560: GenShen: Fix confusing method name

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/517

From stefank at openjdk.org  Mon Oct 21 07:49:23 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 21 Oct 2024 07:49:23 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
Message-ID: <GuER3Aj4WpJqyLXvCAolRt63PdXIQv-x6Q7V4wKc2iM=.1754df87-fcb5-4734-b181-288ce1f5b9b1@github.com>

On Thu, 17 Oct 2024 10:57:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Compact header riscv (#3)
>   
>   Implement compact headers on RISCV
>   ---------
>   
>   Co-authored-by: hamlin <hamlin at rivosinc.com>

The following test crashes `java/lang/StringBuffer/ECoreIndexOf.java#id1` when running with -XX:+UseCompactObjectHeaders.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2425878646

From stuefe at openjdk.org  Mon Oct 21 12:21:17 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Mon, 21 Oct 2024 12:21:17 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v23]
In-Reply-To: <VjSp2vvwfzZWKtreZyiwjxsSblAX5JHCpF3c0mcYUHs=.b154b3dd-a8f2-4fdf-8176-52709f906891@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <0BrAbBTKmpqTGDrc--2znzO8t07yoqabwa6g2K05GHI=.d3c17fd5-4770-4623-8d2f-604816afc033@github.com>
 <VjSp2vvwfzZWKtreZyiwjxsSblAX5JHCpF3c0mcYUHs=.b154b3dd-a8f2-4fdf-8176-52709f906891@github.com>
Message-ID: <7CtwplIs-ILCZhSpPwPtZr-GPCd_XxtOnYwku9zIQTY=.8d8a96e1-9627-4da6-ae57-b692e0580598@github.com>

On Fri, 20 Sep 2024 17:46:21 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'lilliput/JEP-450-temporary-fix-branch-2' into JDK-8305895-v4
>>  - review feedback
>
> src/hotspot/share/memory/metaspace.cpp line 799:
> 
>> 797: 
>> 798:     // Set up compressed class pointer encoding.
>> 799:     // In CDS=off mode, we give the JVM some leeway to choose a favorable base/shift combination.
> 
> I don't know why this comment is here.  Seems out of place.

Its not, but maybe too vague.

There are two ways to initialize CompressedKlassPointers :

- `CompressedKlassPointers::initialize(address, size)` - called here - is used for no CDS case and allows the JVM to freely pick encoding base and shift.
- `CompressedKlassPointers::initialize_for_given_encoding` is called when encoding base and shift are predetermined (when using CDS). Then, the JVM has no freedom at all, it just does sanity checks.

The comment basically says "since here we are not using CDS, we are calling CompressedKlassPointers::initialize(address, size) to give the JVM some freedom when choosing encoding base and shift".

Is this clearer? Should I just remove the code?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1808683312

From stuefe at openjdk.org  Mon Oct 21 12:53:13 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Mon, 21 Oct 2024 12:53:13 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v44]
In-Reply-To: <cQ6B7ZfZdJrEd_dsmQouGG3i45UcX_e_5f1K7fdvTuI=.82c67fe3-671a-4b1e-bfb1-979e65d56b19@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <logRSsOv-ZbKrosBAddqiXOOAcr-tMODEDmGksY6uJs=.ee83f2c3-2b57-41f5-b9f4-9b36811ced37@github.com>
 <cQ6B7ZfZdJrEd_dsmQouGG3i45UcX_e_5f1K7fdvTuI=.82c67fe3-671a-4b1e-bfb1-979e65d56b19@github.com>
Message-ID: <rNbxLonojk0uB0Tg01JpxevDvOxQMG45LludjywTzNU=.3d69f1a5-38cb-441c-90ba-0b027e6bd46e@github.com>

On Wed, 16 Oct 2024 15:37:59 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Problem-list SharedBaseAddress tests on aarch64
>
> src/hotspot/share/oops/compressedKlass.cpp line 185:
> 
>> 183: #endif
>> 184: 
>> 185:   DEBUG_ONLY(sanity_check_after_initialization();)
> 
> This is here twice.

sanity_check_after_initialization is called from both initialization routines, but we only call either one or the other.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1808735324

From ihse at openjdk.org  Mon Oct 21 13:05:13 2024
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Mon, 21 Oct 2024 13:05:13 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
Message-ID: <irVXsRVBlVnhU-iZOsdNTZ7Kg-LU52sApukBUSe3qos=.f6f89542-8ec7-4026-97f6-ca4bde99bf73@github.com>

On Thu, 17 Oct 2024 10:57:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Compact header riscv (#3)
>   
>   Implement compact headers on RISCV
>   ---------
>   
>   Co-authored-by: hamlin <hamlin at rivosinc.com>

Marked as reviewed by ihse (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2382030332

From stefank at openjdk.org  Mon Oct 21 13:05:14 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 21 Oct 2024 13:05:14 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
Message-ID: <ykhwOpVpX0wlLZaUM8N1c6NaMVH0hn9UfCikwoTrHkw=.d30174f6-2138-481c-9929-a475fd187a56@github.com>

On Thu, 17 Oct 2024 10:57:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Compact header riscv (#3)
>   
>   Implement compact headers on RISCV
>   ---------
>   
>   Co-authored-by: hamlin <hamlin at rivosinc.com>

I've managed to reproduce the ECoreIndexOf crash locally by running with -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders. The crash happens on line 773 when reading past the needle.

?      762     __ movq(index, needle_len);
?
?      763     __ andq(index, 0xf);  // nLen % 16
?      764     __ movq(offset, 0x10);
?      765     __ subq(offset, index);  // 16 - (nLen % 16)
?      766     __ movq(index, offset);
?      767     __ shlq(offset, 1);  // * 2
?      768     __ negq(index);      // -(16 - (nLen % 16))
?
?      769     __ xorq(wr_index, wr_index);
?      770
?      771     __ bind(L_top);
?      772     // load needle and expand 
?      773     __ vpmovzxbw(xmm0, Address(needle, index, Address::times_1), Assembler::AVX_256bit);


We're reading this address:

(SEGV_MAPERR), si_addr: 0x00000007cffffffe


which is just before the start of the heap:

Heap address: 0x00000007d0000000, size: 768 MB, Compressed Oops mode: Zero based, Oop shift amount: 3


When this crashed I had:

needle: 0x00000007d000000c
needle_len = 0x12
index = 0xfffffffffffffffe


There has been previous fix to not read past the haystack:
Fix header < 16 bytes in indexOf intrinsic, by @sviswa7
https://github.com/openjdk/jdk/pull/20677/commits/f65ef5dc325212155a50a2fc3a7f4aad18b8d9d0

maybe we need something similar for the needle.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2426614072

From coleenp at openjdk.org  Mon Oct 21 13:05:14 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Mon, 21 Oct 2024 13:05:14 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
Message-ID: <5bn9rl6P9_im5yuUGUuMF6Pn8bcn8nHt1xX6q3b03L4=.0d8ed88a-9599-42b1-9eec-fbab93cf3e37@github.com>

On Thu, 17 Oct 2024 10:57:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Compact header riscv (#3)
>   
>   Implement compact headers on RISCV
>   ---------
>   
>   Co-authored-by: hamlin <hamlin at rivosinc.com>

I think a lot of copyright headers need to be updated.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2426605597

From syan at openjdk.org  Mon Oct 21 13:06:39 2024
From: syan at openjdk.org (SendaoYan)
Date: Mon, 21 Oct 2024 13:06:39 GMT
Subject: RFR: 8342681: TestLoadBypassesNullCheck.java fails improperly
 specified VM option
Message-ID: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>

Hi all,
Newly added test `gc/shenandoah/compiler/TestLoadBypassesNullCheck.java` fails `The unlock option must precede 'StressGCM'.` with release jdk build. This PR fix the test bug.
The change has been verified locally with release/fastdebug/slowdebug on linux-x64. Test-fix only, no risk.

-------------

Commit messages:
 - 8342681: TestLoadBypassesNullCheck.java fails improperly specified VM option

Changes: https://git.openjdk.org/jdk/pull/21609/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21609&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342681
  Stats: 6 lines in 1 file changed: 2 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/21609.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21609/head:pull/21609

PR: https://git.openjdk.org/jdk/pull/21609

From rkennke at openjdk.org  Mon Oct 21 13:56:35 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 21 Oct 2024 13:56:35 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
Message-ID: <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>

On Thu, 17 Oct 2024 10:57:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Compact header riscv (#3)
>   
>   Implement compact headers on RISCV
>   ---------
>   
>   Co-authored-by: hamlin <hamlin at rivosinc.com>

> I've managed to reproduce the ECoreIndexOf crash locally by running with -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders. The crash happens on line 773 when reading past the needle.
> 
> ```
> ?      762     __ movq(index, needle_len);
> ?
> ?      763     __ andq(index, 0xf);  // nLen % 16
> ?      764     __ movq(offset, 0x10);
> ?      765     __ subq(offset, index);  // 16 - (nLen % 16)
> ?      766     __ movq(index, offset);
> ?      767     __ shlq(offset, 1);  // * 2
> ?      768     __ negq(index);      // -(16 - (nLen % 16))
> ?
> ?      769     __ xorq(wr_index, wr_index);
> ?      770
> ?      771     __ bind(L_top);
> ?      772     // load needle and expand 
> ?      773     __ vpmovzxbw(xmm0, Address(needle, index, Address::times_1), Assembler::AVX_256bit);
> ```
> 
> We're reading this address:
> 
> ```
> (SEGV_MAPERR), si_addr: 0x00000007cffffffe
> ```
> 
> which is just before the start of the heap:
> 
> ```
> Heap address: 0x00000007d0000000, size: 768 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
> ```
> 
> When this crashed I had:
> 
> ```
> needle: 0x00000007d000000c
> needle_len = 0x12
> index = 0xfffffffffffffffe
> ```
> 
> There has been previous fix to not read past the haystack: Fix header < 16 bytes in indexOf intrinsic, by @sviswa7 [f65ef5d](https://github.com/openjdk/jdk/commit/f65ef5dc325212155a50a2fc3a7f4aad18b8d9d0)
> 
> maybe we need something similar for the needle.

@sviswa7 @vpaprotsk could you have a look? If we can have a reasonable fix for this soon, we could ship it in this PR, otherwise I'd defer it to a follow-up issue and disable indexOf intrinsic when running with +UseCompactObjectHeaders.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2426754934

From duke at openjdk.org  Mon Oct 21 14:23:53 2024
From: duke at openjdk.org (Volodymyr Paprotski)
Date: Mon, 21 Oct 2024 14:23:53 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
 <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
Message-ID: <-mco1SNkDsaGs1iPfVA_rYxd2rjKseRvjMMMO1KkDog=.ca9caf95-e6c7-4456-ace6-183c4ef45554@github.com>

On Mon, 21 Oct 2024 13:53:58 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Compact header riscv (#3)
>>   
>>   Implement compact headers on RISCV
>>   ---------
>>   
>>   Co-authored-by: hamlin <hamlin at rivosinc.com>
>
>> I've managed to reproduce the ECoreIndexOf crash locally by running with -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders. The crash happens on line 773 when reading past the needle.
>> 
>> ```
>> ?      762     __ movq(index, needle_len);
>> ?
>> ?      763     __ andq(index, 0xf);  // nLen % 16
>> ?      764     __ movq(offset, 0x10);
>> ?      765     __ subq(offset, index);  // 16 - (nLen % 16)
>> ?      766     __ movq(index, offset);
>> ?      767     __ shlq(offset, 1);  // * 2
>> ?      768     __ negq(index);      // -(16 - (nLen % 16))
>> ?
>> ?      769     __ xorq(wr_index, wr_index);
>> ?      770
>> ?      771     __ bind(L_top);
>> ?      772     // load needle and expand 
>> ?      773     __ vpmovzxbw(xmm0, Address(needle, index, Address::times_1), Assembler::AVX_256bit);
>> ```
>> 
>> We're reading this address:
>> 
>> ```
>> (SEGV_MAPERR), si_addr: 0x00000007cffffffe
>> ```
>> 
>> which is just before the start of the heap:
>> 
>> ```
>> Heap address: 0x00000007d0000000, size: 768 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
>> ```
>> 
>> When this crashed I had:
>> 
>> ```
>> needle: 0x00000007d000000c
>> needle_len = 0x12
>> index = 0xfffffffffffffffe
>> ```
>> 
>> There has been previous fix to not read past the haystack: Fix header < 16 bytes in indexOf intrinsic, by @sviswa7 [f65ef5d](https://github.com/openjdk/jdk/commit/f65ef5dc325212155a50a2fc3a7f4aad18b8d9d0)
>> 
>> maybe we need something similar for the needle.
> 
> @sviswa7 @vpaprotsk could you have a look? If we can have a reasonable fix for this soon, we could ship it in this PR, otherwise I'd defer it to a follow-up issue and disable indexOf intrinsic when running with +UseCompactObjectHeaders.

@rkennke looking!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2426828440

From wkemper at openjdk.org  Mon Oct 21 15:56:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 21 Oct 2024 15:56:31 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <AvYjeULgwDZR2HLH06QJ8W9rf_cjhKBPJLaFOvJeEkI=.87b9391c-e058-4c4f-ba52-0e02521159c8@github.com>
References: <AvYjeULgwDZR2HLH06QJ8W9rf_cjhKBPJLaFOvJeEkI=.87b9391c-e058-4c4f-ba52-0e02521159c8@github.com>
Message-ID: <4q8zrEmtGm5xmGHLh24WFelmfOCru0xdgtHPWIZI4GU=.af672d33-792e-40a3-94ed-af378622f5ed@github.com>

On Fri, 18 Oct 2024 14:12:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-24+20

This pull request has now been integrated.

Changeset: 47d77e19
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/47d77e199233297f4285d6eb5bc36ec0cfd4f659
Stats:     24434 lines in 541 files changed: 20214 ins; 1645 del; 2575 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/519

From wkemper at openjdk.org  Mon Oct 21 15:57:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 21 Oct 2024 15:57:38 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v4]
In-Reply-To: <mbqd_3S_N2z-b7NqIWPnl1U0za1AwWY5lXUZK-KgWoI=.10c8544d-b66a-44c5-add5-2924776e7b29@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <uBJ3HKwo99Z9hOLVImVS43JCUFKCzMA-i4OcaSyw34w=.5b551cc9-9919-4e3f-8e8d-227e536dcd3c@github.com>
 <mbqd_3S_N2z-b7NqIWPnl1U0za1AwWY5lXUZK-KgWoI=.10c8544d-b66a-44c5-add5-2924776e7b29@github.com>
Message-ID: <vsDvcAVYiPLoyUWsg6IsSPM6YlFeGQJL-wJfJK8F_dI=.8ab408ba-4300-4e59-9afe-cf3f87a64c81@github.com>

On Fri, 11 Oct 2024 21:17:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 579:
>> 
>>> 577:   st->print("Status: ");
>>> 578:   if (has_forwarded_objects())                 st->print("has forwarded objects, ");
>>> 579:   if (is_concurrent_mark_in_progress())        st->print("marking, ");
>> 
>> What is this printing when not running with generational mode?
>
> It will print "young marking". We can change this.

https://bugs.openjdk.org/browse/JDK-8342564

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1809085061

From shade at openjdk.org  Mon Oct 21 16:02:25 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 21 Oct 2024 16:02:25 GMT
Subject: RFR: 8342681: TestLoadBypassesNullCheck.java fails improperly
 specified VM option
In-Reply-To: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
References: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
Message-ID: <7AWUBcjYaW3RPVrNFaw27Mj8JNn0OaOGK4BsoW7V32Q=.6233d51f-0df5-4cbd-b26c-44c3eb0227fb@github.com>

On Mon, 21 Oct 2024 13:00:23 GMT, SendaoYan <syan at openjdk.org> wrote:

> Hi all,
> Newly added test `gc/shenandoah/compiler/TestLoadBypassesNullCheck.java` fails `The unlock option must precede 'StressGCM'.` with release jdk build. This PR fix the test bug, only add `-XX:+UnlockDiagnosticVMOptions` to make test run passed by release jdk build.
> The change has been verified locally with release/fastdebug/slowdebug on linux-x64. Test-fix only, no risk.

Marked as reviewed by shade (Reviewer).

test/hotspot/jtreg/gc/shenandoah/compiler/TestLoadBypassesNullCheck.java line 33:

> 31:  * @run main/othervm -XX:+UnlockDiagnosticVMOptions -XX:-TieredCompilation -XX:-UseOnStackReplacement -XX:-BackgroundCompilation
> 32:  *                   -XX:+StressGCM -XX:+StressLCM -XX:+UseShenandoahGC -XX:LoopMaxUnroll=0 -XX:StressSeed=270847015
> 33:  *                   TestLoadBypassesNullCheck

I am good with this fix, but if you have time to re-test it, consider reshuffling the arguments a bit, so that we only unlock what we need:


 * @run main/othervm -XX:-TieredCompilation -XX:-UseOnStackReplacement -XX:-BackgroundCompilation 
 *                   -XX:+UseShenandoahGC -XX:LoopMaxUnroll=0
 *                   -XX:+UnlockDiagnosticVMOptions -XX:+StressGCM -XX:+StressLCM -XX:StressSeed=270847015
 *                   TestLoadBypassesNullCheck

-------------

PR Review: https://git.openjdk.org/jdk/pull/21609#pullrequestreview-2382559833
PR Review Comment: https://git.openjdk.org/jdk/pull/21609#discussion_r1809092713

From syan at openjdk.org  Mon Oct 21 16:44:26 2024
From: syan at openjdk.org (SendaoYan)
Date: Mon, 21 Oct 2024 16:44:26 GMT
Subject: RFR: 8342681: TestLoadBypassesNullCheck.java fails improperly
 specified VM option [v2]
In-Reply-To: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
References: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
Message-ID: <zae0V9b4ewNnnyGGFjsBDe0mIwpsFL3oTrSg6P1B5iQ=.66df7883-3c1e-4dcf-8125-6ced76fe4901@github.com>

> Hi all,
> Newly added test `gc/shenandoah/compiler/TestLoadBypassesNullCheck.java` fails `The unlock option must precede 'StressGCM'.` with release jdk build. This PR fix the test bug, only add `-XX:+UnlockDiagnosticVMOptions` to make test run passed by release jdk build.
> The change has been verified locally with release/fastdebug/slowdebug on linux-x64. Test-fix only, no risk.

SendaoYan has updated the pull request incrementally with one additional commit since the last revision:

  only unlock Stress* VM option

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21609/files
  - new: https://git.openjdk.org/jdk/pull/21609/files/43731b9e..b7237e8e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21609&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21609&range=00-01

  Stats: 6 lines in 1 file changed: 2 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/21609.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21609/head:pull/21609

PR: https://git.openjdk.org/jdk/pull/21609

From syan at openjdk.org  Mon Oct 21 16:48:17 2024
From: syan at openjdk.org (SendaoYan)
Date: Mon, 21 Oct 2024 16:48:17 GMT
Subject: RFR: 8342681: TestLoadBypassesNullCheck.java fails improperly
 specified VM option [v2]
In-Reply-To: <7AWUBcjYaW3RPVrNFaw27Mj8JNn0OaOGK4BsoW7V32Q=.6233d51f-0df5-4cbd-b26c-44c3eb0227fb@github.com>
References: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
 <7AWUBcjYaW3RPVrNFaw27Mj8JNn0OaOGK4BsoW7V32Q=.6233d51f-0df5-4cbd-b26c-44c3eb0227fb@github.com>
Message-ID: <KEpK5OPhcm1-7oCo74bAWBhnOz-ilHLe499C8ge19Cg=.cec3ad6f-c742-4da5-9525-877cec3bcae8@github.com>

On Mon, 21 Oct 2024 15:59:43 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   only unlock Stress* VM option
>
> test/hotspot/jtreg/gc/shenandoah/compiler/TestLoadBypassesNullCheck.java line 33:
> 
>> 31:  * @run main/othervm -XX:+UnlockDiagnosticVMOptions -XX:-TieredCompilation -XX:-UseOnStackReplacement -XX:-BackgroundCompilation
>> 32:  *                   -XX:+StressGCM -XX:+StressLCM -XX:+UseShenandoahGC -XX:LoopMaxUnroll=0 -XX:StressSeed=270847015
>> 33:  *                   TestLoadBypassesNullCheck
> 
> I am good with this fix, but if you have time to re-test it, consider reshuffling the arguments a bit, so that we only unlock what we need:
> 
> 
>  * @run main/othervm -XX:-TieredCompilation -XX:-UseOnStackReplacement -XX:-BackgroundCompilation 
>  *                   -XX:+UseShenandoahGC -XX:LoopMaxUnroll=0
>  *                   -XX:+UnlockDiagnosticVMOptions -XX:+StressGCM -XX:+StressLCM -XX:StressSeed=270847015
>  *                   TestLoadBypassesNullCheck

Thanks for your correction, arguments has been argument reshuffled.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21609#discussion_r1809156369

From duke at openjdk.org  Mon Oct 21 16:56:35 2024
From: duke at openjdk.org (Volodymyr Paprotski)
Date: Mon, 21 Oct 2024 16:56:35 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
 <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
Message-ID: <xuz6jeipBqArYVTgblHr2g9l8UCmFrXf8M_3oMJjQZo=.7abd3294-a20e-4605-abc8-d40cd7c6d5a4@github.com>

On Mon, 21 Oct 2024 13:53:58 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Compact header riscv (#3)
>>   
>>   Implement compact headers on RISCV
>>   ---------
>>   
>>   Co-authored-by: hamlin <hamlin at rivosinc.com>
>
>> I've managed to reproduce the ECoreIndexOf crash locally by running with -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders. The crash happens on line 773 when reading past the needle.
>> 
>> ```
>> ?      762     __ movq(index, needle_len);
>> ?
>> ?      763     __ andq(index, 0xf);  // nLen % 16
>> ?      764     __ movq(offset, 0x10);
>> ?      765     __ subq(offset, index);  // 16 - (nLen % 16)
>> ?      766     __ movq(index, offset);
>> ?      767     __ shlq(offset, 1);  // * 2
>> ?      768     __ negq(index);      // -(16 - (nLen % 16))
>> ?
>> ?      769     __ xorq(wr_index, wr_index);
>> ?      770
>> ?      771     __ bind(L_top);
>> ?      772     // load needle and expand 
>> ?      773     __ vpmovzxbw(xmm0, Address(needle, index, Address::times_1), Assembler::AVX_256bit);
>> ```
>> 
>> We're reading this address:
>> 
>> ```
>> (SEGV_MAPERR), si_addr: 0x00000007cffffffe
>> ```
>> 
>> which is just before the start of the heap:
>> 
>> ```
>> Heap address: 0x00000007d0000000, size: 768 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
>> ```
>> 
>> When this crashed I had:
>> 
>> ```
>> needle: 0x00000007d000000c
>> needle_len = 0x12
>> index = 0xfffffffffffffffe
>> ```
>> 
>> There has been previous fix to not read past the haystack: Fix header < 16 bytes in indexOf intrinsic, by @sviswa7 [f65ef5d](https://github.com/openjdk/jdk/commit/f65ef5dc325212155a50a2fc3a7f4aad18b8d9d0)
>> 
>> maybe we need something similar for the needle.
> 
> @sviswa7 @vpaprotsk could you have a look? If we can have a reasonable fix for this soon, we could ship it in this PR, otherwise I'd defer it to a follow-up issue and disable indexOf intrinsic when running with +UseCompactObjectHeaders.

@rkennke Could you post the full command you used please? And perhaps also the seed that gets printed.. having trouble getting it to fail.. 

So far I added a few options and perrmitations of: `./build/linux-x86_64-server-fastdebug/images/jdk/bin/java -Xcomp -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:+EnableX86ECoreOpts -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders test/jdk/java/lang/StringBuffer/ECoreIndexOf.java`  and lo luck.. IndexOf.java test checks "all interesting" lengths of haystack and needle and can't get it to fail either.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2427232140

From shade at openjdk.org  Mon Oct 21 17:16:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 21 Oct 2024 17:16:14 GMT
Subject: RFR: 8342681: TestLoadBypassesNullCheck.java fails improperly
 specified VM option [v2]
In-Reply-To: <zae0V9b4ewNnnyGGFjsBDe0mIwpsFL3oTrSg6P1B5iQ=.66df7883-3c1e-4dcf-8125-6ced76fe4901@github.com>
References: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
 <zae0V9b4ewNnnyGGFjsBDe0mIwpsFL3oTrSg6P1B5iQ=.66df7883-3c1e-4dcf-8125-6ced76fe4901@github.com>
Message-ID: <4WzxRQ2msIu_FM_u_wus9_8y3YrAaG-L3oR812lVcA8=.3508a1ce-7da4-4822-b71e-087a81cf7958@github.com>

On Mon, 21 Oct 2024 16:44:26 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>> Newly added test `gc/shenandoah/compiler/TestLoadBypassesNullCheck.java` fails `The unlock option must precede 'StressGCM'.` with release jdk build. This PR fix the test bug, only add `-XX:+UnlockDiagnosticVMOptions` to make test run passed by release jdk build.
>> The change has been verified locally with release/fastdebug/slowdebug on linux-x64. Test-fix only, no risk.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   only unlock Stress* VM option

Thank you!

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21609#pullrequestreview-2382778280

From rkennke at openjdk.org  Mon Oct 21 18:04:53 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 21 Oct 2024 18:04:53 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
 <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
Message-ID: <BCZjoP4Ao9C1F8kQJmpfooJCnGx3Ef-YNKGHdqZMSCs=.234de599-a258-45ec-9e63-7839f3a48e54@github.com>

On Mon, 21 Oct 2024 13:53:58 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Compact header riscv (#3)
>>   
>>   Implement compact headers on RISCV
>>   ---------
>>   
>>   Co-authored-by: hamlin <hamlin at rivosinc.com>
>
>> I've managed to reproduce the ECoreIndexOf crash locally by running with -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders. The crash happens on line 773 when reading past the needle.
>> 
>> ```
>> ?      762     __ movq(index, needle_len);
>> ?
>> ?      763     __ andq(index, 0xf);  // nLen % 16
>> ?      764     __ movq(offset, 0x10);
>> ?      765     __ subq(offset, index);  // 16 - (nLen % 16)
>> ?      766     __ movq(index, offset);
>> ?      767     __ shlq(offset, 1);  // * 2
>> ?      768     __ negq(index);      // -(16 - (nLen % 16))
>> ?
>> ?      769     __ xorq(wr_index, wr_index);
>> ?      770
>> ?      771     __ bind(L_top);
>> ?      772     // load needle and expand 
>> ?      773     __ vpmovzxbw(xmm0, Address(needle, index, Address::times_1), Assembler::AVX_256bit);
>> ```
>> 
>> We're reading this address:
>> 
>> ```
>> (SEGV_MAPERR), si_addr: 0x00000007cffffffe
>> ```
>> 
>> which is just before the start of the heap:
>> 
>> ```
>> Heap address: 0x00000007d0000000, size: 768 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
>> ```
>> 
>> When this crashed I had:
>> 
>> ```
>> needle: 0x00000007d000000c
>> needle_len = 0x12
>> index = 0xfffffffffffffffe
>> ```
>> 
>> There has been previous fix to not read past the haystack: Fix header < 16 bytes in indexOf intrinsic, by @sviswa7 [f65ef5d](https://github.com/openjdk/jdk/commit/f65ef5dc325212155a50a2fc3a7f4aad18b8d9d0)
>> 
>> maybe we need something similar for the needle.
> 
> @sviswa7 @vpaprotsk could you have a look? If we can have a reasonable fix for this soon, we could ship it in this PR, otherwise I'd defer it to a follow-up issue and disable indexOf intrinsic when running with +UseCompactObjectHeaders.

> @rkennke Could you post the full command you used please? And perhaps also the seed that gets printed.. having trouble getting it to fail..
> 
> So far I added a few options and perrmitations of: `./build/linux-x86_64-server-fastdebug/images/jdk/bin/java -Xcomp -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:+EnableX86ECoreOpts -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders test/jdk/java/lang/StringBuffer/ECoreIndexOf.java` and lo luck.. IndexOf.java test checks "all interesting" lengths of haystack and needle and can't get it to fail either.

I could reproduce on 3rd try with a fastdebug build with:

make test TEST=java/lang/StringBuffer/ECoreIndexOf.java TEST_VM_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders -XX:+UseSerialGC"


It prints:

Seed set to 636754923980405411


It probably depends on GC operation: It would only fail when the array happens to be the very first object in the heap. The relevant GC/heap configs would be:

InitialHeapSize                          = 805306368
MaxHeapSize                              = 805306368
MaxNewSize                               = 268435456


So you should probably also add `-Xmx805306368 -Xms805306368 -Xmn268435456`

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2427375886

From duke at openjdk.org  Mon Oct 21 18:55:52 2024
From: duke at openjdk.org (Volodymyr Paprotski)
Date: Mon, 21 Oct 2024 18:55:52 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <BCZjoP4Ao9C1F8kQJmpfooJCnGx3Ef-YNKGHdqZMSCs=.234de599-a258-45ec-9e63-7839f3a48e54@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
 <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
 <BCZjoP4Ao9C1F8kQJmpfooJCnGx3Ef-YNKGHdqZMSCs=.234de599-a258-45ec-9e63-7839f3a48e54@github.com>
Message-ID: <NN3GcO1kEKz8t-s1nwOkHjwhaO0Gy_Jgyo0oyH-Z_cU=.4fd9a5da-6f34-4343-ba41-3efbcbf826fc@github.com>

On Mon, 21 Oct 2024 18:00:14 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> I've managed to reproduce the ECoreIndexOf crash locally by running with -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders. The crash happens on line 773 when reading past the needle.
>>> 
>>> ```
>>> ?      762     __ movq(index, needle_len);
>>> ?
>>> ?      763     __ andq(index, 0xf);  // nLen % 16
>>> ?      764     __ movq(offset, 0x10);
>>> ?      765     __ subq(offset, index);  // 16 - (nLen % 16)
>>> ?      766     __ movq(index, offset);
>>> ?      767     __ shlq(offset, 1);  // * 2
>>> ?      768     __ negq(index);      // -(16 - (nLen % 16))
>>> ?
>>> ?      769     __ xorq(wr_index, wr_index);
>>> ?      770
>>> ?      771     __ bind(L_top);
>>> ?      772     // load needle and expand 
>>> ?      773     __ vpmovzxbw(xmm0, Address(needle, index, Address::times_1), Assembler::AVX_256bit);
>>> ```
>>> 
>>> We're reading this address:
>>> 
>>> ```
>>> (SEGV_MAPERR), si_addr: 0x00000007cffffffe
>>> ```
>>> 
>>> which is just before the start of the heap:
>>> 
>>> ```
>>> Heap address: 0x00000007d0000000, size: 768 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
>>> ```
>>> 
>>> When this crashed I had:
>>> 
>>> ```
>>> needle: 0x00000007d000000c
>>> needle_len = 0x12
>>> index = 0xfffffffffffffffe
>>> ```
>>> 
>>> There has been previous fix to not read past the haystack: Fix header < 16 bytes in indexOf intrinsic, by @sviswa7 [f65ef5d](https://github.com/openjdk/jdk/commit/f65ef5dc325212155a50a2fc3a7f4aad18b8d9d0)
>>> 
>>> maybe we need something similar for the needle.
>> 
>> @sviswa7 @vpaprotsk could you have a look? If we can have a reasonable fix for this soon, we could ship it in this PR, otherwise I'd defer it to a follow-up issue and disable indexOf intrinsic when running with +UseCompactObjectHeaders.
>
>> @rkennke Could you post the full command you used please? And perhaps also the seed that gets printed.. having trouble getting it to fail..
>> 
>> So far I added a few options and perrmitations of: `./build/linux-x86_64-server-fastdebug/images/jdk/bin/java -Xcomp -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:+EnableX86ECoreOpts -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders test/jdk/java/lang/StringBuffer/ECoreIndexOf.java` and lo luck.. IndexOf.java test checks "all interesting" lengths of haystack and needle and can't get it to fail either.
> 
> I could reproduce on 3rd try with a fastdebug build with:
> 
> make test TEST=java/lang/StringBuffer/ECoreIndexOf.java TEST_VM_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders -XX:+UseSerialGC"
> 
> 
> It prints:
> 
> Seed set to 636754923980405411
> 
> 
> It probably depends on GC operation: It would only fail when the array happens to be the very first object in the heap. The relevant GC/heap configs would be:
> 
> InitialHeapSize                          = 805306368
> MaxHeapSize                              = 805306368
> MaxNewSize                               = 268435456
> 
> 
> So you should probably also add `-Xmx805306368 -Xms805306368 -Xmn268435456`

Thanks @rkennke able to reproduce now.. Sandhya will have a patch soon and I will re-verify

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2427477113

From sviswanathan at openjdk.org  Mon Oct 21 19:26:39 2024
From: sviswanathan at openjdk.org (Sandhya Viswanathan)
Date: Mon, 21 Oct 2024 19:26:39 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <NN3GcO1kEKz8t-s1nwOkHjwhaO0Gy_Jgyo0oyH-Z_cU=.4fd9a5da-6f34-4343-ba41-3efbcbf826fc@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
 <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
 <BCZjoP4Ao9C1F8kQJmpfooJCnGx3Ef-YNKGHdqZMSCs=.234de599-a258-45ec-9e63-7839f3a48e54@github.com>
 <NN3GcO1kEKz8t-s1nwOkHjwhaO0Gy_Jgyo0oyH-Z_cU=.4fd9a5da-6f34-4343-ba41-3efbcbf826fc@github.com>
Message-ID: <QzE3fKHgOAo9Jpsq7O_cer2evaHfByqgdv5bSU09-08=.6c26375c-26a1-40a9-bb8c-432e2a93c860@github.com>

On Mon, 21 Oct 2024 18:52:46 GMT, Volodymyr Paprotski <duke at openjdk.org> wrote:

> Thanks @rkennke able to reproduce now.. Sandhya will have a patch soon and I will re-verify

@rkennke @vpaprotsk Please find attached the patch which should fix the problem.

[smallneedlefix.patch](https://github.com/user-attachments/files/17466073/smallneedlefix.patch)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2427536012

From rkennke at openjdk.org  Mon Oct 21 20:34:41 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 21 Oct 2024 20:34:41 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <BCZjoP4Ao9C1F8kQJmpfooJCnGx3Ef-YNKGHdqZMSCs=.234de599-a258-45ec-9e63-7839f3a48e54@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
 <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
 <BCZjoP4Ao9C1F8kQJmpfooJCnGx3Ef-YNKGHdqZMSCs=.234de599-a258-45ec-9e63-7839f3a48e54@github.com>
Message-ID: <qXNHwb48lt9LL_DePcb6QLEE0d_rsPQGzI_Bz5Swyzc=.e2d2f513-816d-4bf2-9dbd-67fb74522cfd@github.com>

On Mon, 21 Oct 2024 18:00:14 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> I've managed to reproduce the ECoreIndexOf crash locally by running with -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders. The crash happens on line 773 when reading past the needle.
>>> 
>>> ```
>>> ?      762     __ movq(index, needle_len);
>>> ?
>>> ?      763     __ andq(index, 0xf);  // nLen % 16
>>> ?      764     __ movq(offset, 0x10);
>>> ?      765     __ subq(offset, index);  // 16 - (nLen % 16)
>>> ?      766     __ movq(index, offset);
>>> ?      767     __ shlq(offset, 1);  // * 2
>>> ?      768     __ negq(index);      // -(16 - (nLen % 16))
>>> ?
>>> ?      769     __ xorq(wr_index, wr_index);
>>> ?      770
>>> ?      771     __ bind(L_top);
>>> ?      772     // load needle and expand 
>>> ?      773     __ vpmovzxbw(xmm0, Address(needle, index, Address::times_1), Assembler::AVX_256bit);
>>> ```
>>> 
>>> We're reading this address:
>>> 
>>> ```
>>> (SEGV_MAPERR), si_addr: 0x00000007cffffffe
>>> ```
>>> 
>>> which is just before the start of the heap:
>>> 
>>> ```
>>> Heap address: 0x00000007d0000000, size: 768 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
>>> ```
>>> 
>>> When this crashed I had:
>>> 
>>> ```
>>> needle: 0x00000007d000000c
>>> needle_len = 0x12
>>> index = 0xfffffffffffffffe
>>> ```
>>> 
>>> There has been previous fix to not read past the haystack: Fix header < 16 bytes in indexOf intrinsic, by @sviswa7 [f65ef5d](https://github.com/openjdk/jdk/commit/f65ef5dc325212155a50a2fc3a7f4aad18b8d9d0)
>>> 
>>> maybe we need something similar for the needle.
>> 
>> @sviswa7 @vpaprotsk could you have a look? If we can have a reasonable fix for this soon, we could ship it in this PR, otherwise I'd defer it to a follow-up issue and disable indexOf intrinsic when running with +UseCompactObjectHeaders.
>
>> @rkennke Could you post the full command you used please? And perhaps also the seed that gets printed.. having trouble getting it to fail..
>> 
>> So far I added a few options and perrmitations of: `./build/linux-x86_64-server-fastdebug/images/jdk/bin/java -Xcomp -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:+EnableX86ECoreOpts -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders test/jdk/java/lang/StringBuffer/ECoreIndexOf.java` and lo luck.. IndexOf.java test checks "all interesting" lengths of haystack and needle and can't get it to fail either.
> 
> I could reproduce on 3rd try with a fastdebug build with:
> 
> make test TEST=java/lang/StringBuffer/ECoreIndexOf.java TEST_VM_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders -XX:+UseSerialGC"
> 
> 
> It prints:
> 
> Seed set to 636754923980405411
> 
> 
> It probably depends on GC operation: It would only fail when the array happens to be the very first object in the heap. The relevant GC/heap configs would be:
> 
> InitialHeapSize                          = 805306368
> MaxHeapSize                              = 805306368
> MaxNewSize                               = 268435456
> 
> 
> So you should probably also add `-Xmx805306368 -Xms805306368 -Xmn268435456`

> > Thanks @rkennke able to reproduce now.. Sandhya will have a patch soon and I will re-verify
> 
> @rkennke @vpaprotsk Please find attached the patch which should fix the problem.
> 
> [smallneedlefix.patch](https://github.com/user-attachments/files/17466073/smallneedlefix.patch)

Testing now. Runs the reproducer in a loop since half an hour without crashing. I'll let it run overnight, and if @vpaprotsk approves the changes, then I'll intergrate them tomorrow morning.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2427660618

From wkemper at openjdk.org  Mon Oct 21 21:04:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 21 Oct 2024 21:04:14 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one
 header [v2]
In-Reply-To: <14GqBlYL2XDV0k6WC1La5SmYLSAyYcf2FlVE4V75T2s=.bd5ddb3f-e520-44ef-85f9-22d498d5151e@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
 <14GqBlYL2XDV0k6WC1La5SmYLSAyYcf2FlVE4V75T2s=.bd5ddb3f-e520-44ef-85f9-22d498d5151e@github.com>
Message-ID: <Y-Wafdtm1hK0TCP5-pknaeZNekGcx8BipT609RYZRzo=.39b9b5cd-91ed-4498-b327-a90ebffb22d5@github.com>

On Fri, 18 Oct 2024 10:31:23 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix Zero builds

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21579#pullrequestreview-2383303701

From duke at openjdk.org  Mon Oct 21 21:09:38 2024
From: duke at openjdk.org (Volodymyr Paprotski)
Date: Mon, 21 Oct 2024 21:09:38 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v46]
In-Reply-To: <qXNHwb48lt9LL_DePcb6QLEE0d_rsPQGzI_Bz5Swyzc=.e2d2f513-816d-4bf2-9dbd-67fb74522cfd@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TSoqyHiesRchgObQUULLugQ15WdZJlOp21VQTyxrEbQ=.fff7ec53-1145-40e1-90ed-af55b3defb5f@github.com>
 <R_8ZEOsn1sMwBK_6ECrFRv98FrybuIgn1L3tlGFOvcg=.0b947263-39f3-483b-9526-e5fcea4ba603@github.com>
 <BCZjoP4Ao9C1F8kQJmpfooJCnGx3Ef-YNKGHdqZMSCs=.234de599-a258-45ec-9e63-7839f3a48e54@github.com>
 <qXNHwb48lt9LL_DePcb6QLEE0d_rsPQGzI_Bz5Swyzc=.e2d2f513-816d-4bf2-9dbd-67fb74522cfd@github.com>
Message-ID: <fBZr5Zvvl4oTLcO62RFVv9emvEccDwESuB7nNoc8HfI=.a09a0b4b-53a9-4808-9dcd-f0b877e73759@github.com>

On Mon, 21 Oct 2024 20:31:28 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> @rkennke Could you post the full command you used please? And perhaps also the seed that gets printed.. having trouble getting it to fail..
>>> 
>>> So far I added a few options and perrmitations of: `./build/linux-x86_64-server-fastdebug/images/jdk/bin/java -Xcomp -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:+EnableX86ECoreOpts -XX:+UseSerialGC -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders test/jdk/java/lang/StringBuffer/ECoreIndexOf.java` and lo luck.. IndexOf.java test checks "all interesting" lengths of haystack and needle and can't get it to fail either.
>> 
>> I could reproduce on 3rd try with a fastdebug build with:
>> 
>> make test TEST=java/lang/StringBuffer/ECoreIndexOf.java TEST_VM_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders -XX:+UseSerialGC"
>> 
>> 
>> It prints:
>> 
>> Seed set to 636754923980405411
>> 
>> 
>> It probably depends on GC operation: It would only fail when the array happens to be the very first object in the heap. The relevant GC/heap configs would be:
>> 
>> InitialHeapSize                          = 805306368
>> MaxHeapSize                              = 805306368
>> MaxNewSize                               = 268435456
>> 
>> 
>> So you should probably also add `-Xmx805306368 -Xms805306368 -Xmn268435456`
>
>> > Thanks @rkennke able to reproduce now.. Sandhya will have a patch soon and I will re-verify
>> 
>> @rkennke @vpaprotsk Please find attached the patch which should fix the problem.
>> 
>> [smallneedlefix.patch](https://github.com/user-attachments/files/17466073/smallneedlefix.patch)
> 
> Testing now. Runs the reproducer in a loop since half an hour without crashing. I'll let it run overnight, and if @vpaprotsk approves the changes, then I'll intergrate them tomorrow morning.

@rkennke I've been running the patch too, for about 2.5 hours now, looks good to me.

Also looked things over again, looks good. Just to explain/document what I reviewed.. 

- Looked at other uses of the needle (that code didn't change, so not exhaustive claim). Typically size of the needle being less then 16 'doesnt matter'.. i.e. broadcast first char, last char, if first/last character mask matches, switch-table for comparing middle 
     - i.e. no reading headers needed
- The case Sandhya fixes, handles UL special case (i.e. haystack unicode, needle regular). For cases of needle less then 32 bytes, copy the needle to the stack, and expand 8->16 so regular UU code can be used. Previous code looped 256bit loads at a time, now we loop 128 instead.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2427718674

From wkemper at openjdk.org  Mon Oct 21 21:16:50 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 21 Oct 2024 21:16:50 GMT
Subject: RFR: 8337511: Implement JEP-404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:

 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - Merge
 - 8342560: GenShen: Fix confusing method name
   
   Reviewed-by: ysr
 - 8342564: GenShen: Only reference young/old generation names in generational mode
   
   Reviewed-by: xpeng, ysr
 - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
   
   Reviewed-by: kdnilsen, ysr
 - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
   
   Reviewed-by: ysr
 - 8342278: GenShen: Move non-generational mode test out of generational test configuration
   
   Reviewed-by: ysr
 - 8342255: GenShen: Remove unnecessary enum initial values
   
   Reviewed-by: ysr
 - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408

-------------

Changes: https://git.openjdk.org/jdk/pull/21273/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=04
  Stats: 22825 lines in 229 files changed: 21084 ins; 889 del; 852 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From syan at openjdk.org  Tue Oct 22 02:12:22 2024
From: syan at openjdk.org (SendaoYan)
Date: Tue, 22 Oct 2024 02:12:22 GMT
Subject: RFR: 8342681: TestLoadBypassesNullCheck.java fails improperly
 specified VM option [v2]
In-Reply-To: <zae0V9b4ewNnnyGGFjsBDe0mIwpsFL3oTrSg6P1B5iQ=.66df7883-3c1e-4dcf-8125-6ced76fe4901@github.com>
References: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
 <zae0V9b4ewNnnyGGFjsBDe0mIwpsFL3oTrSg6P1B5iQ=.66df7883-3c1e-4dcf-8125-6ced76fe4901@github.com>
Message-ID: <Ca_J8IA7QErDlafzQ4nzz9pRyqlEJbSIBhgyvvd1-ko=.330a9545-caaa-4906-afbc-d633046e1884@github.com>

On Mon, 21 Oct 2024 16:44:26 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>> Newly added test `gc/shenandoah/compiler/TestLoadBypassesNullCheck.java` fails `The unlock option must precede 'StressGCM'.` with release jdk build. This PR fix the test bug, only add `-XX:+UnlockDiagnosticVMOptions` to make test run passed by release jdk build.
>> The change has been verified locally with release/fastdebug/slowdebug on linux-x64. Test-fix only, no risk.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   only unlock Stress* VM option

Thanks for the review.

To make CI test less noisy, I will integrate right now, ignore the 24 hours rule. Test-fix only, no risk.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21609#issuecomment-2428063808
PR Comment: https://git.openjdk.org/jdk/pull/21609#issuecomment-2428065005

From syan at openjdk.org  Tue Oct 22 02:12:22 2024
From: syan at openjdk.org (SendaoYan)
Date: Tue, 22 Oct 2024 02:12:22 GMT
Subject: Integrated: 8342681: TestLoadBypassesNullCheck.java fails improperly
 specified VM option
In-Reply-To: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
References: <MiRErOAQLeXA530ui9cQvk0H7MFcauRmfW15WvoT5yI=.f95d2a38-1ca2-4acb-9ed7-94d094a8ccb6@github.com>
Message-ID: <MC3-ToQ8YXFszBGEI8jqFC1tNiT873NF6f00aIl7pLo=.cedaa22d-b51e-48a7-b34e-70869fd5bff9@github.com>

On Mon, 21 Oct 2024 13:00:23 GMT, SendaoYan <syan at openjdk.org> wrote:

> Hi all,
> Newly added test `gc/shenandoah/compiler/TestLoadBypassesNullCheck.java` fails `The unlock option must precede 'StressGCM'.` with release jdk build. This PR fix the test bug, only add `-XX:+UnlockDiagnosticVMOptions` to make test run passed by release jdk build.
> The change has been verified locally with release/fastdebug/slowdebug on linux-x64. Test-fix only, no risk.

This pull request has now been integrated.

Changeset: 8bcd4920
Author:    SendaoYan <syan at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/8bcd4920f1b03d0ef8e295e53557c629f05ceaa4
Stats:     8 lines in 1 file changed: 4 ins; 0 del; 4 mod

8342681: TestLoadBypassesNullCheck.java fails improperly specified VM option

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/jdk/pull/21609

From rkennke at openjdk.org  Tue Oct 22 07:32:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 22 Oct 2024 07:32:24 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v47]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <PT1OZyKSCB7sEfiFsFugK3GTbr1JATJ1X7xP8XlD9-g=.c35de9eb-e8ab-4739-9957-2f7623500d23@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Fix needle copying in indexOf intrinsic for smaller headers

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/1b907cc8..8c4eb6d6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=46
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=45-46

  Stats: 16 lines in 1 file changed: 0 ins; 0 del; 16 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Tue Oct 22 11:19:19 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 22 Oct 2024 11:19:19 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v48]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <H6cywVFrD2hUSGZd1VCGpMu7OmP0ON0RC2QQaLJFBb8=.3960b3bb-8273-472c-9d3e-655638447762@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 97 commits:

 - Merge tag 'jdk-24+20' into JDK-8305895-v4
   
   Added tag jdk-24+20 for changeset 7a64fbbb
 - Fix needle copying in indexOf intrinsic for smaller headers
 - Compact header riscv (#3)
   
   Implement compact headers on RISCV
   ---------
   
   Co-authored-by: hamlin <hamlin at rivosinc.com>
 - Remove extra sanity check
 - Problem-list SharedBaseAddress tests on aarch64
 - Address comments by @vpaprotsk
 - Fix aarch64.ad
 - Merge tag 'jdk-24+19' into JDK-8305895-v4
   
   Added tag jdk-24+19 for changeset e7c5bf45
 - PPC64 implementation of Compact Object Headers (JEP 450)
 - Increase compiler code stubs size for indexOf intrinsic
 - ... and 87 more: https://git.openjdk.org/jdk/compare/7a64fbbb...e324d95b

-------------

Changes: https://git.openjdk.org/jdk/pull/20677/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=47
  Stats: 5021 lines in 212 files changed: 3472 ins; 847 del; 702 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From stefank at openjdk.org  Tue Oct 22 13:29:42 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 22 Oct 2024 13:29:42 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v48]
In-Reply-To: <H6cywVFrD2hUSGZd1VCGpMu7OmP0ON0RC2QQaLJFBb8=.3960b3bb-8273-472c-9d3e-655638447762@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <H6cywVFrD2hUSGZd1VCGpMu7OmP0ON0RC2QQaLJFBb8=.3960b3bb-8273-472c-9d3e-655638447762@github.com>
Message-ID: <4UtypHyHundxF7XNmcIsoarpmt4EcfgEzSO4uoobf3Q=.0351e5bb-000e-4068-a5e4-3e3db19a61a0@github.com>

On Tue, 22 Oct 2024 11:19:19 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 97 commits:
> 
>  - Merge tag 'jdk-24+20' into JDK-8305895-v4
>    
>    Added tag jdk-24+20 for changeset 7a64fbbb
>  - Fix needle copying in indexOf intrinsic for smaller headers
>  - Compact header riscv (#3)
>    
>    Implement compact headers on RISCV
>    ---------
>    
>    Co-authored-by: hamlin <hamlin at rivosinc.com>
>  - Remove extra sanity check
>  - Problem-list SharedBaseAddress tests on aarch64
>  - Address comments by @vpaprotsk
>  - Fix aarch64.ad
>  - Merge tag 'jdk-24+19' into JDK-8305895-v4
>    
>    Added tag jdk-24+19 for changeset e7c5bf45
>  - PPC64 implementation of Compact Object Headers (JEP 450)
>  - Increase compiler code stubs size for indexOf intrinsic
>  - ... and 87 more: https://git.openjdk.org/jdk/compare/7a64fbbb...e324d95b

We've identified another failure in our testing:

java -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders -XX:TieredStopAtLevel=2 -XX:TLABSize=1 -XX:MinTLABSize=1 ~/tests/HelloWorld.java
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (src/hotspot/share/jfr/support/jfrObjectAllocationSample.cpp:50), pid=775231, tid=775232
#  assert(desired_tlab_size_bytes > alignment_reserve_bytes) failed: invariant
...
V  [libjvm.so+0xf4ec11]  JfrObjectAllocationSample::send_event(Klass const*, unsigned long, bool, Thread*)+0x2d1  (jfrObjectAllocationSample.cpp:50)
V  [libjvm.so+0x5d7899]  AllocTracer::send_allocation_outside_tlab(Klass*, HeapWordImpl**, unsigned long, JavaThread*)+0x39  (allocTracer.cpp:35)
V  [libjvm.so+0x139d6c5]  MemAllocator::Allocation::notify_allocation_jfr_sampler()+0x225  (memAllocator.cpp:214)
V  [libjvm.so+0x139f928]  MemAllocator::allocate() const+0x2a8  (memAllocator.cpp:235)
V  [libjvm.so+0x18379bd]  TypeArrayKlass::allocate_common(int, bool, JavaThread*)+0x13d  (collectedHeap.inline.hpp:41)
V  [libjvm.so+0x14bc5c8]  oopFactory::new_typeArray(BasicType, int, JavaThread*)+0x38  (typeArrayKlass.hpp:68)
V  [libjvm.so+0x8327f1]  Runtime1::new_type_array(JavaThread*, Klass*, int)+0xa1  (c1_Runtime1.cpp:388)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2429286279

From stuefe at openjdk.org  Tue Oct 22 13:56:42 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 22 Oct 2024 13:56:42 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <Na5LWOz_h0ZGmLcETAxDA9wYiEopgWfvNTbYAELExR8=.e30fe7a0-c766-4bb4-aebb-fc6643c832c1@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <spymZDxJXwseFGxL78zfn4V8qe9NpX3gifTgd_eQkiM=.d1af7cf3-b503-4c35-93bb-db00f3c4cfc8@github.com>
 <U1HQ2ijNoadMNlrW_Adxkju4fM8kfAB4hn-a_gkPrnA=.7e0fee38-8534-4dcc-8aac-77d6559687c8@github.com>
 <CNrcAK8SrGCzsreT3SdoxaoGaqKqGb989t-waCJUz64=.0662bccb-661f-4dc9-b8b6-24d615035261@github.com>
 <YyPrYu-6p5wcUwfMQQamwoM1eGp6eUze_yC1lrc7mb0=.f1e5853b-3b6e-4460-8f57-a6f7588e513e@github.com>
 <Na5LWOz_h0ZGmLcETAxDA9wYiEopgWfvNTbYAELExR8=.e30fe7a0-c766-4bb4-aebb-fc6643c832c1@github.com>
Message-ID: <0fDctIMZlpNZ4a5_idrN_w8KnvGfPS49Bw_9WRdjJ9I=.8bedb8be-0b33-468b-b711-9c0b4fb6649e@github.com>

On Thu, 19 Sep 2024 13:34:47 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Do you seen any effects of this in anything other than special-crafted micro benchmarks? I wonder if it would be good enough to hard-code this to be 10 for the first integration of Lilliput.
>
> I will do some benchmarks

I did SpecJBB runs with shift of 6, 8 and 10, respectively, which amounts to Klass alignment of 64, 256 and 1K. Benchmark scores did not show a significant pattern. I did not measure CPU stats though. 

But I still think a dynamically calculated shift makes sense, and I hesitate to change this code at this point. I therefore would like to move this question to followup RFEs if necessary.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1810775878

From rkennke at openjdk.org  Tue Oct 22 14:25:12 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 22 Oct 2024 14:25:12 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v49]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <x0CFyry6L7aq-kozLc85_MEo96bqnhen9RwrExALE4Q=.47b94dc7-94a9-443e-b607-8c873f54daf8@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Update copyright headers

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/e324d95b..19d05e43

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=48
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=47-48

  Stats: 49 lines in 49 files changed: 0 ins; 0 del; 49 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From wkemper at openjdk.org  Tue Oct 22 16:01:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 22 Oct 2024 16:01:47 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <GWpGtSgoRD5u1L7TwA7URqGqlBPMsdQH1_tYFR6uzCY=.309147f4-194a-47eb-882f-c7bfcfcbaf00@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <GWpGtSgoRD5u1L7TwA7URqGqlBPMsdQH1_tYFR6uzCY=.309147f4-194a-47eb-882f-c7bfcfcbaf00@github.com>
Message-ID: <yK_kL5aNjuY2GfOFgToHwrahIuyf97G4ufow0iPSRZA=.9d771ea0-806e-45f9-81f5-99a0374c4502@github.com>

On Wed, 9 Oct 2024 03:26:30 GMT, Liang Mao <lmao at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> Congratulations! Good to see the great progress. I just built this PR for some testing and found something unexpected. I ran the genshen VS shenandoah(default) with jbb2015 on aarch64 N2 with 8 cores and Xmx8g. The critical-jOPS of genshen(5373) is behind shenandoah(6027). Do I miss something on the options?
> ```java -Xmx8g -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational -Xlog:gc*  -XX:MetaspaceSize=1g   -jar specjbb2015.jar -m COMPOSITE```

@mmyxym , Thank you for testing this out! I apologize for not responding to your comment sooner. We run specjbb2015 regularly in our integration pipeline. We see a slight improvement with the generational mode; certainly no regression:

Shen: RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3620, critical-jOPS = 2375
Gen:  RUN RESULT: hbIR (max attempted) = 3934, hbIR (settled) = 3295, max-jOPS = 4013, critical-jOPS = 2470
Shen: RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3667, critical-jOPS = 2397
Gen:  RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3996, critical-jOPS = 2414

These results were produced with these arguments:

-XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:+UnlockDiagnosticV
MOptions -XX:-TieredCompilation -XX:-ShenandoahPacing -XX:+AlwaysPreTouch -XX:+DisableExplicitGC -Xmx10g -Xms10g -XX:ShenandoahGCMode=generational

These runs executed on a Graviton2 host with 4 cores, 16G. I'll run again on a host with more cores and with your exact command line parameters.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2429671754

From rkennke at openjdk.org  Tue Oct 22 16:19:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 22 Oct 2024 16:19:24 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:

 - Update copyright
 - Avoid assert/endless-loop in JFR code

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/19d05e43..1ef6394d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=49
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=48-49

  Stats: 6 lines in 1 file changed: 4 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Tue Oct 22 16:26:39 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 22 Oct 2024 16:26:39 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
Message-ID: <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>

On Tue, 22 Oct 2024 16:19:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code

@egahlin / @mgronlun could you please review the JFR parts of this PR? One change is for getting the right prototype header, the other is for avoiding an endless loop/assert in a corner case.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2429724926

From zgu at openjdk.org  Tue Oct 22 17:15:17 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Tue, 22 Oct 2024 17:15:17 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <GWpGtSgoRD5u1L7TwA7URqGqlBPMsdQH1_tYFR6uzCY=.309147f4-194a-47eb-882f-c7bfcfcbaf00@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <GWpGtSgoRD5u1L7TwA7URqGqlBPMsdQH1_tYFR6uzCY=.309147f4-194a-47eb-882f-c7bfcfcbaf00@github.com>
Message-ID: <b5jJoMTKoxqyNgyMDdLlIg1pr1C1pQMdd-S7fyVHIg0=.5b1ab26d-295c-4935-bac9-c77a7f260884@github.com>

On Wed, 9 Oct 2024 03:26:30 GMT, Liang Mao <lmao at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 478 commits:
>> 
>>  - Fix merge error
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8341099: GenShen: assert(HAS_FWD == _heap->has_forwarded_objects()) failed: Forwarded object status is sane
>>    
>>    Reviewed-by: kdnilsen
>>  - 8341485: GenShen: Make evac tracker a non-product feature and confine it to generational mode
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - Merge
>>  - 8341042: GenShen: Reset mark bitmaps for unaffiliated regions when preparing for a cycle
>>    
>>    Reviewed-by: kdnilsen
>>  - 8339616: GenShen: Introduce new state to distinguish promote-in-place phase as distinct from concurrent evacuation
>>    
>>    Reviewed-by: kdnilsen, shade, ysr
>>  - ... and 468 more: https://git.openjdk.org/jdk/compare/b9db74a6...4db1e0e1
>
> Congratulations! Good to see the great progress. I just built this PR for some testing and found something unexpected. I ran the genshen VS shenandoah(default) with jbb2015 on aarch64 N2 with 8 cores and Xmx8g. The critical-jOPS of genshen(5373) is behind shenandoah(6027). Do I miss something on the options?
> ```java -Xmx8g -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational -Xlog:gc*  -XX:MetaspaceSize=1g   -jar specjbb2015.jar -m COMPOSITE```

> @mmyxym , Thank you for testing this out! I apologize for not responding to your comment sooner. We run specjbb2015 regularly in our integration pipeline. We see a slight improvement with the generational mode; certainly no regression:
> 
> ```
> Shen: RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3620, critical-jOPS = 2375
> Gen:  RUN RESULT: hbIR (max attempted) = 3934, hbIR (settled) = 3295, max-jOPS = 4013, critical-jOPS = 2470
> Shen: RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3667, critical-jOPS = 2397
> Gen:  RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3996, critical-jOPS = 2414
> ```
> 
> These results were produced with these arguments:
> 
> ```
> -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:+UnlockDiagnosticV
> MOptions -XX:-TieredCompilation -XX:-ShenandoahPacing -XX:+AlwaysPreTouch -XX:+DisableExplicitGC -Xmx10g -Xms10g -XX:ShenandoahGCMode=generational
> ```
> 
> These runs executed on a Graviton2 host with 4 cores, 16G. I'll run again on a host with more cores and with your exact command line parameters.

What's the reason to disable tiered compilation?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2429827638

From wkemper at openjdk.org  Tue Oct 22 18:28:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 22 Oct 2024 18:28:23 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
Message-ID: <ABXS9VGZgO5rYX2O_cjToFf5Mv1kDSia2hMvA95BbGM=.01ddfdca-38e4-4440-b677-811a2f9c6805@github.com>

On Mon, 21 Oct 2024 21:16:50 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8342560: GenShen: Fix confusing method name
>    
>    Reviewed-by: ysr
>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>    
>    Reviewed-by: xpeng, ysr
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>    
>    Reviewed-by: ysr
>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>    
>    Reviewed-by: ysr
>  - 8342255: GenShen: Remove unnecessary enum initial values
>    
>    Reviewed-by: ysr
>  - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408

We disabled tiered compilation to force everything to compile through C2 to get more consistent results.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2429963097

From wkemper at openjdk.org  Tue Oct 22 19:42:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 22 Oct 2024 19:42:35 GMT
Subject: RFR: 8342580: GenShen: TestChurnNotifications fails executing in
 unintended test-id modes with ShenandoahGCMode=generational
In-Reply-To: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
References: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
Message-ID: <0Uj9onHkhKhJwYRHoUfM2sb88AyT0r7gmLBT1zkEDKA=.5e37c200-a501-4c18-8bd0-0596ab78c6ce@github.com>

On Fri, 18 Oct 2024 21:33:10 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Generational mode should not be tested with explicit heuristics settings, which are silently ignored. The test failed in these settings because it needs to be told which memory pool to check for notifications, which needs to be specified differently for generational. Ideally, we'd have the test figure this out rather than needing this to be explicitly specified. For now I just tightened the `@requires` clause to run only in the intended modes.
> 
> Related cleanups will follow separately (such as emitting a warning when an incompatible heuristics option is specified, code which may have been lost in a previous refactor). See details in the ticket.
> 
> **Testing** : tested with no specific collector, with Shenandoah, withGenShen, and with other collectors with test explicitly specified. As intended, in the first and second case 6 test-ids are run, in the thirdd case 1 test-id is run, and in the remaining cases, 0 test-ids are run.

LGTM.

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/520#pullrequestreview-2386129245

From stefank at openjdk.org  Tue Oct 22 20:11:27 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 22 Oct 2024 20:11:27 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
Message-ID: <8O2eaSeTC3JyNsCK6Tb-RGi8NzbA17M5S0mnuF_szo0=.f7da9bb1-fd4b-47df-a56c-e6803182dd27@github.com>

On Tue, 22 Oct 2024 16:19:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code

Our testing has found a failure in serviceability/sa/ClhsdbJstackWithConcurrentLock.java when we run C1-only.

I've narrowed it down to be a stale, but seemingly working, implementation of the TLAB data structure. When Lilliput changes the header size this implementation doesn't work anymore and needs to be fixed.

The reproducer for this problem is:

make -C ../build/fastdebug test TEST=serviceability/sa/ClhsdbJstackWithConcurrentLock.java JTREG="JAVA_OPTIONS=-XX:TieredStopAtLevel=2 -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders"


See how the thread reports that the frame holds an AOS, but the list of "Locked ownable synchronizers" is (incorrectly) empty:

"Thread-0" #31 prio=5 tid=0x00007a708c259ad0 nid=1480533 waiting on condition [0x00007a706fefe000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
   JavaThread state: _thread_blocked
 - java.lang.Thread.sleepNanos0(long) @bci=0 (Interpreted frame)
 - java.lang.Thread.sleepNanos(long) @bci=33, line=497 (Interpreted frame)
 - java.lang.Thread.sleep(long) @bci=25, line=528 (Interpreted frame)
 - LingeredAppWithConcurrentLock.lockMethod(java.util.concurrent.locks.Lock) @bci=13, line=38 (Interpreted frame)
	- locked <0x00000000ffd32d88> (a java.util.concurrent.locks.ReentrantLock)
 - LingeredAppWithConcurrentLock.lambda$main$0() @bci=3, line=46 (Interpreted frame)
 - LingeredAppWithConcurrentLock$$Lambda+0x00007a7023001000.run() @bci=0 (Interpreted frame)
 - java.lang.Thread.runWith(java.lang.Object, java.lang.Runnable) @bci=5, line=1589 (Interpreted frame)
 - java.lang.Thread.run() @bci=19, line=1576 (Interpreted frame)

Locked ownable synchronizers:
    - None


This happens because the TLAB ranges become overlapped and that confuses the rest of the SA code that looks for objects in the heap.

I've created a fix for this, which I intend to try to get pushed to openjdk/jdk:
https://github.com/openjdk/jdk/compare/pr/20677...stefank:jdk:8342857_SA_heap_iterator_fix
https://github.com/stefank/jdk/tree/8342857_SA_heap_iterator_fix

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2430157348

From matsaave at openjdk.org  Tue Oct 22 20:27:16 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Tue, 22 Oct 2024 20:27:16 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops
Message-ID: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>

[JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.

-------------

Commit messages:
 - Enabled uncompressed oops on Epsilon and Shenandoah
 - Merge branch 'master' into archive_uncompressed_oops_8341371
 - Added uncompressed oops to tests
 - 8341371: CDS cannot load archived heap objects with -XX:+UseSerialGC -XX:-UseCompressedOops

Changes: https://git.openjdk.org/jdk/pull/21647/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8341371
  Stats: 153 lines in 12 files changed: 117 ins; 21 del; 15 mod
  Patch: https://git.openjdk.org/jdk/pull/21647.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21647/head:pull/21647

PR: https://git.openjdk.org/jdk/pull/21647

From kdnilsen at openjdk.org  Tue Oct 22 21:15:36 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 22 Oct 2024 21:15:36 GMT
Subject: RFR: 8342580: GenShen: TestChurnNotifications fails executing in
 unintended test-id modes with ShenandoahGCMode=generational
In-Reply-To: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
References: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
Message-ID: <8iRulx0MHs_hHEpQKs2tma5Ui1xYUNyXP4Vqa2lveTI=.567bfb55-fea8-4510-b5cf-bab5dfe90b03@github.com>

On Fri, 18 Oct 2024 21:33:10 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Generational mode should not be tested with explicit heuristics settings, which are silently ignored. The test failed in these settings because it needs to be told which memory pool to check for notifications, which needs to be specified differently for generational. Ideally, we'd have the test figure this out rather than needing this to be explicitly specified. For now I just tightened the `@requires` clause to run only in the intended modes.
> 
> Related cleanups will follow separately (such as emitting a warning when an incompatible heuristics option is specified, code which may have been lost in a previous refactor). See details in the ticket.
> 
> **Testing** : tested with no specific collector, with Shenandoah, withGenShen, and with other collectors with test explicitly specified. As intended, in the first and second case 6 test-ids are run, in the thirdd case 1 test-id is run, and in the remaining cases, 0 test-ids are run.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/520#pullrequestreview-2386382512

From wkemper at openjdk.org  Tue Oct 22 23:34:51 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 22 Oct 2024 23:34:51 GMT
Subject: RFR: 8342861: GenShen: Old generation in unexpected state when
 abandoning mixed gc candidates
Message-ID: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>

When the generational mode cannot add any old regions to a mixed collection because they are _all_ pinned, it gives up trying to evacuate these regions. The regions must then be made parsable. However, if this phase is interrupted by a young collection it should no longer try to evacuate any old regions. Doing so triggered an assert when testing with `-XX:ShenandoahCoalesceChance=80` and `XX:+ShenandoahAllocFailureALot`.

-------------

Commit messages:
 - Do not add old regions to collection set once filling has started

Changes: https://git.openjdk.org/shenandoah/pull/522/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=522&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342861
  Stats: 10 lines in 2 files changed: 10 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/522.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/522/head:pull/522

PR: https://git.openjdk.org/shenandoah/pull/522

From iklam at openjdk.org  Wed Oct 23 05:04:05 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Wed, 23 Oct 2024 05:04:05 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops
In-Reply-To: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
Message-ID: <UerwUw4MtQHC8QkhjbWD8aAkhdWVeozIl-1NFSS3YTM=.62746be0-a932-4fb4-9228-ed5992fa5821@github.com>

On Tue, 22 Oct 2024 19:32:03 GMT, Matias Saavedra Silva <matsaave at openjdk.org> wrote:

> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.

Changes requested by iklam (Reviewer).

src/hotspot/share/cds/archiveHeapLoader.cpp line 245:

> 243:   }
> 244: };
> 245: 

This new class looks almost the same as the existing PatchUncompressedEmbeddedPointers class. I think you can change PatchUncompressedEmbeddedPointers to take an additional `intx runtime_offset` parameter in its constructor, which will be `loaded_region->_runtime_offset` or `ArchiveHeapLoader::mapped_heap_delta()`.

src/hotspot/share/cds/filemap.cpp line 2137:

> 2135:   FileMapRegion* r = region_at(MetaspaceShared::hp);
> 2136:   assert(is_aligned(r->mapping_offset(), sizeof(HeapWord)), "must be");
> 2137:   assert(ArchiveHeapLoader::can_use(), "cannot be used by ArchiveHeapLoader::can_load() mode");

The error message needs to be updated.

src/hotspot/share/prims/whitebox.cpp line 2162:

> 2160: 
> 2161: WB_ENTRY(jboolean, WB_CanWriteJavaHeapArchive(JNIEnv* env))
> 2162:   return HeapShared::can_write() && ArchiveHeapLoader::can_use();

Do we still need the `&& ArchiveHeapLoader::can_use()` part?

-------------

PR Review: https://git.openjdk.org/jdk/pull/21647#pullrequestreview-2387142172
PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1811841631
PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1811844617
PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1811846134

From xpeng at openjdk.org  Wed Oct 23 06:29:24 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 06:29:24 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely cleared
 in generational mode
Message-ID: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>

In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well, the crash was reproduce multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:


CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=100" 
``` 

Crash:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
#  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
#
# JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
# Problematic frame:
# V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#


With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.

Logs from instrumentation:

[32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)



The fix is simple, just need to swap the two tests,  `!region->is_affiliated()` should be evaluated prior to  `_generation->contains(region)`

-------------

Commit messages:
 - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode

Changes: https://git.openjdk.org/shenandoah/pull/523/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342848
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/523.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/523/head:pull/523

PR: https://git.openjdk.org/shenandoah/pull/523

From stefank at openjdk.org  Wed Oct 23 11:43:27 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 23 Oct 2024 11:43:27 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
Message-ID: <pYI-64nT97A6U0xwVDOjKoXN4YKAJ3iNzxXFRDRd9Ss=.7334a043-2f5d-40e9-ba20-5bfb2266a4f2@github.com>

On Tue, 22 Oct 2024 16:19:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code

I've published an upstream PR for the SA bug:
https://github.com/openjdk/jdk/pull/21662

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2431837874

From rkennke at openjdk.org  Wed Oct 23 13:03:19 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 23 Oct 2024 13:03:19 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
Message-ID: <rnihqvTSvTSVvQ-B_BekpovR9Si1fw-gjiY-zGziFMs=.09961e51-9651-4067-9da0-f0c8f2dc60fb@github.com>

On Mon, 21 Oct 2024 21:16:50 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8342560: GenShen: Fix confusing method name
>    
>    Reviewed-by: ysr
>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>    
>    Reviewed-by: xpeng, ysr
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>    
>    Reviewed-by: ysr
>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>    
>    Reviewed-by: ysr
>  - 8342255: GenShen: Remove unnecessary enum initial values
>    
>    Reviewed-by: ysr
>  - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408

I've looked at the changes again, especially at all that changed since my last full review. It's looking good to me, now. Please remove the one newline that I've spotted. Thanks for this work, this is really a huge effort!

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 1:

> 1: 

This newline is one too much.

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2388573425
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1812694226

From matsaave at openjdk.org  Wed Oct 23 15:45:09 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Wed, 23 Oct 2024 15:45:09 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops
In-Reply-To: <UerwUw4MtQHC8QkhjbWD8aAkhdWVeozIl-1NFSS3YTM=.62746be0-a932-4fb4-9228-ed5992fa5821@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
 <UerwUw4MtQHC8QkhjbWD8aAkhdWVeozIl-1NFSS3YTM=.62746be0-a932-4fb4-9228-ed5992fa5821@github.com>
Message-ID: <ieV8-KLJR4TSGdhfd_qiAyPula_cxH0E6SAm_5SUa90=.c6ed20c1-7046-4acd-ac89-72eac7e75596@github.com>

On Wed, 23 Oct 2024 05:00:44 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.
>
> src/hotspot/share/prims/whitebox.cpp line 2162:
> 
>> 2160: 
>> 2161: WB_ENTRY(jboolean, WB_CanWriteJavaHeapArchive(JNIEnv* env))
>> 2162:   return HeapShared::can_write() && ArchiveHeapLoader::can_use();
> 
> Do we still need the `&& ArchiveHeapLoader::can_use()` part?

I left it in since it may be a valuable check in case there are further changes or additions to the GCs.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1813067894

From xpeng at openjdk.org  Wed Oct 23 16:11:22 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 16:11:22 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode
In-Reply-To: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
Message-ID: <uNbwtocAYx3GWyZnNt96svGlHtfeW70EVJBvE6FpsuY=.b82460cf-2f7e-4a85-bdb7-d4098c00e090@github.com>

On Wed, 23 Oct 2024 06:07:00 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
> 
> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
> 
> 
> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
> ``` 
> 
> Crash:
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
> #
> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
> # Problematic frame:
> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
> #
> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> 
> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
> 
> Logs from instrumentation:
> 
> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
> 
> ...
> 
> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> 
> 
> The fix is simple, just need to swap the two tests,  `!region->is_affiliated()` ...

The test failure should not be caused by this change, spotted the same failure in other open PR:


java.lang.RuntimeException: expected testPhantom1 to be cleared
	at gc.shenandoah.TestReferenceRefersToShenandoah.fail(TestReferenceRefersToShenandoah.java:155)
	at gc.shenandoah.TestReferenceRefersToShenandoah.expectCleared(TestReferenceRefersToShenandoah.java:166)
	at gc.shenandoah.TestReferenceRefersToShenandoah.testConcurrentCollection(TestReferenceRefersToShenandoah.java:243)
	at gc.shenandoah.TestReferenceRefersToShenandoah.main(TestReferenceRefersToShenandoah.java:340)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:573)
	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
	at java.base/java.lang.Thread.run(Thread.java:1576)

JavaTest Message: Test threw exception: java.lang.RuntimeException: expected testPhantom1 to be cleared
JavaTest Message: shutting down test

` ``

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 80:

> 78:     ShenandoahMarkingContext* const ctx = heap->marking_context();
> 79:     while (region != nullptr) {
> 80:       bool needs_reset = !region->is_affiliated() || _generation->contains(region);

We should really read the affiliation only once for the whole test, but will have to create a new method to achieve that to keep the code clean and encapsulated.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/523#issuecomment-2432746489
PR Review Comment: https://git.openjdk.org/shenandoah/pull/523#discussion_r1813106540

From matsaave at openjdk.org  Wed Oct 23 16:40:44 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Wed, 23 Oct 2024 16:40:44 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v2]
In-Reply-To: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
Message-ID: <yIyDglLtuKhez4DgW5wAX-93AC78MXppqkb8w36Q0-0=.6d467528-b10b-4e6f-958f-348a566e4602@github.com>

> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.

Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision:

  Ioi comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21647/files
  - new: https://git.openjdk.org/jdk/pull/21647/files/da78b148..da433327

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=00-01

  Stats: 30 lines in 2 files changed: 7 ins; 19 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/21647.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21647/head:pull/21647

PR: https://git.openjdk.org/jdk/pull/21647

From ccheung at openjdk.org  Wed Oct 23 16:47:06 2024
From: ccheung at openjdk.org (Calvin Cheung)
Date: Wed, 23 Oct 2024 16:47:06 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v2]
In-Reply-To: <yIyDglLtuKhez4DgW5wAX-93AC78MXppqkb8w36Q0-0=.6d467528-b10b-4e6f-958f-348a566e4602@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
 <yIyDglLtuKhez4DgW5wAX-93AC78MXppqkb8w36Q0-0=.6d467528-b10b-4e6f-958f-348a566e4602@github.com>
Message-ID: <NR7V4oHt9GnNw1rz1y9Kyl14xwXHxHrIZBZVOqx7nXg=.893968bb-b1f4-4354-9d6d-0faaebf0a4df@github.com>

On Wed, 23 Oct 2024 16:40:44 GMT, Matias Saavedra Silva <matsaave at openjdk.org> wrote:

>> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.
>
> Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Ioi comments

I spotted couple of things in the test cases.

test/hotspot/jtreg/runtime/cds/appcds/TestEpsilonGCWithCDS.java line 41:

> 39: 
> 40: // Below is exactly the same as above, except:
> 41: // - requires vm.bits == "64"

I don't see this requirement in the new test.

test/hotspot/jtreg/runtime/cds/appcds/TestEpsilonGCWithCDS.java line 46:

> 44: /*
> 45:  * @test Loading CDS archived heap objects into EpsilonGC
> 46:  * @bug 8234679

Please update the bug id.

test/hotspot/jtreg/runtime/cds/appcds/TestParallelGCWithCDS.java line 41:

> 39: 
> 40: // Below is exactly the same as above, except:
> 41: // - requires vm.bits == "64"

I don't see this requirement in the new test.

test/hotspot/jtreg/runtime/cds/appcds/TestParallelGCWithCDS.java line 46:

> 44:  /*
> 45:  * @test Loading CDS archived heap objects into ParallelGC
> 46:  * @bug 8274788

Please update the bug id.

-------------

Changes requested by ccheung (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21647#pullrequestreview-2389409982
PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1813166147
PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1813167449
PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1813169167
PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1813169654

From matsaave at openjdk.org  Wed Oct 23 16:55:21 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Wed, 23 Oct 2024 16:55:21 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v3]
In-Reply-To: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
Message-ID: <6hF2-lkUNnzJpmou-rgkj9oSQEwlyAr-ltZFV7At4Ck=.27d2542b-7d76-4eb3-b5f0-4690935cf467@github.com>

> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.

Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision:

  Calvin comments and copyright

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21647/files
  - new: https://git.openjdk.org/jdk/pull/21647/files/da433327..35a6fa4d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=01-02

  Stats: 9 lines in 6 files changed: 2 ins; 0 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/21647.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21647/head:pull/21647

PR: https://git.openjdk.org/jdk/pull/21647

From ysr at openjdk.org  Wed Oct 23 17:01:55 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 23 Oct 2024 17:01:55 GMT
Subject: RFR: 8342580: GenShen: TestChurnNotifications fails executing in
 unintended test-id modes with ShenandoahGCMode=generational [v2]
In-Reply-To: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
References: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
Message-ID: <jlFdiW4VHReufHjzEGd0FxdUGROU8E0sy-tiN2ku2eA=.38f04bd4-56f9-48ed-9df4-18c7300a3525@github.com>

> Generational mode should not be tested with explicit heuristics settings, which are silently ignored. The test failed in these settings because it needs to be told which memory pool to check for notifications, which needs to be specified differently for generational. Ideally, we'd have the test figure this out rather than needing this to be explicitly specified. For now I just tightened the `@requires` clause to run only in the intended modes.
> 
> Related cleanups will follow separately (such as emitting a warning when an incompatible heuristics option is specified, code which may have been lost in a previous refactor). See details in the ticket.
> 
> **Testing** : tested with no specific collector, with Shenandoah, withGenShen, and with other collectors with test explicitly specified. As intended, in the first and second case 6 test-ids are run, in the thirdd case 1 test-id is run, and in the remaining cases, 0 test-ids are run.

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:

 - Merge branch 'master' into test_requires
 - Merge branch 'master' into test_requires
 - Don't test generational with heuristics it ignores. Separately, we'll
   fix the code for generational such that the choice of a specific
   heuristic elicits a warning.
 - TestChurnNotifications fails when jtreg run with GenShen

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/520/files
  - new: https://git.openjdk.org/shenandoah/pull/520/files/9cf9504f..3744a7f3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=520&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=520&range=00-01

  Stats: 24445 lines in 544 files changed: 20218 ins; 1645 del; 2582 mod
  Patch: https://git.openjdk.org/shenandoah/pull/520.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/520/head:pull/520

PR: https://git.openjdk.org/shenandoah/pull/520

From matsaave at openjdk.org  Wed Oct 23 17:12:40 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Wed, 23 Oct 2024 17:12:40 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v4]
In-Reply-To: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
Message-ID: <mVmqtudKUR8HLr0Wm1Wo2Dphi3IrUGnmj2YE7ZmKL8c=.bedad5be-e1f8-49d3-8d62-d45ee9dc1124@github.com>

> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.

Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision:

  Added @requires

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21647/files
  - new: https://git.openjdk.org/jdk/pull/21647/files/35a6fa4d..8395f445

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=02-03

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21647.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21647/head:pull/21647

PR: https://git.openjdk.org/jdk/pull/21647

From ccheung at openjdk.org  Wed Oct 23 17:12:40 2024
From: ccheung at openjdk.org (Calvin Cheung)
Date: Wed, 23 Oct 2024 17:12:40 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v3]
In-Reply-To: <6hF2-lkUNnzJpmou-rgkj9oSQEwlyAr-ltZFV7At4Ck=.27d2542b-7d76-4eb3-b5f0-4690935cf467@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
 <6hF2-lkUNnzJpmou-rgkj9oSQEwlyAr-ltZFV7At4Ck=.27d2542b-7d76-4eb3-b5f0-4690935cf467@github.com>
Message-ID: <cAHONda1zBfiE0fsfBSn9B4-WKTt8WFNtPd4GzIWyss=.0c56cfe4-c827-4abd-89fd-6ab852b3dc34@github.com>

On Wed, 23 Oct 2024 16:55:21 GMT, Matias Saavedra Silva <matsaave at openjdk.org> wrote:

>> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.
>
> Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Calvin comments and copyright

Changes requested by ccheung (Reviewer).

test/hotspot/jtreg/runtime/cds/appcds/TestParallelGCWithCDS.java line 50:

> 48:  * @requires vm.gc.Parallel
> 49:  * @requires vm.gc.G1
> 50:  * requires vm.bits == "64"

Missing the `@` before `requires`.

-------------

PR Review: https://git.openjdk.org/jdk/pull/21647#pullrequestreview-2389468004
PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1813197656

From ysr at openjdk.org  Wed Oct 23 17:23:49 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 23 Oct 2024 17:23:49 GMT
Subject: RFR: 8342580: GenShen: TestChurnNotifications fails executing in
 unintended test-id modes with ShenandoahGCMode=generational [v2]
In-Reply-To: <jlFdiW4VHReufHjzEGd0FxdUGROU8E0sy-tiN2ku2eA=.38f04bd4-56f9-48ed-9df4-18c7300a3525@github.com>
References: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
 <jlFdiW4VHReufHjzEGd0FxdUGROU8E0sy-tiN2ku2eA=.38f04bd4-56f9-48ed-9df4-18c7300a3525@github.com>
Message-ID: <PD25QeN5ClfEA46u6Tyz1X0uEvHhl-bRKl_wa9eKwXQ=.d335d50b-6db9-4449-bc6c-345e8a85a573@github.com>

On Wed, 23 Oct 2024 17:01:55 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Generational mode should not be tested with explicit heuristics settings, which are silently ignored. The test failed in these settings because it needs to be told which memory pool to check for notifications, which needs to be specified differently for generational. Ideally, we'd have the test figure this out rather than needing this to be explicitly specified. For now I just tightened the `@requires` clause to run only in the intended modes.
>> 
>> Related cleanups will follow separately (such as emitting a warning when an incompatible heuristics option is specified, code which may have been lost in a previous refactor). See details in the ticket.
>> 
>> **Testing** : tested with no specific collector, with Shenandoah, withGenShen, and with other collectors with test explicitly specified. As intended, in the first and second case 6 test-ids are run, in the thirdd case 1 test-id is run, and in the remaining cases, 0 test-ids are run.
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - Merge branch 'master' into test_requires
>  - Merge branch 'master' into test_requires
>  - Don't test generational with heuristics it ignores. Separately, we'll
>    fix the code for generational such that the choice of a specific
>    heuristic elicits a warning.
>  - TestChurnNotifications fails when jtreg run with GenShen

Thanks for your reviews!

I'm landing this with the latest round of testing after the sync with master still in flight. (The previous round before that sync completed without failures.) I do expect this round of GHA tests to show up failures in TestReferenceRefersToShenandoah, which we are addressing in a sibling ticket (actually tracked in two different open tickets at the moment), and which is obviously unrelated to this change.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/520#issuecomment-2432930481

From ysr at openjdk.org  Wed Oct 23 17:23:49 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 23 Oct 2024 17:23:49 GMT
Subject: Integrated: 8342580: GenShen: TestChurnNotifications fails executing
 in unintended test-id modes with ShenandoahGCMode=generational
In-Reply-To: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
References: <iyphEX1m9QlW9V8rmDXpUEeDtuxXUt2qkbsPv0Okd9Y=.2b46ea11-7aff-4045-ba5f-100a3ab9e5d1@github.com>
Message-ID: <S4oyzzQhWvqg8lqLD9BNEvPVVJ1_wcL3-d8x2dEnkNY=.0ca87b65-9d4c-496c-a5d6-b0589aee0b6c@github.com>

On Fri, 18 Oct 2024 21:33:10 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Generational mode should not be tested with explicit heuristics settings, which are silently ignored. The test failed in these settings because it needs to be told which memory pool to check for notifications, which needs to be specified differently for generational. Ideally, we'd have the test figure this out rather than needing this to be explicitly specified. For now I just tightened the `@requires` clause to run only in the intended modes.
> 
> Related cleanups will follow separately (such as emitting a warning when an incompatible heuristics option is specified, code which may have been lost in a previous refactor). See details in the ticket.
> 
> **Testing** : tested with no specific collector, with Shenandoah, withGenShen, and with other collectors with test explicitly specified. As intended, in the first and second case 6 test-ids are run, in the thirdd case 1 test-id is run, and in the remaining cases, 0 test-ids are run.

This pull request has now been integrated.

Changeset: 49707080
Author:    Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/49707080919c2edf42bef05911e60769aa13aa6d
Stats:     5 lines in 1 file changed: 0 ins; 0 del; 5 mod

8342580: GenShen: TestChurnNotifications fails executing in unintended test-id modes with ShenandoahGCMode=generational

Reviewed-by: wkemper, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/520

From wkemper at openjdk.org  Wed Oct 23 18:16:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 23 Oct 2024 18:16:47 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode
In-Reply-To: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
Message-ID: <UtRW7FMx18lPmf8LgR7x4eSnik7k3E7eYL45b733wag=.0509311a-8b3a-4d3e-b947-e7e6e35b1d87@github.com>

On Wed, 23 Oct 2024 06:07:00 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
> 
> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
> 
> 
> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
> ``` 
> 
> Crash:
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
> #
> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
> # Problematic frame:
> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
> #
> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> 
> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
> 
> Logs from instrumentation:
> 
> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
> 
> ...
> 
> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> 
> 
> The fix is simple, just need to swap the two tests,  `!region->is_affiliated()` ...

Changes requested by wkemper (Committer).

Yes, we have a ticket for this test failure: https://bugs.openjdk.org/browse/JDK-8342734.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/523#pullrequestreview-2389662086
PR Comment: https://git.openjdk.org/shenandoah/pull/523#issuecomment-2433052323

From wkemper at openjdk.org  Wed Oct 23 18:16:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 23 Oct 2024 18:16:47 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode
In-Reply-To: <uNbwtocAYx3GWyZnNt96svGlHtfeW70EVJBvE6FpsuY=.b82460cf-2f7e-4a85-bdb7-d4098c00e090@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
 <uNbwtocAYx3GWyZnNt96svGlHtfeW70EVJBvE6FpsuY=.b82460cf-2f7e-4a85-bdb7-d4098c00e090@github.com>
Message-ID: <gvmiSoLc7WiO1l6ht7Ep7osnyHnPuKgNX_pgZ4BSnUY=.20db3000-d84a-4f76-abc8-a2005f9dbaec@github.com>

On Wed, 23 Oct 2024 16:06:25 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
>> 
>> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
>> 
>> 
>> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
>> ``` 
>> 
>> Crash:
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
>> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
>> #
>> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
>> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
>> # Problematic frame:
>> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
>> #
>> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>> #
>> # An error report file with more information is saved as:
>> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
>> #
>> # If you would like to submit a bug report, please visit:
>> #   https://bugreport.java.com/bugreport/crash.jsp
>> #
>> 
>> 
>> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
>> 
>> Logs from instrumentation:
>> 
>> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
>> 
>> ...
>> 
>> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
>> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> ...
>
> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 80:
> 
>> 78:     ShenandoahMarkingContext* const ctx = heap->marking_context();
>> 79:     while (region != nullptr) {
>> 80:       bool needs_reset = !region->is_affiliated() || _generation->contains(region);
> 
> We should really read the affiliation only once for the whole test, but will have to create a new method to achieve that to keep the code clean and encapsulated.

Hmm, you reckon the affiliation is changing between first testing for `_generation->contains` and then testing for `!region->is_affiliated()`? I suppose this _could_ happen, but the fix feels fragile (not sure we can guarantee the compiler won't reorder the tests). We do have `ShenandoahHeapRegion::affiliation`. What about:

  auto affiliation = region->affiliation();
  bool needs_reset = (affiliation == ShenandoahAffiliation::FREE || _generation->contains(affiliation))

Where `ShenandoahGeneration::contains(ShenandoaAffiliation affiliation)` tests if the affiliation and the generation type match?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/523#discussion_r1813300281

From mdoerr at openjdk.org  Wed Oct 23 18:18:23 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Wed, 23 Oct 2024 18:18:23 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <0fDctIMZlpNZ4a5_idrN_w8KnvGfPS49Bw_9WRdjJ9I=.8bedb8be-0b33-468b-b711-9c0b4fb6649e@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <spymZDxJXwseFGxL78zfn4V8qe9NpX3gifTgd_eQkiM=.d1af7cf3-b503-4c35-93bb-db00f3c4cfc8@github.com>
 <U1HQ2ijNoadMNlrW_Adxkju4fM8kfAB4hn-a_gkPrnA=.7e0fee38-8534-4dcc-8aac-77d6559687c8@github.com>
 <CNrcAK8SrGCzsreT3SdoxaoGaqKqGb989t-waCJUz64=.0662bccb-661f-4dc9-b8b6-24d615035261@github.com>
 <YyPrYu-6p5wcUwfMQQamwoM1eGp6eUze_yC1lrc7mb0=.f1e5853b-3b6e-4460-8f57-a6f7588e513e@github.com>
 <Na5LWOz_h0ZGmLcETAxDA9wYiEopgWfvNTbYAELExR8=.e30fe7a0-c766-4bb4-aebb-fc6643c832c1@github.com>
 <0fDctIMZlpNZ4a5_idrN_w8KnvGfPS49Bw_9WRdjJ9I=.8bedb8be-0b33-468b-b711-9c0b4fb6649e@github.com>
Message-ID: <G2EVWD7HifRRUclJtjvBIZS7CdwuYg4HlUH9QyXMLCw=.b9ff7725-2d53-4ae8-94ee-cb32c3bf1dfa@github.com>

On Tue, 22 Oct 2024 13:53:03 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> I will do some benchmarks
>
> I did SpecJBB runs with shift of 6, 8 and 10, respectively, which amounts to Klass alignment of 64, 256 and 1K. Benchmark scores did not show a significant pattern. I did not measure CPU stats though. 
> 
> But I still think a dynamically calculated shift makes sense, and I hesitate to change this code at this point. I therefore would like to move this question to followup RFEs if necessary.

This code causes test errors in `CompressedClassPointersEncodingScheme.java` on s390 and PPC64. It forces the shift to `log_cacheline` which is 7 on PPC64 and 9 on s390. The test passes when we remove "s > log_cacheline && " from the condition below.
In addition, it doesn't fit to the comment which claims we should avoid shifts larger than the cacheline size. This enforces shifts to be larger (or equal to) than the cacheline size.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1813304646

From iklam at openjdk.org  Wed Oct 23 18:48:06 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Wed, 23 Oct 2024 18:48:06 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v4]
In-Reply-To: <ieV8-KLJR4TSGdhfd_qiAyPula_cxH0E6SAm_5SUa90=.c6ed20c1-7046-4acd-ac89-72eac7e75596@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
 <UerwUw4MtQHC8QkhjbWD8aAkhdWVeozIl-1NFSS3YTM=.62746be0-a932-4fb4-9228-ed5992fa5821@github.com>
 <ieV8-KLJR4TSGdhfd_qiAyPula_cxH0E6SAm_5SUa90=.c6ed20c1-7046-4acd-ac89-72eac7e75596@github.com>
Message-ID: <fgjaUyTTWPgLDhsQqEnCEpl8aracgGcjMaTYRUKjGEo=.c0286838-df81-450e-83ea-da4be6560fd7@github.com>

On Wed, 23 Oct 2024 15:42:26 GMT, Matias Saavedra Silva <matsaave at openjdk.org> wrote:

>> src/hotspot/share/prims/whitebox.cpp line 2162:
>> 
>>> 2160: 
>>> 2161: WB_ENTRY(jboolean, WB_CanWriteJavaHeapArchive(JNIEnv* env))
>>> 2162:   return HeapShared::can_write() && ArchiveHeapLoader::can_use();
>> 
>> Do we still need the `&& ArchiveHeapLoader::can_use()` part?
>
> I left it in since it may be a valuable check in case there are further changes or additions to the GCs.

But the code does not do what its name suggests. If this is not required now, I think we should remove the can_load part so the code is not confusing.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21647#discussion_r1813352439

From xpeng at openjdk.org  Wed Oct 23 18:54:28 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 18:54:28 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode
In-Reply-To: <gvmiSoLc7WiO1l6ht7Ep7osnyHnPuKgNX_pgZ4BSnUY=.20db3000-d84a-4f76-abc8-a2005f9dbaec@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
 <uNbwtocAYx3GWyZnNt96svGlHtfeW70EVJBvE6FpsuY=.b82460cf-2f7e-4a85-bdb7-d4098c00e090@github.com>
 <gvmiSoLc7WiO1l6ht7Ep7osnyHnPuKgNX_pgZ4BSnUY=.20db3000-d84a-4f76-abc8-a2005f9dbaec@github.com>
Message-ID: <yGDaG2FSM2BqiDi1OwQNRo429GtFgk6G0S8cnXM4uEE=.06dd8ad0-a4fc-4330-8767-cf22eebe24b4@github.com>

On Wed, 23 Oct 2024 18:11:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 80:
>> 
>>> 78:     ShenandoahMarkingContext* const ctx = heap->marking_context();
>>> 79:     while (region != nullptr) {
>>> 80:       bool needs_reset = !region->is_affiliated() || _generation->contains(region);
>> 
>> We should really read the affiliation only once for the whole test, but will have to create a new method to achieve that to keep the code clean and encapsulated.
>
> Hmm, you reckon the affiliation is changing between first testing for `_generation->contains` and then testing for `!region->is_affiliated()`? I suppose this _could_ happen, but the fix feels fragile (not sure we can guarantee the compiler won't reorder the tests). We do have `ShenandoahHeapRegion::affiliation`. What about:
> 
>   auto affiliation = region->affiliation();
>   bool needs_reset = (affiliation == ShenandoahAffiliation::FREE || _generation->contains(affiliation))
> 
> Where `ShenandoahGeneration::contains(ShenandoaAffiliation affiliation)` tests if the affiliation and the generation type match?

I didn't notice we have `ShenandoahGeneration::contains(ShenandoaAffiliation affiliation) `, I'll update the PR, thanks.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/523#discussion_r1813364499

From egahlin at openjdk.org  Wed Oct 23 19:31:26 2024
From: egahlin at openjdk.org (Erik Gahlin)
Date: Wed, 23 Oct 2024 19:31:26 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
Message-ID: <kg1P7IgX_T7-UHKt5vu-cy5LgmEGCF6DXQskxCjsMPo=.fd01234b-ac4d-48ce-97df-01acd1c1fa7d@github.com>

On Tue, 22 Oct 2024 16:19:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code

Marked as reviewed by egahlin (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2389952721

From egahlin at openjdk.org  Wed Oct 23 19:31:26 2024
From: egahlin at openjdk.org (Erik Gahlin)
Date: Wed, 23 Oct 2024 19:31:26 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
 <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
Message-ID: <Dfb8NRaKopl215q7WQ68VBR4axwO7_dNTle1TCdZPZ0=.70d2d9bd-f0b0-431c-a472-f1e7f2ecfca5@github.com>

On Tue, 22 Oct 2024 16:22:20 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> @egahlin / @mgronlun could you please review the JFR parts of this PR? One change is for getting the right prototype header, the other is for avoiding an endless loop/assert in a corner case.

JFR changes look reasonable.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2433263488

From xpeng at openjdk.org  Wed Oct 23 20:16:36 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 20:16:36 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode [v2]
In-Reply-To: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
Message-ID: <fbLbnsJm4WLl39XbfLtc_frxQxy8b7GYdTz2FJCNyf4=.2ff6c06a-1fdd-4ff2-b51b-e89ff4de9397@github.com>

> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
> 
> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
> 
> 
> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
> ``` 
> 
> Crash:
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
> #
> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
> # Problematic frame:
> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
> #
> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> 
> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
> 
> Logs from instrumentation:
> 
> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
> 
> ...
> 
> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> 
> 
> The fix is simple, just need to swap the two tests,  `!region->is_affiliated()` ...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Add ShenandoahGeneration::contains(ShenandoaAffiliation affiliation)

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/523/files
  - new: https://git.openjdk.org/shenandoah/pull/523/files/8cf581ec..548c6152

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=00-01

  Stats: 18 lines in 8 files changed: 17 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/523.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/523/head:pull/523

PR: https://git.openjdk.org/shenandoah/pull/523

From xpeng at openjdk.org  Wed Oct 23 20:21:59 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 20:21:59 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode [v3]
In-Reply-To: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
Message-ID: <9eB4o-kKd6bwdj6xa67wCRzjBTLj0RhrCPdcZlM99YY=.6a0d886b-f6ea-47ca-af2a-12afba04f0ce@github.com>

> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
> 
> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
> 
> 
> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
> ``` 
> 
> Crash:
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
> #
> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
> # Problematic frame:
> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
> #
> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> 
> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
> 
> Logs from instrumentation:
> 
> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
> 
> ...
> 
> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> 
> 
> The fix is simple, just need to swap the two tests,  `!region->is_affiliated()` ...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  fix build warning

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/523/files
  - new: https://git.openjdk.org/shenandoah/pull/523/files/548c6152..a798bcda

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/523.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/523/head:pull/523

PR: https://git.openjdk.org/shenandoah/pull/523

From xpeng at openjdk.org  Wed Oct 23 20:28:34 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 20:28:34 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode [v4]
In-Reply-To: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
Message-ID: <amuMX265_Z0VPvtYK4whhmFasJHa3uuA6baJvojIqBU=.820c9f19-0073-4ab0-997a-159f2db4e1de@github.com>

> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
> 
> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
> 
> 
> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
> ``` 
> 
> Crash:
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
> #
> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
> # Problematic frame:
> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
> #
> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> 
> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
> 
> Logs from instrumentation:
> 
> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
> 
> ...
> 
> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> 
> 
> The fix is simple, just need to swap the two tests,  `!region->is_affiliated()` ...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Fix build error

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/523/files
  - new: https://git.openjdk.org/shenandoah/pull/523/files/a798bcda..ef777526

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=02-03

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/523.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/523/head:pull/523

PR: https://git.openjdk.org/shenandoah/pull/523

From xpeng at openjdk.org  Wed Oct 23 20:28:34 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 20:28:34 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode [v4]
In-Reply-To: <yGDaG2FSM2BqiDi1OwQNRo429GtFgk6G0S8cnXM4uEE=.06dd8ad0-a4fc-4330-8767-cf22eebe24b4@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
 <uNbwtocAYx3GWyZnNt96svGlHtfeW70EVJBvE6FpsuY=.b82460cf-2f7e-4a85-bdb7-d4098c00e090@github.com>
 <gvmiSoLc7WiO1l6ht7Ep7osnyHnPuKgNX_pgZ4BSnUY=.20db3000-d84a-4f76-abc8-a2005f9dbaec@github.com>
 <yGDaG2FSM2BqiDi1OwQNRo429GtFgk6G0S8cnXM4uEE=.06dd8ad0-a4fc-4330-8767-cf22eebe24b4@github.com>
Message-ID: <iu1yd827sHvqxBPzPrQ8B5g4dNBd_9j5DbgP1nyuLXc=.a1abdd47-fdcd-46a7-926f-537cec0d9a82@github.com>

On Wed, 23 Oct 2024 18:51:42 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Hmm, you reckon the affiliation is changing between first testing for `_generation->contains` and then testing for `!region->is_affiliated()`? I suppose this _could_ happen, but the fix feels fragile (not sure we can guarantee the compiler won't reorder the tests). We do have `ShenandoahHeapRegion::affiliation`. What about:
>> 
>>   auto affiliation = region->affiliation();
>>   bool needs_reset = (affiliation == ShenandoahAffiliation::FREE || _generation->contains(affiliation))
>> 
>> Where `ShenandoahGeneration::contains(ShenandoaAffiliation affiliation)` tests if the affiliation and the generation type match?
>
> I didn't notice we have `ShenandoahGeneration::contains(ShenandoaAffiliation affiliation) `, I'll update the PR, thanks.

I have added `ShenandoahGeneration::contains(ShenandoaAffiliation affiliation)` and updated the PR.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/523#discussion_r1813484827

From wkemper at openjdk.org  Wed Oct 23 21:17:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 23 Oct 2024 21:17:35 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode [v4]
In-Reply-To: <amuMX265_Z0VPvtYK4whhmFasJHa3uuA6baJvojIqBU=.820c9f19-0073-4ab0-997a-159f2db4e1de@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
 <amuMX265_Z0VPvtYK4whhmFasJHa3uuA6baJvojIqBU=.820c9f19-0073-4ab0-997a-159f2db4e1de@github.com>
Message-ID: <pIc-VFwvGM3zQYinveku8eTi9Oiy9shPu3v7ef6htWM=.0ca14ab3-77ea-4eeb-ab57-fcee672b0daa@github.com>

On Wed, 23 Oct 2024 20:28:34 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
>> 
>> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
>> 
>> 
>> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
>> ``` 
>> 
>> Crash:
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
>> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
>> #
>> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
>> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
>> # Problematic frame:
>> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
>> #
>> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>> #
>> # An error report file with more information is saved as:
>> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
>> #
>> # If you would like to submit a bug report, please visit:
>> #   https://bugreport.java.com/bugreport/crash.jsp
>> #
>> 
>> 
>> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
>> 
>> Logs from instrumentation:
>> 
>> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
>> 
>> ...
>> 
>> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
>> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> ...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix build error

Okay with me to /integrate after fixing include order in `shenandoahGeneration.hpp`

src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp line 29:

> 27: 
> 28: #include "memory/allocation.hpp"
> 29: #include "gc/shenandoah/shenandoahAffiliation.hpp"

Nit, but `shenandoah/shenandoahAffiliation.hpp` should come between `heuristics/shenandoahSpaceInfo.hpp` and `shenandoah/shenandoahGenerationType.hpp`.

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/523#pullrequestreview-2390663492
PR Review Comment: https://git.openjdk.org/shenandoah/pull/523#discussion_r1813607434

From wkemper at openjdk.org  Wed Oct 23 21:28:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 23 Oct 2024 21:28:54 GMT
Subject: RFR: 8342919: GenShen: Fix whitespace
Message-ID: <fL7MHEEmENuLFRqqnn3-w1uKw7ONRQU_3kj8SQC1IS4=.1ce1a32a-52bb-46e7-9ce0-5537078c31f3@github.com>

There is an errant new line at the start of shFreeset.hpp.

-------------

Commit messages:
 - Fix whitespace

Changes: https://git.openjdk.org/shenandoah/pull/524/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=524&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342919
  Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/524.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/524/head:pull/524

PR: https://git.openjdk.org/shenandoah/pull/524

From xpeng at openjdk.org  Wed Oct 23 21:34:26 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 21:34:26 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode [v5]
In-Reply-To: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
Message-ID: <VON0A4ccOh2AXO0E2ujrHHRGBDESdBmAahBMqtXGsfM=.0300e16e-ffb6-408d-8713-44666058451e@github.com>

> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
> 
> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
> 
> 
> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
> ``` 
> 
> Crash:
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
> #
> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
> # Problematic frame:
> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
> #
> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> 
> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
> 
> Logs from instrumentation:
> 
> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
> 
> ...
> 
> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> 
> 
> The fix is simple, just need to swap the two tests,  `!region->is_affiliated()` ...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  fix order of includes

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/523/files
  - new: https://git.openjdk.org/shenandoah/pull/523/files/ef777526..302bf4f0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=04
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=523&range=03-04

  Stats: 2 lines in 1 file changed: 1 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/523.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/523/head:pull/523

PR: https://git.openjdk.org/shenandoah/pull/523

From duke at openjdk.org  Wed Oct 23 21:34:32 2024
From: duke at openjdk.org (duke)
Date: Wed, 23 Oct 2024 21:34:32 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode [v4]
In-Reply-To: <amuMX265_Z0VPvtYK4whhmFasJHa3uuA6baJvojIqBU=.820c9f19-0073-4ab0-997a-159f2db4e1de@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
 <amuMX265_Z0VPvtYK4whhmFasJHa3uuA6baJvojIqBU=.820c9f19-0073-4ab0-997a-159f2db4e1de@github.com>
Message-ID: <e82cXARX0p6w1EaiPcFiiOM9UWC1KEL256SZcwEBLnQ=.835ea00f-941f-4d14-a5d2-add7b5320f5f@github.com>

On Wed, 23 Oct 2024 20:28:34 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
>> 
>> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
>> 
>> 
>> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
>> ``` 
>> 
>> Crash:
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
>> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
>> #
>> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
>> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
>> # Problematic frame:
>> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
>> #
>> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>> #
>> # An error report file with more information is saved as:
>> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
>> #
>> # If you would like to submit a bug report, please visit:
>> #   https://bugreport.java.com/bugreport/crash.jsp
>> #
>> 
>> 
>> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
>> 
>> Logs from instrumentation:
>> 
>> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
>> 
>> ...
>> 
>> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
>> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> ...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix build error

@pengxiaolong 
Your change (at version 302bf4f021ef5d5b0aa39009168aecf3a2be9f2d) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/523#issuecomment-2433500910

From xpeng at openjdk.org  Wed Oct 23 21:34:53 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 21:34:53 GMT
Subject: RFR: 8342919: GenShen: Fix whitespace
In-Reply-To: <fL7MHEEmENuLFRqqnn3-w1uKw7ONRQU_3kj8SQC1IS4=.1ce1a32a-52bb-46e7-9ce0-5537078c31f3@github.com>
References: <fL7MHEEmENuLFRqqnn3-w1uKw7ONRQU_3kj8SQC1IS4=.1ce1a32a-52bb-46e7-9ce0-5537078c31f3@github.com>
Message-ID: <41EKLpAksPL6EOYS0H7-zdVatHIdR0M9ikGoVLSqL5I=.3f758946-3b09-4f9c-8877-4724cd814834@github.com>

On Wed, 23 Oct 2024 21:23:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> There is an errant new line at the start of shFreeset.hpp.

Marked as reviewed by xpeng (no project role).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/524#pullrequestreview-2390691357

From xpeng at openjdk.org  Wed Oct 23 21:34:37 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 21:34:37 GMT
Subject: RFR: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode [v4]
In-Reply-To: <pIc-VFwvGM3zQYinveku8eTi9Oiy9shPu3v7ef6htWM=.0ca14ab3-77ea-4eeb-ab57-fcee672b0daa@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
 <amuMX265_Z0VPvtYK4whhmFasJHa3uuA6baJvojIqBU=.820c9f19-0073-4ab0-997a-159f2db4e1de@github.com>
 <pIc-VFwvGM3zQYinveku8eTi9Oiy9shPu3v7ef6htWM=.0ca14ab3-77ea-4eeb-ab57-fcee672b0daa@github.com>
Message-ID: <wBYrtf5UNkGsmQBKIHIe1p3LBKmlccsnB2qFkZQ8Z9I=.e90d02d8-4683-4467-a386-80a78c265eac@github.com>

On Wed, 23 Oct 2024 21:13:39 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix build error
>
> src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp line 29:
> 
>> 27: 
>> 28: #include "memory/allocation.hpp"
>> 29: #include "gc/shenandoah/shenandoahAffiliation.hpp"
> 
> Nit, but `shenandoah/shenandoahAffiliation.hpp` should come between `heuristics/shenandoahSpaceInfo.hpp` and `shenandoah/shenandoahGenerationType.hpp`.

Fixed, thanks.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/523#discussion_r1813654683

From xpeng at openjdk.org  Wed Oct 23 22:41:21 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 23 Oct 2024 22:41:21 GMT
Subject: Integrated: 8342848: Shenandoah: Marking bitmap may not be completely
 cleared in generational mode
In-Reply-To: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
References: <zEVPFYS6fDGalfRdDv4YClxFaVhlLEJBAfeIEVXQMmY=.d2d47327-06dc-44e1-9cc9-bd412bd35234@github.com>
Message-ID: <d3YyRTgXXMB8j1vRGxsLWyYc4RFKo2TVLPLiMhDK0aI=.e519d735-8432-4461-9cd4-576eddf6e4b4@github.com>

On Wed, 23 Oct 2024 06:07:00 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> In the investigation of the crashe I saw in PR https://github.com/openjdk/shenandoah/pull/516, I happened to reproduce the crash GenShen TIP as well.  
> 
> The crash was reproduced multi times on both AWS r7g-4xlarge and r7i-4xlarge instances by running test below repeatedly:
> 
> 
> CONF=linux-aarch64-server-fastdebug  make clean test TEST=gc/stress/gcold/TestGCOldWithShenandoah.java#generational JTREG="REPEAT_COUNT=1000" 
> ``` 
> 
> Crash:
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error (/home/xlpeng/repos/jdk-xlpeng/src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp:642), pid=24134, tid=24158
> #  assert(_generation->is_bitmap_clear()) failed: need clear marking bitmap
> #
> # JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-adhoc.xlpeng.jdk-xlpeng)
> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 24-internal-adhoc.xlpeng.jdk-xlpeng, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-aarch64)
> # Problematic frame:
> # V  [libjvm.so+0x15eadc4]  ShenandoahConcurrentGC::op_init_mark()+0x358
> #
> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /local/home/xlpeng/repos/jdk-xlpeng/build/linux-aarch64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_gc_stress_gcold_TestGCOldWithShenandoah_java_generational/scratch/0/hs_err_pid24134.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://bugreport.java.com/bugreport/crash.jsp
> #
> 
> 
> With logging/instrumentation,  it seems to be caused by the one line  code `bool needs_reset = _generation->contains(region) || !region->is_affiliated(); `, considering bitmap reset is a concurrent operation, if is possible mutator thread changes the affiliation from FREE to YOUNG when  bitmap reset is running, both `_generation->contains(region)` and `!region->is_affiliated()` can be false when affiliation is FREE and mutator is updating it at the same time.
> 
> Logs from instrumentation:
> 
> [32.793s][info][gc          ] GC(19) Not reseting bitmap for YOUNG region (0x0000ffff8c1a6100)(affiliation before test: FREE)
> 
> ...
> 
> [32.807s][info][gc,task     ] GC(20) Using 8 of 8 workers for init marking
> [32.808s][info][gc          ] GC(20) Region (0x0000ffff8c1a6100) doesn't have clear bitmap, [1, 1, 1]
> 
> 
> The fix is simple, just need to swap the two tests,  `!region->is_affiliated()` ...

This pull request has now been integrated.

Changeset: 55c6f678
Author:    Xiaolong Peng <xpeng at openjdk.org>
Committer: William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/55c6f678722627722fe840acd8a21fb93329d3b1
Stats:     19 lines in 8 files changed: 18 ins; 0 del; 1 mod

8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/523

From wkemper at openjdk.org  Wed Oct 23 23:13:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 23 Oct 2024 23:13:00 GMT
Subject: RFR: 8342861: GenShen: Old generation in unexpected state when
 abandoning mixed gc candidates [v2]
In-Reply-To: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>
References: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>
Message-ID: <Vf-zaq-YReHezC2zAj_xNWJ7o7bhCks_RAscD5Ms4d4=.b7c99650-4ad7-43fd-aac8-93452d4b4dab@github.com>

> When the generational mode cannot add any old regions to a mixed collection because they are _all_ pinned, it gives up trying to evacuate these regions. The regions must then be made parsable. However, if this phase is interrupted by a young collection it should no longer try to evacuate any old regions. Doing so triggered an assert when testing with `-XX:ShenandoahCoalesceChance=80` and `XX:+ShenandoahAllocFailureALot`.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Problem list failing reference processing test

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/522/files
  - new: https://git.openjdk.org/shenandoah/pull/522/files/1356221f..8532d08f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=522&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=522&range=00-01

  Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/522.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/522/head:pull/522

PR: https://git.openjdk.org/shenandoah/pull/522

From kdnilsen at openjdk.org  Wed Oct 23 23:17:18 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Oct 2024 23:17:18 GMT
Subject: RFR: 8342919: GenShen: Fix whitespace
In-Reply-To: <fL7MHEEmENuLFRqqnn3-w1uKw7ONRQU_3kj8SQC1IS4=.1ce1a32a-52bb-46e7-9ce0-5537078c31f3@github.com>
References: <fL7MHEEmENuLFRqqnn3-w1uKw7ONRQU_3kj8SQC1IS4=.1ce1a32a-52bb-46e7-9ce0-5537078c31f3@github.com>
Message-ID: <ZuSJkqCGKR1QNog7LO2GOfmskdX_mGTnwza67m8SC9I=.a042fdb5-cfcc-4cd5-ab3b-dbc6c350c930@github.com>

On Wed, 23 Oct 2024 21:23:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> There is an errant new line at the start of shFreeset.hpp.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/524#pullrequestreview-2390835772

From wkemper at openjdk.org  Wed Oct 23 23:19:48 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 23 Oct 2024 23:19:48 GMT
Subject: RFR: 8342924: GenShen: Problem list
 gc/shenandoah/TestReferenceRefersToShenandoah.java
Message-ID: <PttRJho0RngGcu4JJAaROEl7mej1zpROn1DCxSV_IHM=.6c878e45-d4ac-4950-80c4-fd76a5704200@github.com>

Skip gc/shenandoah/TestReferenceRefersToShenandoah.java while we work on solution.

-------------

Commit messages:
 - Problem list failing reference processing test

Changes: https://git.openjdk.org/shenandoah/pull/525/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=525&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342924
  Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/525.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/525/head:pull/525

PR: https://git.openjdk.org/shenandoah/pull/525

From kdnilsen at openjdk.org  Wed Oct 23 23:19:48 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Oct 2024 23:19:48 GMT
Subject: RFR: 8342924: GenShen: Problem list
 gc/shenandoah/TestReferenceRefersToShenandoah.java
In-Reply-To: <PttRJho0RngGcu4JJAaROEl7mej1zpROn1DCxSV_IHM=.6c878e45-d4ac-4950-80c4-fd76a5704200@github.com>
References: <PttRJho0RngGcu4JJAaROEl7mej1zpROn1DCxSV_IHM=.6c878e45-d4ac-4950-80c4-fd76a5704200@github.com>
Message-ID: <0-_HPj64y-2HtRY6Wk7ulVJnRwvZIHn8_IMZ3IdLYjs=.c00031b4-1e31-45ed-8f88-32d44435ff88@github.com>

On Wed, 23 Oct 2024 23:14:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Skip gc/shenandoah/TestReferenceRefersToShenandoah.java while we work on solution.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/525#pullrequestreview-2390836630

From ysr at openjdk.org  Wed Oct 23 23:44:26 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 23 Oct 2024 23:44:26 GMT
Subject: RFR: 8342924: GenShen: Problem list
 gc/shenandoah/TestReferenceRefersToShenandoah.java
In-Reply-To: <PttRJho0RngGcu4JJAaROEl7mej1zpROn1DCxSV_IHM=.6c878e45-d4ac-4950-80c4-fd76a5704200@github.com>
References: <PttRJho0RngGcu4JJAaROEl7mej1zpROn1DCxSV_IHM=.6c878e45-d4ac-4950-80c4-fd76a5704200@github.com>
Message-ID: <Ie3HhdTv4XeJPO_00GrPnBtJK9Yrwp9zFBo4rpFikfA=.402eeeb8-82e3-4912-b555-162b631db5cd@github.com>

On Wed, 23 Oct 2024 23:14:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Skip gc/shenandoah/TestReferenceRefersToShenandoah.java while we work on solution.

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/525#pullrequestreview-2390871724

From wkemper at openjdk.org  Wed Oct 23 23:49:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 23 Oct 2024 23:49:20 GMT
Subject: Integrated: 8342924: GenShen: Problem list
 gc/shenandoah/TestReferenceRefersToShenandoah.java
In-Reply-To: <PttRJho0RngGcu4JJAaROEl7mej1zpROn1DCxSV_IHM=.6c878e45-d4ac-4950-80c4-fd76a5704200@github.com>
References: <PttRJho0RngGcu4JJAaROEl7mej1zpROn1DCxSV_IHM=.6c878e45-d4ac-4950-80c4-fd76a5704200@github.com>
Message-ID: <dz3uV-a-9gcNJQDy6Y8GzR4duD9n-kLFZKr13OmGWUQ=.c0afff52-79f3-46c0-823f-72a35e2dd5e0@github.com>

On Wed, 23 Oct 2024 23:14:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Skip gc/shenandoah/TestReferenceRefersToShenandoah.java while we work on solution.

This pull request has now been integrated.

Changeset: 863ebda8
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/863ebda853c66e616eccf18ef577ce74047d74b1
Stats:     2 lines in 1 file changed: 1 ins; 0 del; 1 mod

8342924: GenShen: Problem list gc/shenandoah/TestReferenceRefersToShenandoah.java

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/525

From xpeng at openjdk.org  Thu Oct 24 00:07:56 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 24 Oct 2024 00:07:56 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v5]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <9nJiHrjgsKJEcf76iUom8_Lgg5MZYx16qaIrP9umNp0=.ea7ef6cf-195f-4972-8e4b-3e758928e4f0@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 28 commits:

 - Merge branch 'shenandoah' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION
 - Address review comments
 - Reset bitmap after ShenandoahOldGC
 - Not rest bitmap after cycle if it needs to bootstrap old gc
 - typo in comments
 - Minimize the regions to walk when reset bitmap
 - comments
 - bug fix
 - ... and 18 more: https://git.openjdk.org/shenandoah/compare/863ebda8...34b64ee7

-------------

Changes: https://git.openjdk.org/shenandoah/pull/516/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=04
  Stats: 106 lines in 8 files changed: 96 ins; 2 del; 8 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From wkemper at openjdk.org  Thu Oct 24 00:17:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 00:17:45 GMT
Subject: RFR: 8342927: GenShen: Guarantee slices of time for coalesce and
 filling
Message-ID: <EFh4LKe-Qi5mvssTqby9oDzs6kBVH2-L6PJJRhk-et0=.387c94b9-2569-43f1-b2c2-6db309244aa3@github.com>

By design, young collections may preempt old collections. However, in some cases, this leads to starvation of old collection cycles. When old generation collections cannot make progress, the system continues to get "backed up" until an eventual degenerated or full GC cleans out the old generation. Genshen already has a mechanism to guarantee progress for old marking. It should be extended to also provide for coalescing and filling.

-------------

Commit messages:
 - Guarantee time slice for old marking and preparing for old marking

Changes: https://git.openjdk.org/shenandoah/pull/526/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=526&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342927
  Stats: 12 lines in 2 files changed: 3 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/shenandoah/pull/526.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/526/head:pull/526

PR: https://git.openjdk.org/shenandoah/pull/526

From kdnilsen at openjdk.org  Thu Oct 24 00:26:19 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 24 Oct 2024 00:26:19 GMT
Subject: RFR: 8342927: GenShen: Guarantee slices of time for coalesce and
 filling
In-Reply-To: <EFh4LKe-Qi5mvssTqby9oDzs6kBVH2-L6PJJRhk-et0=.387c94b9-2569-43f1-b2c2-6db309244aa3@github.com>
References: <EFh4LKe-Qi5mvssTqby9oDzs6kBVH2-L6PJJRhk-et0=.387c94b9-2569-43f1-b2c2-6db309244aa3@github.com>
Message-ID: <AY74jn80CV44G7oOY6T4Hv7ij6fjc83J_2cLFKNy0gM=.921fc434-1ff2-4f45-b23c-d54649b764d0@github.com>

On Thu, 24 Oct 2024 00:13:11 GMT, William Kemper <wkemper at openjdk.org> wrote:

> By design, young collections may preempt old collections. However, in some cases, this leads to starvation of old collection cycles. When old generation collections cannot make progress, the system continues to get "backed up" until an eventual degenerated or full GC cleans out the old generation. Genshen already has a mechanism to guarantee progress for old marking. It should be extended to also provide for coalescing and filling.

Super.  Thanks.

src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 561:

> 559:                                                                             \
> 560:   product(uintx, ShenandoahMinimumOldTimeMs, 100, EXPERIMENTAL,         \
> 561:          "Minimum amount of time in milliseconds to run old marking "       \

May want to adjust the "description": Minimum time to run old marking or old cleanup (C&F could be considered cleanup - or just say C&F)...

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/526#pullrequestreview-2390913091
PR Review Comment: https://git.openjdk.org/shenandoah/pull/526#discussion_r1814048346

From lmesnik at openjdk.org  Thu Oct 24 05:15:29 2024
From: lmesnik at openjdk.org (Leonid Mesnik)
Date: Thu, 24 Oct 2024 05:15:29 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
Message-ID: <kIFHgHW4e-LKs1cklS5gCfulvfjqP8uEHm2fvQNDLJM=.6e53020e-50ff-484b-b0ba-626dccf75c10@github.com>

On Tue, 22 Oct 2024 16:19:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code

Not actually review, just confirming that my request fulfilled. And no more issues arised during PIT

-------------

Marked as reviewed by lmesnik (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2391292083

From lmesnik at openjdk.org  Thu Oct 24 05:15:30 2024
From: lmesnik at openjdk.org (Leonid Mesnik)
Date: Thu, 24 Oct 2024 05:15:30 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v6]
In-Reply-To: <YlpC3YUY4jE9Qb4Et5SqUc3q16AkDrY3b9-mXSMKzSg=.dd4d0fd4-ffac-42b8-83f5-639f3a930f8c@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <BGmZ04yibLn_KtPyQrA3sQv5OtMonrcDy3aN0vAj8NA=.5de6cae3-3a05-4a74-a15c-aade4176bdeb@github.com>
 <JEzJGLkDHhgNGFns5-xLFV1TGH2-EkzWd568IcAO1Rg=.b48dc38e-57a9-4e4f-b512-fc86efdb77c4@github.com>
 <YlpC3YUY4jE9Qb4Et5SqUc3q16AkDrY3b9-mXSMKzSg=.dd4d0fd4-ffac-42b8-83f5-639f3a930f8c@github.com>
Message-ID: <Q1kbjcs2NnrkL51fHs7B8y-Kyx-zVRor8ZKmRoXmXvw=.3ef65ffe-2edb-4640-874e-6b389926e12e@github.com>

On Fri, 30 Aug 2024 07:37:35 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> make/Images.gmk line 135:
>> 
>>> 133: #
>>> 134: # Param1 - VM variant (e.g., server, client, zero, ...)
>>> 135: # Param2 - _nocoops, _coh, _nocoops_coh, or empty
>> 
>> The -XX:+UseCompactObjectHeaders ssems to incompatible withe zero vm. The zero vm build start failing while generating shared archive with +UseCompactObjectHeaders. Generation should be disabled by default for zero to don't break the build.
>
> No, zero works with +COH, but a small change is needed. I'll post a suggestion inline.

no objection anymore

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1814259543

From stuefe at openjdk.org  Thu Oct 24 09:15:30 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Thu, 24 Oct 2024 09:15:30 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <G2EVWD7HifRRUclJtjvBIZS7CdwuYg4HlUH9QyXMLCw=.b9ff7725-2d53-4ae8-94ee-cb32c3bf1dfa@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <spymZDxJXwseFGxL78zfn4V8qe9NpX3gifTgd_eQkiM=.d1af7cf3-b503-4c35-93bb-db00f3c4cfc8@github.com>
 <U1HQ2ijNoadMNlrW_Adxkju4fM8kfAB4hn-a_gkPrnA=.7e0fee38-8534-4dcc-8aac-77d6559687c8@github.com>
 <CNrcAK8SrGCzsreT3SdoxaoGaqKqGb989t-waCJUz64=.0662bccb-661f-4dc9-b8b6-24d615035261@github.com>
 <YyPrYu-6p5wcUwfMQQamwoM1eGp6eUze_yC1lrc7mb0=.f1e5853b-3b6e-4460-8f57-a6f7588e513e@github.com>
 <Na5LWOz_h0ZGmLcETAxDA9wYiEopgWfvNTbYAELExR8=.e30fe7a0-c766-4bb4-aebb-fc6643c832c1@github.com>
 <0fDctIMZlpNZ4a5_idrN_w8KnvGfPS49Bw_9WRdjJ9I=.8bedb8be-0b33-468b-b711-9c0b4fb6649e@github.com>
 <G2EVWD7HifRRUclJtjvBIZS7CdwuYg4HlUH9QyXMLCw=.b9ff7725-2d53-4ae8-94ee-cb32c3bf1dfa@github.com>
Message-ID: <vMlI9KnBvvHNvxT__7149la64QKN1CrHlr3OoX17x-E=.e06e392f-0e55-465d-b33e-a69d7f6ba5b3@github.com>

On Wed, 23 Oct 2024 18:14:50 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:

> This code causes test errors in `CompressedClassPointersEncodingScheme.java` on s390 and PPC64. It forces the shift to `log_cacheline` which is 7 on PPC64 and 9 on s390. The test passes when we remove "s > log_cacheline && " from the condition below.

It's a bit late. We are close to pushing. While it should be harmless to drop below alignment to below cache line size, this would be a change affecting all platforms and would require all tests repeated. 

PPC/s390 are not targeted by the JEP. There had never been a discussion I am aware of that these platforms have to be clean with +COH. While it's nice that the changes had been contributed, I don't think that test errors on these platforms should hold up pushing this RFE. Therefore, if needed, we should just omit +COH part of the test for PPC/S390.

But then, what exactly is the error? If it's just the test assuming that cache line size is log 6, then the test should be fixed for ppc, not hotspot.

> In addition, it doesn't fit to the comment which claims we should avoid shifts larger than the cacheline size. This enforces shifts to be larger (or equal to) than the cacheline size.

?? The comment is correct. We try to avoid hyper alignment, hence we drop the shift to - if possible - log 2 cache line size. If it's equal to log 2 cache line size, we succeeded.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1814598543

From stuefe at openjdk.org  Thu Oct 24 09:25:35 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Thu, 24 Oct 2024 09:25:35 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <vMlI9KnBvvHNvxT__7149la64QKN1CrHlr3OoX17x-E=.e06e392f-0e55-465d-b33e-a69d7f6ba5b3@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <spymZDxJXwseFGxL78zfn4V8qe9NpX3gifTgd_eQkiM=.d1af7cf3-b503-4c35-93bb-db00f3c4cfc8@github.com>
 <U1HQ2ijNoadMNlrW_Adxkju4fM8kfAB4hn-a_gkPrnA=.7e0fee38-8534-4dcc-8aac-77d6559687c8@github.com>
 <CNrcAK8SrGCzsreT3SdoxaoGaqKqGb989t-waCJUz64=.0662bccb-661f-4dc9-b8b6-24d615035261@github.com>
 <YyPrYu-6p5wcUwfMQQamwoM1eGp6eUze_yC1lrc7mb0=.f1e5853b-3b6e-4460-8f57-a6f7588e513e@github.com>
 <Na5LWOz_h0ZGmLcETAxDA9wYiEopgWfvNTbYAELExR8=.e30fe7a0-c766-4bb4-aebb-fc6643c832c1@github.com>
 <0fDctIMZlpNZ4a5_idrN_w8KnvGfPS49Bw_9WRdjJ9I=.8bedb8be-0b33-468b-b711-9c0b4fb6649e@github.com>
 <G2EVWD7HifRRUclJtjvBIZS7CdwuYg4HlUH9QyXMLCw=.b9ff7725-2d53-4ae8-94ee-cb32c3bf1dfa@github.com>
 <vMlI9KnBvvHNvxT__7149la64QKN1CrHlr3
 OoX17x-E=.e06e392f-0e55-465d-b33e-a69d7f6ba5b3@github.com>
Message-ID: <cQxx13f6ujrDdrDqN_ynyMHKAEOJsKWwPpNDs3da_Mo=.d2fcfe0c-77d1-4e9c-8022-d103fd91fa0c@github.com>

On Thu, 24 Oct 2024 09:12:34 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> But then, what exactly is the error? If it's just the test assuming that cache line size is log 6, then the test should be fixed for ppc, not hotspot.

that is the problem, test assumes log2 of 6 for chacheline size

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1814617136

From amitkumar at openjdk.org  Thu Oct 24 09:31:31 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Thu, 24 Oct 2024 09:31:31 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <cQxx13f6ujrDdrDqN_ynyMHKAEOJsKWwPpNDs3da_Mo=.d2fcfe0c-77d1-4e9c-8022-d103fd91fa0c@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <spymZDxJXwseFGxL78zfn4V8qe9NpX3gifTgd_eQkiM=.d1af7cf3-b503-4c35-93bb-db00f3c4cfc8@github.com>
 <U1HQ2ijNoadMNlrW_Adxkju4fM8kfAB4hn-a_gkPrnA=.7e0fee38-8534-4dcc-8aac-77d6559687c8@github.com>
 <CNrcAK8SrGCzsreT3SdoxaoGaqKqGb989t-waCJUz64=.0662bccb-661f-4dc9-b8b6-24d615035261@github.com>
 <YyPrYu-6p5wcUwfMQQamwoM1eGp6eUze_yC1lrc7mb0=.f1e5853b-3b6e-4460-8f57-a6f7588e513e@github.com>
 <Na5LWOz_h0ZGmLcETAxDA9wYiEopgWfvNTbYAELExR8=.e30fe7a0-c766-4bb4-aebb-fc6643c832c1@github.com>
 <0fDctIMZlpNZ4a5_idrN_w8KnvGfPS49Bw_9WRdjJ9I=.8bedb8be-0b33-468b-b711-9c0b4fb6649e@github.com>
 <G2EVWD7HifRRUclJtjvBIZS7CdwuYg4HlUH9QyXMLCw=.b9ff7725-2d53-4ae8-94ee-cb32c3bf1dfa@github.com>
 <vMlI9KnBvvHNvxT__7149la64QKN1CrHlr3
 OoX17x-E=.e06e392f-0e55-465d-b33e-a69d7f6ba5b3@github.com>
 <cQxx13f6ujrDdrDqN_ynyMHKAEOJsKWwPpNDs3da_Mo=.d2fcfe0c-77d1-4e9c-8022-d103fd91fa0c@github.com>
Message-ID: <XuC0_RaUpR0_JvlEzq8-j7oqovvyqNHFQyWPmDQamqA=.a5e8730c-555a-4872-919b-c39fa0873f97@github.com>

On Thu, 24 Oct 2024 09:22:34 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>>> This code causes test errors in `CompressedClassPointersEncodingScheme.java` on s390 and PPC64. It forces the shift to `log_cacheline` which is 7 on PPC64 and 9 on s390. The test passes when we remove "s > log_cacheline && " from the condition below.
>> 
>> It's a bit late. We are close to pushing. While it should be harmless to drop below alignment to below cache line size, this would be a change affecting all platforms and would require all tests repeated. 
>> 
>> PPC/s390 are not targeted by the JEP. There had never been a discussion I am aware of that these platforms have to be clean with +COH. While it's nice that the changes had been contributed, I don't think that test errors on these platforms should hold up pushing this RFE. Therefore, if needed, we should just omit +COH part of the test for PPC/S390.
>> 
>> But then, what exactly is the error? If it's just the test assuming that cache line size is log 6, then the test should be fixed for ppc, not hotspot.
>> 
>>> In addition, it doesn't fit to the comment which claims we should avoid shifts larger than the cacheline size. This enforces shifts to be larger (or equal to) than the cacheline size.
>> 
>> ?? The comment is correct. We try to avoid hyper alignment, hence we drop the shift to - if possible - log 2 cache line size. If it's equal to log 2 cache line size, we succeeded.
>
>> But then, what exactly is the error? If it's just the test assuming that cache line size is log 6, then the test should be fixed for ppc, not hotspot.
> 
> that is the problem, test assumes log2 of 6 for chacheline size

PPC log2 will be `7` (`DEFAULT_CACHE_LINE_SIZE? = 128`) and for S390x it will be `8` (`DEFAULT_CACHE_LINE_SIZE? = 256`). 

So I guess this change should be fine for now : 

diff --git a/test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java b/test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java
index e04e716315a..c1be59e77ab 100644
--- a/test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java
+++ b/test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java
@@ -108,7 +108,9 @@ public static void main(String[] args) throws Exception {
 
         long ccsSize = 128 * M;
         int expectedShift = 6;
-        test(forceAddress, true, ccsSize, forceAddress, expectedShift);
+        if (!Platform.isPPC() && !Platform.isS390x()) {
+            test(forceAddress, true, ccsSize, forceAddress, expectedShift);
+        }
 
         ccsSize = 512 * M;
         expectedShift = 8;

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1814627120

From amitkumar at openjdk.org  Thu Oct 24 09:46:31 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Thu, 24 Oct 2024 09:46:31 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
 <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
Message-ID: <ISW5WbcZr7Rr8Zg462H--apDVxKgiDKzVWGlXfhlgek=.2ea3202f-29e6-47f1-987c-52e11add973d@github.com>

On Tue, 22 Oct 2024 16:22:20 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Update copyright
>>  - Avoid assert/endless-loop in JFR code
>
> @egahlin / @mgronlun could you please review the JFR parts of this PR? One change is for getting the right prototype header, the other is for avoiding an endless loop/assert in a corner case.

@rkennke Please include s390x implementation from here: https://github.com/offamitkumar/jdk/commit/e67e332ce6b3b09e723c08b11146ebe0cc16f0fd. 

This also disables this test on s390x & PPC for now, but if that's not what we want then I can revert changes done to `CompressedClassPointersEncodingScheme.java` file.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2434795830

From mdoerr at openjdk.org  Thu Oct 24 09:57:41 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Thu, 24 Oct 2024 09:57:41 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <XuC0_RaUpR0_JvlEzq8-j7oqovvyqNHFQyWPmDQamqA=.a5e8730c-555a-4872-919b-c39fa0873f97@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <spymZDxJXwseFGxL78zfn4V8qe9NpX3gifTgd_eQkiM=.d1af7cf3-b503-4c35-93bb-db00f3c4cfc8@github.com>
 <U1HQ2ijNoadMNlrW_Adxkju4fM8kfAB4hn-a_gkPrnA=.7e0fee38-8534-4dcc-8aac-77d6559687c8@github.com>
 <CNrcAK8SrGCzsreT3SdoxaoGaqKqGb989t-waCJUz64=.0662bccb-661f-4dc9-b8b6-24d615035261@github.com>
 <YyPrYu-6p5wcUwfMQQamwoM1eGp6eUze_yC1lrc7mb0=.f1e5853b-3b6e-4460-8f57-a6f7588e513e@github.com>
 <Na5LWOz_h0ZGmLcETAxDA9wYiEopgWfvNTbYAELExR8=.e30fe7a0-c766-4bb4-aebb-fc6643c832c1@github.com>
 <0fDctIMZlpNZ4a5_idrN_w8KnvGfPS49Bw_9WRdjJ9I=.8bedb8be-0b33-468b-b711-9c0b4fb6649e@github.com>
 <G2EVWD7HifRRUclJtjvBIZS7CdwuYg4HlUH9QyXMLCw=.b9ff7725-2d53-4ae8-94ee-cb32c3bf1dfa@github.com>
 <vMlI9KnBvvHNvxT__7149la64QKN1CrHlr3
 OoX17x-E=.e06e392f-0e55-465d-b33e-a69d7f6ba5b3@github.com>
 <cQxx13f6ujrDdrDqN_ynyMHKAEOJsKWwPpNDs3da_Mo=.d2fcfe0c-77d1-4e9c-8022-d103fd91fa0c@github.com>
 <XuC0_RaUpR0_JvlEzq8-j7oqovvyqNHFQyWPmDQamqA=.a5e8730c-555a-4872-919b-c39fa0873f97@github.com>
Message-ID: <EhhpOu63Iz3L0zJ24gmprx32GB024wOfF8dYejNwtBY=.95d2a304-9033-4fe0-9f20-939608451b9e@github.com>

On Thu, 24 Oct 2024 09:28:13 GMT, Amit Kumar <amitkumar at openjdk.org> wrote:

>>> But then, what exactly is the error? If it's just the test assuming that cache line size is log 6, then the test should be fixed for ppc, not hotspot.
>> 
>> that is the problem, test assumes log2 of 6 for chacheline size
>
> PPC log2 will be `7` (`DEFAULT_CACHE_LINE_SIZE? = 128`) and for S390x it will be `8` (`DEFAULT_CACHE_LINE_SIZE? = 256`). 
> 
> So I guess this change should be fine for now : 
> 
> diff --git a/test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java b/test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java
> index e04e716315a..c1be59e77ab 100644
> --- a/test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java
> +++ b/test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java
> @@ -108,7 +108,9 @@ public static void main(String[] args) throws Exception {
>  
>          long ccsSize = 128 * M;
>          int expectedShift = 6;
> -        test(forceAddress, true, ccsSize, forceAddress, expectedShift);
> +        if (!Platform.isPPC() && !Platform.isS390x()) {
> +            test(forceAddress, true, ccsSize, forceAddress, expectedShift);
> +        }
>  
>          ccsSize = 512 * M;
>          expectedShift = 8;

As I understand the comment, it says alignment <= cache line size. But the implementation makes alignment >= cache line size. "hyper alignment" means alignment > cache line size?
If we want alignment = cache line size, I'll be ok with disabling the +COH part of the test on PPC64 and s390. Correct, the problem is that the test assumes that log cache line size = 6.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1814666149

From stuefe at openjdk.org  Thu Oct 24 10:05:29 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Thu, 24 Oct 2024 10:05:29 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v21]
In-Reply-To: <EhhpOu63Iz3L0zJ24gmprx32GB024wOfF8dYejNwtBY=.95d2a304-9033-4fe0-9f20-939608451b9e@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <TCFshDmOrJjN0zTh1X9BGEa_rczyMqRbyDIiYsFr_5I=.ea051371-4c86-461e-8721-83b32caf808f@github.com>
 <qPL-XuWVkfNYFmfYatyQZwbHismNzW_jXdvkDvIiVNc=.d4fd26a8-f509-494a-bd3d-0c6f37ba5269@github.com>
 <spymZDxJXwseFGxL78zfn4V8qe9NpX3gifTgd_eQkiM=.d1af7cf3-b503-4c35-93bb-db00f3c4cfc8@github.com>
 <U1HQ2ijNoadMNlrW_Adxkju4fM8kfAB4hn-a_gkPrnA=.7e0fee38-8534-4dcc-8aac-77d6559687c8@github.com>
 <CNrcAK8SrGCzsreT3SdoxaoGaqKqGb989t-waCJUz64=.0662bccb-661f-4dc9-b8b6-24d615035261@github.com>
 <YyPrYu-6p5wcUwfMQQamwoM1eGp6eUze_yC1lrc7mb0=.f1e5853b-3b6e-4460-8f57-a6f7588e513e@github.com>
 <Na5LWOz_h0ZGmLcETAxDA9wYiEopgWfvNTbYAELExR8=.e30fe7a0-c766-4bb4-aebb-fc6643c832c1@github.com>
 <0fDctIMZlpNZ4a5_idrN_w8KnvGfPS49Bw_9WRdjJ9I=.8bedb8be-0b33-468b-b711-9c0b4fb6649e@github.com>
 <G2EVWD7HifRRUclJtjvBIZS7CdwuYg4HlUH9QyXMLCw=.b9ff7725-2d53-4ae8-94ee-cb32c3bf1dfa@github.com>
 <vMlI9KnBvvHNvxT__7149la64QKN1CrHlr3
 OoX17x-E=.e06e392f-0e55-465d-b33e-a69d7f6ba5b3@github.com>
 <cQxx13f6ujrDdrDqN_ynyMHKAEOJsKWwPpNDs3da_Mo=.d2fcfe0c-77d1-4e9c-8022-d103fd91fa0c@github.com>
 <XuC0_RaUpR0_JvlEzq8-j7oqovvyqNHFQyWPmDQamqA=.a5e8730c-555a-4872-919b-c39fa0873f97@github.com>
 <EhhpOu63Iz3L0zJ24gmprx32GB024wOfF8dYejNwtBY=.95d2a304-9033-4fe0-9f20-939608451b9e@github.com>
Message-ID: <jukDFOOSL9NVHA-RzXo6E9nhIV66vCf0lVnBtqu73ug=.ea8a5277-c7d3-4eb0-84b0-55960e4f3d38@github.com>

On Thu, 24 Oct 2024 09:54:05 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:

> As I understand the comment, it says alignment <= cache line size. But the implementation makes alignment >= cache line size. "hyper alignment" means alignment > cache line size?

Correct. since encoding range must cover the full klass range, and we only have 22bit nklass, shift is larger. at most 10. but since that causes hyper aligning, we try to get away with smaller shifts if klass range is smaller.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1814680330

From shade at openjdk.org  Thu Oct 24 10:44:48 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 24 Oct 2024 10:44:48 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v7]
In-Reply-To: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
Message-ID: <_iuE0SMhZnZ8tw5oEjRRehxeaGEC8abMF0QZrIxLjlw=.c90a9135-8b1e-414b-a177-f168f707840f@github.com>

> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
> 
> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
> 
> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
> 
> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
> 
> Additional testing:
>  - [ ] `jdk_jfr` pass by default
>  - [ ] `jdk_jfr` now passes with `-XX:+UseShenandoah`

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:

 - Redo the whole thing by disabling Leak Prof with Shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Just exclude the tests for Shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Tighten up comments prose
 - Roman's review: more precise GC state check, more includes
 - Referencing INCLUDE_SHENANDOAHGC nominally requires macro.hpp
 - Try to make tests faster / more robust
 - ... and 1 more: https://git.openjdk.org/jdk/compare/53b27962...d8f77743

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20328/files
  - new: https://git.openjdk.org/jdk/pull/20328/files/515c1869..d8f77743

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=06
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=05-06

  Stats: 176277 lines in 1496 files changed: 144251 ins; 25413 del; 6613 mod
  Patch: https://git.openjdk.org/jdk/pull/20328.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20328/head:pull/20328

PR: https://git.openjdk.org/jdk/pull/20328

From shade at openjdk.org  Thu Oct 24 10:44:52 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 24 Oct 2024 10:44:52 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v6]
In-Reply-To: <i1tUBBIz9RyNNTRPbueZ8v63O3_kZYLOpzUazNdsSxQ=.aaa88873-a779-41af-b8a1-04ab606b877e@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <i1tUBBIz9RyNNTRPbueZ8v63O3_kZYLOpzUazNdsSxQ=.aaa88873-a779-41af-b8a1-04ab606b877e@github.com>
Message-ID: <aPRHkxjjJQRIaKHXurSCCq3IQVAhju7ZjmvCGriii28=.580f4b99-9a74-4787-ba6e-f47b2a8f095a@github.com>

On Tue, 1 Oct 2024 15:19:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [ ] `jdk_jfr` pass by default
>>  - [ ] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Just exclude the tests for Shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Tighten up comments prose
>  - Roman's review: more precise GC state check, more includes
>  - Referencing INCLUDE_SHENANDOAHGC nominally requires macro.hpp
>  - Try to make tests faster / more robust
>  - Fix
>    
>    Fix
>    
>    Make JFR tests more reliable with new behavior with Shenandoah
>    
>    More precisely, only skip when Shenandoah already has forwarded objects
>    
>    Revert "More precisely, only skip when Shenandoah already has forwarded objects"
>    
>    This reverts commit 403824d4b27e091807a838c2c86b3228f75fe056.

I redid the PR to summarily disable Leak Profiler with Shenandoah. This patch makes sure enabling JFR with Shenandoah does not break the VM. Moving forward, we would probably rewrite Leak Profiler to avoid dependency on mark words, but that would be a larger endeavor, and I would like to have a reliable VM sooner :)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2434920739

From shade at openjdk.org  Thu Oct 24 10:44:52 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 24 Oct 2024 10:44:52 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v4]
In-Reply-To: <A0I-NPSQf1oIRb9Km19Jbcs3cmcbfvrBvwQhptjRGis=.a218c37c-6e3f-49e8-8ba2-412d1ff25160@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <BS9u0insN4_spkY3CJFaogMCV0pwtyG4Tt8xmhx-DyI=.c99e029b-519f-4953-84ee-19f6dcb0799b@github.com>
 <kLeRuEq9-fa5xXviC-jGjysRHqhY4KADIAmdcDjNmbQ=.f4604455-3d99-4c58-8280-aac9318af6c5@github.com>
 <iWl8uzGD-AnuV9MrnQmFBvcxf1HbjMW734we_w6OkgA=.8e415657-81de-4788-8f59-48578133c831@github.com>
 <yZAdQ70Lmu9wj8Ae74XJBRvNbHFcibMaSeaPayotFPA=.e1ec6862-feb3-4ca4-bda0-d9e08413135e@github.com>
 <G31KUF4vHDGFz6Daz3y4hXZRu6Cikmlzo0O82h6lXSU=.d65c903b-79f4-4271-a194-470e62879634@github.com>
 <A0I-NPSQf1oIRb9Km19Jbcs3cmcbfvrBvwQhptjRGis=.a218c37c-6e3f-49e8-8ba2-412d1ff25160@github.com>
Message-ID: <vntS_-uoTf6xtHJcA7dBWOlD2ml9nlrTATA_jBeDyLI=.b6576b1c-36f8-412e-8652-28c26b0a4253@github.com>

On Tue, 30 Jul 2024 11:13:22 GMT, Markus Gr?nlund <mgronlun at openjdk.org> wrote:

>> All right, I think I misunderstood how sampling works. I assumed it happens from the GC cycle, but now I see it happens from the allocation path. Indeed sampler holds the object in weak oop storage, and they are supposed to be accessible immediately after the allocation. 
>> 
>> I tightened up comments to reflect this understanding. Looks better, or?
>
> I don't know. The backoff mechanism now introduced is to handle the special case where the LeakProfiler dump operation, even though it is performed under an exclusive safepoint, happens to to coincide with some phase of Shenandoah GC - a lot of surface for handling this special case.
> 
> Also, need this not be categorical for all OldObject tests?

Yes. I think we go for a more comprehensive fix: https://github.com/openjdk/jdk/pull/20328#issuecomment-2434920739

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20328#discussion_r1814734821

From shade at openjdk.org  Thu Oct 24 14:05:30 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 24 Oct 2024 14:05:30 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v8]
In-Reply-To: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
Message-ID: <HGiAwQ-KL3iYJqL6Akmrkorn3TghWjm1Je_l5yOYoC0=.cb0f7258-08df-4c66-825f-799624ac4b21@github.com>

> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
> 
> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
> 
> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
> 
> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
> 
> Additional testing:
>  - [x] `jdk_jfr` pass by default
>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Disable Shenandoah-targeted test specially

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20328/files
  - new: https://git.openjdk.org/jdk/pull/20328/files/d8f77743..0876a529

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=06-07

  Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/20328.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20328/head:pull/20328

PR: https://git.openjdk.org/jdk/pull/20328

From rkennke at openjdk.org  Thu Oct 24 14:05:40 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 24 Oct 2024 14:05:40 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v51]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <V0bl0ewTz3kmPqZhynFqVFjlKjtpzxLCax-Yua5wCTQ=.326f5fee-58bd-4085-a4e8-3848f23f9cbe@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/1ef6394d..aadd7b8e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=50
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=49-50

  Stats: 16 lines in 1 file changed: 2 ins; 0 del; 14 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Thu Oct 24 14:19:11 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 24 Oct 2024 14:19:11 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v52]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <cjjVE9UklTmD7ofyudyMkhY9MWtD2MGmtPpZczGgm6w=.62f0f8eb-6c63-4ac2-8bcd-30d23240e989@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  s390 port

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/aadd7b8e..c2f6d202

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=51
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=50-51

  Stats: 151 lines in 9 files changed: 113 ins; 17 del; 21 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From kdnilsen at openjdk.org  Thu Oct 24 15:32:28 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 24 Oct 2024 15:32:28 GMT
Subject: RFR: 8342861: GenShen: Old generation in unexpected state when
 abandoning mixed gc candidates [v2]
In-Reply-To: <Vf-zaq-YReHezC2zAj_xNWJ7o7bhCks_RAscD5Ms4d4=.b7c99650-4ad7-43fd-aac8-93452d4b4dab@github.com>
References: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>
 <Vf-zaq-YReHezC2zAj_xNWJ7o7bhCks_RAscD5Ms4d4=.b7c99650-4ad7-43fd-aac8-93452d4b4dab@github.com>
Message-ID: <lnwZniFRpBK6BPhhb_76_LqWRUGIJx9uyCBvELzfozs=.6a8b76b2-fcb4-4335-b9a4-bd957c97de27@github.com>

On Wed, 23 Oct 2024 23:13:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> When the generational mode cannot add any old regions to a mixed collection because they are _all_ pinned, it gives up trying to evacuate these regions. The regions must then be made parsable. However, if this phase is interrupted by a young collection it should no longer try to evacuate any old regions. Doing so triggered an assert when testing with `-XX:ShenandoahCoalesceChance=80` and `XX:+ShenandoahAllocFailureALot`.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Problem list failing reference processing test

Marked as reviewed by kdnilsen (Committer).

Wondering if maybe we should introduce a new test as a regression check: Before, we triggered an assert when testing with -XX:ShenandoahCoalesceChance=80 and XX:+ShenandoahAllocFailureALot.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/522#pullrequestreview-2393048211
PR Comment: https://git.openjdk.org/shenandoah/pull/522#issuecomment-2435612663

From wkemper at openjdk.org  Thu Oct 24 17:17:05 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 17:17:05 GMT
Subject: RFR: 8342927: GenShen: Guarantee slices of time for coalesce and
 filling [v2]
In-Reply-To: <EFh4LKe-Qi5mvssTqby9oDzs6kBVH2-L6PJJRhk-et0=.387c94b9-2569-43f1-b2c2-6db309244aa3@github.com>
References: <EFh4LKe-Qi5mvssTqby9oDzs6kBVH2-L6PJJRhk-et0=.387c94b9-2569-43f1-b2c2-6db309244aa3@github.com>
Message-ID: <hr5K39oGLLld5zStOfNtv5gyc5zQGalwPzo3wISMZp4=.6072bf2f-1a97-4f9e-87c5-c35899339ac7@github.com>

> By design, young collections may preempt old collections. However, in some cases, this leads to starvation of old collection cycles. When old generation collections cannot make progress, the system continues to get "backed up" until an eventual degenerated or full GC cleans out the old generation. Genshen already has a mechanism to guarantee progress for old marking. It should be extended to also provide for coalescing and filling.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Update description of ShenandoahMinimumOldTimeMs

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/526/files
  - new: https://git.openjdk.org/shenandoah/pull/526/files/e3ee59ba..a0aac0b6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=526&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=526&range=00-01

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/526.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/526/head:pull/526

PR: https://git.openjdk.org/shenandoah/pull/526

From wkemper at openjdk.org  Thu Oct 24 18:24:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 18:24:31 GMT
Subject: Integrated: 8342927: GenShen: Guarantee slices of time for coalesce
 and filling
In-Reply-To: <EFh4LKe-Qi5mvssTqby9oDzs6kBVH2-L6PJJRhk-et0=.387c94b9-2569-43f1-b2c2-6db309244aa3@github.com>
References: <EFh4LKe-Qi5mvssTqby9oDzs6kBVH2-L6PJJRhk-et0=.387c94b9-2569-43f1-b2c2-6db309244aa3@github.com>
Message-ID: <zKKDjiebtmPK9D-GNJaxAFs4Ave7q40WdF9MfJd261k=.40e209fc-881b-467c-a718-7023208093ba@github.com>

On Thu, 24 Oct 2024 00:13:11 GMT, William Kemper <wkemper at openjdk.org> wrote:

> By design, young collections may preempt old collections. However, in some cases, this leads to starvation of old collection cycles. When old generation collections cannot make progress, the system continues to get "backed up" until an eventual degenerated or full GC cleans out the old generation. Genshen already has a mechanism to guarantee progress for old marking. It should be extended to also provide for coalescing and filling.

This pull request has now been integrated.

Changeset: 0eceb357
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/0eceb3574e17fd03fd739ce944eefb5c535d036c
Stats:     14 lines in 2 files changed: 3 ins; 0 del; 11 mod

8342927: GenShen: Guarantee slices of time for coalesce and filling

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/526

From wkemper at openjdk.org  Thu Oct 24 18:24:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 18:24:31 GMT
Subject: RFR: 8342927: GenShen: Guarantee slices of time for coalesce and
 filling [v2]
In-Reply-To: <AY74jn80CV44G7oOY6T4Hv7ij6fjc83J_2cLFKNy0gM=.921fc434-1ff2-4f45-b23c-d54649b764d0@github.com>
References: <EFh4LKe-Qi5mvssTqby9oDzs6kBVH2-L6PJJRhk-et0=.387c94b9-2569-43f1-b2c2-6db309244aa3@github.com>
 <AY74jn80CV44G7oOY6T4Hv7ij6fjc83J_2cLFKNy0gM=.921fc434-1ff2-4f45-b23c-d54649b764d0@github.com>
Message-ID: <lrNbvwODT1rvcbCXYBzCl5HUhGLJVVJECMpQW-jRUoA=.ebdead75-2cec-41b4-871e-e7a9f62ab510@github.com>

On Thu, 24 Oct 2024 00:23:02 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Update description of ShenandoahMinimumOldTimeMs
>
> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 561:
> 
>> 559:                                                                             \
>> 560:   product(uintx, ShenandoahMinimumOldTimeMs, 100, EXPERIMENTAL,         \
>> 561:          "Minimum amount of time in milliseconds to run old marking "       \
> 
> May want to adjust the "description": Minimum time to run old marking or old cleanup (C&F could be considered cleanup - or just say C&F)...

Yep, good catch.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/526#discussion_r1815524355

From wkemper at openjdk.org  Thu Oct 24 18:38:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 18:38:42 GMT
Subject: RFR: 8342927: GenShen: Guarantee slices of time for coalesce and
 filling
Message-ID: <KrbQkCzCQPd-YSKXyz7t6x4BgS4bUCoI1ee2QyoAP28=.af55be60-e704-458d-8a73-9c9cbe705334@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 0eceb3574e17fd03fd739ce944eefb5c535d036c

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/132/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=132&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342927
  Stats: 14 lines in 2 files changed: 3 ins; 0 del; 11 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/132.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/132/head:pull/132

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/132

From mli at openjdk.org  Thu Oct 24 19:01:36 2024
From: mli at openjdk.org (Hamlin Li)
Date: Thu, 24 Oct 2024 19:01:36 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v51]
In-Reply-To: <V0bl0ewTz3kmPqZhynFqVFjlKjtpzxLCax-Yua5wCTQ=.326f5fee-58bd-4085-a4e8-3848f23f9cbe@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <V0bl0ewTz3kmPqZhynFqVFjlKjtpzxLCax-Yua5wCTQ=.326f5fee-58bd-4085-a4e8-3848f23f9cbe@github.com>
Message-ID: <6cP6tvH2d8TU7TEuAxZoAtXFHg2jhtLEpOogKSCIeDE=.d2c3cce9-bb23-48c8-8829-8edd14249842@github.com>

On Thu, 24 Oct 2024 14:05:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test

test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java line 107:

> 105:         // the encoding range. We expect the encoding Base to start at the class space start - but to enforce that,
> 106:         // we choose a high address.
> 107:         if (Platform.isAArch64() || Platform.isX64()) {

@rkennke please also enable riscv for this test `CompressedClassPointersEncodingScheme.java`, it passed in my environment. Thanks!

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1815565554

From xpeng at openjdk.org  Thu Oct 24 20:06:58 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 24 Oct 2024 20:06:58 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v6]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <vYqcuDkiiWTZu34mz3KGJODuiM7Caw0PpLYeE4CqR3k=.7d20f487-e08c-4c88-a27c-7f7828d0d811@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 29 commits:

 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Merge branch 'shenandoah' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION
 - Address review comments
 - Reset bitmap after ShenandoahOldGC
 - Not rest bitmap after cycle if it needs to bootstrap old gc
 - typo in comments
 - Minimize the regions to walk when reset bitmap
 - comments
 - ... and 19 more: https://git.openjdk.org/shenandoah/compare/0eceb357...cf22fbc5

-------------

Changes: https://git.openjdk.org/shenandoah/pull/516/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=05
  Stats: 106 lines in 8 files changed: 96 ins; 2 del; 8 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From matsaave at openjdk.org  Thu Oct 24 20:15:39 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Thu, 24 Oct 2024 20:15:39 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v5]
In-Reply-To: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
Message-ID: <5fbkBeoXHfCfKGVVIPG0zexDYgEbuia9fEXtfBhqEFE=.d01ef3dc-992b-4109-8aba-0f3be19adc8c@github.com>

> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.

Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision:

  Removed can_use from whitebox

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21647/files
  - new: https://git.openjdk.org/jdk/pull/21647/files/8395f445..5400c057

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=03-04

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21647.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21647/head:pull/21647

PR: https://git.openjdk.org/jdk/pull/21647

From rkennke at openjdk.org  Thu Oct 24 21:04:51 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 24 Oct 2024 21:04:51 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v53]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <LDg0GvC9lXkYPS90d9w8QfkTWlztUa5tlsgblGywCc4=.78ab2a17-2e28-4c27-b026-cd694ec02e21@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Enable riscv in CompressedClassPointersEncodingScheme test

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/c2f6d202..434c6817

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=52
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=51-52

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Thu Oct 24 21:04:51 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 24 Oct 2024 21:04:51 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v51]
In-Reply-To: <6cP6tvH2d8TU7TEuAxZoAtXFHg2jhtLEpOogKSCIeDE=.d2c3cce9-bb23-48c8-8829-8edd14249842@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <V0bl0ewTz3kmPqZhynFqVFjlKjtpzxLCax-Yua5wCTQ=.326f5fee-58bd-4085-a4e8-3848f23f9cbe@github.com>
 <6cP6tvH2d8TU7TEuAxZoAtXFHg2jhtLEpOogKSCIeDE=.d2c3cce9-bb23-48c8-8829-8edd14249842@github.com>
Message-ID: <YfqWH6yBeIGE9yXnmP1j70P48QMaVrUa-RJGwVTw1yU=.70c9bb75-d223-4795-9438-1e76c315ffe3@github.com>

On Thu, 24 Oct 2024 18:58:03 GMT, Hamlin Li <mli at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>
> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointersEncodingScheme.java line 107:
> 
>> 105:         // the encoding range. We expect the encoding Base to start at the class space start - but to enforce that,
>> 106:         // we choose a high address.
>> 107:         if (Platform.isAArch64() || Platform.isX64()) {
> 
> @rkennke please also enable riscv for this test `CompressedClassPointersEncodingScheme.java`, it passed in my environment. Thanks!

Done.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1815690759

From xpeng at openjdk.org  Thu Oct 24 21:12:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 24 Oct 2024 21:12:20 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v7]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <qWN7i5cTvlQMdiJd-1IX_ZdP7247fArau_AmSin3Fx0=.49ffbbf7-5a18-443c-b811-c999cb302c0f@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:

 - Capture Top At Mark Start for *this* generation
 - Revert "set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION"
   
   This reverts commit d4fddac5a1e7321ff3b4441bd5cd096e949cd56a.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/516/files
  - new: https://git.openjdk.org/shenandoah/pull/516/files/cf22fbc5..8d068b33

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=06
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=05-06

  Stats: 12 lines in 2 files changed: 2 ins; 5 del; 5 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From xpeng at openjdk.org  Thu Oct 24 21:45:40 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 24 Oct 2024 21:45:40 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v8]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <wHR1sTeJRta3Bgn3hHjmp1C_RRC8B3AXAU9wIyPsPi0=.cabd092b-36ed-44ba-ace4-6713e38f2a96@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Not reset bitmap after ShenandoahOldGC

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/516/files
  - new: https://git.openjdk.org/shenandoah/pull/516/files/8d068b33..e7328c0e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=07
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=06-07

  Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From wkemper at openjdk.org  Thu Oct 24 22:52:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 22:52:13 GMT
Subject: git: openjdk/shenandoah: master: 8342919: GenShen: Fix whitespace
Message-ID: <a16fcaec-cc59-485d-9378-16e2c058b9f9@openjdk.org>

Changeset: 3d485cec
Branch: master
Author:    William Kemper <wkemper at openjdk.org>
Date:      2024-10-24 22:49:18 +0000
URL:       https://git.openjdk.org/shenandoah/commit/3d485ceca843ca17dd70cacb35cda98bbbdd58b5

8342919: GenShen: Fix whitespace

Reviewed-by: xpeng, kdnilsen

! src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp


From wkemper at openjdk.org  Thu Oct 24 22:54:51 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 22:54:51 GMT
Subject: RFR: 8342734: GenShen: Test failure
 gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
Message-ID: <FUAdC4dunV4vwvZm91RKAwe8DPeR5AGJnclZZtWznG8=.ad1d3bfd-6b03-4813-8887-b9a107334a1c@github.com>

Recent change introduced a regression: https://bugs.openjdk.org/browse/JDK-8342560 that caused old regions to not belong to global collections.

-------------

Commit messages:
 - Allow active global collections to contain old regions

Changes: https://git.openjdk.org/shenandoah/pull/527/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=527&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342734
  Stats: 4 lines in 2 files changed: 0 ins; 2 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/527.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/527/head:pull/527

PR: https://git.openjdk.org/shenandoah/pull/527

From ysr at openjdk.org  Thu Oct 24 22:54:51 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 24 Oct 2024 22:54:51 GMT
Subject: RFR: 8342734: GenShen: Test failure
 gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
In-Reply-To: <FUAdC4dunV4vwvZm91RKAwe8DPeR5AGJnclZZtWznG8=.ad1d3bfd-6b03-4813-8887-b9a107334a1c@github.com>
References: <FUAdC4dunV4vwvZm91RKAwe8DPeR5AGJnclZZtWznG8=.ad1d3bfd-6b03-4813-8887-b9a107334a1c@github.com>
Message-ID: <ynw3EDMMe5hgk7sQQl4bdlG1AMpJ056YqG8IH3WQD-0=.90dcbae2-7c0d-4b4a-9ddd-6153b30f0c1c@github.com>

On Thu, 24 Oct 2024 21:56:02 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Recent change introduced a regression: https://bugs.openjdk.org/browse/JDK-8342560 that caused old regions to not belong to global collections.

The change looks good; sorry for missing the change (or at least asking about it) in the previous PR.

I do have a couple of documentation requests just to make this method more easy to understand and to prevent future bugs from changes here.
1. Provide a 1-line documentation comment for the `is_in_active_generation(oop)` method in the `.hpp` file.
2. Change the comments at lines 372, 375, 378, instead of "Foo regions are in bar and baz generations, not in foobar generation", say something like "Foo regions are included in bar and baz _GCs_, not in foobar _GCs_"

I think that will make the comments easier to read and the method easier to understand (and maintain).

Since the suggestion is how to improve documentation to improve readability/maintainability, any changes to documentation comments that you make do not require my re-review/approval.

Reviewed.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/527#pullrequestreview-2393837830

From wkemper at openjdk.org  Thu Oct 24 22:55:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 22:55:21 GMT
Subject: Integrated: 8342919: GenShen: Fix whitespace
In-Reply-To: <fL7MHEEmENuLFRqqnn3-w1uKw7ONRQU_3kj8SQC1IS4=.1ce1a32a-52bb-46e7-9ce0-5537078c31f3@github.com>
References: <fL7MHEEmENuLFRqqnn3-w1uKw7ONRQU_3kj8SQC1IS4=.1ce1a32a-52bb-46e7-9ce0-5537078c31f3@github.com>
Message-ID: <RnCr-ijhy3an-jVLWfKMPv3PZwUJGcAMNF4bSE6EIaE=.114a5aef-fdc1-409b-b0a2-ac3037d0331d@github.com>

On Wed, 23 Oct 2024 21:23:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> There is an errant new line at the start of shFreeset.hpp.

This pull request has now been integrated.

Changeset: 3d485cec
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/3d485ceca843ca17dd70cacb35cda98bbbdd58b5
Stats:     1 line in 1 file changed: 0 ins; 1 del; 0 mod

8342919: GenShen: Fix whitespace

Reviewed-by: xpeng, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/524

From wkemper at openjdk.org  Thu Oct 24 22:57:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 24 Oct 2024 22:57:41 GMT
Subject: Integrated: 8342927: GenShen: Guarantee slices of time for coalesce
 and filling
In-Reply-To: <KrbQkCzCQPd-YSKXyz7t6x4BgS4bUCoI1ee2QyoAP28=.af55be60-e704-458d-8a73-9c9cbe705334@github.com>
References: <KrbQkCzCQPd-YSKXyz7t6x4BgS4bUCoI1ee2QyoAP28=.af55be60-e704-458d-8a73-9c9cbe705334@github.com>
Message-ID: <2KGpHpuwBTCDqINra-YMS9pfgEwTyUqBhdPHz-S3CMM=.eec0b7c9-7cc4-4527-b919-91e1cd959aa2@github.com>

On Thu, 24 Oct 2024 18:33:04 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: 9dcd1f23
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/9dcd1f23c44ac110d8c94f121166157696da2d5d
Stats:     14 lines in 2 files changed: 3 ins; 0 del; 11 mod

8342927: GenShen: Guarantee slices of time for coalesce and filling

Backport-of: 0eceb3574e17fd03fd739ce944eefb5c535d036c

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/132

From iklam at openjdk.org  Thu Oct 24 23:27:06 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Thu, 24 Oct 2024 23:27:06 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v5]
In-Reply-To: <5fbkBeoXHfCfKGVVIPG0zexDYgEbuia9fEXtfBhqEFE=.d01ef3dc-992b-4109-8aba-0f3be19adc8c@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
 <5fbkBeoXHfCfKGVVIPG0zexDYgEbuia9fEXtfBhqEFE=.d01ef3dc-992b-4109-8aba-0f3be19adc8c@github.com>
Message-ID: <rc_bjF4F8qmA7fKlcJ6OtFwIx2i1yPGayged8l493wM=.03c0cfd4-9c4f-4159-9b24-3e16d6e2d287@github.com>

On Thu, 24 Oct 2024 20:15:39 GMT, Matias Saavedra Silva <matsaave at openjdk.org> wrote:

>> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.
>
> Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removed can_use from whitebox

Marked as reviewed by iklam (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21647#pullrequestreview-2393868829

From wkemper at openjdk.org  Fri Oct 25 14:17:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Oct 2024 14:17:47 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <RR3YHLUnETwIF7iOChyhiE5YPTKAo9SmwtJlO7qdbPg=.7c87fbab-380f-4681-88cf-2abea3017242@github.com>

Merges tag jdk-24+21

-------------

Commit messages:
 - 8342681: TestLoadBypassesNullCheck.java fails improperly specified VM option
 - 8342609: jpackage test helper function incorrectly removes a directory instead of its contents only
 - 8342715: x86 unused orw instruction encoding could be removed
 - 8342011: Conditionally compile ReservedHeapSpace compressed heap support
 - 8338570: sun/font/HBShaper - mismatch in return type of FFM upcall function description and native invocation
 - 8340477: Remove JDK1.1 compatible behavior for "EST", "MST", and "HST" time zones
 - 8342698: Fix order of @param tags in module java.base
 - 8335271: Specify the MessageFormat ArgumentIndex Implementation Limit
 - 8339507: Test generation tool and gtest for testing APX encoding of extended gpr instructions
 - 8342376: More reliable OOM handling in ExceptionDuringDumpAtObjectsInitPhase test
 - ... and 52 more: https://git.openjdk.org/shenandoah/compare/7a64fbbb...8bcd4920

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=528&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/528/files
  Stats: 94693 lines in 422 files changed: 92101 ins; 1977 del; 615 mod
  Patch: https://git.openjdk.org/shenandoah/pull/528.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/528/head:pull/528

PR: https://git.openjdk.org/shenandoah/pull/528

From wkemper at openjdk.org  Fri Oct 25 17:17:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Oct 2024 17:17:08 GMT
Subject: RFR: 8342734: GenShen: Test failure
 gc/shenandoah/TestReferenceRefersToShenandoah.java#generational [v2]
In-Reply-To: <FUAdC4dunV4vwvZm91RKAwe8DPeR5AGJnclZZtWznG8=.ad1d3bfd-6b03-4813-8887-b9a107334a1c@github.com>
References: <FUAdC4dunV4vwvZm91RKAwe8DPeR5AGJnclZZtWznG8=.ad1d3bfd-6b03-4813-8887-b9a107334a1c@github.com>
Message-ID: <K03fIUSGP6dUD4aT17jB0RJwfp5u0MZSBboD6m1hPww=.7f3ca895-3829-402b-9a8d-31b1342547fd@github.com>

> Recent change introduced a regression: https://bugs.openjdk.org/browse/JDK-8342560 that caused old regions to not belong to global collections.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Comment improvements

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/527/files
  - new: https://git.openjdk.org/shenandoah/pull/527/files/238be83d..65af911f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=527&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=527&range=00-01

  Stats: 4 lines in 2 files changed: 1 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/527.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/527/head:pull/527

PR: https://git.openjdk.org/shenandoah/pull/527

From wkemper at openjdk.org  Fri Oct 25 17:26:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Oct 2024 17:26:25 GMT
Subject: Integrated: 8342734: GenShen: Test failure
 gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
In-Reply-To: <FUAdC4dunV4vwvZm91RKAwe8DPeR5AGJnclZZtWznG8=.ad1d3bfd-6b03-4813-8887-b9a107334a1c@github.com>
References: <FUAdC4dunV4vwvZm91RKAwe8DPeR5AGJnclZZtWznG8=.ad1d3bfd-6b03-4813-8887-b9a107334a1c@github.com>
Message-ID: <JXiZ706s5x902QDU1B9OiE0yrITcuN6-Tg3oe9mcv_M=.10698fda-a7aa-4757-bd8c-5c6c26ebddd9@github.com>

On Thu, 24 Oct 2024 21:56:02 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Recent change introduced a regression: https://bugs.openjdk.org/browse/JDK-8342560 that caused old regions to not belong to global collections.

This pull request has now been integrated.

Changeset: 1b751ad1
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/1b751ad1daf507624fd56a57c4b72ba79ef4f9cb
Stats:     7 lines in 3 files changed: 1 ins; 2 del; 4 mod

8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/527

From wkemper at openjdk.org  Fri Oct 25 17:28:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Oct 2024 17:28:46 GMT
Subject: RFR: 8342861: GenShen: Old generation in unexpected state when
 abandoning mixed gc candidates [v3]
In-Reply-To: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>
References: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>
Message-ID: <1BbZjHu71-smEmeE1Vqf-4zFLlK0T3gFkyrRYJX9ZH4=.560e57b0-88ad-4bdb-9ea0-60c8d3b8eece@github.com>

> When the generational mode cannot add any old regions to a mixed collection because they are _all_ pinned, it gives up trying to evacuate these regions. The regions must then be made parsable. However, if this phase is interrupted by a young collection it should no longer try to evacuate any old regions. Doing so triggered an assert when testing with `-XX:ShenandoahCoalesceChance=80` and `XX:+ShenandoahAllocFailureALot`.

William Kemper has updated the pull request incrementally with five additional commits since the last revision:

 - 8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
   
   Reviewed-by: ysr
 - 8342919: GenShen: Fix whitespace
   
   Reviewed-by: xpeng, kdnilsen
 - 8342927: GenShen: Guarantee slices of time for coalesce and filling
   
   Reviewed-by: kdnilsen
 - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode
   
   Reviewed-by: wkemper
 - 8342580: GenShen: TestChurnNotifications fails executing in unintended test-id modes with ShenandoahGCMode=generational
   
   Reviewed-by: wkemper, kdnilsen

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/522/files
  - new: https://git.openjdk.org/shenandoah/pull/522/files/8532d08f..8f486949

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=522&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=522&range=01-02

  Stats: 46 lines in 15 files changed: 22 ins; 3 del; 21 mod
  Patch: https://git.openjdk.org/shenandoah/pull/522.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/522/head:pull/522

PR: https://git.openjdk.org/shenandoah/pull/522

From wkemper at openjdk.org  Fri Oct 25 18:44:53 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Oct 2024 18:44:53 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <RR3YHLUnETwIF7iOChyhiE5YPTKAo9SmwtJlO7qdbPg=.7c87fbab-380f-4681-88cf-2abea3017242@github.com>
References: <RR3YHLUnETwIF7iOChyhiE5YPTKAo9SmwtJlO7qdbPg=.7c87fbab-380f-4681-88cf-2abea3017242@github.com>
Message-ID: <6xLW-0H8XkTTJTgLIZdd5Uon9y48Zd00LxkiAsC5DxM=.beda393f-c2ee-40d9-80cd-63c12737129b@github.com>

> Merges tag jdk-24+21

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 63 commits:

 - Merge tag jdk-24+21
 - 8342681: TestLoadBypassesNullCheck.java fails improperly specified VM option
   
   Reviewed-by: shade
 - 8342609: jpackage test helper function incorrectly removes a directory instead of its contents only
   
   Reviewed-by: almatvee
 - 8342715: x86 unused orw instruction encoding could be removed
   
   Reviewed-by: sviswanathan, jbhateja, kvn
 - 8342011: Conditionally compile ReservedHeapSpace compressed heap support
   
   Reviewed-by: stefank, dholmes
 - 8338570: sun/font/HBShaper - mismatch in return type of FFM upcall function description and native invocation
   
   Reviewed-by: serb, aivanov
 - 8340477: Remove JDK1.1 compatible behavior for "EST", "MST", and "HST" time zones
   
   Reviewed-by: iris, jlu, joehw
 - 8342698: Fix order of @param tags in module java.base
   
   Reviewed-by: iris
 - 8335271: Specify the MessageFormat ArgumentIndex Implementation Limit
   
   Reviewed-by: darcy, iris, naoto, rriggs
 - 8339507: Test generation tool and gtest for testing APX encoding of extended gpr instructions
   
   Reviewed-by: jbhateja, sviswanathan, kvn
 - ... and 53 more: https://git.openjdk.org/shenandoah/compare/1b751ad1...707d0c97

-------------

Changes: https://git.openjdk.org/shenandoah/pull/528/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=528&range=01
  Stats: 94702 lines in 423 files changed: 92105 ins; 1978 del; 619 mod
  Patch: https://git.openjdk.org/shenandoah/pull/528.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/528/head:pull/528

PR: https://git.openjdk.org/shenandoah/pull/528

From matsaave at openjdk.org  Fri Oct 25 20:43:43 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Fri, 25 Oct 2024 20:43:43 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v6]
In-Reply-To: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
Message-ID: <MFflLf0HgZIq3DI7mUwW9vZDhhdTuCYK857CMRRMm-I=.a7dbc68c-c6d2-4be3-95df-12924cea4f86@github.com>

> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.

Matias Saavedra Silva has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:

 - Added @requires to ParallelGC test
 - Merge branch 'master' into archive_uncompressed_oops_8341371
 - Removed can_use from whitebox
 - Added @requires
 - Calvin comments and copyright
 - Ioi comments
 - Enabled uncompressed oops on Epsilon and Shenandoah
 - Merge branch 'master' into archive_uncompressed_oops_8341371
 - Added uncompressed oops to tests
 - 8341371: CDS cannot load archived heap objects with -XX:+UseSerialGC -XX:-UseCompressedOops

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21647/files
  - new: https://git.openjdk.org/jdk/pull/21647/files/5400c057..6d82f7db

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21647&range=04-05

  Stats: 108047 lines in 174 files changed: 4473 ins; 102169 del; 1405 mod
  Patch: https://git.openjdk.org/jdk/pull/21647.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21647/head:pull/21647

PR: https://git.openjdk.org/jdk/pull/21647

From ccheung at openjdk.org  Fri Oct 25 21:10:09 2024
From: ccheung at openjdk.org (Calvin Cheung)
Date: Fri, 25 Oct 2024 21:10:09 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v6]
In-Reply-To: <MFflLf0HgZIq3DI7mUwW9vZDhhdTuCYK857CMRRMm-I=.a7dbc68c-c6d2-4be3-95df-12924cea4f86@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
 <MFflLf0HgZIq3DI7mUwW9vZDhhdTuCYK857CMRRMm-I=.a7dbc68c-c6d2-4be3-95df-12924cea4f86@github.com>
Message-ID: <cPiN2iZsy1U04G-aR8JYVoaGomMwdSdPKkrBoiUNXVM=.7238511d-ae9a-4b3d-b985-8200799a0335@github.com>

On Fri, 25 Oct 2024 20:43:43 GMT, Matias Saavedra Silva <matsaave at openjdk.org> wrote:

>> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.
>
> Matias Saavedra Silva has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:
> 
>  - Added @requires to ParallelGC test
>  - Merge branch 'master' into archive_uncompressed_oops_8341371
>  - Removed can_use from whitebox
>  - Added @requires
>  - Calvin comments and copyright
>  - Ioi comments
>  - Enabled uncompressed oops on Epsilon and Shenandoah
>  - Merge branch 'master' into archive_uncompressed_oops_8341371
>  - Added uncompressed oops to tests
>  - 8341371: CDS cannot load archived heap objects with -XX:+UseSerialGC -XX:-UseCompressedOops

Marked as reviewed by ccheung (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21647#pullrequestreview-2396503546

From zgu at openjdk.org  Sat Oct 26 01:39:15 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Sat, 26 Oct 2024 01:39:15 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
Message-ID: <p7YVH0moyGyDlINK9wVZK4n-7Z5KCyI6kvmX8wkcgWs=.35c31ce4-595f-48a0-9902-6db8a3a70738@github.com>

On Mon, 21 Oct 2024 21:16:50 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8342560: GenShen: Fix confusing method name
>    
>    Reviewed-by: ysr
>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>    
>    Reviewed-by: xpeng, ysr
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>    
>    Reviewed-by: ysr
>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>    
>    Reviewed-by: ysr
>  - 8342255: GenShen: Remove unnecessary enum initial values
>    
>    Reviewed-by: ysr
>  - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408

src/hotspot/share/gc/shenandoah/shenandoahAgeCensus.cpp line 243:

> 241: void ShenandoahAgeCensus::update_tenuring_threshold() {
> 242:   if (!ShenandoahGenerationalAdaptiveTenuring) {
> 243:     _tenuring_threshold[_epoch] = InitialTenuringThreshold;

`InitialTenuringThreshold` is currently only used by parallel GC, so that the flag's constraint is only implemented in parallel GC [here](https://github.com/openjdk/jdk/blob/master/src/hotspot/share/gc/shared/jvmFlagConstraintsGC.cpp#L177) . Now, Shenandoah reuses the flag, you may want to consider to parallel GC's implementation into `shared`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1817529349

From zgu at openjdk.org  Sat Oct 26 02:08:16 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Sat, 26 Oct 2024 02:08:16 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
Message-ID: <Rku187KbA-0dpP0fxJhyUnvODEyUPjHLWlEgkS-KGEU=.36fd6343-7cdd-4168-9944-fa52552139d2@github.com>

On Mon, 21 Oct 2024 21:16:50 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8342560: GenShen: Fix confusing method name
>    
>    Reviewed-by: ysr
>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>    
>    Reviewed-by: xpeng, ysr
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>    
>    Reviewed-by: ysr
>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>    
>    Reviewed-by: ysr
>  - 8342255: GenShen: Remove unnecessary enum initial values
>    
>    Reviewed-by: ysr
>  - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 701:

> 699:       // Seed the collection set with resource area-allocated
> 700:       // preselected regions, which are removed when we exit this scope.
> 701:       ResourceMark rm;

You can fold `ResourceMark` into `ShenandoahCollectionSetPreselector` class.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1817564685

From xpeng at openjdk.org  Sat Oct 26 02:21:09 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Sat, 26 Oct 2024 02:21:09 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v9]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <7Ncw9rE_Gf74ZbDebfDDpcmB0_fIM9t2QH2slmvh8nY=.69c4cb79-3cfd-4310-9cd2-482139c01a0e@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 33 commits:

 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Not reset bitmap after ShenandoahOldGC
 - Capture Top At Mark Start for *this* generation
 - Revert "set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION"
   
   This reverts commit d4fddac5a1e7321ff3b4441bd5cd096e949cd56a.
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Merge branch 'shenandoah' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION
 - Address review comments
 - Reset bitmap after ShenandoahOldGC
 - ... and 23 more: https://git.openjdk.org/shenandoah/compare/1b751ad1...79ecb90f

-------------

Changes: https://git.openjdk.org/shenandoah/pull/516/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=08
  Stats: 106 lines in 6 files changed: 91 ins; 2 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From ysr at openjdk.org  Sat Oct 26 08:31:54 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 26 Oct 2024 08:31:54 GMT
Subject: RFR: 8342570: gc.*.gcUntilOld() assumes that full gc always promotes
 objects to old generation in a generational heap
Message-ID: <yqzamEyfEumBhErqpIl68siks3PPQ2zC2j-uftrb42g=.74b1361b-f2e5-432e-9dee-17f693f54c99@github.com>

This is a draft while we go over the issues in the ticket.

-------------

Commit messages:
 - Remove debugging debris
 - Merge branch 'master' into test_cleanup
 - Changes identical to PR 527 for JDK-8342734.
 - Merge branch 'master' into test_cleanup
 - Merge branch 'master' into test_cleanup
 - Debugging cruft, not for checkin.
 - Tweak documentation comment, explicating that non-generational
 - Move the override of CollectedHeap::full_gc_promotes_to_old() from the
 - Merge branch 'master' into test_cleanup
 - JDK-8342570  gc.*.gcUntilOld() assumes that full gc always promotes objects to old generation in a generational heap

Changes: https://git.openjdk.org/shenandoah/pull/521/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=521&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342570
  Stats: 51 lines in 10 files changed: 39 ins; 8 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/521.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/521/head:pull/521

PR: https://git.openjdk.org/shenandoah/pull/521

From xpeng at openjdk.org  Mon Oct 28 06:07:15 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Mon, 28 Oct 2024 06:07:15 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v10]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <B2HfKZWAi-PYo2e3Zcf_67OS8dkO6J-7KzZyk7vFD8U=.3591544b-1ed4-4875-b723-d78552a9fb10@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  set_mark_incomplete should be always set after resetting bitmap

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/516/files
  - new: https://git.openjdk.org/shenandoah/pull/516/files/79ecb90f..0c334b65

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=09
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=08-09

  Stats: 6 lines in 2 files changed: 4 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From xpeng at openjdk.org  Mon Oct 28 07:13:27 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Mon, 28 Oct 2024 07:13:27 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v11]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <S9vWKOqFABrRyjRQH6yfPatbSzdj6SPTyr-lLTHg-GA=.050af3f1-1e40-44ac-8d7f-d79d11047e24@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  fix test failures

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/516/files
  - new: https://git.openjdk.org/shenandoah/pull/516/files/0c334b65..7c21b919

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=10
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=09-10

  Stats: 3 lines in 1 file changed: 1 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From matsaave at openjdk.org  Mon Oct 28 14:35:36 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Mon, 28 Oct 2024 14:35:36 GMT
Subject: RFR: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops [v5]
In-Reply-To: <rc_bjF4F8qmA7fKlcJ6OtFwIx2i1yPGayged8l493wM=.03c0cfd4-9c4f-4159-9b24-3e16d6e2d287@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
 <5fbkBeoXHfCfKGVVIPG0zexDYgEbuia9fEXtfBhqEFE=.d01ef3dc-992b-4109-8aba-0f3be19adc8c@github.com>
 <rc_bjF4F8qmA7fKlcJ6OtFwIx2i1yPGayged8l493wM=.03c0cfd4-9c4f-4159-9b24-3e16d6e2d287@github.com>
Message-ID: <3pBDeuislEBY3UadwJ3_8XvwzFG5izewA7P3WiWBuXc=.51a7bc87-88e8-46e1-821a-0b5f146ef842@github.com>

On Thu, 24 Oct 2024 23:24:37 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removed can_use from whitebox
>
> Marked as reviewed by iklam (Reviewer).

Thanks for the reviews @iklam and @calvinccheung!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21647#issuecomment-2441756486

From matsaave at openjdk.org  Mon Oct 28 14:35:36 2024
From: matsaave at openjdk.org (Matias Saavedra Silva)
Date: Mon, 28 Oct 2024 14:35:36 GMT
Subject: Integrated: 8341371: CDS cannot load archived heap objects with
 -XX:+UseSerialGC -XX:-UseCompressedOops
In-Reply-To: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
References: <zQspbeQP93SOHiEU17eTA0gX6yup52TJZQuH-7QCToA=.aedf38ff-7049-413a-b629-8bdd64e59c1f@github.com>
Message-ID: <Di8whYa2f92v2HbvwEdc2083gSx3kCt3h9Si96wCx-U=.d93fd9c2-f3e2-45c1-96e2-29303670b017@github.com>

On Tue, 22 Oct 2024 19:32:03 GMT, Matias Saavedra Silva <matsaave at openjdk.org> wrote:

> [JDK-8298614](https://bugs.openjdk.org/browse/JDK-8298614) enabled writing the archived heap region with SerialGC and ParallelGC, but archives written with XX:-UseCompressedOops can't be loaded at runtime. This patch enables heap loading for SerialGC and ParallelGC as well as EpsilonGC and Shenandoah since they were enabled in [JDK-8341913](https://bugs.openjdk.org/browse/JDK-8341913). Related tests were updated to check for the new conditions. Verified with tier 1-5 tests.

This pull request has now been integrated.

Changeset: 9f6211bc
Author:    Matias Saavedra Silva <matsaave at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/9f6211bcf1b46e4bfba2d128d9eb8457bc0cde51
Stats:     149 lines in 12 files changed: 107 ins; 21 del; 21 mod

8341371: CDS cannot load archived heap objects with -XX:+UseSerialGC -XX:-UseCompressedOops

Reviewed-by: ccheung, iklam

-------------

PR: https://git.openjdk.org/jdk/pull/21647

From kdnilsen at openjdk.org  Mon Oct 28 16:04:13 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 28 Oct 2024 16:04:13 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one
 header [v2]
In-Reply-To: <14GqBlYL2XDV0k6WC1La5SmYLSAyYcf2FlVE4V75T2s=.bd5ddb3f-e520-44ef-85f9-22d498d5151e@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
 <14GqBlYL2XDV0k6WC1La5SmYLSAyYcf2FlVE4V75T2s=.bd5ddb3f-e520-44ef-85f9-22d498d5151e@github.com>
Message-ID: <qnctyF5Uhrz0R2CBL7sYR2vIz96WuXtrgpjlnjQ7n8Y=.1ce51750-64eb-4bf2-9ba2-deda532335d0@github.com>

On Fri, 18 Oct 2024 10:31:23 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix Zero builds

Thanks.

-------------

Marked as reviewed by kdnilsen (Author).

PR Review: https://git.openjdk.org/jdk/pull/21579#pullrequestreview-2399529979

From xpeng at openjdk.org  Mon Oct 28 18:24:40 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Mon, 28 Oct 2024 18:24:40 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v12]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <QPQrEXhVVY9tzAWsVOmX57EPuKURppryBIfSSWg2f_Q=.2cd00175-ce20-4475-a9a3-48918a57ecd8@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Move set_mark_incomplete into reset_mark_bitmap

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/516/files
  - new: https://git.openjdk.org/shenandoah/pull/516/files/7c21b919..df91d467

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=11
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=10-11

  Stats: 9 lines in 2 files changed: 5 ins; 3 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From stefank at openjdk.org  Tue Oct 29 12:16:33 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 29 Oct 2024 12:16:33 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v53]
In-Reply-To: <LDg0GvC9lXkYPS90d9w8QfkTWlztUa5tlsgblGywCc4=.78ab2a17-2e28-4c27-b026-cd694ec02e21@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <LDg0GvC9lXkYPS90d9w8QfkTWlztUa5tlsgblGywCc4=.78ab2a17-2e28-4c27-b026-cd694ec02e21@github.com>
Message-ID: <IoELh3tHR8eEK8ZJfXgd1VBulpypTeLXAS7Nqkw_uJ4=.c2e38997-4be2-412d-a01d-5017d970a49f@github.com>

On Thu, 24 Oct 2024 21:04:51 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Enable riscv in CompressedClassPointersEncodingScheme test

src/hotspot/share/oops/markWord.inline.hpp line 29:

> 27: 
> 28: #include "oops/compressedOops.inline.hpp"
> 29: #include "oops/markWord.hpp"

I found this nit while looking around the code.
Suggestion:

#include "oops/markWord.hpp"

#include "oops/compressedOops.inline.hpp"

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1820682388

From amitkumar at openjdk.org  Tue Oct 29 13:12:34 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Tue, 29 Oct 2024 13:12:34 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
 <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
Message-ID: <BMnEQkyopmAU46K23jp9I-EuIjqrWgd3eNnQTgqTMVE=.055b1538-96bc-47e4-836b-f2f294b7686c@github.com>

On Tue, 22 Oct 2024 16:22:20 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Update copyright
>>  - Avoid assert/endless-loop in JFR code
>
> @egahlin / @mgronlun could you please review the JFR parts of this PR? One change is for getting the right prototype header, the other is for avoiding an endless loop/assert in a corner case.

@rkennke can you include this small update for s390x as well: 

```diff 
diff --git a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
index 0f7e5c9f457..476e3d5daa4 100644
--- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
+++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
@@ -174,8 +174,11 @@ void C1_MacroAssembler::try_allocate(
 void C1_MacroAssembler::initialize_header(Register obj, Register klass, Register len, Register Rzero, Register t1) {
   assert_different_registers(obj, klass, len, t1, Rzero);
   if (UseCompactObjectHeaders) {
-    z_lg(t1, Address(klass, in_bytes(Klass::prototype_header_offset())));
-    z_stg(t1, Address(obj, oopDesc::mark_offset_in_bytes()));
+    z_mvc(
+          Address(obj, oopDesc::mark_offset_in_bytes()),  /* move to */
+          Address(klass, in_bytes(Klass::prototype_header_offset())), /* move from */
+          sizeof(markWord) /* how much to move */
+        );
   } else {
     load_const_optimized(t1, (intx)markWord::prototype().value());
     z_stg(t1, Address(obj, oopDesc::mark_offset_in_bytes()));
diff --git a/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp b/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
index 378d5e4cfe1..c5713161bf9 100644
--- a/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
+++ b/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
@@ -46,7 +46,7 @@ void C2_MacroAssembler::load_narrow_klass_compact_c2(Register dst, Address src)
   // The incoming address is pointing into obj-start + klass_offset_in_bytes. We need to extract
   // obj-start, so that we can load from the object's mark-word instead.
   z_lg(dst, src.plus_disp(-oopDesc::klass_offset_in_bytes()));
-  z_srlg(dst, dst, markWord::klass_shift); // TODO: could be z_sra
+  z_srlg(dst, dst, markWord::klass_shift);
 }
 
 //------------------------------------------------------
diff --git a/src/hotspot/cpu/s390/templateTable_s390.cpp b/src/hotspot/cpu/s390/templateTable_s390.cpp
index 3cb1aba810d..5b8f7a20478 100644
--- a/src/hotspot/cpu/s390/templateTable_s390.cpp
+++ b/src/hotspot/cpu/s390/templateTable_s390.cpp
@@ -3980,8 +3980,11 @@ void TemplateTable::_new() {
     // Initialize object header only.
     __ bind(initialize_header);
     if (UseCompactObjectHeaders) {
-      __ z_lg(tmp, Address(iklass, in_bytes(Klass::prototype_header_offset())));
-      __ z_stg(tmp, Address(RallocatedObject, oopDesc::mark_offset_in_bytes()));
+      __ z_mvc(
+            Address(RallocatedObject, oopDesc::mark_offset_in_bytes()), // move to
+            Address(iklass, in_bytes(Klass::prototype_header_offset())), // move from
+            sizeof(markWord) // how much to move
+          );
     } else {
       __ store_const(Address(RallocatedObject, oopDesc::mark_offset_in_bytes()),
                      (long) markWord::prototype().value());

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2444180131

From wkemper at openjdk.org  Tue Oct 29 16:56:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 16:56:35 GMT
Subject: RFR: 8342861: GenShen: Old generation in unexpected state when
 abandoning mixed gc candidates [v2]
In-Reply-To: <lnwZniFRpBK6BPhhb_76_LqWRUGIJx9uyCBvELzfozs=.6a8b76b2-fcb4-4335-b9a4-bd957c97de27@github.com>
References: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>
 <Vf-zaq-YReHezC2zAj_xNWJ7o7bhCks_RAscD5Ms4d4=.b7c99650-4ad7-43fd-aac8-93452d4b4dab@github.com>
 <lnwZniFRpBK6BPhhb_76_LqWRUGIJx9uyCBvELzfozs=.6a8b76b2-fcb4-4335-b9a4-bd957c97de27@github.com>
Message-ID: <6_4WP5GLZc7vfByaCNEs0-xQ9k3uAh-nBn7sYIHlpmI=.73d00d74-9d25-4e88-8045-0aca03f6abca@github.com>

On Thu, 24 Oct 2024 15:29:32 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Problem list failing reference processing test
>
> Wondering if maybe we should introduce a new test as a regression check: Before, we triggered an assert when testing with -XX:ShenandoahCoalesceChance=80 and XX:+ShenandoahAllocFailureALot.

@kdnilsen - I will follow up in a different PR with a test that runs with `-XX:ShenandoahCoalesceChance=80` enabled. Filed https://bugs.openjdk.org/browse/JDK-8343217

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/522#issuecomment-2444835987

From wkemper at openjdk.org  Tue Oct 29 16:56:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 16:56:35 GMT
Subject: Integrated: 8342861: GenShen: Old generation in unexpected state when
 abandoning mixed gc candidates
In-Reply-To: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>
References: <JlR2F_PxNGqyIeI7TBBkhEFjQgV5qgfoIXdooxlPoFA=.c4da3c6e-6c8c-4b17-a296-20dc487e37a7@github.com>
Message-ID: <i1WxX01tUV0MJ8ulG3IzmKajHvunF_Ke33go3qCExtk=.734c77cd-8401-4410-854f-71c43a266123@github.com>

On Tue, 22 Oct 2024 23:29:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When the generational mode cannot add any old regions to a mixed collection because they are _all_ pinned, it gives up trying to evacuate these regions. The regions must then be made parsable. However, if this phase is interrupted by a young collection it should no longer try to evacuate any old regions. Doing so triggered an assert when testing with `-XX:ShenandoahCoalesceChance=80` and `XX:+ShenandoahAllocFailureALot`.

This pull request has now been integrated.

Changeset: 278d3c9a
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/278d3c9adc0882ec9d2f582a5e18263b6bf83b77
Stats:     10 lines in 2 files changed: 10 ins; 0 del; 0 mod

8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/522

From simonis at openjdk.org  Tue Oct 29 17:02:08 2024
From: simonis at openjdk.org (Volker Simonis)
Date: Tue, 29 Oct 2024 17:02:08 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one
 header [v2]
In-Reply-To: <14GqBlYL2XDV0k6WC1La5SmYLSAyYcf2FlVE4V75T2s=.bd5ddb3f-e520-44ef-85f9-22d498d5151e@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
 <14GqBlYL2XDV0k6WC1La5SmYLSAyYcf2FlVE4V75T2s=.bd5ddb3f-e520-44ef-85f9-22d498d5151e@github.com>
Message-ID: <EToLNCZ0bc7XaNioNtWktCprhkMV31ncylSbrzZO1cs=.93606986-b5bc-4692-91cf-0fed03632954@github.com>

On Fri, 18 Oct 2024 10:31:23 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix Zero builds

In general looks good. The only thing I'd recommend (if you haven't done that already anyway) would be to configure with `--disable-precompiled-headers` and see if the build still works (i.e. if all the inline functions are still all available in all the compilation units).

-------------

Marked as reviewed by simonis (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21579#pullrequestreview-2402576093

From wkemper at openjdk.org  Tue Oct 29 17:22:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 17:22:45 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <RR3YHLUnETwIF7iOChyhiE5YPTKAo9SmwtJlO7qdbPg=.7c87fbab-380f-4681-88cf-2abea3017242@github.com>
References: <RR3YHLUnETwIF7iOChyhiE5YPTKAo9SmwtJlO7qdbPg=.7c87fbab-380f-4681-88cf-2abea3017242@github.com>
Message-ID: <61Sdy5VqbU3AlSUK1XueihFDbsvKBFoqPPFR9xgyaJE=.0ea25035-eaf6-4c36-85bc-a2fbd0f00adb@github.com>

On Fri, 25 Oct 2024 14:12:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-24+21

This pull request has now been integrated.

Changeset: 99272621
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/9927262173288e7baa1c04c06f9be61c0ceac716
Stats:     94702 lines in 423 files changed: 92105 ins; 1978 del; 619 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/528

From xpeng at openjdk.org  Tue Oct 29 17:42:51 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 29 Oct 2024 17:42:51 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v13]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <aG-w3FbMWH3MEDMPcERd8G4-HfoNaEIOUeXBK9_JEeI=.8196571f-67c8-4a5c-ad78-d46478314535@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 37 commits:

 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Move set_mark_incomplete into reset_mark_bitmap
 - fix test failures
 - set_mark_incomplete should be always set after resetting bitmap
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Not reset bitmap after ShenandoahOldGC
 - Capture Top At Mark Start for *this* generation
 - Revert "set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION"
   
   This reverts commit d4fddac5a1e7321ff3b4441bd5cd096e949cd56a.
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Merge branch 'shenandoah' into genshen-JDK-8338737
 - ... and 27 more: https://git.openjdk.org/shenandoah/compare/99272621...36618788

-------------

Changes: https://git.openjdk.org/shenandoah/pull/516/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=12
  Stats: 109 lines in 6 files changed: 94 ins; 2 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From shade at openjdk.org  Tue Oct 29 17:59:04 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 29 Oct 2024 17:59:04 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one
 header [v2]
In-Reply-To: <EToLNCZ0bc7XaNioNtWktCprhkMV31ncylSbrzZO1cs=.93606986-b5bc-4692-91cf-0fed03632954@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
 <14GqBlYL2XDV0k6WC1La5SmYLSAyYcf2FlVE4V75T2s=.bd5ddb3f-e520-44ef-85f9-22d498d5151e@github.com>
 <EToLNCZ0bc7XaNioNtWktCprhkMV31ncylSbrzZO1cs=.93606986-b5bc-4692-91cf-0fed03632954@github.com>
Message-ID: <6v0izwPV9oylWKVifIesMz0EK9Qo7DHkLo3_ahumLow=.98d9b2f5-aecb-45bc-98bc-5faf0503a1ca@github.com>

On Tue, 29 Oct 2024 16:59:03 GMT, Volker Simonis <simonis at openjdk.org> wrote:

> In general looks good. The only thing I'd recommend (if you haven't done that already anyway) would be to configure with `--disable-precompiled-headers` and see if the build still works (i.e. if all the inline functions are still all available in all the compilation units).

Yup, thanks! GHA also does it for us.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21579#issuecomment-2444976068

From shade at openjdk.org  Tue Oct 29 18:18:01 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 29 Oct 2024 18:18:01 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one
 header [v3]
In-Reply-To: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
Message-ID: <vRij8RPQ9PbB2cQq5dIzlNNnmZdL-kH1ai_Ha7JjDuI=.85b9b9a1-eebf-41b1-b06d-de02e31bdb38@github.com>

> Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:

 - Merge branch 'master' into JDK-8342588-shenandoah-shared-closures
 - Fix Zero builds
 - Fix

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21579/files
  - new: https://git.openjdk.org/jdk/pull/21579/files/c9af382c..831372cf

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21579&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21579&range=01-02

  Stats: 44445 lines in 716 files changed: 21494 ins; 20581 del; 2370 mod
  Patch: https://git.openjdk.org/jdk/pull/21579.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21579/head:pull/21579

PR: https://git.openjdk.org/jdk/pull/21579

From wkemper at openjdk.org  Tue Oct 29 18:20:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 18:20:33 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <p7YVH0moyGyDlINK9wVZK4n-7Z5KCyI6kvmX8wkcgWs=.35c31ce4-595f-48a0-9902-6db8a3a70738@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
 <p7YVH0moyGyDlINK9wVZK4n-7Z5KCyI6kvmX8wkcgWs=.35c31ce4-595f-48a0-9902-6db8a3a70738@github.com>
Message-ID: <XvVvRtymTU72I7j6s2i-gJS6rWLAPGzuJbFAzW8JS_s=.0be3a91a-6bdf-4960-a8a6-b7d994d4c354@github.com>

On Sat, 26 Oct 2024 01:33:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8342560: GenShen: Fix confusing method name
>>    
>>    Reviewed-by: ysr
>>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>>    
>>    Reviewed-by: xpeng, ysr
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>>    
>>    Reviewed-by: ysr
>>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>>    
>>    Reviewed-by: ysr
>>  - 8342255: GenShen: Remove unnecessary enum initial values
>>    
>>    Reviewed-by: ysr
>>  - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408
>
> src/hotspot/share/gc/shenandoah/shenandoahAgeCensus.cpp line 243:
> 
>> 241: void ShenandoahAgeCensus::update_tenuring_threshold() {
>> 242:   if (!ShenandoahGenerationalAdaptiveTenuring) {
>> 243:     _tenuring_threshold[_epoch] = InitialTenuringThreshold;
> 
> `InitialTenuringThreshold` is currently only used by parallel GC, so that the flag's constraint is only implemented in parallel GC [here](https://github.com/openjdk/jdk/blob/master/src/hotspot/share/gc/shared/jvmFlagConstraintsGC.cpp#L177) . Now, Shenandoah reuses the flag, you may want to consider to parallel GC's implementation into `shared`.

Good catch. https://bugs.openjdk.org/browse/JDK-8343226

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1821337048

From wkemper at openjdk.org  Tue Oct 29 18:46:17 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 18:46:17 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <Rku187KbA-0dpP0fxJhyUnvODEyUPjHLWlEgkS-KGEU=.36fd6343-7cdd-4168-9944-fa52552139d2@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
 <Rku187KbA-0dpP0fxJhyUnvODEyUPjHLWlEgkS-KGEU=.36fd6343-7cdd-4168-9944-fa52552139d2@github.com>
Message-ID: <Of3TjfZ9Xf9VO9HbrRRe_Xi0n_1Rt6a9L5HpoP6Boss=.b2b19aef-a099-42f8-94ee-05c5a23c7fb9@github.com>

On Sat, 26 Oct 2024 02:05:28 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8342560: GenShen: Fix confusing method name
>>    
>>    Reviewed-by: ysr
>>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>>    
>>    Reviewed-by: xpeng, ysr
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>>    
>>    Reviewed-by: ysr
>>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>>    
>>    Reviewed-by: ysr
>>  - 8342255: GenShen: Remove unnecessary enum initial values
>>    
>>    Reviewed-by: ysr
>>  - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408
>
> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 701:
> 
>> 699:       // Seed the collection set with resource area-allocated
>> 700:       // preselected regions, which are removed when we exit this scope.
>> 701:       ResourceMark rm;
> 
> You can fold `ResourceMark` into `ShenandoahCollectionSetPreselector` class.

This makes sense: https://bugs.openjdk.org/browse/JDK-8343227

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1821368739

From wkemper at openjdk.org  Tue Oct 29 18:50:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 18:50:45 GMT
Subject: RFR: 8343226: Shenandoah: Move tenuring threshold constraint functions
 into shared constraints
Message-ID: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>

This change anticipates the [generational mode for Shenandoah](https://github.com/openjdk/jdk/pull/21273), which makes use of the `InitialTenuringThreshold` and `MaxTenuringThreshold` flags.

-------------

Commit messages:
 - Move tenuring flag contraints, make them available to Shenandoah

Changes: https://git.openjdk.org/shenandoah/pull/529/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=529&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8343226
  Stats: 115 lines in 4 files changed: 6 ins; 100 del; 9 mod
  Patch: https://git.openjdk.org/shenandoah/pull/529.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/529/head:pull/529

PR: https://git.openjdk.org/shenandoah/pull/529

From wkemper at openjdk.org  Tue Oct 29 18:52:02 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 18:52:02 GMT
Subject: RFR: 8343227: GenShen: Fold resource mark into management of
 preselected regions
Message-ID: <z0e1fEGWhB9QruTiwoB5VGnNvzve9rgvQqK43r1TwFw=.eb69df94-7bef-49bc-b7d1-ed7b981986dd@github.com>

`ShenandoahCollectionSetPreselector` manages a resource scoped array. It makes sense to move the `ResourceMark` into this class.

-------------

Commit messages:
 - Move resource mark into collection set preselector

Changes: https://git.openjdk.org/shenandoah/pull/530/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=530&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8343227
  Stats: 3 lines in 2 files changed: 2 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/530.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/530/head:pull/530

PR: https://git.openjdk.org/shenandoah/pull/530

From wkemper at openjdk.org  Tue Oct 29 19:01:18 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 19:01:18 GMT
Subject: RFR: 8342861: GenShen: Old generation in unexpected state when
 abandoning mixed gc candidates
Message-ID: <OilmKA8XA4tLMn_ILQUW8dnCqqKQVVmDHH1C8oPrMyo=.9ec57fea-74a4-4264-bd74-10636b28cd1a@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 278d3c9adc0882ec9d2f582a5e18263b6bf83b77

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/133/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=133&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342861
  Stats: 10 lines in 2 files changed: 10 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/133.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/133/head:pull/133

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/133

From wkemper at openjdk.org  Tue Oct 29 21:21:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 21:21:08 GMT
Subject: RFR: 8343227: GenShen: Fold resource mark into management of
 preselected regions [v2]
In-Reply-To: <z0e1fEGWhB9QruTiwoB5VGnNvzve9rgvQqK43r1TwFw=.eb69df94-7bef-49bc-b7d1-ed7b981986dd@github.com>
References: <z0e1fEGWhB9QruTiwoB5VGnNvzve9rgvQqK43r1TwFw=.eb69df94-7bef-49bc-b7d1-ed7b981986dd@github.com>
Message-ID: <9i2bBOokA4jUxMQKECIftbLRNgs2jueVGlzzV4Epmhg=.dc2d913e-9d2f-4091-afb3-fc95a8b490e9@github.com>

> `ShenandoahCollectionSetPreselector` manages a resource scoped array. It makes sense to move the `ResourceMark` into this class.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix builds

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/530/files
  - new: https://git.openjdk.org/shenandoah/pull/530/files/b6a136f8..c2a06a5f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=530&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=530&range=00-01

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/530.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/530/head:pull/530

PR: https://git.openjdk.org/shenandoah/pull/530

From wkemper at openjdk.org  Tue Oct 29 23:48:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Oct 2024 23:48:29 GMT
Subject: Integrated: 8342861: GenShen: Old generation in unexpected state when
 abandoning mixed gc candidates
In-Reply-To: <OilmKA8XA4tLMn_ILQUW8dnCqqKQVVmDHH1C8oPrMyo=.9ec57fea-74a4-4264-bd74-10636b28cd1a@github.com>
References: <OilmKA8XA4tLMn_ILQUW8dnCqqKQVVmDHH1C8oPrMyo=.9ec57fea-74a4-4264-bd74-10636b28cd1a@github.com>
Message-ID: <_7ltIILSEkl7y6z8XB_u0CjH48dg510q6h65I_GRUVM=.a51db302-8675-4748-9cc7-51bf82e19dec@github.com>

On Tue, 29 Oct 2024 18:56:19 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: d03588ad
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/d03588ad79119be43088e0482c4617492c9a13d0
Stats:     10 lines in 2 files changed: 10 ins; 0 del; 0 mod

8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates

Backport-of: 278d3c9adc0882ec9d2f582a5e18263b6bf83b77

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/133

From lmao at openjdk.org  Wed Oct 30 02:12:22 2024
From: lmao at openjdk.org (Liang Mao)
Date: Wed, 30 Oct 2024 02:12:22 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <ABXS9VGZgO5rYX2O_cjToFf5Mv1kDSia2hMvA95BbGM=.01ddfdca-38e4-4440-b677-811a2f9c6805@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
 <ABXS9VGZgO5rYX2O_cjToFf5Mv1kDSia2hMvA95BbGM=.01ddfdca-38e4-4440-b677-811a2f9c6805@github.com>
Message-ID: <L8N4NbS_URQkt0cLY8s3UajQEFoL1L_fg25K4pntzS0=.fa6cbb88-e95d-4f81-ac04-2a9b1174a5ac@github.com>

On Tue, 22 Oct 2024 18:25:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - Merge
>>  - 8342560: GenShen: Fix confusing method name
>>    
>>    Reviewed-by: ysr
>>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>>    
>>    Reviewed-by: xpeng, ysr
>>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>>    
>>    Reviewed-by: ysr
>>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>>    
>>    Reviewed-by: ysr
>>  - 8342255: GenShen: Remove unnecessary enum initial values
>>    
>>    Reviewed-by: ysr
>>  - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408
>
> We disabled tiered compilation to force everything to compile through C2 to get more consistent results.

@earthling-amzn ?thanks for providing your options. Looks like disabling pacing would help the score. 

-XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:+UnlockDiagnosticVMOptions -XX:-TieredCompilation -XX:-ShenandoahPacing -XX:+AlwaysPreTouch -XX:+DisableExplicitGC

I guess 4 cores may not be a target configuration for pauseless GC since generational pauseless GC requires at least 2 concurrent GC threads that already occupy half of CPU resource. I did a rough test on 8 cores and 16 cores according to your options. GenShen doesn't have obvious advantages with 8 cores and Shenandoah seems to outperform genshen significantly with 16 cores. Would you please help verify the result in your environment?

8 cores -Xmx8g -Xms8g:

GenShen:
RUN RESULT: hbIR (max attempted) = 9640, hbIR (settled) = 8050, max-jOPS = 8098, critical-jOPS = 6132
RUN RESULT: hbIR (max attempted) = 8944, hbIR (settled) = 8574, max-jOPS = 7781, critical-jOPS = 6130

Shenandoah:
RUN RESULT: hbIR (max attempted) = 9640, hbIR (settled) = 8050, max-jOPS = 7616, critical-jOPS = 6194
RUN RESULT: hbIR (max attempted) = 9640, hbIR (settled) = 8050, max-jOPS = 7712, critical-jOPS = 6262

16 cores -Xmx20g -Xms20g:

GenShen:
RUN RESULT: hbIR (max attempted) = 19881, hbIR (settled) = 16584, max-jOPS = 17694, critical-jOPS = 13274
RUN RESULT: hbIR (max attempted) = 19881, hbIR (settled) = 18724, max-jOPS = 17694, critical-jOPS = 13445
Shenandoah:
RUN RESULT: hbIR (max attempted) = 23838, hbIR (settled) = 20446, max-jOPS = 18594, critical-jOPS = 15441
RUN RESULT: hbIR (max attempted) = 20138, hbIR (settled) = 19989, max-jOPS = 18728, critical-jOPS = 14967

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2445672323

From shade at openjdk.org  Wed Oct 30 07:20:43 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 30 Oct 2024 07:20:43 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one
 header [v4]
In-Reply-To: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
Message-ID: <721dXjSoRCUw4xhyWZOfDBKv-nlMyfLKx6AnTwdsVR4=.c7e0f527-0692-4f7a-99ac-d6b446b6866e@github.com>

> Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:

 - Merge branch 'master' into JDK-8342588-shenandoah-shared-closures
 - Merge branch 'master' into JDK-8342588-shenandoah-shared-closures
 - Fix Zero builds
 - Fix

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21579/files
  - new: https://git.openjdk.org/jdk/pull/21579/files/831372cf..ad14c6d0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21579&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21579&range=02-03

  Stats: 798 lines in 31 files changed: 553 ins; 132 del; 113 mod
  Patch: https://git.openjdk.org/jdk/pull/21579.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21579/head:pull/21579

PR: https://git.openjdk.org/jdk/pull/21579

From shade at openjdk.org  Wed Oct 30 10:27:16 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 30 Oct 2024 10:27:16 GMT
Subject: RFR: 8342588: Shenandoah: Pull shared closures together in one
 header [v4]
In-Reply-To: <721dXjSoRCUw4xhyWZOfDBKv-nlMyfLKx6AnTwdsVR4=.c7e0f527-0692-4f7a-99ac-d6b446b6866e@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
 <721dXjSoRCUw4xhyWZOfDBKv-nlMyfLKx6AnTwdsVR4=.c7e0f527-0692-4f7a-99ac-d6b446b6866e@github.com>
Message-ID: <vNfvVIpY3VFAHDnaQ1pVoB87Wh5oFuFXRONHGVK2rR4=.0d93a8fb-9ff9-4974-9f3d-6450e4b7d837@github.com>

On Wed, 30 Oct 2024 07:20:43 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8342588-shenandoah-shared-closures
>  - Merge branch 'master' into JDK-8342588-shenandoah-shared-closures
>  - Fix Zero builds
>  - Fix

Thanks all!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21579#issuecomment-2446471913

From shade at openjdk.org  Wed Oct 30 10:27:17 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 30 Oct 2024 10:27:17 GMT
Subject: Integrated: 8342588: Shenandoah: Pull shared closures together in one
 header
In-Reply-To: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
References: <jrhYA2e60xAZ6ytXp0Tojw0gL8Fs-ZJ5yitPn2Bmfp8=.3635c0be-86bb-41d8-a951-75b2bc43759d@github.com>
Message-ID: <uQ5ZMhvodkzaA8P-YQzjV10SxZregmUyhQ6RpJFoNR4=.8e7ce81c-7c5a-4502-b942-bd2b5666cda5@github.com>

On Fri, 18 Oct 2024 09:14:47 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Currently we have shenandoahClosures.hpp and shenandoahOopClosures.hpp, each of which have some closure inside. There is no reason to have this separation, and we should really just merge everything in one header. I also did a bit of mechanical renames. There should be no semantic change with this patch.
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

This pull request has now been integrated.

Changeset: 9003524c
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/9003524c8bbe5041b8d51deb6000b688a29027f8
Stats:     452 lines in 16 files changed: 192 ins; 234 del; 26 mod

8342588: Shenandoah: Pull shared closures together in one header

Reviewed-by: wkemper, kdnilsen, simonis

-------------

PR: https://git.openjdk.org/jdk/pull/21579

From kdnilsen at openjdk.org  Wed Oct 30 17:01:29 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 30 Oct 2024 17:01:29 GMT
Subject: RFR: 8343226: Shenandoah: Move tenuring threshold constraint
 functions into shared constraints
In-Reply-To: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
References: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
Message-ID: <bcrnSZnyZNIbh3C9TyR0AKv28Mp9jfEjFIA-whUCTuE=.57efbbc6-95c1-43c8-93b1-a45a31a8542d@github.com>

On Tue, 29 Oct 2024 18:45:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change anticipates the [generational mode for Shenandoah](https://github.com/openjdk/jdk/pull/21273), which makes use of the `InitialTenuringThreshold` and `MaxTenuringThreshold` flags.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/529#pullrequestreview-2405651285

From kdnilsen at openjdk.org  Wed Oct 30 17:02:40 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 30 Oct 2024 17:02:40 GMT
Subject: RFR: 8343227: GenShen: Fold resource mark into management of
 preselected regions [v2]
In-Reply-To: <9i2bBOokA4jUxMQKECIftbLRNgs2jueVGlzzV4Epmhg=.dc2d913e-9d2f-4091-afb3-fc95a8b490e9@github.com>
References: <z0e1fEGWhB9QruTiwoB5VGnNvzve9rgvQqK43r1TwFw=.eb69df94-7bef-49bc-b7d1-ed7b981986dd@github.com>
 <9i2bBOokA4jUxMQKECIftbLRNgs2jueVGlzzV4Epmhg=.dc2d913e-9d2f-4091-afb3-fc95a8b490e9@github.com>
Message-ID: <PeNXg9Y2V29egIiMKWOyBTLBqlI07aTA5XM_uDBUdrU=.6418d55c-c8ff-425e-b1e0-5f14e5751523@github.com>

On Tue, 29 Oct 2024 21:21:08 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> `ShenandoahCollectionSetPreselector` manages a resource scoped array. It makes sense to move the `ResourceMark` into this class.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix builds

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/530#pullrequestreview-2405654663

From wkemper at openjdk.org  Wed Oct 30 17:19:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 30 Oct 2024 17:19:29 GMT
Subject: Integrated: 8343227: GenShen: Fold resource mark into management of
 preselected regions
In-Reply-To: <z0e1fEGWhB9QruTiwoB5VGnNvzve9rgvQqK43r1TwFw=.eb69df94-7bef-49bc-b7d1-ed7b981986dd@github.com>
References: <z0e1fEGWhB9QruTiwoB5VGnNvzve9rgvQqK43r1TwFw=.eb69df94-7bef-49bc-b7d1-ed7b981986dd@github.com>
Message-ID: <YztE_9ONwaOgrobTjReFSBIxs2ekyoBpBRRO1z9Rtlo=.b4f5f609-65d2-43eb-9575-1666fe386127@github.com>

On Tue, 29 Oct 2024 18:47:09 GMT, William Kemper <wkemper at openjdk.org> wrote:

> `ShenandoahCollectionSetPreselector` manages a resource scoped array. It makes sense to move the `ResourceMark` into this class.

This pull request has now been integrated.

Changeset: bbc2841c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/bbc2841c2597f84544e4506b065bc464db072118
Stats:     4 lines in 2 files changed: 3 ins; 1 del; 0 mod

8343227: GenShen: Fold resource mark into management of preselected regions

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/530

From wkemper at openjdk.org  Wed Oct 30 20:15:52 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 30 Oct 2024 20:15:52 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <L8N4NbS_URQkt0cLY8s3UajQEFoL1L_fg25K4pntzS0=.fa6cbb88-e95d-4f81-ac04-2a9b1174a5ac@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
 <ABXS9VGZgO5rYX2O_cjToFf5Mv1kDSia2hMvA95BbGM=.01ddfdca-38e4-4440-b677-811a2f9c6805@github.com>
 <L8N4NbS_URQkt0cLY8s3UajQEFoL1L_fg25K4pntzS0=.fa6cbb88-e95d-4f81-ac04-2a9b1174a5ac@github.com>
Message-ID: <0ymPZeZsrtfMJd1-5w_4Ciw0tAxMtSoaS8mGoOO9Kr8=.f4b59791-21bd-4e51-aefb-ffaeccddeec6@github.com>

On Wed, 30 Oct 2024 02:09:46 GMT, Liang Mao <lmao at openjdk.org> wrote:

>> We disabled tiered compilation to force everything to compile through C2 to get more consistent results.
>
> @earthling-amzn ?thanks for providing your options. Looks like disabling pacing would help the score. 
> 
> -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:+UnlockDiagnosticVMOptions -XX:-TieredCompilation -XX:-ShenandoahPacing -XX:+AlwaysPreTouch -XX:+DisableExplicitGC
> 
> I guess 4 cores may not be a target configuration for pauseless GC since generational pauseless GC requires at least 2 concurrent GC threads that already occupy half of CPU resource. I did a rough test on 8 cores and 16 cores according to your options. GenShen doesn't have obvious advantages with 8 cores and Shenandoah seems to outperform genshen significantly with 16 cores. Would you please help verify the result in your environment?
> 
> 8 cores -Xmx8g -Xms8g:
> 
> GenShen:
> RUN RESULT: hbIR (max attempted) = 9640, hbIR (settled) = 8050, max-jOPS = 8098, critical-jOPS = 6132
> RUN RESULT: hbIR (max attempted) = 8944, hbIR (settled) = 8574, max-jOPS = 7781, critical-jOPS = 6130
> 
> Shenandoah:
> RUN RESULT: hbIR (max attempted) = 9640, hbIR (settled) = 8050, max-jOPS = 7616, critical-jOPS = 6194
> RUN RESULT: hbIR (max attempted) = 9640, hbIR (settled) = 8050, max-jOPS = 7712, critical-jOPS = 6262
> 
> 16 cores -Xmx20g -Xms20g:
> 
> GenShen:
> RUN RESULT: hbIR (max attempted) = 19881, hbIR (settled) = 16584, max-jOPS = 17694, critical-jOPS = 13274
> RUN RESULT: hbIR (max attempted) = 19881, hbIR (settled) = 18724, max-jOPS = 17694, critical-jOPS = 13445
> Shenandoah:
> RUN RESULT: hbIR (max attempted) = 23838, hbIR (settled) = 20446, max-jOPS = 18594, critical-jOPS = 15441
> RUN RESULT: hbIR (max attempted) = 20138, hbIR (settled) = 19989, max-jOPS = 18728, critical-jOPS = 14967

@mmyxym - I disabled Shenandoah's pacer, and find that the generational mode out performs the non-generational mode:

Shen: RUN RESULT: hbIR (max attempted) = 16584, hbIR (settled) = 13837, max-jOPS = 11609, critical-jOPS = 11062
Shen: RUN RESULT: hbIR (max attempted) = 16584, hbIR (settled) = 13837, max-jOPS = 11609, critical-jOPS = 10425

Gen:  RUN RESULT: hbIR (max attempted) = 16584, hbIR (settled) = 14144, max-jOPS = 13267, critical-jOPS = 12087
Gen:  RUN RESULT: hbIR (max attempted) = 16584, hbIR (settled) = 13837, max-jOPS = 13267, critical-jOPS = 12151

The shared options for these runs were:

"-Xmx8g -Xms8g -XX:ActiveProcessorCount=8 -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:MetaspaceSize=1g -XX:-ShenandoahPacing

This is an aarch64 host with 16 cores and 64G of memory.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2448245419

From wkemper at openjdk.org  Thu Oct 31 14:26:57 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 31 Oct 2024 14:26:57 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <30xsGR0ErdQXpuA4zCWPUe675JpRzxIAsZMwsNvkkCY=.086b0d90-509f-4bbe-9aaf-ff81223a743a@github.com>

Merges tag jdk-21.0.6+1

-------------

Commit messages:
 - 8342701: [PPC64] TestOSRLotsOfLocals.java crashes
 - 8334560: [PPC64]: postalloc_expand_java_dynamic_call_sched does not copy all fields
 - 8328379: Convert URLDragTest.html applet test to main
 - 8195675: Call to insertText with single character from custom Input Method ignored
 - 8325506: Ensure randomness is only read from provided SecureRandom object
 - 8320673: PageFormat/CustomPaper.java has no Pass/Fail buttons; multiple instructions
 - 8322754: click JComboBox when dialog about to close causes IllegalComponentStateException
 - 8225220: When the Tab Policy is checked,the scroll button direction displayed incorrectly.
 - 8316428: G1: Nmethod count statistics only count last code root set iterated
 - 8333824: Unused ClassValue in VarHandles
 - ... and 123 more: https://git.openjdk.org/shenandoah-jdk21u/compare/dfcd8d2e...7dc0f7a6

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=134&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/134/files
  Stats: 15749 lines in 328 files changed: 9595 ins; 5011 del; 1143 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/134.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/134/head:pull/134

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/134

From rkennke at openjdk.org  Thu Oct 31 18:07:06 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 31 Oct 2024 18:07:06 GMT
Subject: RFR: 8343226: Shenandoah: Move tenuring threshold constraint
 functions into shared constraints
In-Reply-To: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
References: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
Message-ID: <8BdjieENGaCdv9NMRA5HAsGnDq8lv2DtGcvzrpHbRSg=.69b23081-167f-4924-8fba-d0854e8c325a@github.com>

On Tue, 29 Oct 2024 18:45:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change anticipates the [generational mode for Shenandoah](https://github.com/openjdk/jdk/pull/21273), which makes use of the `InitialTenuringThreshold` and `MaxTenuringThreshold` flags.

Looks good to me. Thanks!

-------------

Marked as reviewed by rkennke (Lead).

PR Review: https://git.openjdk.org/shenandoah/pull/529#pullrequestreview-2408729533

From xpeng at openjdk.org  Thu Oct 31 20:46:22 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 31 Oct 2024 20:46:22 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v14]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <ytE8eHPKFnh58mPCzjIAl-2jwV0p4Y--b4uLEtNCyqc=.3ca02bc7-c477-4e29-9507-304fe0470527@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 38 commits:

 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Move set_mark_incomplete into reset_mark_bitmap
 - fix test failures
 - set_mark_incomplete should be always set after resetting bitmap
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Not reset bitmap after ShenandoahOldGC
 - Capture Top At Mark Start for *this* generation
 - Revert "set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION"
   
   This reverts commit d4fddac5a1e7321ff3b4441bd5cd096e949cd56a.
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - ... and 28 more: https://git.openjdk.org/shenandoah/compare/bbc2841c...0b583764

-------------

Changes: https://git.openjdk.org/shenandoah/pull/516/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=13
  Stats: 109 lines in 6 files changed: 94 ins; 2 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516