From wkemper at openjdk.org  Fri Nov  1 14:17:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 1 Nov 2024 14:17:33 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <Ud_kMqN00A5s2_QlTbjgfmvZ5fwh03cTtZW4vEmiStY=.ab6d2b5c-95ab-4332-aaa6-a50ff0ddecaa@github.com>

Merges tag jdk-24+22

-------------

Commit messages:
 - 8342156: C2: Compilation failure with fewer arguments after JDK-8329032
 - 8342642: Class loading failure due to archived map issue in ModuleLoaderMap.Mapper
 - 8342181: Update tests to use stronger Key and Salt size
 - 8337276: jcmd man page update for PID in output filenames
 - 8342635: javax/swing/JFileChooser/FileSystemView/WindowsDefaultIconSizeTest.java creates tmp file in src dir
 - 8341788: Fix ExceptionOccurred in hotspot
 - 8341927: Replace hardcoded security providers with new test.provider.name system property
 - 8343103: Enable debug logging for vmTestbase/nsk/jvmti/scenarios/sampling/SP05/sp05t003/TestDescription.java
 - 8202100: Merge vm/share/InMemoryJavaCompiler w/ jdk/test/lib/compiler/InMemoryJavaCompiler
 - 8341692: Implement JEP 490: ZGC: Remove the Non-Generational Mode
 - ... and 112 more: https://git.openjdk.org/shenandoah/compare/8bcd4920...388d44fb

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=531&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/531/files
  Stats: 166108 lines in 1348 files changed: 19132 ins; 143002 del; 3974 mod
  Patch: https://git.openjdk.org/shenandoah/pull/531.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/531/head:pull/531

PR: https://git.openjdk.org/shenandoah/pull/531

From ysr at openjdk.org  Fri Nov  1 18:09:52 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 1 Nov 2024 18:09:52 GMT
Subject: RFR: 8343226: Shenandoah: Move tenuring threshold constraint
 functions into shared constraints
In-Reply-To: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
References: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
Message-ID: <Qv0gnbySyhq9WV2NNBVPCV1O1nxxa488xZWKusp-KJQ=.79e18c4e-0c1f-42b0-a454-d7904b951fcc@github.com>

On Tue, 29 Oct 2024 18:45:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change anticipates the [generational mode for Shenandoah](https://github.com/openjdk/jdk/pull/21273), which makes use of the `InitialTenuringThreshold` and `MaxTenuringThreshold` flags.

The change looks fine, but a few related comments. As far as I could tell:

1. With adaptive tenuring, GenShen makes use of neither of these options
2. Without adaptive tenuring, GenShen makes use of just InitialTenuringThreshold
3. G1 uses MaxTenuringThreshold, but not InitialTenuringThreshold

Given 2 & 3, I wonder if a comparison between them makes sense. It might, but worth checking the ways in which these are used. Given 3, then, would it need to also use the constraints function?

None of these are new issues (they predate your changes), so just leaving these comments for you to think about.

The changes look fine to me otherwise.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/529#pullrequestreview-2410631926

From gziemski at openjdk.org  Sat Nov  2 14:48:01 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Sat, 2 Nov 2024 14:48:01 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
Message-ID: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>

We use mtNone value in several functions default parameters, which may show up in NMT reports.

We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.

Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.

TODO: update copyrights.

Testing: undergoing MARCH5 tier1-5 ...

-------------

Commit messages:
 - do not allow default parameter for mtNone

Changes: https://git.openjdk.org/jdk/pull/21843/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21843&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8328944
  Stats: 193 lines in 47 files changed: 23 ins; 1 del; 169 mod
  Patch: https://git.openjdk.org/jdk/pull/21843.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21843/head:pull/21843

PR: https://git.openjdk.org/jdk/pull/21843

From dholmes at openjdk.org  Mon Nov  4 01:27:38 2024
From: dholmes at openjdk.org (David Holmes)
Date: Mon, 4 Nov 2024 01:27:38 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <LAMkJJGkzhMIfFElINOqOLuJA-0z72zL5fQ7W9j-xGU=.e6356df8-3bbf-4111-8bb3-a252790b131c@github.com>

On Fri, 1 Nov 2024 20:44:50 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

> We use mtNone value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
> 
> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
> 
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

Seems okay in principle, but you will need someone familiar with each modified area to confirm that the chosen tags are appropriate - I'm unsure about the use of `mtClass` versus `mtClassShared` for example.

-------------

PR Review: https://git.openjdk.org/jdk/pull/21843#pullrequestreview-2412003334

From stuefe at openjdk.org  Mon Nov  4 08:36:42 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Mon, 4 Nov 2024 08:36:42 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>

On Fri, 1 Nov 2024 20:44:50 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

> We use mtNone value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
> 
> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
> 
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

Had a cursory look. The direction is okay. Some general comments:

1) We have two cases: a) where today we report mtNone and b) where today we report a tag other than mtNone, but the tag was set after the initial reservation

For the latter case, if the case is simple (the reservation was for a single thing, and its purpose was clear at reservation) there must have existed a subsequent NMT registration, which you should then remove since it should not be necessary anymore. I will point them out if I stumble over them, but may miss a bunch.

2) About 
> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.

I think this is neither realistic nor desirable. mtNone means "don't know yet; will do later". This is perfectly valid for cases where
- you allocate something on behalf of someone else; obviously, you don't know what he intends to do with it
- when you reserve a range that is later shared by multiple tags.

If we move to more fine granular tags, or something like hierarchical tags, the latter point will become more important. Therefore, mtNone has its place.

src/hotspot/cpu/aarch64/compressedKlass_aarch64.cpp line 72:

> 70:     assert(immediate > 0 && Assembler::operand_valid_for_logical_immediate(/*is32*/false, immediate),
> 71:            "Invalid immediate %d " UINT64_FORMAT, index, immediate);
> 72:     result = os::attempt_reserve_memory_at((char*)immediate, size, false, mtMetaspace);

mtMetaspace is wrong. There is no right tag apart from mtNone, since this address range is a combination of mtClass+mtClassShared, depending on how class space is configured. mtNone signals "will be set later", which is the correct course here IHMO.

The tag is set later, see metaspace.cpp and metaspaceShared.cpp

src/hotspot/os/bsd/gc/z/zPhysicalMemoryBacking_bsd.cpp line 82:

> 80: 
> 81:   // Reserve address space for backing memory
> 82:   _base = (uintptr_t)os::reserve_memory(max_capacity, false, mtGC);

This is Java heap.

src/hotspot/share/cds/archiveBuilder.cpp line 330:

> 328:   size_t buffer_size = estimate_archive_size();
> 329:   ReservedSpace rs(buffer_size, MetaspaceShared::core_region_alignment(),
> 330:                    os::vm_page_size(), nullptr, mtMetaspace);

This is not Metaspace. Possibly mtClassShared, though that one is reserved for the runtime mapping of the CDS archive; this is the dumptime image. mtInternal would be the best fit, IMHO.

src/hotspot/share/cds/metaspaceShared.cpp line 1387:

> 1385:       total_space_rs = ReservedSpace(total_range_size, base_address_alignment,
> 1386:                                      os::vm_page_size(), (char*)base_address, mtClass);
> 1387:     } else {

All the changes here in `MetaspaceShared::reserve_address_space_for_archives` are unnecessary; we tag the address region already. No need to do this multiple times, and in every case branch. The ReservedSpace objects here are not used for much and mostly ephemeral anyway - they are only used to reserve memory here, but after that have not much meaning, and apart from the class space rs don't even survive the invocation.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 249:

> 247:             _bitmap_bytes_per_slice, bitmap_page_size);
> 248: 
> 249:   ReservedSpace bitmap(_bitmap_size, bitmap_page_size, mtGC);

remove nmt registration below?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 269:

> 267: 
> 268:   if (ShenandoahVerify) {
> 269:     ReservedSpace verify_bitmap(_bitmap_size, bitmap_page_size, mtGC);

remove nmt registration below?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 288:

> 286: 
> 287:   ReservedSpace aux_bitmap(_bitmap_size, aux_bitmap_page_size, mtGC);
> 288:   os::trace_page_sizes_for_requested_size("Aux Bitmap",

remove nmt registration below?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 306:

> 304: 
> 305:   ReservedSpace region_storage(region_storage_size, region_page_size, mtGC);
> 306:   os::trace_page_sizes_for_requested_size("Region Storage",

remove nmt registration below?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 340:

> 338: 
> 339:     if (_collection_set == nullptr) {
> 340:       cset_rs = ReservedSpace(cset_size, cset_align, os::vm_page_size(), nullptr, mtGC);

remove NMT registration in ShenandoahCollectionSet?

src/hotspot/share/jfr/recorder/storage/jfrVirtualMemory.cpp line 107:

> 105:   _rs = ReservedSpace(reservation_size_request_bytes,
> 106:                       os::vm_allocation_granularity(), os::vm_page_size(),
> 107:                       nullptr, mtTracing);

remove NMT registraton?

src/hotspot/share/memory/heap.cpp line 231:

> 229:   // reserve space for _segmap
> 230:   ReservedSpace seg_rs(reserved_segments_size, mtCode);
> 231:   if (!_segmap.initialize(seg_rs, committed_segments_size)) {

remove NMT registration

src/hotspot/share/memory/metaspace.cpp line 605:

> 603:     assert(is_aligned(result, Metaspace::reserve_alignment()), "Alignment too small for metaspace");
> 604:     rs = ReservedSpace::space_for_range(result, size, Metaspace::reserve_alignment(),
> 605:                                                       os::vm_page_size(), false, false, mtMetaspace);

Please revert to mtNone; same reason as given in metaspaceShared.cpp. You can add a comment that tags are adjusted afterward, when the distribution of mtClass and mtClassShared is clear.

src/hotspot/share/memory/metaspace.cpp line 746:

> 744:       }
> 745:       rs = ReservedSpace(size, Metaspace::reserve_alignment(),
> 746:                          os::vm_page_size() /* large */, (char*)base, mtMetaspace);

mtClass, but actually not needed, since we set mtClass below. Again, not necessary to do this for every branch individually we take here when reserving class space memory.

src/hotspot/share/memory/metaspace.cpp line 775:

> 773: 
> 774:     // Mark metaspace as such
> 775:     MemTracker::record_virtual_memory_tag((address)rs.base(), mtMetaspace);

No, please revert. mtClass is correct here.

src/hotspot/share/memory/metaspace/testHelpers.cpp line 82:

> 80:     // have reserve limit -> non-expandable context
> 81:     _rs = ReservedSpace(reserve_limit * BytesPerWord, Metaspace::reserve_alignment(),
> 82:                         os::vm_page_size(), nullptr, MemTag::mtMetaspace);

Give this mtTest

src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 258:

> 256:   ReservedSpace rs(word_size * BytesPerWord,
> 257:                    Settings::virtual_space_node_reserve_alignment_words() * BytesPerWord,
> 258:                    os::vm_page_size(), nullptr, mtMetaspace);

remove tracking below.

This is actually wrong, and only works out of accident correctly, since this constructor just happens to be called for mtMetaspace; more correct would be to hand in the tag. This is on my plate, there is a JBS issue somewhere.

src/hotspot/share/memory/virtualspace.hpp line 50:

> 48:   // ReservedSpace
> 49:   ReservedSpace(char* base, size_t size, size_t alignment,
> 50:                 size_t page_size, bool special, bool executable, MemTag mem_tag);

The growth in complexity is unfortunate.

src/hotspot/share/memory/virtualspace.hpp line 67:

> 65: 
> 66:   void reserve(size_t size, size_t alignment, size_t page_size,
> 67:                char* requested_address, bool executable, MemTag mem_tag);

Why do the instance methods get MemTag? Why is MemTag not stored as a (const) attribute of ReservedSpace? 

But then, we would have double accounting - we'd store the tag both in NMT and in ReservedSpace instances. If we change them post-reservation, we would need to change both. Sigh.

So we pass in MemTag just for the ReservedSpace to pass it into os::reserve_memory. Seeing how often ReservedSpace is used in situations where the tag is not clear at reservation time, I wonder whether we are not better off letting the user of ReservedSpace register the NMT tag post reservation like they do today.

src/hotspot/share/nmt/memReporter.cpp line 83:

> 81:   if (mem_tag != mtNone) {
> 82:     out->print("(%s" SIZE_FORMAT "%s type=%u", alloc_type,
> 83:       amount_in_current_scale(amount), scale, (unsigned)mem_tag);

No, please revert.

src/hotspot/share/nmt/memReporter.cpp line 252:

> 250:    // report malloc'd memory
> 251:   if (amount_in_current_scale(MAX2(malloc_memory->malloc_size(), pk_malloc)) > 0) {
> 252:     print_malloc(malloc_memory->malloc_counter(), mtNMT);

Please revert, the original was correct. print_malloc is used for both summary and detail reports; for summary, we pass in mtNone deliberately since the report happens under a common tag.

src/hotspot/share/nmt/virtualMemoryTracker.cpp line 364:

> 362:       // thread does not detach from VM before exits, and leads to
> 363:       // leak JavaThread object
> 364:       if ((reserved_rgn->mem_tag() == mtThreadStack) && (reserved_rgn->mem_tag() == mem_tag)) {

Please explain the changes in this function, especially the changed conditions. The added conditions for tag equality seem wrong to me.

src/hotspot/share/runtime/safepointMechanism.cpp line 61:

> 59:     const size_t page_size = os::vm_page_size();
> 60:     const size_t allocation_size = 2 * page_size;
> 61:     char* polling_page = os::reserve_memory(allocation_size, false, mtSafepoint);

remove registration

src/hotspot/share/utilities/debug.cpp line 712:

> 710: 
> 711: void initialize_assert_poison() {
> 712:   char* page = os::reserve_memory(os::vm_page_size(), false, mtInternal);

remove registration

test/hotspot/jtreg/runtime/NMT/MallocRoundingReportTest.java line 60:

> 58:                 NMTTestUtils.runJcmdSummaryReportAndCheckOutput(
> 59:                         "Test (reserved=" + numKB + "KB, committed=" + numKB + "KB)",
> 60:                         "(malloc=" + numKB + "KB type=12 #1) (at peak)"

Again, this is wrong. The memory had not been reserved for NMT usage, so it should not be tagged with mtNMT (12)

test/hotspot/jtreg/runtime/NMT/MallocRoundingReportTest.java line 68:

> 66:                 NMTTestUtils.runJcmdSummaryReportAndCheckOutput(
> 67:                         "Test (reserved=0KB, committed=0KB)",
> 68:                         "(malloc=0KB type=12) (peak=" + numKB + "KB #1)"

ditto

-------------

PR Review: https://git.openjdk.org/jdk/pull/21843#pullrequestreview-2412301432
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827274229
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827277342
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827280468
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827292469
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827310565
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827310673
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827310815
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827310928
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827312298
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827312806
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827312951
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827314192
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827314524
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827314999
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827316399
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827318234
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827329282
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827323214
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827334776
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827339410
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827348792
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827361075
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827361821
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827363362
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1827363506

From jsjolen at openjdk.org  Mon Nov  4 13:15:29 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Mon, 4 Nov 2024 13:15:29 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
Message-ID: <nrLkAasFTenrFs1uh7KzbGpK1qryUe6FiPvjLXVT3fg=.bb0515c7-3bf4-40b7-b19d-11a5b2575b4b@github.com>

On Mon, 4 Nov 2024 08:34:00 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> > Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
> 
> I think this is neither realistic nor desirable. mtNone means "don't know yet; will do later". This is perfectly valid for cases where
> 
>     * you allocate something on behalf of someone else; obviously, you don't know what he intends to do with it
> 
>     * when you reserve a range that is later shared by multiple tags.
> 
> 
> If we move to more fine granular tags, or something like hierarchical tags, the latter point will become more important. Therefore, mtNone has its place.

It seems to me like the first case isn't a case at all, as that someone else can always give you a tag for the memory. With more granular or hierarchical tags, the case for `mtNone` seems even rarer. We can then always have a top-level tag for the category: `mtCardTable <: mtGC`.

Now I'm just thinking out loud: `mtNone` could be renamed to `mtAny` in the case that we do have hierarchical tags, as to represent `top` in a partially ordered set.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2454684099

From stuefe at openjdk.org  Mon Nov  4 13:33:31 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Mon, 4 Nov 2024 13:33:31 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <nrLkAasFTenrFs1uh7KzbGpK1qryUe6FiPvjLXVT3fg=.bb0515c7-3bf4-40b7-b19d-11a5b2575b4b@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
 <nrLkAasFTenrFs1uh7KzbGpK1qryUe6FiPvjLXVT3fg=.bb0515c7-3bf4-40b7-b19d-11a5b2575b4b@github.com>
Message-ID: <nbcE0gSU0NjoqBaY5i6dvWmYFMuHNoi9V53byvYs31Y=.5c45db29-cb3f-4f4a-b61e-42f91a0a7089@github.com>

On Mon, 4 Nov 2024 13:12:52 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> > > Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
> > 
> > 
> > I think this is neither realistic nor desirable. mtNone means "don't know yet; will do later". This is perfectly valid for cases where
> > ```
> > * you allocate something on behalf of someone else; obviously, you don't know what he intends to do with it
> > 
> > * when you reserve a range that is later shared by multiple tags.
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > If we move to more fine granular tags, or something like hierarchical tags, the latter point will become more important. Therefore, mtNone has its place.
> 
> It seems to me like the first case isn't a case at all, as that someone else can always give you a tag for the memory. 

The requirement of having to funnel MemTags through call chains makes the code brittle and inflexible to work with. You need to modify every single call frame to marshall that tag through to the eventual place where it gets tracked. 

Apart from that hassle, you may not even know. An allocator can allocate a range en-block and hand it out piecemeal to different users. Currently, that whole range is then tagged as belonging to the allocator, which is a bit useless tbh. E.g. the full class space range is tagged as mtClass. Every ResourceArea chunk is tagged as RA. That is very coarse-granular, and there is no way to have finer-grained tagging. I often wished for that at customer sites.

Bottomline, allocation operations on the system level don't necessarily map 1:1 to NMT semantic tagging. We should not hard-code this requirement into NMT. It limits the future usefulness of NMT, and/or limits how we allow ourselves to allocate memory (which should always take the most efficient route and not be limited by what NMT can or can't do).

> With more granular or hierarchical tags, the case for `mtNone` seems even rarer. We can then always have a top-level tag for the category: `mtCardTable <: mtGC`.
> 
> Now I'm just thinking out loud: `mtNone` could be renamed to `mtAny` in the case that we do have hierarchical tags, as to represent `top` in a partially ordered set.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2454725202

From stefank at openjdk.org  Mon Nov  4 13:44:46 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 4 Nov 2024 13:44:46 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v5]
In-Reply-To: <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <FMQHoGfcWxoMldgl3JRvFvUKqOuu1N-YSv6qSkRmBjA=.9bea9619-d7f3-4834-bbf3-5a05e73c17a3@github.com>
Message-ID: <aOgHVX3-yTHBlK-wxHAupcIVBDErW-NNTAy7Zk7oobA=.9916e11a-20ec-4a7c-bce3-76a01534066e@github.com>

On Mon, 21 Oct 2024 21:16:50 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 492 commits:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8342560: GenShen: Fix confusing method name
>    
>    Reviewed-by: ysr
>  - 8342564: GenShen: Only reference young/old generation names in generational mode
>    
>    Reviewed-by: xpeng, ysr
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8342214: GenShen: Reduce code duplication in shFreeSet with iterator abstraction
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342239: GenShen: Revert changes in adaptive heuristic to avoid overflow on 32 bit
>    
>    Reviewed-by: ysr
>  - 8342278: GenShen: Move non-generational mode test out of generational test configuration
>    
>    Reviewed-by: ysr
>  - 8342255: GenShen: Remove unnecessary enum initial values
>    
>    Reviewed-by: ysr
>  - ... and 482 more: https://git.openjdk.org/jdk/compare/71583222...2a2aa408

The changes to the shared code is minimal and looks good.

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2413104483

From wkemper at openjdk.org  Mon Nov  4 17:24:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 4 Nov 2024 17:24:54 GMT
Subject: RFR: 8343226: Shenandoah: Move tenuring threshold constraint
 functions into shared constraints
In-Reply-To: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
References: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
Message-ID: <qniNArwEEM0IhrX5pfG8Lpx4-jhrdMudoK96tMXDStE=.bb836ba9-39ca-4f76-9bcd-8148fba2d86e@github.com>

On Tue, 29 Oct 2024 18:45:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change anticipates the [generational mode for Shenandoah](https://github.com/openjdk/jdk/pull/21273), which makes use of the `InitialTenuringThreshold` and `MaxTenuringThreshold` flags.

In light of Ramki's review, I'm going to abandon this PR. I'd forgotten that genshen has `ShenandoahGenerationalMaxTenuringAge`.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/529#issuecomment-2455285696

From wkemper at openjdk.org  Mon Nov  4 17:24:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 4 Nov 2024 17:24:54 GMT
Subject: Withdrawn: 8343226: Shenandoah: Move tenuring threshold constraint
 functions into shared constraints
In-Reply-To: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
References: <dD4wknotzYnSGxRzOYtkWdm4j8oG2KTAn3NldNq6yz0=.79ababa2-7fa6-4f07-8fc4-8712cf249992@github.com>
Message-ID: <FdSo8q8Y_1yfvKdd1yZjcMTu8jmQj-WfnwVOuAxdJHU=.053d937f-35ef-4f27-90a7-5b4d7d23f149@github.com>

On Tue, 29 Oct 2024 18:45:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change anticipates the [generational mode for Shenandoah](https://github.com/openjdk/jdk/pull/21273), which makes use of the `InitialTenuringThreshold` and `MaxTenuringThreshold` flags.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/529

From shade at openjdk.org  Mon Nov  4 19:13:44 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 4 Nov 2024 19:13:44 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v9]
In-Reply-To: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
Message-ID: <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>

> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
> 
> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
> 
> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
> 
> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
> 
> Additional testing:
>  - [x] `jdk_jfr` pass by default
>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits:

 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Disable Shenandoah-targeted test specially
 - Redo the whole thing by disabling Leak Prof with Shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Just exclude the tests for Shenandoah
 - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
 - Tighten up comments prose
 - Roman's review: more precise GC state check, more includes
 - ... and 3 more: https://git.openjdk.org/jdk/compare/23fa1a33...d47b1665

-------------

Changes: https://git.openjdk.org/jdk/pull/20328/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=08
  Stats: 67 lines in 34 files changed: 39 ins; 0 del; 28 mod
  Patch: https://git.openjdk.org/jdk/pull/20328.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20328/head:pull/20328

PR: https://git.openjdk.org/jdk/pull/20328

From wkemper at openjdk.org  Mon Nov  4 19:52:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 4 Nov 2024 19:52:30 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 503 commits:

 - Merge openjdk/jdk tip into great-genshen-pr-redux
 - Merge remote-tracking branch 'jdk/master' into merge-latest
 - Merge remote-tracking branch 'jdk/master' into merge-latest
 - Merge
 - 8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates
   
   Reviewed-by: kdnilsen
 - 8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
   
   Reviewed-by: ysr
 - 8342919: GenShen: Fix whitespace
   
   Reviewed-by: xpeng, kdnilsen
 - 8342927: GenShen: Guarantee slices of time for coalesce and filling
   
   Reviewed-by: kdnilsen
 - 8342924: GenShen: Problem list gc/shenandoah/TestReferenceRefersToShenandoah.java
   
   Reviewed-by: kdnilsen, ysr
 - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode
   
   Reviewed-by: wkemper
 - ... and 493 more: https://git.openjdk.org/jdk/compare/1c448347...19b25bc3

-------------

Changes: https://git.openjdk.org/jdk/pull/21273/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=05
  Stats: 22859 lines in 229 files changed: 21116 ins; 886 del; 857 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From shade at openjdk.org  Tue Nov  5 09:08:42 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 5 Nov 2024 09:08:42 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v9]
In-Reply-To: <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>
Message-ID: <rJdfmQK49lsyiXbjk-tftLXVgoX0j8vn9DhIdTcMFj8=.75f618bb-0cca-4734-bd64-4bf1eb652ca5@github.com>

On Mon, 4 Nov 2024 19:13:44 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [x] `jdk_jfr` pass by default
>>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits:
> 
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Disable Shenandoah-targeted test specially
>  - Redo the whole thing by disabling Leak Prof with Shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Just exclude the tests for Shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Tighten up comments prose
>  - Roman's review: more precise GC state check, more includes
>  - ... and 3 more: https://git.openjdk.org/jdk/compare/23fa1a33...d47b1665

Local testing passes. Please re-review!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2456610619

From stuefe at openjdk.org  Tue Nov  5 09:18:28 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 5 Nov 2024 09:18:28 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <qkd1hyV_e3dXmHrp2zU0LSNxefRoGyaF4cy3i1MglFE=.304dbaff-6651-4598-8f42-02c97a9949fd@github.com>

On Fri, 1 Nov 2024 20:44:50 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

> We use mtNone value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
> 
> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
> 
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

In case this does not come across: I really welcome fixing unnecessary "mtNone" uses. I would like to see a nuanced approach though. The rule should be to try to use the correct tag at reservation. But exceptions should be allowed:
- if the tag is not clear at reservation, any other tag than mtNone (or the proposed mtAny) would be wrong
- if it makes the code a lot more brittle because reservation is "smeared" across many different mmap calls in many conditional branches. In that case, it is easier to tag the whole range after all that reservation business is done, outside the conditional branches.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2456635960

From egahlin at openjdk.org  Tue Nov  5 10:00:33 2024
From: egahlin at openjdk.org (Erik Gahlin)
Date: Tue, 5 Nov 2024 10:00:33 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v9]
In-Reply-To: <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>
Message-ID: <OpH0rr94M2CpLqqb0sFmj93_zosl5gKoZ6yU_jMoi1Q=.cbe241c3-cf28-4cb2-987e-66fb79a87b01@github.com>

On Mon, 4 Nov 2024 19:13:44 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [x] `jdk_jfr` pass by default
>>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits:
> 
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Disable Shenandoah-targeted test specially
>  - Redo the whole thing by disabling Leak Prof with Shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Just exclude the tests for Shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Tighten up comments prose
>  - Roman's review: more precise GC state check, more includes
>  - ... and 3 more: https://git.openjdk.org/jdk/compare/23fa1a33...d47b1665

I'm not sure why all these tests are run with Shenandoah (or any specific GC). The purpose of these unit tests is to check the Leak Profiler implementation, for example, that the object age is written correctly or that array information is serialized properly. It doesn't matter which GC, compiler, etc. is being used.

When the JFR tests were initially written, the purpose of the jtreg "jfr" tag was to filter out the JFR tests so they don't receive external flags. Since then, I think vm.flagless has been added. It may be more appropriate.

If the interaction with a certain GC needs to be tested, it's better to write a dedicated test for that, like TestG1.java and TestZ.java. If such a test doesn't work, it can be put on the ProblemList.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2456731236

From shade at openjdk.org  Tue Nov  5 10:11:30 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 5 Nov 2024 10:11:30 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v9]
In-Reply-To: <OpH0rr94M2CpLqqb0sFmj93_zosl5gKoZ6yU_jMoi1Q=.cbe241c3-cf28-4cb2-987e-66fb79a87b01@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>
 <OpH0rr94M2CpLqqb0sFmj93_zosl5gKoZ6yU_jMoi1Q=.cbe241c3-cf28-4cb2-987e-66fb79a87b01@github.com>
Message-ID: <fZqNhjd65vvDwq-DPn37epwg16H9U1avMrjyV5ksaO4=.a00ce81e-08e2-4a2e-820f-928f29b433a2@github.com>

On Tue, 5 Nov 2024 09:58:10 GMT, Erik Gahlin <egahlin at openjdk.org> wrote:

> I'm not sure why all these tests are run with Shenandoah (or any specific GC).

It is common to run tests with specific VM options overridden/amended for extensive testing. For example, `make test TEST=jdk_jfr TEST_VM_OPTS=-XX:+UseShenandoahGC`. This is why OpenJDK test suites generally accept `@requires` filters that can test if we are running in a particular configuration the test is not supposed to work in.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2456755843

From shade at openjdk.org  Tue Nov  5 10:28:32 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 5 Nov 2024 10:28:32 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v9]
In-Reply-To: <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>
Message-ID: <NKGoPfIweYusCGPkoH7oiHTXUmZSp5Hpjh_QZRb0KbI=.2b6d47de-4b78-4f33-bc71-711a62f2b74a@github.com>

On Mon, 4 Nov 2024 19:13:44 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [x] `jdk_jfr` pass by default
>>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits:
> 
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Disable Shenandoah-targeted test specially
>  - Redo the whole thing by disabling Leak Prof with Shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Just exclude the tests for Shenandoah
>  - Merge branch 'master' into JDK-8279016-jfr-leak-profiler-shenandoah
>  - Tighten up comments prose
>  - Roman's review: more precise GC state check, more includes
>  - ... and 3 more: https://git.openjdk.org/jdk/compare/23fa1a33...d47b1665

I can redo this for Shenandoah-specific problem lists, for sure. ZGC does GC-specific problem lists, Shenandoah can do some as well.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2456793615

From shade at openjdk.org  Tue Nov  5 11:10:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 5 Nov 2024 11:10:14 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v10]
In-Reply-To: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
Message-ID: <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>

> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
> 
> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
> 
> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
> 
> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
> 
> Additional testing:
>  - [x] `jdk_jfr` pass by default
>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Use Shenandoah ProblemLists

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20328/files
  - new: https://git.openjdk.org/jdk/pull/20328/files/d47b1665..2d1987b9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=09
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20328&range=08-09

  Stats: 115 lines in 34 files changed: 64 ins; 23 del; 28 mod
  Patch: https://git.openjdk.org/jdk/pull/20328.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20328/head:pull/20328

PR: https://git.openjdk.org/jdk/pull/20328

From shade at openjdk.org  Tue Nov  5 11:10:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 5 Nov 2024 11:10:14 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v9]
In-Reply-To: <NKGoPfIweYusCGPkoH7oiHTXUmZSp5Hpjh_QZRb0KbI=.2b6d47de-4b78-4f33-bc71-711a62f2b74a@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <PQbEdgJz8NGlAfUof-qZfFlQcjhfSaiSSinm99MLnGg=.67b1201f-e660-4a82-af7d-f36617a609e1@github.com>
 <NKGoPfIweYusCGPkoH7oiHTXUmZSp5Hpjh_QZRb0KbI=.2b6d47de-4b78-4f33-bc71-711a62f2b74a@github.com>
Message-ID: <cK1t2IW5ZAUBat24prRRXrz9mUHu2V1K3srk45Gqhkk=.d9868d79-e9f3-4e16-a619-55d1fa64ff5a@github.com>

On Tue, 5 Nov 2024 10:26:12 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> I can redo this for Shenandoah-specific problem lists, for sure. ZGC does GC-specific problem lists, Shenandoah can do some as well. But we will always have to remember to add new LeakProfiler tests there.

Did so, see new commit.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2456881779

From egahlin at openjdk.org  Tue Nov  5 11:42:30 2024
From: egahlin at openjdk.org (Erik Gahlin)
Date: Tue, 5 Nov 2024 11:42:30 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v10]
In-Reply-To: <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
Message-ID: <iuyeKxbePN9KpLEhZFdxUtk3pYVA3SjVNYZ5c07v8ak=.1234b33f-3ae4-41cd-ac70-cc53330ea0df@github.com>

On Tue, 5 Nov 2024 11:10:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [x] `jdk_jfr` pass by default
>>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use Shenandoah ProblemLists

Marked as reviewed by egahlin (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20328#pullrequestreview-2415386469

From stuefe at openjdk.org  Tue Nov  5 16:40:54 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Tue, 5 Nov 2024 16:40:54 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v53]
In-Reply-To: <LDg0GvC9lXkYPS90d9w8QfkTWlztUa5tlsgblGywCc4=.78ab2a17-2e28-4c27-b026-cd694ec02e21@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <LDg0GvC9lXkYPS90d9w8QfkTWlztUa5tlsgblGywCc4=.78ab2a17-2e28-4c27-b026-cd694ec02e21@github.com>
Message-ID: <5EgL-mJp75JLOxEccrrGVxbfS6QdUywRSfsOcgx4zl8=.3c283bf3-3e2e-4fe2-bce5-c30d7d4e2da4@github.com>

On Thu, 24 Oct 2024 21:04:51 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Enable riscv in CompressedClassPointersEncodingScheme test

Went again through all the changes, with focus on runtime code. Still good.

-------------

Marked as reviewed by stuefe (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2416155892

From rkennke at openjdk.org  Tue Nov  5 16:49:01 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 5 Nov 2024 16:49:01 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
 <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
Message-ID: <gRFjP3PNXLsbhcvc4ooX0bbLDWxThzosVQfz5u4EvwY=.72906fe2-2716-4078-888a-67f50bd9b726@github.com>

On Tue, 22 Oct 2024 16:22:20 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Update copyright
>>  - Avoid assert/endless-loop in JFR code
>
> @egahlin / @mgronlun could you please review the JFR parts of this PR? One change is for getting the right prototype header, the other is for avoiding an endless loop/assert in a corner case.

> @rkennke can you include this small update for s390x as well:
> 
> ```diff
> diff --git a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
> index 0f7e5c9f457..476e3d5daa4 100644
> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
> @@ -174,8 +174,11 @@ void C1_MacroAssembler::try_allocate(
>  void C1_MacroAssembler::initialize_header(Register obj, Register klass, Register len, Register Rzero, Register t1) {
>    assert_different_registers(obj, klass, len, t1, Rzero);
>    if (UseCompactObjectHeaders) {
> -    z_lg(t1, Address(klass, in_bytes(Klass::prototype_header_offset())));
> -    z_stg(t1, Address(obj, oopDesc::mark_offset_in_bytes()));
> +    z_mvc(
> +          Address(obj, oopDesc::mark_offset_in_bytes()),  /* move to */
> +          Address(klass, in_bytes(Klass::prototype_header_offset())), /* move from */
> +          sizeof(markWord) /* how much to move */
> +        );
>    } else {
>      load_const_optimized(t1, (intx)markWord::prototype().value());
>      z_stg(t1, Address(obj, oopDesc::mark_offset_in_bytes()));
> diff --git a/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp b/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
> index 378d5e4cfe1..c5713161bf9 100644
> --- a/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
> +++ b/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
> @@ -46,7 +46,7 @@ void C2_MacroAssembler::load_narrow_klass_compact_c2(Register dst, Address src)
>    // The incoming address is pointing into obj-start + klass_offset_in_bytes. We need to extract
>    // obj-start, so that we can load from the object's mark-word instead.
>    z_lg(dst, src.plus_disp(-oopDesc::klass_offset_in_bytes()));
> -  z_srlg(dst, dst, markWord::klass_shift); // TODO: could be z_sra
> +  z_srlg(dst, dst, markWord::klass_shift);
>  }
>  
>  //------------------------------------------------------
> diff --git a/src/hotspot/cpu/s390/templateTable_s390.cpp b/src/hotspot/cpu/s390/templateTable_s390.cpp
> index 3cb1aba810d..5b8f7a20478 100644
> --- a/src/hotspot/cpu/s390/templateTable_s390.cpp
> +++ b/src/hotspot/cpu/s390/templateTable_s390.cpp
> @@ -3980,8 +3980,11 @@ void TemplateTable::_new() {
>      // Initialize object header only.
>      __ bind(initialize_header);
>      if (UseCompactObjectHeaders) {
> -      __ z_lg(tmp, Address(iklass, in_bytes(Klass::prototype_header_offset())));
> -      __ z_stg(tmp, Address(RallocatedObject, oopDesc::mark_offset_in_bytes()));
> +      __ z_mvc(
> +            Address(RallocatedObject, oopDesc::mark_offset_in_bytes()), // move to
> +            Address(iklass, in_bytes(Klass::prototype_header_offset())), // move from
> +            sizeof(markWord) // how much to move
> +          );
>      } else {
>        __ store_const(Address(RallocatedObject, oopDesc::mark_offset_in_bytes()),
>                       (long) markWord::prototype().value());
> ```

Hi Amit,
sorry I only now get to reply to this, I have been traveling. What does the change do? Is it critical? Would it be possible to fix it after I intergrated the JEP? Because any change that I do now invalidates existing reviews, and might delay integration, and we're already running pretty close to RDP1. If at all possible, I would prefer to take it after I intergrated the JEP - we can have fixes well after RDP1, but not new features. If you agree, then please file a follow-up issue.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2457674486

From amitkumar at openjdk.org  Tue Nov  5 16:49:01 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Tue, 5 Nov 2024 16:49:01 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <gRFjP3PNXLsbhcvc4ooX0bbLDWxThzosVQfz5u4EvwY=.72906fe2-2716-4078-888a-67f50bd9b726@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
 <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
 <gRFjP3PNXLsbhcvc4ooX0bbLDWxThzosVQfz5u4EvwY=.72906fe2-2716-4078-888a-67f50bd9b726@github.com>
Message-ID: <UBbadM9MPxieW45wUnR96RxRW-egNTbBqOearMzO5Xo=.55e56cd3-35e6-4587-bd9e-c0ccd5190e1f@github.com>

On Tue, 5 Nov 2024 16:43:35 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>Hi Amit,
sorry I only now get to reply to this, I have been traveling. What does the change do? Is it critical? Would it be possible to fix it after I intergrated the JEP? Because any change that I do now invalidates existing reviews, and might delay integration, and we're already running pretty close to RDP1. If at all possible, I would prefer to take it after I intergrated the JEP - we can have fixes well after RDP1, but not new features. If you agree, then please file a follow-up issue.

That's perfectly fine. I will do it with separate RFE :-)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2457680086

From rkennke at openjdk.org  Tue Nov  5 17:35:30 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 5 Nov 2024 17:35:30 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v10]
In-Reply-To: <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
Message-ID: <zmylpfwvDWrLxFi72FigUuMGjyK4XLOQgjBNd533PTk=.8564a9e4-be66-4beb-b410-dc57958f600d@github.com>

On Tue, 5 Nov 2024 11:10:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [x] `jdk_jfr` pass by default
>>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use Shenandoah ProblemLists

Looks good to me, thanks!

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20328#pullrequestreview-2416296694

From rkennke at openjdk.org  Tue Nov  5 20:00:31 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 5 Nov 2024 20:00:31 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v54]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <icfqZVcRx3LUK8imw_WCGODbZID8SmHluf0PLA1jc_0=.dcc9389d-12d2-4edc-bce5-67339ae0d33e@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 104 commits:

 - Merge tag 'jdk-24+22' into JDK-8305895-v4
   
   Added tag jdk-24+22 for changeset 388d44fb
 - Enable riscv in CompressedClassPointersEncodingScheme test
 - s390 port
 - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
 - Update copyright
 - Avoid assert/endless-loop in JFR code
 - Update copyright headers
 - Merge tag 'jdk-24+20' into JDK-8305895-v4
   
   Added tag jdk-24+20 for changeset 7a64fbbb
 - Fix needle copying in indexOf intrinsic for smaller headers
 - Compact header riscv (#3)
   
   Implement compact headers on RISCV
   ---------
   
   Co-authored-by: hamlin <hamlin at rivosinc.com>
 - ... and 94 more: https://git.openjdk.org/jdk/compare/388d44fb...b945822a

-------------

Changes: https://git.openjdk.org/jdk/pull/20677/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=53
  Stats: 5214 lines in 218 files changed: 3587 ins; 864 del; 763 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From wkemper at openjdk.org  Tue Nov  5 23:53:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 5 Nov 2024 23:53:13 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <Ud_kMqN00A5s2_QlTbjgfmvZ5fwh03cTtZW4vEmiStY=.ab6d2b5c-95ab-4332-aaa6-a50ff0ddecaa@github.com>
References: <Ud_kMqN00A5s2_QlTbjgfmvZ5fwh03cTtZW4vEmiStY=.ab6d2b5c-95ab-4332-aaa6-a50ff0ddecaa@github.com>
Message-ID: <bvGEFFxeuOdztH_IFR_eKpBYcDzErpQA-28ziEdiKmg=.ddfc5519-b25d-41eb-b93e-bb4fa236b174@github.com>

> Merges tag jdk-24+22

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 125 commits:

 - Merge tag 'jdk-24+22' into merge-jdk-24+22
   
   Added tag jdk-24+22 for changeset 388d44fb
 - 8342156: C2: Compilation failure with fewer arguments after JDK-8329032
   
   Co-authored-by: Christian Hagedorn <chagedorn at openjdk.org>
   Reviewed-by: rcastanedalo, chagedorn, kvn
 - 8342642: Class loading failure due to archived map issue in ModuleLoaderMap.Mapper
   
   Reviewed-by: iklam, shade, alanb
 - 8342181: Update tests to use stronger Key and Salt size
   
   Reviewed-by: rhalade, ascarpino
 - 8337276: jcmd man page update for PID in output filenames
   
   Reviewed-by: cjplummer, szaldana
 - 8342635: javax/swing/JFileChooser/FileSystemView/WindowsDefaultIconSizeTest.java creates tmp file in src dir
   8342634: javax/imageio/plugins/wbmp/WBMPStreamTruncateTest.java creates temp file in src dir
   
   Reviewed-by: erikj, serb
 - 8341788: Fix ExceptionOccurred in hotspot
   
   Reviewed-by: dholmes
 - 8341927: Replace hardcoded security providers with new test.provider.name system property
   
   Reviewed-by: mullan, ascarpino, rhalade
 - 8343103: Enable debug logging for vmTestbase/nsk/jvmti/scenarios/sampling/SP05/sp05t003/TestDescription.java
   
   Reviewed-by: cjplummer
 - Merge remote-tracking branch 'origin/merge-latest' into merge-jdk-24+22
 - ... and 115 more: https://git.openjdk.org/shenandoah/compare/bbc2841c...42f15b62

-------------

Changes: https://git.openjdk.org/shenandoah/pull/531/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=531&range=01
  Stats: 166126 lines in 1352 files changed: 19140 ins; 143008 del; 3978 mod
  Patch: https://git.openjdk.org/shenandoah/pull/531.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/531/head:pull/531

PR: https://git.openjdk.org/shenandoah/pull/531

From wkemper at openjdk.org  Tue Nov  5 23:59:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 5 Nov 2024 23:59:35 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <30xsGR0ErdQXpuA4zCWPUe675JpRzxIAsZMwsNvkkCY=.086b0d90-509f-4bbe-9aaf-ff81223a743a@github.com>
References: <30xsGR0ErdQXpuA4zCWPUe675JpRzxIAsZMwsNvkkCY=.086b0d90-509f-4bbe-9aaf-ff81223a743a@github.com>
Message-ID: <10y9Sse1Ce23f-MRI3dewLYlkaYAg6oxHYZh50jR8DI=.3994cbb4-91bd-46ca-90fe-0126e2cf819a@github.com>

> Merges tag jdk-21.0.6+1

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 134 commits:

 - Merge branch 'master' into merge-jdk-21.0.6+1
 - 8342701: [PPC64] TestOSRLotsOfLocals.java crashes
   
   Backport-of: 3bba0f3dc8faf83a3aadcd704ae2ae4967e6daa4
 - 8334560: [PPC64]: postalloc_expand_java_dynamic_call_sched does not copy all fields
   
   Backport-of: 13dce296fc3924b269757ce1279c57afe18faeeb
 - 8328379: Convert URLDragTest.html applet test to main
   
   Backport-of: fbeac98c84078a566c572abeba07c49e94bbf26b
 - 8195675: Call to insertText with single character from custom Input Method ignored
   
   Backport-of: b8f2ec9091f9f7e5f4611991d04dd8aa113b94fd
 - 8325506: Ensure randomness is only read from provided SecureRandom object
   
   Reviewed-by: lucy
   Backport-of: b87d9cf2c9d905c15f4c957d42361b1a72974edf
 - 8320673: PageFormat/CustomPaper.java has no Pass/Fail buttons; multiple instructions
   
   Backport-of: 0ff2deab5d6b8228ddfed9ae08820d2adf2330c2
 - 8322754: click JComboBox when dialog about to close causes IllegalComponentStateException
   
   Backport-of: e44276989fc6358065412be7567d0141c84f1282
 - 8225220: When the Tab Policy is checked,the scroll button direction displayed incorrectly.
   
   Backport-of: ed5b8c3a7bb6de27ab5050db494b08d5e5dd1c44
 - 8316428: G1: Nmethod count statistics only count last code root set iterated
   
   Backport-of: fab372d3a23b17f64ae4306e28bdb0bc511f4912
 - ... and 124 more: https://git.openjdk.org/shenandoah-jdk21u/compare/d03588ad...2e0f974f

-------------

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/134/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=134&range=01
  Stats: 15747 lines in 326 files changed: 9595 ins; 5010 del; 1142 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/134.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/134/head:pull/134

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/134

From stuefe at openjdk.org  Wed Nov  6 09:13:47 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Wed, 6 Nov 2024 09:13:47 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v50]
In-Reply-To: <gRFjP3PNXLsbhcvc4ooX0bbLDWxThzosVQfz5u4EvwY=.72906fe2-2716-4078-888a-67f50bd9b726@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <F8pi_T7-BZh5Junuk22zJ5d9PnBRUs94urSreD5oH1c=.63e9a189-5d9a-45ca-a70c-53cf677627d9@github.com>
 <QirAyr5mX2p_jDtPINJsXF8Dv5p2Y_T5Dj2aTqEX-5I=.e1d5c692-81f8-4611-acde-eea1ad6313cd@github.com>
 <gRFjP3PNXLsbhcvc4ooX0bbLDWxThzosVQfz5u4EvwY=.72906fe2-2716-4078-888a-67f50bd9b726@github.com>
Message-ID: <W673dm9fAdXbGYJZ-HO68TcwCc_88Ufltd1K0NY_W9s=.f3cf778a-816c-4060-8d87-541ea970a708@github.com>

On Tue, 5 Nov 2024 16:43:35 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> @egahlin / @mgronlun could you please review the JFR parts of this PR? One change is for getting the right prototype header, the other is for avoiding an endless loop/assert in a corner case.
>
>> @rkennke can you include this small update for s390x as well:
>> 
>> ```diff
>> diff --git a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
>> index 0f7e5c9f457..476e3d5daa4 100644
>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp
>> @@ -174,8 +174,11 @@ void C1_MacroAssembler::try_allocate(
>>  void C1_MacroAssembler::initialize_header(Register obj, Register klass, Register len, Register Rzero, Register t1) {
>>    assert_different_registers(obj, klass, len, t1, Rzero);
>>    if (UseCompactObjectHeaders) {
>> -    z_lg(t1, Address(klass, in_bytes(Klass::prototype_header_offset())));
>> -    z_stg(t1, Address(obj, oopDesc::mark_offset_in_bytes()));
>> +    z_mvc(
>> +          Address(obj, oopDesc::mark_offset_in_bytes()),  /* move to */
>> +          Address(klass, in_bytes(Klass::prototype_header_offset())), /* move from */
>> +          sizeof(markWord) /* how much to move */
>> +        );
>>    } else {
>>      load_const_optimized(t1, (intx)markWord::prototype().value());
>>      z_stg(t1, Address(obj, oopDesc::mark_offset_in_bytes()));
>> diff --git a/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp b/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
>> index 378d5e4cfe1..c5713161bf9 100644
>> --- a/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
>> +++ b/src/hotspot/cpu/s390/c2_MacroAssembler_s390.cpp
>> @@ -46,7 +46,7 @@ void C2_MacroAssembler::load_narrow_klass_compact_c2(Register dst, Address src)
>>    // The incoming address is pointing into obj-start + klass_offset_in_bytes. We need to extract
>>    // obj-start, so that we can load from the object's mark-word instead.
>>    z_lg(dst, src.plus_disp(-oopDesc::klass_offset_in_bytes()));
>> -  z_srlg(dst, dst, markWord::klass_shift); // TODO: could be z_sra
>> +  z_srlg(dst, dst, markWord::klass_shift);
>>  }
>>  
>>  //------------------------------------------------------
>> diff --git a/src/hotspot/cpu/s390/templateTable_s390.cpp b/src/hotspot/cpu/s390/templateTable_s390.cpp
>> index 3cb1aba810d..5b8f7a20478 100644
>> --- a/src/hotspot/cpu/s390/templateTable_s390.cpp
>> +++ b/src/hotspot/cpu/s390/templateTable_s390.cpp
>> @@ -3980,8 +3980,11 @@ void TemplateTable::_new() {
>>      // Initialize object header only.
>>      __ bind(initialize_header);
>>      if (UseCompactObjectHeaders) {
>> -      __ z_lg(tmp, Address(iklass, in_bytes(Klass::prototype_header_offset())));
>> -      __ z_stg(tmp, Address(RallocatedObject, oo...

Merge is good. @rkennke patch for the new test errors due to removal of non-generational ZGC:
https://gist.github.com/tstuefe/321b769d3b281198b767b68e18bb7271

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2459069232

From rkennke at openjdk.org  Wed Nov  6 09:13:46 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 6 Nov 2024 09:13:46 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v55]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <nq6MmWoH2fhaSljcr9xEwuf5M1v6qEcSsqtFCA7CRD4=.0c837bd6-2f2d-4074-abec-29a72c4eab18@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Fix gen-ZGC removal

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/20677/files
  - new: https://git.openjdk.org/jdk/pull/20677/files/b945822a..1ea4de16

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=54
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=53-54

  Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From shade at openjdk.org  Wed Nov  6 15:11:34 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 6 Nov 2024 15:11:34 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v10]
In-Reply-To: <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
Message-ID: <5z-PvTLk9E_v6GctWI5eFQvqmWE8JzR067d9t2F7qVA=.6c3835ef-64f8-4dfa-8708-e159a7f96c45@github.com>

On Tue, 5 Nov 2024 11:10:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [x] `jdk_jfr` pass by default
>>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use Shenandoah ProblemLists

@mgronlun -- are you good with the current version?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2460016159

From mgronlun at openjdk.org  Wed Nov  6 16:29:34 2024
From: mgronlun at openjdk.org (Markus =?UTF-8?B?R3LDtm5sdW5k?=)
Date: Wed, 6 Nov 2024 16:29:34 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v10]
In-Reply-To: <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
Message-ID: <X0wVu4MDE0N1jR7sAnfPTrDZ2klCEVyZTvj3tCXBDXw=.338ceb16-d75e-45e9-a40c-ee4c56b72d05@github.com>

On Tue, 5 Nov 2024 11:10:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [x] `jdk_jfr` pass by default
>>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use Shenandoah ProblemLists

Marked as reviewed by mgronlun (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20328#pullrequestreview-2418848912

From mgronlun at openjdk.org  Wed Nov  6 16:29:35 2024
From: mgronlun at openjdk.org (Markus =?UTF-8?B?R3LDtm5sdW5k?=)
Date: Wed, 6 Nov 2024 16:29:35 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v10]
In-Reply-To: <X0wVu4MDE0N1jR7sAnfPTrDZ2klCEVyZTvj3tCXBDXw=.338ceb16-d75e-45e9-a40c-ee4c56b72d05@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
 <X0wVu4MDE0N1jR7sAnfPTrDZ2klCEVyZTvj3tCXBDXw=.338ceb16-d75e-45e9-a40c-ee4c56b72d05@github.com>
Message-ID: <46r2aomzO7Wl6pg-RWl4Xq3YbzM5mwWPuJetSxS11DY=.0b834520-ddec-4789-9d55-639f9679ba59@github.com>

On Wed, 6 Nov 2024 16:26:13 GMT, Markus Gr?nlund <mgronlun at openjdk.org> wrote:

>> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Use Shenandoah ProblemLists
>
> Marked as reviewed by mgronlun (Reviewer).

> @mgronlun -- are you good with the current version?

Ok. Thank you.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2460237237

From shade at openjdk.org  Wed Nov  6 16:41:37 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 6 Nov 2024 16:41:37 GMT
Subject: RFR: 8279016: JFR Leak Profiler is broken with Shenandoah [v10]
In-Reply-To: <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
 <Pakh_-zY0xVcnH6aDreNItT2pakp3tRMLoZ-X_c6f7w=.29811737-4fdc-47f6-a1ad-30cc287e1984@github.com>
Message-ID: <OXbDR-AiazvcQQ_to04wqaaK2fnw2nWXQSVkAo5_PGE=.1b0b9eac-389b-4064-9df6-8ba5094791b7@github.com>

On Tue, 5 Nov 2024 11:10:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
>> 
>> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
>> 
>> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
>> 
>> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
>> 
>> Additional testing:
>>  - [x] `jdk_jfr` pass by default
>>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use Shenandoah ProblemLists

Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20328#issuecomment-2460263678

From shade at openjdk.org  Wed Nov  6 16:41:38 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 6 Nov 2024 16:41:38 GMT
Subject: Integrated: 8279016: JFR Leak Profiler is broken with Shenandoah
In-Reply-To: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
References: <schCFz3_D2X157-vK6_OSAJGwXEyp9bycNYDvy6sOEM=.d988e2c3-d502-4cd2-94bc-69c810323df4@github.com>
Message-ID: <cRaVhHg8aamRxcqtRBT7Af0DIUyTd_1nnILpTuj_qHs=.b9f661a7-15a2-4a94-96e5-463de6021a34@github.com>

On Thu, 25 Jul 2024 11:26:45 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> While testing unrelated Shenandoah patch, I caught a GC assert when Leak Profiler was running ([JDK-8337194](https://bugs.openjdk.org/browse/JDK-8337194)). 
> 
> Leak Profiler is notorious in using the mark words for its own needs. We have been trying to mitigate its impact on GCs by moving to separate bitsets for tracking marked objects, or by treating "marked without fwdptr" as "JFR marked" and handling it. But this is not reliable, since things like [putting indexes in mark word](https://github.com/openjdk/jdk/blob/3baff2af6a30cc6cb2e0d4391db1cf7e61c33f64/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L275-L280) sneak in. This is okay for Leak Profiler alone, since it restores the mark words after the operation completes, but that is still not enough when GC is already running.
> 
> I say we side-step this whack-a-mole by cleanly bailing from JFR op, when we know it is unsafe to do. I thought to use `VM_Operation::doit_prologue`, but I think GC start may sneak in between checking in prologue and op start.
> 
> This realistically only affects Shenandoah. All other STW collectors would never see what Leak Profiler did with mark words. ZGC would not see it, since it does not care about mark words for its own operation.
> 
> Additional testing:
>  - [x] `jdk_jfr` pass by default
>  - [x] `jdk_jfr` now passes with `-XX:+UseShenandoah`

This pull request has now been integrated.

Changeset: 0be7118b
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/0be7118b2f761b416ebf8cbb11473d51e80be409
Stats:     80 lines in 4 files changed: 80 ins; 0 del; 0 mod

8279016: JFR Leak Profiler is broken with Shenandoah

Reviewed-by: egahlin, rkennke, mgronlun, wkemper

-------------

PR: https://git.openjdk.org/jdk/pull/20328

From wkemper at openjdk.org  Wed Nov  6 19:40:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 6 Nov 2024 19:40:45 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <30xsGR0ErdQXpuA4zCWPUe675JpRzxIAsZMwsNvkkCY=.086b0d90-509f-4bbe-9aaf-ff81223a743a@github.com>
References: <30xsGR0ErdQXpuA4zCWPUe675JpRzxIAsZMwsNvkkCY=.086b0d90-509f-4bbe-9aaf-ff81223a743a@github.com>
Message-ID: <c3J5NfDwcS8TqEUWpN6F0ubVnQKKcdRFA6aQAJlfZiI=.c5ffed2a-7c55-43ed-827a-fba51e42ebb0@github.com>

On Thu, 31 Oct 2024 14:20:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.6+1

This pull request has now been integrated.

Changeset: 916e6849
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/916e684911cc8d38718df4ccd1c9a772018e7c1e
Stats:     15747 lines in 326 files changed: 9595 ins; 5010 del; 1142 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/134

From wkemper at openjdk.org  Wed Nov  6 19:40:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 6 Nov 2024 19:40:31 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <Ud_kMqN00A5s2_QlTbjgfmvZ5fwh03cTtZW4vEmiStY=.ab6d2b5c-95ab-4332-aaa6-a50ff0ddecaa@github.com>
References: <Ud_kMqN00A5s2_QlTbjgfmvZ5fwh03cTtZW4vEmiStY=.ab6d2b5c-95ab-4332-aaa6-a50ff0ddecaa@github.com>
Message-ID: <mlH-9scYwwPok4vuNEngsvJsCo4jcYqLRrkInlFJEqQ=.8c39e612-45d6-4ae0-9506-0894c41703f3@github.com>

On Fri, 1 Nov 2024 14:12:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-24+22

This pull request has now been integrated.

Changeset: 20d13722
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/20d13722fbc56c94f03ac416bf7bf3139752de8e
Stats:     166126 lines in 1352 files changed: 19140 ins; 143008 del; 3978 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/531

From ssubramaniam at openjdk.org  Wed Nov  6 20:40:04 2024
From: ssubramaniam at openjdk.org (Satyen Subramaniam)
Date: Wed, 6 Nov 2024 20:40:04 GMT
Subject: Integrated: 8343649: Shenandoah: ShenandoahEvacInfo event does not
 follow JFR guidelines
Message-ID: <gNCieCicx3BUAFAttfCDCkk5aUxn5E4nYLa4EECdoRs=.fe2b0b80-775e-4f47-b37c-db280fb0ef2a@github.com>

Renaming ShenandoahEvacInfo event to ShenandoahEvacuationInformation (to comply with JFR guidelines) and adding test for event functionality.

Tested with tier 1, tier 2, and tier 3 tests.

-------------

Commit messages:
 - Adding ShenandoahEvacuationInformation Event Tests
 - Renaming ShenandoahEvacInfo to ShenandoahEvacuationInformation

Changes: https://git.openjdk.org/shenandoah/pull/533/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=533&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8343649
  Stats: 127 lines in 9 files changed: 114 ins; 0 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah/pull/533.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/533/head:pull/533

PR: https://git.openjdk.org/shenandoah/pull/533

From wkemper at openjdk.org  Wed Nov  6 20:40:04 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 6 Nov 2024 20:40:04 GMT
Subject: Integrated: 8343649: Shenandoah: ShenandoahEvacInfo event does not
 follow JFR guidelines
In-Reply-To: <gNCieCicx3BUAFAttfCDCkk5aUxn5E4nYLa4EECdoRs=.fe2b0b80-775e-4f47-b37c-db280fb0ef2a@github.com>
References: <gNCieCicx3BUAFAttfCDCkk5aUxn5E4nYLa4EECdoRs=.fe2b0b80-775e-4f47-b37c-db280fb0ef2a@github.com>
Message-ID: <1vPNjDJhNqPuP3whj7Hpy2NLzQGldFI5cr3fsybw9L4=.d30fa77a-00c3-424d-b795-7f0bf16f6291@github.com>

On Wed, 6 Nov 2024 17:52:10 GMT, Satyen Subramaniam <ssubramaniam at openjdk.org> wrote:

> Renaming ShenandoahEvacInfo event to ShenandoahEvacuationInformation (to comply with JFR guidelines) and adding test for event functionality.
> 
> Tested with tier 1, tier 2, and tier 3 tests.

Thank you.

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/533#pullrequestreview-2419363501

From ssubramaniam at openjdk.org  Wed Nov  6 20:40:05 2024
From: ssubramaniam at openjdk.org (Satyen Subramaniam)
Date: Wed, 6 Nov 2024 20:40:05 GMT
Subject: Integrated: 8343649: Shenandoah: ShenandoahEvacInfo event does not
 follow JFR guidelines
In-Reply-To: <gNCieCicx3BUAFAttfCDCkk5aUxn5E4nYLa4EECdoRs=.fe2b0b80-775e-4f47-b37c-db280fb0ef2a@github.com>
References: <gNCieCicx3BUAFAttfCDCkk5aUxn5E4nYLa4EECdoRs=.fe2b0b80-775e-4f47-b37c-db280fb0ef2a@github.com>
Message-ID: <EVZ3UBL-Kj6eRvg-lctN0xlZ6Bp7kxyFxRtmdaKqgl0=.627db40d-fcb3-4826-9506-fcc576c98098@github.com>

On Wed, 6 Nov 2024 17:52:10 GMT, Satyen Subramaniam <ssubramaniam at openjdk.org> wrote:

> Renaming ShenandoahEvacInfo event to ShenandoahEvacuationInformation (to comply with JFR guidelines) and adding test for event functionality.
> 
> Tested with tier 1, tier 2, and tier 3 tests.

This pull request has now been integrated.

Changeset: 2b505eeb
Author:    Satyen Subramaniam <ssubramaniam at openjdk.org>
Committer: William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/2b505eebe5006673f99c58922434f43aab4b7b77
Stats:     127 lines in 9 files changed: 114 ins; 0 del; 13 mod

8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/533

From wkemper at openjdk.org  Wed Nov  6 20:56:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 6 Nov 2024 20:56:35 GMT
Subject: Integrated: 8343649: Shenandoah: ShenandoahEvacInfo event does not
 follow JFR guidelines
Message-ID: <h9qtLylwkB9jEghIQnUNZNNuMYUrQQkYllp5FaIsgFA=.8fb77c58-9f41-4e04-82f7-b821003596a8@github.com>

Clean backport.

-------------

Commit messages:
 - Backport 2b505eebe5006673f99c58922434f43aab4b7b77

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/135/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=135&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8343649
  Stats: 127 lines in 9 files changed: 114 ins; 0 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/135.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/135/head:pull/135

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/135

From wkemper at openjdk.org  Wed Nov  6 20:56:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 6 Nov 2024 20:56:36 GMT
Subject: Integrated: 8343649: Shenandoah: ShenandoahEvacInfo event does not
 follow JFR guidelines
In-Reply-To: <h9qtLylwkB9jEghIQnUNZNNuMYUrQQkYllp5FaIsgFA=.8fb77c58-9f41-4e04-82f7-b821003596a8@github.com>
References: <h9qtLylwkB9jEghIQnUNZNNuMYUrQQkYllp5FaIsgFA=.8fb77c58-9f41-4e04-82f7-b821003596a8@github.com>
Message-ID: <g67v_5CGm5J4KIAdilEC0vcEXSZuLk4VC7xTYAS1XPE=.203cd14c-b516-4d25-b2c3-bd0227b81cca@github.com>

On Wed, 6 Nov 2024 20:49:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Clean backport.

This pull request has now been integrated.

Changeset: c0ea97ad
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/c0ea97ad77a1e80ae96cecb9b9ddb672f029b1f1
Stats:     127 lines in 9 files changed: 114 ins; 0 del; 13 mod

8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines

Backport-of: 2b505eebe5006673f99c58922434f43aab4b7b77

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/135

From syan at openjdk.org  Thu Nov  7 13:05:24 2024
From: syan at openjdk.org (SendaoYan)
Date: Thu, 7 Nov 2024 13:05:24 GMT
Subject: RFR: 8343742: Test TestShenandoah.java fails Could not find
 leaking object [v2]
In-Reply-To: <vRLmMn5I3U4qn8Iq0tWJwPxuiDxef1IFyOvtc2KSXbg=.6286a4e2-57c4-468b-a958-81550ec465f9@github.com>
References: <vRLmMn5I3U4qn8Iq0tWJwPxuiDxef1IFyOvtc2KSXbg=.6286a4e2-57c4-468b-a958-81550ec465f9@github.com>
Message-ID: <pJ2ajtcfaNvQqTvynf8uIT2PKPURdtzBQV8oVchUKZ0=.447c229e-9fd2-4616-affc-60f780b50b12@github.com>

> Hi all,
>  Test `jdk/jfr/event/oldobject/TestShenandoah.java` run timed out after [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016), because of infinite loop of `Could not find leaking object, retrying...`
> In [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016), this test has been Problemlisted by [ProblemList-shenandoah.txt
> ](https://github.com/openjdk/jdk/blame/master/test/jdk/ProblemList-shenandoah.txt#L52), because of the LeakProfiler tests not suitable for run with `-XX:+UseShenandoahGC` after [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016).
> But in the VM option `-XX:+UseShenandoahGC` set inside the test tag cause the Problemlist mechanism will not take effect. The problemlist only take effect when the VM option set from command line such as `make TEST JTREG="JAVA_OPTIONS=-XX:+UseShenandoahGC"`.
> So in this PR move Problemlist entry `jdk/jfr/event/oldobject/TestShenandoah.java` from `test/jdk/ProblemList-shenandoah.txt` to `test/jdk/ProblemList.txt` to make this test could by excluded normally.
> The other optional solution is delete the VM option `-XX:+UseShenandoahGC` in test file to make test run passed. After [JDK-8342951](https://bugs.openjdk.org/browse/JDK-8342951) has been resolved, the change should be backouted. This PR choice the previous one solution.
> 
> Test-fix only, no risk.

SendaoYan has updated the pull request incrementally with one additional commit since the last revision:

  Move Problemlist entry jdk/jfr/event/oldobject/TestShenandoah.java from test/jdk/ProblemList-shenandoah.txt to test/jdk/ProblemList.txt

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21951/files
  - new: https://git.openjdk.org/jdk/pull/21951/files/f54ed654..30a73e8a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21951&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21951&range=00-01

  Stats: 4 lines in 3 files changed: 1 ins; 1 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/21951.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21951/head:pull/21951

PR: https://git.openjdk.org/jdk/pull/21951

From shade at openjdk.org  Thu Nov  7 13:24:42 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 7 Nov 2024 13:24:42 GMT
Subject: RFR: 8343742: Test TestShenandoah.java fails Could not find
 leaking object [v2]
In-Reply-To: <pJ2ajtcfaNvQqTvynf8uIT2PKPURdtzBQV8oVchUKZ0=.447c229e-9fd2-4616-affc-60f780b50b12@github.com>
References: <vRLmMn5I3U4qn8Iq0tWJwPxuiDxef1IFyOvtc2KSXbg=.6286a4e2-57c4-468b-a958-81550ec465f9@github.com>
 <pJ2ajtcfaNvQqTvynf8uIT2PKPURdtzBQV8oVchUKZ0=.447c229e-9fd2-4616-affc-60f780b50b12@github.com>
Message-ID: <w4txW4TlSBRHALAGwZ5OR_4ey_gl1P4Z4_gozgVzXS4=.0d45cf1c-ac5f-4fac-9970-b30db1572542@github.com>

On Thu, 7 Nov 2024 13:05:24 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>>  Test `jdk/jfr/event/oldobject/TestShenandoah.java` run timed out after [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016), because of infinite loop of `Could not find leaking object, retrying...`
>> In [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016), this test has been Problemlisted by [ProblemList-shenandoah.txt
>> ](https://github.com/openjdk/jdk/blame/master/test/jdk/ProblemList-shenandoah.txt#L52), because of the LeakProfiler tests not suitable for run with `-XX:+UseShenandoahGC` after [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016).
>> But in the VM option `-XX:+UseShenandoahGC` set inside the test tag cause the Problemlist mechanism will not take effect. The problemlist only take effect when the VM option set from command line such as `make TEST JTREG="JAVA_OPTIONS=-XX:+UseShenandoahGC"`.
>> So in this PR move Problemlist entry `jdk/jfr/event/oldobject/TestShenandoah.java` from `test/jdk/ProblemList-shenandoah.txt` to `test/jdk/ProblemList.txt` to make this test could by excluded normally.
>> The other optional solution is delete the VM option `-XX:+UseShenandoahGC` in test file to make test run passed. After [JDK-8342951](https://bugs.openjdk.org/browse/JDK-8342951) has been resolved, the change should be backouted. This PR choice the previous one solution.
>> 
>> Test-fix only, no risk.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Move Problemlist entry jdk/jfr/event/oldobject/TestShenandoah.java from test/jdk/ProblemList-shenandoah.txt to test/jdk/ProblemList.txt

Already here: https://github.com/openjdk/jdk/pull/21947

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21951#issuecomment-2462228896

From wkemper at openjdk.org  Thu Nov  7 14:25:52 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 7 Nov 2024 14:25:52 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <9L6OzvZLrJmoj11yH3cSSrQc7Y0HaNADZA7r_9pe_1c=.9e1c7a0b-0efd-4269-b576-f0e751b4e919@github.com>

Merges tag jdk-21.0.6+2

-------------

Commit messages:
 - 8342681: TestLoadBypassesNullCheck.java fails improperly specified VM option
 - 8341235: Improve default instruction frame title in PassFailJFrame
 - 8340799: Add border inside instruction frame in PassFailJFrame
 - 8340812: LambdaForm customization via MethodHandle::updateForm is not thread safe
 - 8339803: Acknowledge case insensitive unambiguous keywords in tzdata files
 - 8306446: java/lang/management/ThreadMXBean/Locks.java transient failures
 - 8333728: ubsan: shenandoahFreeSet.cpp:1347:24: runtime error: division by zero
 - 8342496: C2/Shenandoah: SEGV in compiled code when running jcstress
 - 8340214: C2 compilation asserts with "no node with a side effect" in PhaseIdealLoop::try_sink_out_of_loop
 - 8337066: Repeated call of StringBuffer.reverse with double byte string returns wrong result
 - ... and 27 more: https://git.openjdk.org/shenandoah-jdk21u/compare/7dc0f7a6...bb60a427

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=136&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/136/files
  Stats: 12428 lines in 75 files changed: 11328 ins; 720 del; 380 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/136.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/136/head:pull/136

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/136

From syan at openjdk.org  Thu Nov  7 15:54:48 2024
From: syan at openjdk.org (SendaoYan)
Date: Thu, 7 Nov 2024 15:54:48 GMT
Subject: RFR: 8343742: Test TestShenandoah.java fails Could not find
 leaking object [v2]
In-Reply-To: <pJ2ajtcfaNvQqTvynf8uIT2PKPURdtzBQV8oVchUKZ0=.447c229e-9fd2-4616-affc-60f780b50b12@github.com>
References: <vRLmMn5I3U4qn8Iq0tWJwPxuiDxef1IFyOvtc2KSXbg=.6286a4e2-57c4-468b-a958-81550ec465f9@github.com>
 <pJ2ajtcfaNvQqTvynf8uIT2PKPURdtzBQV8oVchUKZ0=.447c229e-9fd2-4616-affc-60f780b50b12@github.com>
Message-ID: <t-p3XwiCjpdt_3A0RDsQH2pD87DrPNNc9QFZ5i7DsXM=.7b09aa3b-a492-4873-b826-6bae09a8ed9f@github.com>

On Thu, 7 Nov 2024 13:05:24 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>>  Test `jdk/jfr/event/oldobject/TestShenandoah.java` run timed out after [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016), because of infinite loop of `Could not find leaking object, retrying...`
>> In [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016), this test has been Problemlisted by [ProblemList-shenandoah.txt
>> ](https://github.com/openjdk/jdk/blame/master/test/jdk/ProblemList-shenandoah.txt#L52), because of the LeakProfiler tests not suitable for run with `-XX:+UseShenandoahGC` after [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016).
>> But in the VM option `-XX:+UseShenandoahGC` set inside the test tag cause the Problemlist mechanism will not take effect. The problemlist only take effect when the VM option set from command line such as `make TEST JTREG="JAVA_OPTIONS=-XX:+UseShenandoahGC"`.
>> So in this PR move Problemlist entry `jdk/jfr/event/oldobject/TestShenandoah.java` from `test/jdk/ProblemList-shenandoah.txt` to `test/jdk/ProblemList.txt` to make this test could by excluded normally.
>> The other optional solution is delete the VM option `-XX:+UseShenandoahGC` in test file to make test run passed. After [JDK-8342951](https://bugs.openjdk.org/browse/JDK-8342951) has been resolved, the change should be backouted. This PR choice the previous one solution.
>> 
>> Test-fix only, no risk.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Move Problemlist entry jdk/jfr/event/oldobject/TestShenandoah.java from test/jdk/ProblemList-shenandoah.txt to test/jdk/ProblemList.txt

> Already here: #21947

Close as duplicated

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21951#issuecomment-2462586477

From syan at openjdk.org  Thu Nov  7 15:54:49 2024
From: syan at openjdk.org (SendaoYan)
Date: Thu, 7 Nov 2024 15:54:49 GMT
Subject: Withdrawn: 8343742: Test TestShenandoah.java fails Could not find
 leaking object
In-Reply-To: <vRLmMn5I3U4qn8Iq0tWJwPxuiDxef1IFyOvtc2KSXbg=.6286a4e2-57c4-468b-a958-81550ec465f9@github.com>
References: <vRLmMn5I3U4qn8Iq0tWJwPxuiDxef1IFyOvtc2KSXbg=.6286a4e2-57c4-468b-a958-81550ec465f9@github.com>
Message-ID: <g_NwpGN5yxeVOzXHi4dpcdTwPNSYZzIXBK8ikRdSnhY=.3c39ac02-8c17-4c4b-b594-b9274ebd18f6@github.com>

On Thu, 7 Nov 2024 12:40:15 GMT, SendaoYan <syan at openjdk.org> wrote:

> Hi all,
>  Test `jdk/jfr/event/oldobject/TestShenandoah.java` run timed out after [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016), because of infinite loop of `Could not find leaking object, retrying...`
> In [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016), this test has been Problemlisted by [ProblemList-shenandoah.txt
> ](https://github.com/openjdk/jdk/blame/master/test/jdk/ProblemList-shenandoah.txt#L52), because of the LeakProfiler tests not suitable for run with `-XX:+UseShenandoahGC` after [JDK-8279016](https://bugs.openjdk.org/browse/JDK-8279016).
> But in the VM option `-XX:+UseShenandoahGC` set inside the test tag cause the Problemlist mechanism will not take effect. The problemlist only take effect when the VM option set from command line such as `make TEST JTREG="JAVA_OPTIONS=-XX:+UseShenandoahGC"`.
> So in this PR move Problemlist entry `jdk/jfr/event/oldobject/TestShenandoah.java` from `test/jdk/ProblemList-shenandoah.txt` to `test/jdk/ProblemList.txt` to make this test could by excluded normally.
> The other optional solution is delete the VM option `-XX:+UseShenandoahGC` in test file to make test run passed. After [JDK-8342951](https://bugs.openjdk.org/browse/JDK-8342951) has been resolved, the change should be backouted. This PR choice the previous one solution.
> 
> Test-fix only, no risk.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/21951

From rkennke at openjdk.org  Thu Nov  7 16:58:36 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 7 Nov 2024 16:58:36 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v56]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <lJKBNSVsH-5V7iNUrTmNaY2-GhzE8j8tycQ-7dfQzMA=.831c66f6-2635-4b65-9ec2-7b7d740499cb@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 106 commits:

 - Merge tag 'jdk-25+23' into JDK-8305895-v4
   
   Added tag jdk-24+23 for changeset c0e6c3b9
 - Fix gen-ZGC removal
 - Merge tag 'jdk-24+22' into JDK-8305895-v4
   
   Added tag jdk-24+22 for changeset 388d44fb
 - Enable riscv in CompressedClassPointersEncodingScheme test
 - s390 port
 - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
 - Update copyright
 - Avoid assert/endless-loop in JFR code
 - Update copyright headers
 - Merge tag 'jdk-24+20' into JDK-8305895-v4
   
   Added tag jdk-24+20 for changeset 7a64fbbb
 - ... and 96 more: https://git.openjdk.org/jdk/compare/c0e6c3b9...4d282247

-------------

Changes: https://git.openjdk.org/jdk/pull/20677/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=55
  Stats: 5212 lines in 218 files changed: 3585 ins; 864 del; 763 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Thu Nov  7 17:25:40 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 7 Nov 2024 17:25:40 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:

 - Merge branch 'master' into JDK-8305895-v4
 - Merge tag 'jdk-25+23' into JDK-8305895-v4
   
   Added tag jdk-24+23 for changeset c0e6c3b9
 - Fix gen-ZGC removal
 - Merge tag 'jdk-24+22' into JDK-8305895-v4
   
   Added tag jdk-24+22 for changeset 388d44fb
 - Enable riscv in CompressedClassPointersEncodingScheme test
 - s390 port
 - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
 - Update copyright
 - Avoid assert/endless-loop in JFR code
 - Update copyright headers
 - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

-------------

Changes: https://git.openjdk.org/jdk/pull/20677/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20677&range=56
  Stats: 5212 lines in 218 files changed: 3585 ins; 864 del; 763 mod
  Patch: https://git.openjdk.org/jdk/pull/20677.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20677/head:pull/20677

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Thu Nov  7 17:33:11 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 7 Nov 2024 17:33:11 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <B6rd8C0LEZTiWWNJdT_NtFxZZRxc25iKvQ8ik7ajNpc=.ca812f0e-c6b3-42b4-99ed-fedfd4af02a0@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

I'd like to prepare for integration now. I merged from master and resolved some conflicts. I am now running at least tier1 on aarch64 x x86_64 x -UCOH x +UCOH, possibly tier2 .. 4, too (time permitting). In the meantime, could you please re-approve the PR? I hope it doesn't catch any more conflicts until we're ready for intergration.
As soon as the JEP is targeted (sometime today, I think), tests are clean and approvals are there, I would like to integrate.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2462834035

From coleenp at openjdk.org  Thu Nov  7 17:46:11 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Thu, 7 Nov 2024 17:46:11 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <qD67yqKQH8BEsRN81lCEF-7MTL1QZ6jdgaoE84Uz6UQ=.13bc8c86-2b8d-4b9e-a03a-7cc71b704c96@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

Reapproving.  Please wait for GHA to complete, when JEP is targeted to integrate.  Thanks!

-------------

Marked as reviewed by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2421741026

From stefank at openjdk.org  Thu Nov  7 17:53:10 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 7 Nov 2024 17:53:10 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <EozZCSlj-uiC3afO_V4tozAqODxo7nXdO8khiZkO5y0=.c287c3d3-714a-431e-8f35-e31edf6e58c8@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

Marked as reviewed by stefank (Reviewer).

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2417620293
PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2421753879

From rkennke at openjdk.org  Thu Nov  7 21:27:09 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 7 Nov 2024 21:27:09 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <2xoAD2r5G_6IHT9gt8-uSkN_hPiRmIkJ6VhkB1GarfI=.4e3c65db-3aab-4926-b1fc-fc78599b2885@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

GHA failures look like one unrelated timeout and one unrelated infra problem. Please confirm.
I also run tier1 on x86_64 x aarch64 x -UCOH x + UCOH, with nothing sticking out (same timeout observed, though).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2463245179

From stuefe at openjdk.org  Fri Nov  8 07:02:51 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 8 Nov 2024 07:02:51 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <JbXIqq9xSUb8Nr7074DIqLmaBvqorTolJI6tLp-nNj0=.056e5beb-2d10-4a80-9ef1-170725dd38c7@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

Merge looks good. build errors on MacOS unrelated.

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2422830379

From wkemper at openjdk.org  Fri Nov  8 14:16:56 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 8 Nov 2024 14:16:56 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <0hRViAZGsn_y1-Zmr29ALAKpqibVo8ZBc0B7wcBjaeg=.40ce2420-ae03-4ca3-9013-851b64cc4384@github.com>

Merges tag jdk-24+23

-------------

Commit messages:
 - 8343214: Fix encoding errors in APX New Data Destination Instructions Support
 - 8279016: JFR Leak Profiler is broken with Shenandoah
 - 8341408: Implement JEP 488: Primitive Types in Patterns, instanceof, and switch (Second Preview)
 - 8341834: C2 compilation fails with "bad AD file" due to Replicate
 - 8343068: C2: CastX2P Ideal transformation not always applied
 - 8339303: C2: dead node after failing to match cloned address expression
 - 8331341: secondary_super_cache does not scale well: C1 and interpreter
 - 8318442: java/net/httpclient/ManyRequests2.java fails intermittently on Linux
 - 8343502: RISC-V: SIGBUS in updateBytesCRC32 after JDK-8339738
 - 8342943: Replace predicate walking and cloning code for main/post loops with a predicate visitor
 - ... and 89 more: https://git.openjdk.org/shenandoah/compare/388d44fb...c0e6c3b9

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/534/files
  Stats: 135556 lines in 783 files changed: 105964 ins; 10455 del; 19137 mod
  Patch: https://git.openjdk.org/shenandoah/pull/534.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/534/head:pull/534

PR: https://git.openjdk.org/shenandoah/pull/534

From stuefe at openjdk.org  Fri Nov  8 16:10:56 2024
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 8 Nov 2024 16:10:56 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <jP_Idrs6KSyEkPU5YQi2wyKtnxsoaR6VGKgcl0ArpKo=.89ce2f58-aa31-4c51-9dbe-b2ba12be0c27@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

Marked as reviewed by stuefe (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2424199289

From phh at openjdk.org  Fri Nov  8 16:15:14 2024
From: phh at openjdk.org (Paul Hohensee)
Date: Fri, 8 Nov 2024 16:15:14 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <hzcN_BjnNSq3yHBbR3VuFp0sESFxrE4mGqgTvqG3M5U=.6c701fee-5289-49e1-880a-5dd804451333@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

Marked as reviewed by phh (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2424210008

From stefank at openjdk.org  Fri Nov  8 16:26:28 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 8 Nov 2024 16:26:28 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <VbbkfAVctCeJOW5If5yoDTGaAy1mz9uiLFzVauEzotM=.c7e03057-b5a5-4dc1-b99a-700066dd6fa4@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2424260100

From coleenp at openjdk.org  Fri Nov  8 16:26:28 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Fri, 8 Nov 2024 16:26:28 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <r4b0USRHo5knfhNqR0drNETEjAfTBZjmL-fXikTeWfc=.93a5a8a8-5762-4236-8b63-9807830f4042@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

Still looks good.  Nice work!

-------------

Marked as reviewed by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20677#pullrequestreview-2424274474

From rkennke at openjdk.org  Fri Nov  8 17:24:05 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 8 Nov 2024 17:24:05 GMT
Subject: Integrated: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental)
In-Reply-To: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
Message-ID: <JdMfVI6US88mJIogR2QGQqMzY55iUjzbTGbs2WqNppg=.97f680a3-1e31-4f7c-b8d0-b7955ab7a01f@github.com>

On Thu, 22 Aug 2024 13:35:08 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> This is the main body of the JEP 450: Compact Object Headers (Experimental).
> 
> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
> 
> Main changes:
>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>  - Arrays will now store their length at offset 8.
>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _coh variants of CDS archiv...

This pull request has now been integrated.

Changeset: 44ec501a
Author:    Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/44ec501a41f4794259dd03cd168838e79334890e
Stats:     5212 lines in 218 files changed: 3585 ins; 864 del; 763 mod

8305895: Implement JEP 450: Compact Object Headers (Experimental)

Co-authored-by: Sandhya Viswanathan <sviswanathan at openjdk.org>
Co-authored-by: Martin Doerr <mdoerr at openjdk.org>
Co-authored-by: Hamlin Li <mli at openjdk.org>
Co-authored-by: Thomas Stuefe <stuefe at openjdk.org>
Co-authored-by: Amit Kumar <amitkumar at openjdk.org>
Co-authored-by: Stefan Karlsson <stefank at openjdk.org>
Co-authored-by: Coleen Phillimore <coleenp at openjdk.org>
Co-authored-by: Axel Boldt-Christmas <aboldtch at openjdk.org>
Reviewed-by: coleenp, stefank, stuefe, phh, ihse, lmesnik, tschatzl, matsaave, rcastanedalo, vpaprotski, yzheng, egahlin

-------------

PR: https://git.openjdk.org/jdk/pull/20677

From rkennke at openjdk.org  Fri Nov  8 17:45:40 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 8 Nov 2024 17:45:40 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
Message-ID: <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>

On Wed, 18 Sep 2024 12:22:34 GMT, Yudi Zheng <yzheng at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - CompressedKlassPointers::is_encodable shall be callable with -UseCCP
>>  - Johan review feedback
>
> Could you please cherry pick https://github.com/mur47x111/jdk/commit/c45ebc2a89d0b25a3dd8cc46386e37a635ff9af2 for the JVMCI support?

@mur47x111 it's now intergrated in jdk24. do your magic in Graal ;-)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2465413222

From yzheng at openjdk.org  Fri Nov  8 17:52:05 2024
From: yzheng at openjdk.org (Yudi Zheng)
Date: Fri, 8 Nov 2024 17:52:05 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
Message-ID: <iKhNQYZW_syT0KHfABZHufGjwiG7YcCsLdpq5SdNZf4=.8ec52749-5884-4e96-9afa-0e1d710ef2ea@github.com>

On Fri, 8 Nov 2024 17:42:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Could you please cherry pick https://github.com/mur47x111/jdk/commit/c45ebc2a89d0b25a3dd8cc46386e37a635ff9af2 for the JVMCI support?
>
> @mur47x111 it's now intergrated in jdk24. do your magic in Graal ;-)

@rkennke It is in the merge queue

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2465423342

From wkemper at openjdk.org  Mon Nov 11 16:58:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 11 Nov 2024 16:58:46 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <0hRViAZGsn_y1-Zmr29ALAKpqibVo8ZBc0B7wcBjaeg=.40ce2420-ae03-4ca3-9013-851b64cc4384@github.com>
References: <0hRViAZGsn_y1-Zmr29ALAKpqibVo8ZBc0B7wcBjaeg=.40ce2420-ae03-4ca3-9013-851b64cc4384@github.com>
Message-ID: <DZqzUpnTYDEl7C6nbT_23QpwXpZEE0FxW2gKVYgVdPo=.0612f4ca-bacd-4a48-89a4-e6cebe9ffc34@github.com>

> Merges tag jdk-24+23

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/534/files
  - new: https://git.openjdk.org/shenandoah/pull/534/files/c0e6c3b9..c0e6c3b9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=534&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=534&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/534.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/534/head:pull/534

PR: https://git.openjdk.org/shenandoah/pull/534

From wkemper at openjdk.org  Mon Nov 11 17:02:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 11 Nov 2024 17:02:03 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <0hRViAZGsn_y1-Zmr29ALAKpqibVo8ZBc0B7wcBjaeg=.40ce2420-ae03-4ca3-9013-851b64cc4384@github.com>
References: <0hRViAZGsn_y1-Zmr29ALAKpqibVo8ZBc0B7wcBjaeg=.40ce2420-ae03-4ca3-9013-851b64cc4384@github.com>
Message-ID: <3WX00JwD98JzHHOnnTVph-691A9QhkmvAEIQqHt89iI=.c55fe659-3b71-4346-865d-a3d5b6a2fe99@github.com>

On Fri, 8 Nov 2024 14:11:49 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-24+23

This pull request has now been integrated.

Changeset: 3825ae3e
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/3825ae3e91ce3b8e095c50569140064a416e837c
Stats:     135556 lines in 783 files changed: 105964 ins; 10455 del; 19137 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/534

From wkemper at openjdk.org  Mon Nov 11 17:03:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 11 Nov 2024 17:03:54 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <9L6OzvZLrJmoj11yH3cSSrQc7Y0HaNADZA7r_9pe_1c=.9e1c7a0b-0efd-4269-b576-f0e751b4e919@github.com>
References: <9L6OzvZLrJmoj11yH3cSSrQc7Y0HaNADZA7r_9pe_1c=.9e1c7a0b-0efd-4269-b576-f0e751b4e919@github.com>
Message-ID: <souRqhl-oPIVaoLLqlPs8IuozMNk-xVs1wiiYKgXyRw=.f86f8a3e-cf90-444c-80a0-81fbd0fc3a9c@github.com>

> Merges tag jdk-21.0.6+2

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 38 commits:

 - Merge branch 'master' into merge-jdk-21.0.6+2
 - 8342681: TestLoadBypassesNullCheck.java fails improperly specified VM option
   
   Backport-of: 8bcd4920f1b03d0ef8e295e53557c629f05ceaa4
 - 8341235: Improve default instruction frame title in PassFailJFrame
   
   Backport-of: 4ba170c403ae85576f84dafd4a157ba0db99873f
 - 8340799: Add border inside instruction frame in PassFailJFrame
   
   Backport-of: 520060f79a3cedb8f93e6bbd0e9b2823eaabf79a
 - 8340812: LambdaForm customization via MethodHandle::updateForm is not thread safe
   
   Backport-of: 47c10694c66bc131c8a5e1572340415b8daaba08
 - 8339803: Acknowledge case insensitive unambiguous keywords in tzdata files
   
   Reviewed-by: mbaesken
   Backport-of: 35a94b769761bd923fe6db03be672f05c1a74c38
 - 8306446: java/lang/management/ThreadMXBean/Locks.java transient failures
   
   Backport-of: 4ae5a3e39b681bfd001df1483d8a6d1fce0bc7f8
 - 8333728: ubsan: shenandoahFreeSet.cpp:1347:24: runtime error: division by zero
   
   Reviewed-by: mdoerr, lucy
   Backport-of: b5575942027281166676678e2081b024ec572644
 - 8342496: C2/Shenandoah: SEGV in compiled code when running jcstress
   
   Backport-of: 680dc5d896f4f7b01b3cf800d548e32bb2ef8c81
 - 8340214: C2 compilation asserts with "no node with a side effect" in PhaseIdealLoop::try_sink_out_of_loop
   
   Backport-of: ff2f39f24018436556a8956ec55da433dc697437
 - ... and 28 more: https://git.openjdk.org/shenandoah-jdk21u/compare/c0ea97ad...c1187db1

-------------

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/136/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=136&range=01
  Stats: 12425 lines in 74 files changed: 11328 ins; 718 del; 379 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/136.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/136/head:pull/136

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/136

From ysr at openjdk.org  Tue Nov 12 02:34:43 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 12 Nov 2024 02:34:43 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v3]
In-Reply-To: <b5jJoMTKoxqyNgyMDdLlIg1pr1C1pQMdd-S7fyVHIg0=.5b1ab26d-295c-4935-bac9-c77a7f260884@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pLB45ckLnKHk0gDs4o_MV-7E2C2sogJOerr5Vu84CTI=.b9c439de-980d-42cd-aeef-575744f3c414@github.com>
 <GWpGtSgoRD5u1L7TwA7URqGqlBPMsdQH1_tYFR6uzCY=.309147f4-194a-47eb-882f-c7bfcfcbaf00@github.com>
 <b5jJoMTKoxqyNgyMDdLlIg1pr1C1pQMdd-S7fyVHIg0=.5b1ab26d-295c-4935-bac9-c77a7f260884@github.com>
Message-ID: <tH9PShOmA75rMQWwojEZq8kWHFJIHBmru9WkHERGdjQ=.59f89619-4997-4eeb-874d-f11337a5ce8d@github.com>

On Tue, 22 Oct 2024 17:12:38 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Congratulations! Good to see the great progress. I just built this PR for some testing and found something unexpected. I ran the genshen VS shenandoah(default) with jbb2015 on aarch64 N2 with 8 cores and Xmx8g. The critical-jOPS of genshen(5373) is behind shenandoah(6027). Do I miss something on the options?
>> ```java -Xmx8g -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational -Xlog:gc*  -XX:MetaspaceSize=1g   -jar specjbb2015.jar -m COMPOSITE```
>
>> @mmyxym , Thank you for testing this out! I apologize for not responding to your comment sooner. We run specjbb2015 regularly in our integration pipeline. We see a slight improvement with the generational mode; certainly no regression:
>> 
>> ```
>> Shen: RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3620, critical-jOPS = 2375
>> Gen:  RUN RESULT: hbIR (max attempted) = 3934, hbIR (settled) = 3295, max-jOPS = 4013, critical-jOPS = 2470
>> Shen: RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3667, critical-jOPS = 2397
>> Gen:  RUN RESULT: hbIR (max attempted) = 4701, hbIR (settled) = 3934, max-jOPS = 3996, critical-jOPS = 2414
>> ```
>> 
>> These results were produced with these arguments:
>> 
>> ```
>> -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:+UnlockDiagnosticV
>> MOptions -XX:-TieredCompilation -XX:-ShenandoahPacing -XX:+AlwaysPreTouch -XX:+DisableExplicitGC -Xmx10g -Xms10g -XX:ShenandoahGCMode=generational
>> ```
>> 
>> These runs executed on a Graviton2 host with 4 cores, 16G. I'll run again on a host with more cores and with your exact command line parameters.
> 
> What's the reason to disable tiered compilation?

@zhengyu123 and @shipilev : could you please take another look and leave your review feedback and/or approval? We'd like to land the changes this week if possible. Currently internal testing is in progress after merging with the latest changes from tip. Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2469466977

From kdnilsen at openjdk.org  Tue Nov 12 17:32:00 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 12 Nov 2024 17:32:00 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <m0nT9Z9rWxv035VXKArefrumnP31iQF8YAke5XLF1y8=.7b7b8328-5dce-40d1-9ea0-2badd8aeac3a@github.com>

On Mon, 11 Nov 2024 17:31:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 555:

> 553:     while (region != nullptr) {
> 554:       {
> 555:         ShenandoahHeapLocker locker(heap->lock());

Was it a bug that previous version of this code did not acquire the heap lock?

Is the lock required for the entirety of time that we are clearing the bitmap?  Or is it just required to get a trustworthy check on is_bitmap_slice_committed()?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1837047649

From wkemper at openjdk.org  Tue Nov 12 17:32:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Nov 2024 17:32:00 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread
Message-ID: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>

Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

-------------

Commit messages:
 - Check for safepoint when stopping (stopping thread is java thread)
 - Fix ridiculous typo
 - Merge remote-tracking branch 'jdk/master' into shen-uncommit-thread
 - Fix shutdown protocol
 - Take heap lock when uncommitting bitmaps, uncommit thread joins STS.
 - Little bit of cleanup
 - WIP: checkpoint before sync up
 - WIP: checkpoint

Changes: https://git.openjdk.org/jdk/pull/22019/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342444
  Stats: 319 lines in 6 files changed: 229 ins; 74 del; 16 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From wkemper at openjdk.org  Tue Nov 12 17:32:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Nov 2024 17:32:01 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread
In-Reply-To: <seTWtGW6jGmYoZhVAChkayjal9FEEI-94YJC5IJF3Yg=.4978ecfc-4bad-4df5-b7c3-a3473a4dcf52@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <m0nT9Z9rWxv035VXKArefrumnP31iQF8YAke5XLF1y8=.7b7b8328-5dce-40d1-9ea0-2badd8aeac3a@github.com>
 <seTWtGW6jGmYoZhVAChkayjal9FEEI-94YJC5IJF3Yg=.4978ecfc-4bad-4df5-b7c3-a3473a4dcf52@github.com>
Message-ID: <dLO84w60iGOGN53ams38IG8QHsiS2LJv7kStQQ_2jhE=.e3177cfd-8e38-40b8-bc41-d49ef134943b@github.com>

On Mon, 11 Nov 2024 18:26:29 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 555:
>> 
>>> 553:     while (region != nullptr) {
>>> 554:       {
>>> 555:         ShenandoahHeapLocker locker(heap->lock());
>> 
>> Was it a bug that previous version of this code did not acquire the heap lock?
>> 
>> Is the lock required for the entirety of time that we are clearing the bitmap?  Or is it just required to get a trustworthy check on is_bitmap_slice_committed()?
>
> After reading more of this PR, I believe we need the heap lock to get a reliable signal of bitmap_slice_committed().  But I believe we do not need the heap lock for ctx->clear_bitmap(region) so would prefer to move that outside the lock, unless I am misunderstanding.

Hmm, I'm not sure we can do that. Prior to this change, the control thread performed both clearing the bitmap and uncommitting the region's bitmap, so they could never happen concurrently. With this change, a separate thread could perform the uncommit. Consider:

1. Control thread takes heap lock, observes that bitmap slice for region A is committed
2. Control thread releases heap lock, begins clearing bitmap (writing zeros to bitmap slice)
3. Uncommit thread takes heap lock, believes it must uncommit region A
4. Uncommit thread uncommits bitmap slice for region A
5. Segfault in Control Thread

I do _believe_ if we had a per region lock, it would be useful here. Holding a lock over the entire heap for this feels like overkill. Or, we could schedule the uncommit so that it does not occur during a GC cycle.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1837215468

From wkemper at openjdk.org  Tue Nov 12 17:32:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Nov 2024 17:32:00 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <zWGMtaqfZd7MJecqwAWQV4ePtAYI3zYLlVmZabprS4I=.294083be-9598-4646-8666-c9e2557192da@github.com>

On Mon, 11 Nov 2024 17:31:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

I modified the testing pipelines to set `-Xms4g -Xmx10g -XX:+ShenandoahUncommit`. All performance and stress tests completed successfully on x86 and aarch64. Marking this as ready for review.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22019#issuecomment-2471152157

From kdnilsen at openjdk.org  Tue Nov 12 17:32:01 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 12 Nov 2024 17:32:01 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread
In-Reply-To: <m0nT9Z9rWxv035VXKArefrumnP31iQF8YAke5XLF1y8=.7b7b8328-5dce-40d1-9ea0-2badd8aeac3a@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <m0nT9Z9rWxv035VXKArefrumnP31iQF8YAke5XLF1y8=.7b7b8328-5dce-40d1-9ea0-2badd8aeac3a@github.com>
Message-ID: <seTWtGW6jGmYoZhVAChkayjal9FEEI-94YJC5IJF3Yg=.4978ecfc-4bad-4df5-b7c3-a3473a4dcf52@github.com>

On Mon, 11 Nov 2024 18:19:33 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 555:
> 
>> 553:     while (region != nullptr) {
>> 554:       {
>> 555:         ShenandoahHeapLocker locker(heap->lock());
> 
> Was it a bug that previous version of this code did not acquire the heap lock?
> 
> Is the lock required for the entirety of time that we are clearing the bitmap?  Or is it just required to get a trustworthy check on is_bitmap_slice_committed()?

After reading more of this PR, I believe we need the heap lock to get a reliable signal of bitmap_slice_committed().  But I believe we do not need the heap lock for ctx->clear_bitmap(region) so would prefer to move that outside the lock, unless I am misunderstanding.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1837053704

From wkemper at openjdk.org  Tue Nov 12 17:42:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Nov 2024 17:42:25 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <9L6OzvZLrJmoj11yH3cSSrQc7Y0HaNADZA7r_9pe_1c=.9e1c7a0b-0efd-4269-b576-f0e751b4e919@github.com>
References: <9L6OzvZLrJmoj11yH3cSSrQc7Y0HaNADZA7r_9pe_1c=.9e1c7a0b-0efd-4269-b576-f0e751b4e919@github.com>
Message-ID: <nu3QomD5I_0SgRmm-jK4g4AITfdRdRZth82Q0AVC3UQ=.f6abea44-8523-4c3e-bde4-ef9511c9b9b8@github.com>

On Thu, 7 Nov 2024 14:21:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.6+2

This pull request has now been integrated.

Changeset: f79ec117
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/f79ec1178d5cdd076e12afe32dcf7085d15f482b
Stats:     12425 lines in 74 files changed: 11328 ins; 718 del; 379 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/136

From wkemper at openjdk.org  Tue Nov 12 18:11:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Nov 2024 18:11:25 GMT
Subject: RFR: 8344050: Shenandoah: Retire GC LABs concurrently
Message-ID: <M8PzzuQu4nydYYd1B4YMFF8YjKeRuOewMEyXjhtx-I0=.77de1863-2c80-471b-965d-74e30c3aadc8@github.com>

Test results show a consistent reduction in the time stopped at `init-update-refs` (as expected). Other metrics and benchmark scores are not significantly affected. This change itself does not completely eliminate the safepoint.

Results comparing `-XX:ShenandoahGCMode=satb` to `shenandoah/master` on `aarch64` (results for generational mode and other platforms are similar):

-51.91% sunflow/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.020ms (+/-  0.00ms)         70
  Test:         0.013ms (+/-  0.00ms)         20

-50.70% luindex/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.019ms (+/-  0.00ms)         70
  Test:         0.013ms (+/-  0.00ms)         20

-50.50% avrora/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.017ms (+/-  0.00ms)        144
  Test:         0.012ms (+/-  0.00ms)         42

-49.79% fop/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.022ms (+/-  0.00ms)        572
  Test:         0.015ms (+/-  0.00ms)        164

-47.99% batik/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.024ms (+/-  0.01ms)        420
  Test:         0.016ms (+/-  0.00ms)        120

-42.54% graphchi/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.020ms (+/-  0.00ms)         70
  Test:         0.014ms (+/-  0.00ms)         20

-42.36% xalan/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.017ms (+/-  0.00ms)        213
  Test:         0.012ms (+/-  0.00ms)         60

-23.63% scimark.lu.large/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.045ms (+/-  0.00ms)         54
  Test:         0.037ms (+/-  0.00ms)         12

-20.66% serial/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.041ms (+/-  0.00ms)        140
  Test:         0.034ms (+/-  0.00ms)         40

-20.31% crypto.aes/shenandoahinitupdaterefs_stopped p=0.00000
  Control:      0.041ms (+/-  0.00ms)         69
  Test:         0.034ms (+/-  0.01ms)         20

-------------

Commit messages:
 - Add comments to new method
 - Anticipate future work with better names for things
 - Fix safepoint worker handling
 - Restore gc labs to safepoint workers
 - Retire mutator gc labs with handshake
 - Retire gc labs concurrently

Changes: https://git.openjdk.org/shenandoah/pull/535/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=535&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344050
  Stats: 101 lines in 3 files changed: 72 ins; 29 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/535.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/535/head:pull/535

PR: https://git.openjdk.org/shenandoah/pull/535

From shade at openjdk.org  Tue Nov 12 18:20:59 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 12 Nov 2024 18:20:59 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>

On Mon, 11 Nov 2024 17:31:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

Cursory review:

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 91:

> 89: #include "runtime/stackWatermarkSet.hpp"
> 90: #include "runtime/vmThread.hpp"
> 91: #include "gc/shenandoah/shenandoahUncommitThread.hpp"

Includes are usually in alpbabetical order :)

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1345:

> 1343:   if (_uncommit_thread != nullptr) {
> 1344:     tcl->do_thread(_uncommit_thread);
> 1345:   }

New line after new block?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1932:

> 1930:   if (_uncommit_thread != nullptr) {
> 1931:     _uncommit_thread->stop();
> 1932:   }

Are there limits on proper sequencing here? Can we shutdown uncommit thread before cancelling the GC and waiting for control thread to exit? This would save end-to-end time for short commands, as we would hide the uncommit thread shutdown in the shadow of control thread exiting.

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 55:

> 53: 
> 54:     if (soft_max_changed || explicit_gc_requested || current - last_shrink_time > shrink_period) {
> 55:       double shrink_before = (soft_max_changed || explicit_gc_requested) ? current : current - ((double) ShenandoahUncommitDelay / 1000.0);

Suggestion:

      double shrink_before = (soft_max_changed || explicit_gc_requested) ? 
         current : 
         current - ((double) ShenandoahUncommitDelay / 1000.0);

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 69:

> 67:       MonitorLocker locker(&_lock, Mutex::_no_safepoint_check_flag);
> 68:       if (!_stop_requested.is_set()) {
> 69:         locker.wait((int64_t )shrink_period);

Suggestion:

        locker.wait((int64_t)shrink_period);

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 139:

> 137:     _heap->notify_heap_changed();
> 138:     double elapsed = os::elapsedTime() - start;
> 139:     log_info(gc)("Uncommitted " SIZE_FORMAT " regions, in %.3fs", count, elapsed);

If we can, can we match the current log format? E.g. print `Concurrent uncommit`, with appropriately formatted timestamp? I think we also want `log_info(gc,start)` at the beginning of the method. I think `ShenandoahConcurrentPhase` helper did all that, can we still use it?

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.hpp line 37:

> 35:   ShenandoahSharedFlag _explicit_gc_requested;
> 36:   ShenandoahSharedFlag _stop_requested;
> 37:   Monitor _lock;

Which one of these can be `const`?

-------------

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2430298661
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838524990
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838527316
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838530829
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838568788
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838569201
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838537328
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838569638

From shade at openjdk.org  Tue Nov 12 18:20:59 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 12 Nov 2024 18:20:59 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread
In-Reply-To: <dLO84w60iGOGN53ams38IG8QHsiS2LJv7kStQQ_2jhE=.e3177cfd-8e38-40b8-bc41-d49ef134943b@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <m0nT9Z9rWxv035VXKArefrumnP31iQF8YAke5XLF1y8=.7b7b8328-5dce-40d1-9ea0-2badd8aeac3a@github.com>
 <seTWtGW6jGmYoZhVAChkayjal9FEEI-94YJC5IJF3Yg=.4978ecfc-4bad-4df5-b7c3-a3473a4dcf52@github.com>
 <dLO84w60iGOGN53ams38IG8QHsiS2LJv7kStQQ_2jhE=.e3177cfd-8e38-40b8-bc41-d49ef134943b@github.com>
Message-ID: <Si5UjES5jyaglpGVCt4FhxxegutJ_abxY-iDOj8pj-U=.258887b1-ef26-440d-92c5-c931f4330e06@github.com>

On Mon, 11 Nov 2024 21:18:41 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> After reading more of this PR, I believe we need the heap lock to get a reliable signal of bitmap_slice_committed().  But I believe we do not need the heap lock for ctx->clear_bitmap(region) so would prefer to move that outside the lock, unless I am misunderstanding.
>
> Hmm, I'm not sure we can do that. Prior to this change, the control thread performed both clearing the bitmap and uncommitting the region's bitmap, so they could never happen concurrently. With this change, a separate thread could perform the uncommit. Consider:
> 
> 1. Control thread takes heap lock, observes that bitmap slice for region A is committed
> 2. Control thread releases heap lock, begins clearing bitmap (writing zeros to bitmap slice)
> 3. Uncommit thread takes heap lock, believes it must uncommit region A
> 4. Uncommit thread uncommits bitmap slice for region A
> 5. Segfault in Control Thread
> 
> I do _believe_ if we had a per region lock, it would be useful here. Holding a lock over the entire heap for this feels like overkill. Or, we could schedule the uncommit so that it does not occur during a GC cycle.

So, wait a sec. This code is in `ShenandoahResetBitmapTask`, so it can run in parallel. Putting a lock here inhibits parallelism. I understand the failure mode, but I think we should really be optimizing for the case when `ShenandoahUncommit` is not enabled (e.g. `-Xmx` == `-Xms`).

Sounds like there is a hassle in allowing concurrent uncommit to overlap with the GC cycle. In addition to this particular problem, we might be stealing cycles from the GC threads and take additional TTSP lag to park the uncommitter for the in-cycle GC pauses. I have no clear solution for this yet, but I think we need to explore if we can suspend the uncommit before going into GC cycle...

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838568215

From wkemper at openjdk.org  Tue Nov 12 19:02:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Nov 2024 19:02:37 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v2]
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <J8aB31ViJ6JlvDz64PXOSzV2QgmzhOUH32vj5K4qFKo=.d230208f-aa16-44e9-a48a-e5ca5e5f8c71@github.com>

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Style and formatting fixes
 - Alphabetize includes in shenandoahHeap.cpp

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22019/files
  - new: https://git.openjdk.org/jdk/pull/22019/files/e6684365..7301871e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=00-01

  Stats: 23 lines in 3 files changed: 10 ins; 6 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From wkemper at openjdk.org  Tue Nov 12 19:27:19 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Nov 2024 19:27:19 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v2]
In-Reply-To: <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
Message-ID: <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>

On Tue, 12 Nov 2024 17:46:26 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Style and formatting fixes
>>  - Alphabetize includes in shenandoahHeap.cpp
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1932:
> 
>> 1930:   if (_uncommit_thread != nullptr) {
>> 1931:     _uncommit_thread->stop();
>> 1932:   }
> 
> Are there limits on proper sequencing here? Can we shutdown uncommit thread before cancelling the GC and waiting for control thread to exit? This would save end-to-end time for short commands, as we would hide the uncommit thread shutdown in the shadow of control thread exiting.

I'm not sure the order matters here. `ConcurrentGCThread::stop` waits until the target thread sets `_has_terminated`.

> src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 139:
> 
>> 137:     _heap->notify_heap_changed();
>> 138:     double elapsed = os::elapsedTime() - start;
>> 139:     log_info(gc)("Uncommitted " SIZE_FORMAT " regions, in %.3fs", count, elapsed);
> 
> If we can, can we match the current log format? E.g. print `Concurrent uncommit`, with appropriately formatted timestamp? I think we also want `log_info(gc,start)` at the beginning of the method. I think `ShenandoahConcurrentPhase` helper did all that, can we still use it?

We can restore the log messages, but I don't think `ShenandoahConcurrentPhase` and friends will like being used outside of a cycle. I'll look into it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838644895
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838646433

From wkemper at openjdk.org  Tue Nov 12 19:27:18 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 12 Nov 2024 19:27:18 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v2]
In-Reply-To: <Si5UjES5jyaglpGVCt4FhxxegutJ_abxY-iDOj8pj-U=.258887b1-ef26-440d-92c5-c931f4330e06@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <m0nT9Z9rWxv035VXKArefrumnP31iQF8YAke5XLF1y8=.7b7b8328-5dce-40d1-9ea0-2badd8aeac3a@github.com>
 <seTWtGW6jGmYoZhVAChkayjal9FEEI-94YJC5IJF3Yg=.4978ecfc-4bad-4df5-b7c3-a3473a4dcf52@github.com>
 <dLO84w60iGOGN53ams38IG8QHsiS2LJv7kStQQ_2jhE=.e3177cfd-8e38-40b8-bc41-d49ef134943b@github.com>
 <Si5UjES5jyaglpGVCt4FhxxegutJ_abxY-iDOj8pj-U=.258887b1-ef26-440d-92c5-c931f4330e06@github.com>
Message-ID: <TP6J3QB781DC0NGZsi2SBMgiqcSJzZO5dex5ros5SY0=.a00ee441-9a31-4bee-bd75-869ebd7cb9b5@github.com>

On Tue, 12 Nov 2024 18:17:29 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Hmm, I'm not sure we can do that. Prior to this change, the control thread performed both clearing the bitmap and uncommitting the region's bitmap, so they could never happen concurrently. With this change, a separate thread could perform the uncommit. Consider:
>> 
>> 1. Control thread takes heap lock, observes that bitmap slice for region A is committed
>> 2. Control thread releases heap lock, begins clearing bitmap (writing zeros to bitmap slice)
>> 3. Uncommit thread takes heap lock, believes it must uncommit region A
>> 4. Uncommit thread uncommits bitmap slice for region A
>> 5. Segfault in Control Thread
>> 
>> I do _believe_ if we had a per region lock, it would be useful here. Holding a lock over the entire heap for this feels like overkill. Or, we could schedule the uncommit so that it does not occur during a GC cycle.
>
> So, wait a sec. This code is in `ShenandoahResetBitmapTask`, so it can run in parallel. Putting a lock here inhibits parallelism. I understand the failure mode, but I think we should really be optimizing for the case when `ShenandoahUncommit` is not enabled (e.g. `-Xmx` == `-Xms`).
> 
> Sounds like there is a hassle in allowing concurrent uncommit to overlap with the GC cycle. In addition to this particular problem, we might be stealing cycles from the GC threads and take additional TTSP lag to park the uncommitter for the in-cycle GC pauses. I have no clear solution for this yet, but I think we need to explore if we can suspend the uncommit before going into GC cycle...

We could have the control and uncommit threads coordinate their efforts. In the worst case, it could mean delaying concurrent reset while the control thread waits for the uncommit thread to yield.

We could also try a more targeted lock only for the region's bitmap slice, but it doesn't seem right that one thread would be trying to clear a bitmap, while the other is trying to uncommit it. A lock could preserve technical correctness, but contention here would just mean that one thread would have wasted its time (either clearing a bitmap that is then uncommitted, or attempting to clear a bitmap that was first uncommitted (in this case, we would need the control thread to detect this and skip the region)).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1838641673

From kdnilsen at openjdk.org  Tue Nov 12 20:48:28 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 12 Nov 2024 20:48:28 GMT
Subject: RFR: 8344050: Shenandoah: Retire GC LABs concurrently
In-Reply-To: <M8PzzuQu4nydYYd1B4YMFF8YjKeRuOewMEyXjhtx-I0=.77de1863-2c80-471b-965d-74e30c3aadc8@github.com>
References: <M8PzzuQu4nydYYd1B4YMFF8YjKeRuOewMEyXjhtx-I0=.77de1863-2c80-471b-965d-74e30c3aadc8@github.com>
Message-ID: <KKjsCF_OMGGbb1YlRqsZPae5FdCefH2U7kcMLsTbUXc=.2c4e0e9a-18f3-4729-8d47-ede2a562852a@github.com>

On Tue, 12 Nov 2024 18:07:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Test results show a consistent reduction in the time stopped at `init-update-refs` (as expected). Other metrics and benchmark scores are not significantly affected. This change itself does not completely eliminate the safepoint.
> 
> Results comparing `-XX:ShenandoahGCMode=satb` to `shenandoah/master` on `aarch64` (results for generational mode and other platforms are similar):
> 
> -51.91% sunflow/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.020ms (+/-  0.00ms)         70
>   Test:         0.013ms (+/-  0.00ms)         20
> 
> -50.70% luindex/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.019ms (+/-  0.00ms)         70
>   Test:         0.013ms (+/-  0.00ms)         20
> 
> -50.50% avrora/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.017ms (+/-  0.00ms)        144
>   Test:         0.012ms (+/-  0.00ms)         42
> 
> -49.79% fop/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.022ms (+/-  0.00ms)        572
>   Test:         0.015ms (+/-  0.00ms)        164
> 
> -47.99% batik/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.024ms (+/-  0.01ms)        420
>   Test:         0.016ms (+/-  0.00ms)        120
> 
> -42.54% graphchi/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.020ms (+/-  0.00ms)         70
>   Test:         0.014ms (+/-  0.00ms)         20
> 
> -42.36% xalan/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.017ms (+/-  0.00ms)        213
>   Test:         0.012ms (+/-  0.00ms)         60
> 
> -23.63% scimark.lu.large/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.045ms (+/-  0.00ms)         54
>   Test:         0.037ms (+/-  0.00ms)         12
> 
> -20.66% serial/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.041ms (+/-  0.00ms)        140
>   Test:         0.034ms (+/-  0.00ms)         40
> 
> -20.31% crypto.aes/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.041ms (+/-  0.00ms)         69
>   Test:         0.034ms (+/-  0.01ms)         20

Marked as reviewed by kdnilsen (Committer).

Nice improvement.  Thanks.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/535#pullrequestreview-2430659233
PR Comment: https://git.openjdk.org/shenandoah/pull/535#issuecomment-2471548658

From shade at openjdk.org  Wed Nov 13 12:48:04 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 13 Nov 2024 12:48:04 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
Message-ID: <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>

On Mon, 4 Nov 2024 19:52:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 503 commits:
> 
>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge
>  - 8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates
>    
>    Reviewed-by: kdnilsen
>  - 8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
>    
>    Reviewed-by: ysr
>  - 8342919: GenShen: Fix whitespace
>    
>    Reviewed-by: xpeng, kdnilsen
>  - 8342927: GenShen: Guarantee slices of time for coalesce and filling
>    
>    Reviewed-by: kdnilsen
>  - 8342924: GenShen: Problem list gc/shenandoah/TestReferenceRefersToShenandoah.java
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode
>    
>    Reviewed-by: wkemper
>  - ... and 493 more: https://git.openjdk.org/jdk/compare/1c448347...19b25bc3

I read about half, here is a dump of my current observations. I'll read the second half a bit later.

src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 474:

> 472:     // elision safe.
> 473:     return;
> 474:   }

Why is this safe for Shenandoah? I suspect it needs `CardTableBarrierSet::on_slowpath_allocation_exit` to work. `G1BarrierSetC2` gets it by subclassing `CardTableBarrierSetC2`. But `ShenandoahBarrierSetC2` does not. Should it?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.cpp line 67:

> 65:     _region_data[i].clear();
> 66:   }
> 67: #endif

I understand this is to make sure `union_tag` works well. But why don't we extend `clear` to initialize all fields, and do this block without `ASSERT`? This does not look like frequently used path. Generally, doing these inits only for debug modes might hide some assertion failures that would indicate a problem in product builds.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.hpp line 118:

> 116: #ifdef ASSERT
> 117:       assert(_union_tag != is_uninitialized, "Cannot fetch region from uninialized RegionData");
> 118: #endif

Style: No point in wrapping single-line `assert`-s in `#ifdef ASSERT`.

src/hotspot/share/gc/shenandoah/shenandoahAsserts.hpp line 238:

> 236: #define shenandoah_assert_control_or_vm_thread_at_safepoint()
> 237: #define shenandoah_assert_generational()
> 238: #define shenandoah_assert_generations_reconciled()                                                             \

There seems to be dangling `` on this line.

src/hotspot/share/gc/shenandoah/shenandoahCardStats.hpp line 47:

> 45:   CARD_STAT_UPDATE_REFS,
> 46:   MAX_CARD_STAT_LOG_TYPE
> 47: };

These are in global namespace. There is a risk they would clash with some other symbol elsewhere. Do you need them to be global, or can they go into `ShenandoahCardStats`?

src/hotspot/share/gc/shenandoah/shenandoahStackWatermark.cpp line 77:

> 75:     return reinterpret_cast<OopClosure*>(context);
> 76:   } else {
> 77:     if (_heap->is_concurrent_weak_root_in_progress()) {

Comprehension check: We flips this because in generational mode we _can_ have both concurrent weak roots and concurrent mark in progress at the same time, and we need to prioritize evacs, right?

src/hotspot/share/gc/shenandoah/shenandoahUtils.hpp line 50:

> 48:   switch (generation_type) {                                              \
> 49:     case NON_GEN:                                                         \
> 50:       return prefix " (NON-GENERATIONAL)" postfix;                        \

In the interest of keeping the non-generational Shenandoah logging intact, should we drop ` (NON-GENERATIONAL)` here?

src/hotspot/share/gc/shenandoah/shenandoahVMOperations.cpp line 83:

> 81: void VM_ShenandoahInitMark::doit() {
> 82:   ShenandoahGCPauseMark mark(_gc_id, "Init Mark", SvcGCMarker::CONCURRENT);
> 83:   set_active_generation();

Why is it here, and not down in `entry_*` methods, probably in helper classes?

src/hotspot/share/gc/shenandoah/shenandoahVMOperations.hpp line 41:

> 39: //   - VM_ShenandoahInitUpdateRefs: initiate update references
> 40: //   - VM_ShenandoahFinalUpdateRefs: finish up update references
> 41: //   - VM_ShenandoahFinalRoots

If we add it here, let's provide a comment:


// - VM_ShenandoahFinalRoots: finish up the roots, shortcut cycle

src/hotspot/share/jfr/metadata/metadata.xml line 1221:

> 1219:     <Field type="ulong" contentType="bytes" name="immediateBytes" label="Immediate Bytes" />
> 1220:   </Event>
> 1221:   

Not sure if we _need_ this JFR event as part of GenShen? This bit is what forces us to do CSR, right?

test/hotspot/jtreg/ProblemList.txt line 98:

> 96: gc/TestAlwaysPreTouchBehavior.java#Epsilon 8334513 generic-all
> 97: gc/stress/gclocker/TestExcessGCLockerCollections.java 8229120 generic-all
> 98: 

This change is unnecessary.

test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 61:

> 59:                 // case that the previously instantiated NastyThread accumulated more than SheanndoahNoProgressThreshold
> 60:                 // unproductive GC cycles before failing, the main thread may not try a Full GC before it experiences
> 61:                 // OutOfMemoryError exception.

Do you really need comments in this test?

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2429755115
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1839969170
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1839985542
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1840008742
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1840173090
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1840153151
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1840187886
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1840195628
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1840199905
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1840201386
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1840096925
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1838665756
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1838196649

From wkemper at openjdk.org  Wed Nov 13 19:10:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 19:10:12 GMT
Subject: RFR: 8344050: Shenandoah: Retire GC LABs concurrently [v2]
In-Reply-To: <M8PzzuQu4nydYYd1B4YMFF8YjKeRuOewMEyXjhtx-I0=.77de1863-2c80-471b-965d-74e30c3aadc8@github.com>
References: <M8PzzuQu4nydYYd1B4YMFF8YjKeRuOewMEyXjhtx-I0=.77de1863-2c80-471b-965d-74e30c3aadc8@github.com>
Message-ID: <on_-E8AEbHEsRdG8qIP-PVvhY1lTI82opr2v40guElU=.a08f6178-58e1-423f-b2ff-69b9a455f45b@github.com>

> Test results show a consistent reduction in the time stopped at `init-update-refs` (as expected). Other metrics and benchmark scores are not significantly affected. This change itself does not completely eliminate the safepoint.
> 
> Results comparing `-XX:ShenandoahGCMode=satb` to `shenandoah/master` on `aarch64` (results for generational mode and other platforms are similar):
> 
> -51.91% sunflow/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.020ms (+/-  0.00ms)         70
>   Test:         0.013ms (+/-  0.00ms)         20
> 
> -50.70% luindex/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.019ms (+/-  0.00ms)         70
>   Test:         0.013ms (+/-  0.00ms)         20
> 
> -50.50% avrora/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.017ms (+/-  0.00ms)        144
>   Test:         0.012ms (+/-  0.00ms)         42
> 
> -49.79% fop/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.022ms (+/-  0.00ms)        572
>   Test:         0.015ms (+/-  0.00ms)        164
> 
> -47.99% batik/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.024ms (+/-  0.01ms)        420
>   Test:         0.016ms (+/-  0.00ms)        120
> 
> -42.54% graphchi/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.020ms (+/-  0.00ms)         70
>   Test:         0.014ms (+/-  0.00ms)         20
> 
> -42.36% xalan/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.017ms (+/-  0.00ms)        213
>   Test:         0.012ms (+/-  0.00ms)         60
> 
> -23.63% scimark.lu.large/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.045ms (+/-  0.00ms)         54
>   Test:         0.037ms (+/-  0.00ms)         12
> 
> -20.66% serial/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.041ms (+/-  0.00ms)        140
>   Test:         0.034ms (+/-  0.00ms)         40
> 
> -20.31% crypto.aes/shenandoahinitupdaterefs_stopped p=0.00000
>   Control:      0.041ms (+/-  0.00ms)         69
>   Test:         0.034ms (+/-  0.01ms)         20

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix builds

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/535/files
  - new: https://git.openjdk.org/shenandoah/pull/535/files/8297f4d3..64b71fe7

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=535&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=535&range=00-01

  Stats: 3 lines in 1 file changed: 0 ins; 1 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/535.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/535/head:pull/535

PR: https://git.openjdk.org/shenandoah/pull/535

From wkemper at openjdk.org  Wed Nov 13 19:32:53 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 19:32:53 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:

 - Merge branch 'merge-latest' into great-genshen-pr-redux
 - Use new CompactHeader forwarding APIs in generational mode
 - Merge remote-tracking branch 'jdk/master' into merge-latest
 - Merge
 - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
   
   Reviewed-by: wkemper
 - Merge
 - 8343227: GenShen: Fold resource mark into management of preselected regions
   
   Reviewed-by: kdnilsen
 - Merge openjdk/jdk tip into great-genshen-pr-redux
 - Merge remote-tracking branch 'jdk/master' into merge-latest
 - Merge remote-tracking branch 'jdk/master' into merge-latest
 - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8

-------------

Changes: https://git.openjdk.org/jdk/pull/21273/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=06
  Stats: 22977 lines in 231 files changed: 21234 ins; 886 del; 857 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From xpeng at openjdk.org  Wed Nov 13 21:52:03 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 13 Nov 2024 21:52:03 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v15]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <hS-ZmvCxVjFajJ22EEPJGo0sIMrgPP38SGVx7hYoLjw=.14adce79-dded-40b0-836b-b4c0ef4240d1@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 39 commits:

 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Move set_mark_incomplete into reset_mark_bitmap
 - fix test failures
 - set_mark_incomplete should be always set after resetting bitmap
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Not reset bitmap after ShenandoahOldGC
 - Capture Top At Mark Start for *this* generation
 - Revert "set need_bitmap_reset flag when change affiliation of heap region from OLD_GENERATION"
   
   This reverts commit d4fddac5a1e7321ff3b4441bd5cd096e949cd56a.
 - ... and 29 more: https://git.openjdk.org/shenandoah/compare/3825ae3e...40719d9c

-------------

Changes: https://git.openjdk.org/shenandoah/pull/516/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=14
  Stats: 109 lines in 6 files changed: 94 ins; 2 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From gziemski at openjdk.org  Wed Nov 13 22:21:12 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Wed, 13 Nov 2024 22:21:12 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <0P1eFuX2RUUcKEhqyjZwvOOh8t2fz_mkM0zHTuM78Zc=.93971045-a0fd-4972-946d-160dfc3e0f42@github.com>

On Fri, 1 Nov 2024 20:44:50 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

> We use mtNone value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
> 
> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
> 
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

Thank you very much everyone for feedback, and especially Thomas for so much details.

I'm working through the feedback, but have some thoughts on the subject of tags that I would like to share.

Here is how I would love to see NMT tags used:

- An object starts its life with "mtNone" tag. I probably would prefer to call it "mtUnset" or "mtNull", but "mtNone" is OK too.

- When the type is known at the allocation time, then use the corresponding tag and we are done.

- When the type is NOT known at the allocation time, then we set the tag to a new tag named "mtAllocated", "mtUnknown" or "mtTransition" or something like that. Later, when we know exactly who owns such tagged memory object, we set it to its final value.

This way we avoid the overloaded meaning of "mtNone" we have right now - either unset or truly unknown, or developer unfamiliar with NMT to know (or just plain too lazy) to propgate the tag value down.

In this fix I'm not trying to fix all of it, I'm just trying to fix the particular case in the issue and further clean up the code a bit to make it one step closer towards the ultimate goal - making sure that we tag memory (if it's reasonable to figure it out).



Last general thought on the tag transition from one value to another (not to be addressed here, but later), which we currently allow. It would be nice to have a general NMT API that controls how the tag values are allowed to change (similar to VirtualMemoryTracker::add_reserved_region). So instead of "mtAllocated" (as proposed) or "mtNone" (as we currently have), we would use a union of the subset of the tags, where the final value can be only one out of the union value (suddenly there is a use case here for tags to be "real" logical flags)

- Currently we have: "mtNone" (which currently means both unset and unknown) allowed to be changed to any tag.

- Partial fix, that I could do in this change: "mtNone" allowed to be changed to final tag, or to "mtAllocated". "mtAllocated" is allowed to be changed to any tag (this addresses the duality of "mtNone")

- Ultimate fix (to be addressed in a followup): "mtNone" allowed to be changed to final tag, or to "mtAllocated" (if final tag could be anything), or to some union tag (if the final tag is expected to be one of a subset). Union tag can only be changed to one member of the uinion members.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2474934177

From wkemper at openjdk.org  Wed Nov 13 23:10:51 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 23:10:51 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
Message-ID: <ZQT5Q_SYQs_RBf-nIVS3WfNqBc7Wdyl999yPf2AH144=.b730b700-dc8c-4c47-9663-a9e240226831@github.com>

On Tue, 12 Nov 2024 14:25:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 503 commits:
>> 
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
>>    
>>    Reviewed-by: ysr
>>  - 8342919: GenShen: Fix whitespace
>>    
>>    Reviewed-by: xpeng, kdnilsen
>>  - 8342927: GenShen: Guarantee slices of time for coalesce and filling
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342924: GenShen: Problem list gc/shenandoah/TestReferenceRefersToShenandoah.java
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode
>>    
>>    Reviewed-by: wkemper
>>  - ... and 493 more: https://git.openjdk.org/jdk/compare/1c448347...19b25bc3
>
> test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 61:
> 
>> 59:                 // case that the previously instantiated NastyThread accumulated more than SheanndoahNoProgressThreshold
>> 60:                 // unproductive GC cycles before failing, the main thread may not try a Full GC before it experiences
>> 61:                 // OutOfMemoryError exception.
> 
> Do you really need comments in this test?

This test was a sore spot for us and we spent quite some time to make it pass. I don't think the code is very clear on its face about what the test expects, so I'd prefer to keep this comment.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841302676

From wkemper at openjdk.org  Wed Nov 13 23:24:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 23:24:32 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
Message-ID: <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>

On Wed, 13 Nov 2024 12:23:26 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 503 commits:
>> 
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
>>    
>>    Reviewed-by: ysr
>>  - 8342919: GenShen: Fix whitespace
>>    
>>    Reviewed-by: xpeng, kdnilsen
>>  - 8342927: GenShen: Guarantee slices of time for coalesce and filling
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342924: GenShen: Problem list gc/shenandoah/TestReferenceRefersToShenandoah.java
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode
>>    
>>    Reviewed-by: wkemper
>>  - ... and 493 more: https://git.openjdk.org/jdk/compare/1c448347...19b25bc3
>
> src/hotspot/share/gc/shenandoah/shenandoahAsserts.hpp line 238:
> 
>> 236: #define shenandoah_assert_control_or_vm_thread_at_safepoint()
>> 237: #define shenandoah_assert_generational()
>> 238: #define shenandoah_assert_generations_reconciled()                                                             \
> 
> There seems to be dangling `` on this line.

Good catch.

> src/hotspot/share/gc/shenandoah/shenandoahStackWatermark.cpp line 77:
> 
>> 75:     return reinterpret_cast<OopClosure*>(context);
>> 76:   } else {
>> 77:     if (_heap->is_concurrent_weak_root_in_progress()) {
> 
> Comprehension check: We flips this because in generational mode we _can_ have both concurrent weak roots and concurrent mark in progress at the same time, and we need to prioritize evacs, right?

Correct. Although no old marking threads will ever run during a young collection.

> src/hotspot/share/gc/shenandoah/shenandoahUtils.hpp line 50:
> 
>> 48:   switch (generation_type) {                                              \
>> 49:     case NON_GEN:                                                         \
>> 50:       return prefix " (NON-GENERATIONAL)" postfix;                        \
> 
> In the interest of keeping the non-generational Shenandoah logging intact, should we drop ` (NON-GENERATIONAL)` here?

No objections here. @ysramakrishna , @kdnilsen ?

> src/hotspot/share/gc/shenandoah/shenandoahVMOperations.cpp line 83:
> 
>> 81: void VM_ShenandoahInitMark::doit() {
>> 82:   ShenandoahGCPauseMark mark(_gc_id, "Init Mark", SvcGCMarker::CONCURRENT);
>> 83:   set_active_generation();
> 
> Why is it here, and not down in `entry_*` methods, probably in helper classes?

I think it's here for the same reason `propagate_gc_state_to_java_threads` is here. Having it here makes it easier to see that this critical synchronization step happens for every safepoint.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841308265
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841308915
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841310699
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841313192

From wkemper at openjdk.org  Wed Nov 13 23:35:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 23:35:28 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
 <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
Message-ID: <Evjqt6250LX2iCBc27CbgEowf3-Jj6_ZXxHM4x4dkR0=.90bb32c1-0c37-4c1a-ad37-17e42c12faf4@github.com>

On Wed, 13 Nov 2024 23:14:23 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahAsserts.hpp line 238:
>> 
>>> 236: #define shenandoah_assert_control_or_vm_thread_at_safepoint()
>>> 237: #define shenandoah_assert_generational()
>>> 238: #define shenandoah_assert_generations_reconciled()                                                             \
>> 
>> There seems to be dangling `` on this line.
>
> Good catch.

https://bugs.openjdk.org/browse/JDK-8344151

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841320809

From wkemper at openjdk.org  Wed Nov 13 23:35:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 23:35:28 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
Message-ID: <m21YB0_w5lit0qYpvJOLIfRAo7YiWJAsomxGuKv2Qqw=.192f9f54-8491-469a-8f90-08211fcb3516@github.com>

On Wed, 13 Nov 2024 11:01:59 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 503 commits:
>> 
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
>>    
>>    Reviewed-by: ysr
>>  - 8342919: GenShen: Fix whitespace
>>    
>>    Reviewed-by: xpeng, kdnilsen
>>  - 8342927: GenShen: Guarantee slices of time for coalesce and filling
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342924: GenShen: Problem list gc/shenandoah/TestReferenceRefersToShenandoah.java
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode
>>    
>>    Reviewed-by: wkemper
>>  - ... and 493 more: https://git.openjdk.org/jdk/compare/1c448347...19b25bc3
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.hpp line 118:
> 
>> 116: #ifdef ASSERT
>> 117:       assert(_union_tag != is_uninitialized, "Cannot fetch region from uninialized RegionData");
>> 118: #endif
> 
> Style: No point in wrapping single-line `assert`-s in `#ifdef ASSERT`.

https://bugs.openjdk.org/browse/JDK-8344151

> src/hotspot/share/gc/shenandoah/shenandoahVMOperations.hpp line 41:
> 
>> 39: //   - VM_ShenandoahInitUpdateRefs: initiate update references
>> 40: //   - VM_ShenandoahFinalUpdateRefs: finish up update references
>> 41: //   - VM_ShenandoahFinalRoots
> 
> If we add it here, let's provide a comment:
> 
> 
> // - VM_ShenandoahFinalRoots: finish up the roots, shortcut cycle

https://bugs.openjdk.org/browse/JDK-8344151

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841320875
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841320596

From wkemper at openjdk.org  Wed Nov 13 23:35:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 23:35:29 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
Message-ID: <xpTSPx1490LZ6wj97-WNahmwjM9vKZoc9Cn_u4aZTVk=.bc2806a2-5514-4d75-adb7-b84d4488af35@github.com>

On Tue, 12 Nov 2024 19:41:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> test/hotspot/jtreg/ProblemList.txt line 98:
> 
>> 96: gc/TestAlwaysPreTouchBehavior.java#Epsilon 8334513 generic-all
>> 97: gc/stress/gclocker/TestExcessGCLockerCollections.java 8229120 generic-all
>> 98: 
> 
> This change is unnecessary.

https://bugs.openjdk.org/browse/JDK-8344151

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841321018

From duke at openjdk.org  Wed Nov 13 23:36:32 2024
From: duke at openjdk.org (duke)
Date: Wed, 13 Nov 2024 23:36:32 GMT
Subject: Withdrawn: 8340381: Shenandoah: Class mirrors verification should
 check forwarded objects
In-Reply-To: <9vV2xnuP2lgRCLLbB5LWnIg26HtPjS7BOIyt0qaLkwg=.d7975d49-c70b-43e5-89cb-ef1b4f86ac52@github.com>
References: <9vV2xnuP2lgRCLLbB5LWnIg26HtPjS7BOIyt0qaLkwg=.d7975d49-c70b-43e5-89cb-ef1b4f86ac52@github.com>
Message-ID: <9mAsbnJFW9K4S36cpIm1a4tKbrVHtBLRgsOIYipLQL4=.da564665-6894-465b-ac65-fe548c8807ed@github.com>

On Wed, 18 Sep 2024 13:48:43 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> The from-space objects can be effectively dead, and their backlinks to `InstanceKlass*` not updated anymore. So they can point to garbage. 
> 
> Additional testing:
>  - [x] Some previously failing reproducers are not failing anymore
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/21064

From wkemper at openjdk.org  Wed Nov 13 23:37:10 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 23:37:10 GMT
Subject: RFR: 8344151: GenShen: Improve comments,
 clean up unnecessary and unintended changes
Message-ID: <wow-94HiLm0410FNuFD30qf13kdW6t-oKEDSDUzUGDo=.96708f45-d1df-450b-b9bc-0e90ed31da6f@github.com>

Address a few style nits from: https://github.com/openjdk/jdk/pull/21273

-------------

Commit messages:
 - Style and formatting fixes, revert unnecessary changes

Changes: https://git.openjdk.org/shenandoah/pull/536/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=536&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344151
  Stats: 10 lines in 4 files changed: 1 ins; 6 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/536.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/536/head:pull/536

PR: https://git.openjdk.org/shenandoah/pull/536

From wkemper at openjdk.org  Wed Nov 13 23:44:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 23:44:43 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
Message-ID: <JtL_0nomC_nVnmfWWiI7o6qeUddhcxu1ex3xk9l5B3M=.abece391-d25a-4a1a-ab9b-5572145a8102@github.com>

On Wed, 13 Nov 2024 10:57:08 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 503 commits:
>> 
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
>>    
>>    Reviewed-by: ysr
>>  - 8342919: GenShen: Fix whitespace
>>    
>>    Reviewed-by: xpeng, kdnilsen
>>  - 8342927: GenShen: Guarantee slices of time for coalesce and filling
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342924: GenShen: Problem list gc/shenandoah/TestReferenceRefersToShenandoah.java
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode
>>    
>>    Reviewed-by: wkemper
>>  - ... and 493 more: https://git.openjdk.org/jdk/compare/1c448347...19b25bc3
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.cpp line 67:
> 
>> 65:     _region_data[i].clear();
>> 66:   }
>> 67: #endif
> 
> I understand this is to make sure `union_tag` works well. But why don't we extend `clear` to initialize all fields, and do this block without `ASSERT`? This does not look like frequently used path. Generally, doing these inits only for debug modes might hide some assertion failures that would indicate a problem in product builds.

https://bugs.openjdk.org/browse/JDK-8344152

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841326159

From wkemper at openjdk.org  Wed Nov 13 23:47:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 13 Nov 2024 23:47:46 GMT
Subject: RFR: 8344152: GenShen: Initialize RegionData in product builds
Message-ID: <GDLnpATgoVomDKfLCwxSf0gWUNaVheO_PbxYxAzFQhM=.c40ff124-eebf-4a27-aa83-a7f40b53b1ec@github.com>

Hygiene improvement.

-------------

Commit messages:
 - Initialize RegionData fields in product builds

Changes: https://git.openjdk.org/shenandoah/pull/537/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=537&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344152
  Stats: 8 lines in 2 files changed: 4 ins; 4 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/537.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/537/head:pull/537

PR: https://git.openjdk.org/shenandoah/pull/537

From shade at openjdk.org  Thu Nov 14 10:43:15 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 14 Nov 2024 10:43:15 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <ZQT5Q_SYQs_RBf-nIVS3WfNqBc7Wdyl999yPf2AH144=.b730b700-dc8c-4c47-9663-a9e240226831@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
 <ZQT5Q_SYQs_RBf-nIVS3WfNqBc7Wdyl999yPf2AH144=.b730b700-dc8c-4c47-9663-a9e240226831@github.com>
Message-ID: <ZEY-2SaxBhTuReNx5ua_0sVMPFYiq8D1hkXHjkp-QjA=.eb42068d-f17d-49e1-9d4a-5bd79ed408ab@github.com>

On Wed, 13 Nov 2024 23:07:02 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 61:
>> 
>>> 59:                 // case that the previously instantiated NastyThread accumulated more than SheanndoahNoProgressThreshold
>>> 60:                 // unproductive GC cycles before failing, the main thread may not try a Full GC before it experiences
>>> 61:                 // OutOfMemoryError exception.
>> 
>> Do you really need comments in this test?
>
> This test was a sore spot for us and we spent quite some time to make it pass. I don't think the code is very clear on its face about what the test expects, so I'd prefer to keep this comment.

OK, fine!

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1841996383

From wkemper at openjdk.org  Thu Nov 14 14:25:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 14 Nov 2024 14:25:42 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <fCQhKSyRmZIprJNkE78hBD3RTOrmVEb-bYviw3Y4tp8=.e0f8aa9f-507c-4c94-a124-7abfd825be5d@github.com>

Merges tag jdk-21.0.6+3

-------------

Commit messages:
 - 8341927: Replace hardcoded security providers with new test.provider.name system property
 - 8335142: compiler/c1/TestTraceLinearScanLevel.java occasionally times out with -Xcomp
 - 8333235: vmTestbase/nsk/jdb/kill/kill001/kill001.java fails with C1
 - 8332112: Update nsk.share.Log to don't print summary during VM shutdown hook
 - 8330278: Have SSLSocketTemplate.doClientSide use loopback address
 - 8240343: JDI stopListening/stoplis001 "FAILED: listening is successfully stopped without starting listening"
 - 8327924: Simplify TrayIconScalingTest.java
 - 8325038: runtime/cds/appcds/ProhibitedPackage.java can fail with UseLargePages
 - 8308429: jvmti/StopThread/stopthrd007 failed with "NoClassDefFoundError: Could not initialize class jdk.internal.misc.VirtualThreads"
 - 8320575: generic type information lost on mandated parameters of record's compact constructors
 - ... and 1 more: https://git.openjdk.org/shenandoah-jdk21u/compare/bb60a427...9bdf2a61

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/137/files
  Stats: 1634 lines in 274 files changed: 696 ins; 179 del; 759 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/137.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/137/head:pull/137

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/137

From shade at openjdk.org  Thu Nov 14 14:42:38 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 14 Nov 2024 14:42:38 GMT
Subject: RFR: 8344151: GenShen: Improve comments,
 clean up unnecessary and unintended changes
In-Reply-To: <wow-94HiLm0410FNuFD30qf13kdW6t-oKEDSDUzUGDo=.96708f45-d1df-450b-b9bc-0e90ed31da6f@github.com>
References: <wow-94HiLm0410FNuFD30qf13kdW6t-oKEDSDUzUGDo=.96708f45-d1df-450b-b9bc-0e90ed31da6f@github.com>
Message-ID: <ypxdmbf1ZzF7yQHEudyfkHNOtB1FVSFC_0sD06zx0Oo=.b6eb65b3-0d69-434d-941d-4a7e3a5b12cb@github.com>

On Wed, 13 Nov 2024 23:31:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Address a few style nits from: https://github.com/openjdk/jdk/pull/21273

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/536#pullrequestreview-2436280559

From shade at openjdk.org  Thu Nov 14 19:34:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 14 Nov 2024 19:34:14 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
 <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
Message-ID: <O3oMAse5GA7hCUCsshJh3clb3lJsPRg6b2t9SUbdOi0=.dae0df8c-661e-452d-90ff-eeb1b740a392@github.com>

On Wed, 13 Nov 2024 23:17:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahUtils.hpp line 50:
>> 
>>> 48:   switch (generation_type) {                                              \
>>> 49:     case NON_GEN:                                                         \
>>> 50:       return prefix " (NON-GENERATIONAL)" postfix;                        \
>> 
>> In the interest of keeping the non-generational Shenandoah logging intact, should we drop ` (NON-GENERATIONAL)` here?
>
> No objections here. @ysramakrishna , @kdnilsen ?

See, with current code, default Shenandoah prints this:


[0.025s][info][gc] GC(0) Concurrent reset (NON-GENERATIONAL) (unload classes) 0.135ms
[0.025s][info][gc] GC(0) Pause Init Mark (NON-GENERATIONAL) (unload classes) 0.018ms
[0.025s][info][gc] GC(0) Concurrent marking roots 0.097ms
[0.029s][info][gc] GC(0) Concurrent marking (NON-GENERATIONAL) (unload classes) 4.050ms
[0.029s][info][gc] GC(0) Pause Final Mark (NON-GENERATIONAL) (unload classes) 0.123ms
[0.029s][info][gc] GC(0) Concurrent thread roots 0.205ms
[0.029s][info][gc] GC(0) Concurrent weak references (NON-GENERATIONAL) (unload classes) 0.014ms
[0.029s][info][gc] GC(0) Concurrent weak roots (NON-GENERATIONAL) (unload classes) 0.047ms
[0.029s][info][gc] GC(0) Concurrent cleanup (NON-GENERATIONAL) (unload classes) 40M->40M(2064M) 0.006ms
[0.029s][info][gc] GC(0) Concurrent class unloading 0.035ms
[0.029s][info][gc] GC(0) Concurrent strong roots 0.119ms


This is not great: experimental GenShen should not affect the logging for current product code. There is a risk we would be breaking some GC parsers if we change it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1842764603

From shade at openjdk.org  Thu Nov 14 19:34:13 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 14 Nov 2024 19:34:13 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
Message-ID: <g4cgdAfi3TZNYo8sDLY3nYiK4rB5PKlASOm1e2P_1d0=.aeec0d91-f45b-47ae-8d15-bb5eb4c7ddf1@github.com>

On Wed, 13 Nov 2024 19:32:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
> 
>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>  - Use new CompactHeader forwarding APIs in generational mode
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge
>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>    
>    Reviewed-by: wkemper
>  - Merge
>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>    
>    Reviewed-by: kdnilsen
>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8

More reviews...

src/hotspot/share/gc/shenandoah/shenandoahGenerationSizer.cpp line 141:

> 139:   dst->increase_capacity(bytes_to_transfer);
> 140:   const size_t new_size = dst->max_capacity();
> 141:   log_info(gc)("Transfer " SIZE_FORMAT " region(s) from %s to %s, yielding increased size: " PROPERFMT,

This should not be `log_info(gc)`. In fact, please look at other places where you print logging. Generally, `log_info(gc)` should provide _only_ the high-level GC info: which phases were running. Everything else should go under `log_debug(gc)`.

src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp line 140:

> 138:       break;
> 139:     }
> 140:     case OLD:

Are we not doing OLD STW mark? Deserves a comment.

src/hotspot/share/gc/shenandoah/shenandoahYoungGeneration.hpp line 54:

> 52:   void parallel_heap_region_iterate(ShenandoahHeapRegionClosure* cl) override;
> 53: 
> 54:   void parallel_region_iterate_free(ShenandoahHeapRegionClosure* cl) override;

Is this a sibling of `parallel_heap_region_iterate`? Shouldn't these be called `parallel_heap_region_iterate_free`?

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2436882261
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1842779639
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1842701030
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1842704295

From shade at openjdk.org  Thu Nov 14 19:34:15 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 14 Nov 2024 19:34:15 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <O3oMAse5GA7hCUCsshJh3clb3lJsPRg6b2t9SUbdOi0=.dae0df8c-661e-452d-90ff-eeb1b740a392@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
 <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
 <O3oMAse5GA7hCUCsshJh3clb3lJsPRg6b2t9SUbdOi0=.dae0df8c-661e-452d-90ff-eeb1b740a392@github.com>
Message-ID: <zvE0h9uweb8EUGvlN1IzBd588uanXbqLmlNlNP3AS-4=.4099afcb-3e2c-4a2a-a9e3-46aaf157496d@github.com>

On Thu, 14 Nov 2024 19:12:25 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> No objections here. @ysramakrishna , @kdnilsen ?
>
> See, with current code, default Shenandoah prints this:
> 
> 
> [0.025s][info][gc] GC(0) Concurrent reset (NON-GENERATIONAL) (unload classes) 0.135ms
> [0.025s][info][gc] GC(0) Pause Init Mark (NON-GENERATIONAL) (unload classes) 0.018ms
> [0.025s][info][gc] GC(0) Concurrent marking roots 0.097ms
> [0.029s][info][gc] GC(0) Concurrent marking (NON-GENERATIONAL) (unload classes) 4.050ms
> [0.029s][info][gc] GC(0) Pause Final Mark (NON-GENERATIONAL) (unload classes) 0.123ms
> [0.029s][info][gc] GC(0) Concurrent thread roots 0.205ms
> [0.029s][info][gc] GC(0) Concurrent weak references (NON-GENERATIONAL) (unload classes) 0.014ms
> [0.029s][info][gc] GC(0) Concurrent weak roots (NON-GENERATIONAL) (unload classes) 0.047ms
> [0.029s][info][gc] GC(0) Concurrent cleanup (NON-GENERATIONAL) (unload classes) 40M->40M(2064M) 0.006ms
> [0.029s][info][gc] GC(0) Concurrent class unloading 0.035ms
> [0.029s][info][gc] GC(0) Concurrent strong roots 0.119ms
> 
> 
> This is not great: experimental GenShen should not affect the logging for current product code. There is a risk we would be breaking some GC parsers if we change it.

While we are at it, we should probably replace `YOUNG` -> `Young`, etc, and figure out if `Transition from`, `Heuristics request`, and ` Periodic Sample:` should be in `gc=debug`,  so logs are cleaner:


[7.514s][info][gc] Transition from: normal to: idle
[7.617s][info][gc] Trigger (YOUNG): Average GC time (123.52 ms) is above the time for average allocation rate (2868 MB/s) to deplete free headroom (350M) (margin of error = 1.79)
[7.617s][info][gc] Transition from: idle to: normal
[7.617s][info][gc] Heuristics request for young collection accepted
[7.618s][info][gc] GC(31) Concurrent reset (YOUNG) 1.095ms
[7.618s][info][gc] GC(31) Pause Init Mark (YOUNG) 0.026ms
[7.618s][info][gc] GC(31) Concurrent remembered set scanning 0.148ms
[7.618s][info][gc] GC(31) Concurrent marking roots 0.217ms
[7.680s][info][gc] GC(31) Concurrent marking (YOUNG) 61.990ms
[7.681s][info][gc] GC(31) Transfer 1 region(s) from OLD to YOUNG, yielding increased size: 1021M
[7.681s][info][gc] GC(31) Transfer 1 region(s) from YOUNG to OLD, yielding increased size: 3584K
[7.681s][info][gc] GC(31) Pause Final Mark (YOUNG) 0.246ms
[7.681s][info][gc] GC(31) Concurrent thread roots 0.041ms
[7.728s][info][gc] GC(31) Concurrent weak references (YOUNG) 47.278ms
[7.728s][info][gc] GC(31) Concurrent weak roots (YOUNG) 0.114ms
[7.728s][info][gc] GC(31) Concurrent cleanup (YOUNG) 674M->658M(1024M) 0.014ms
[7.728s][info][gc] GC(31) Concurrent strong roots 0.217ms
[7.730s][info][gc] GC(31) Concurrent evacuation 1.879ms
[7.730s][info][gc] GC(31) Pause Init Update Refs 0.010ms
[7.730s][info][gc] GC(31) Transfer 1 region(s) from OLD to YOUNG, yielding increased size: 1021M
[7.739s][info][gc] GC(31) Concurrent update references 8.651ms
[7.739s][info][gc] GC(31) Concurrent update thread roots 0.069ms
[7.739s][info][gc] GC(31) Transfer 1 region(s) from YOUNG to OLD, yielding increased size: 3584K
[7.739s][info][gc] GC(31) Pause Final Update Refs 0.076ms
[7.739s][info][gc] GC(31) Concurrent cleanup (YOUNG) 690M->135M(1024M) 0.078ms

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1842769207

From mdoerr at openjdk.org  Thu Nov 14 23:43:50 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Thu, 14 Nov 2024 23:43:50 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
Message-ID: <2m-YJkwNcpIEDdVElZOw2uqbTD6eOoBM0cRGoSFdADM=.86118c91-b8dd-4da9-b691-5ffe0643f51d@github.com>

On Wed, 13 Nov 2024 19:32:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
> 
>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>  - Use new CompactHeader forwarding APIs in generational mode
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge
>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>    
>    Reviewed-by: wkemper
>  - Merge
>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>    
>    Reviewed-by: kdnilsen
>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8

The new test TestAllocOutOfMemory.java#large failed on some PPC64 machines:

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"

java.lang.RuntimeException: 'java.lang.OutOfMemoryError: Java heap space' missing from stdout/stderr

Probably not a show-stopper. Should the test be adapted or disabled for this platform?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2477628609

From wkemper at openjdk.org  Fri Nov 15 00:15:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 00:15:59 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <2m-YJkwNcpIEDdVElZOw2uqbTD6eOoBM0cRGoSFdADM=.86118c91-b8dd-4da9-b691-5ffe0643f51d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <2m-YJkwNcpIEDdVElZOw2uqbTD6eOoBM0cRGoSFdADM=.86118c91-b8dd-4da9-b691-5ffe0643f51d@github.com>
Message-ID: <hcdKWXi5qFuSFTahjURLpCg28_M9sFtwSvfza1OH3IY=.b86fa453-6e22-4ee3-9d7e-fe34b68237c2@github.com>

On Thu, 14 Nov 2024 23:39:08 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> The new test TestAllocOutOfMemory.java#large failed on some PPC64 machines:
> 
> Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
> 
> java.lang.RuntimeException: 'java.lang.OutOfMemoryError: Java heap space' missing from stdout/stderr
> 
> Probably not a show-stopper. Should the test be adapted or disabled for this platform?

@TheRealMDoerr - do you have the logs from the test failure? If so, could you open a ticket with them in JBS? Thank you!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2477671400

From wkemper at openjdk.org  Fri Nov 15 00:46:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 00:46:11 GMT
Subject: RFR: 8344260: GenShen: Excess unaffiliated regions reserved for old
 will not be transferred to young
Message-ID: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>

Accidental assignment to new (and unread) variable.

-------------

Commit messages:
 - Fix to have excess unaffiliated old regions transferred to young

Changes: https://git.openjdk.org/shenandoah/pull/538/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=538&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344260
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/538.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/538/head:pull/538

PR: https://git.openjdk.org/shenandoah/pull/538

From wkemper at openjdk.org  Fri Nov 15 01:27:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 01:27:59 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <g4cgdAfi3TZNYo8sDLY3nYiK4rB5PKlASOm1e2P_1d0=.aeec0d91-f45b-47ae-8d15-bb5eb4c7ddf1@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <g4cgdAfi3TZNYo8sDLY3nYiK4rB5PKlASOm1e2P_1d0=.aeec0d91-f45b-47ae-8d15-bb5eb4c7ddf1@github.com>
Message-ID: <YLp8UkhzZFUONHcSh6hnC-paH1TpZk8fnJAXB1M-9gI=.fdd9ca3b-3551-4eb4-96ad-e38e5e0f8be8@github.com>

On Thu, 14 Nov 2024 19:26:12 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationSizer.cpp line 141:
> 
>> 139:   dst->increase_capacity(bytes_to_transfer);
>> 140:   const size_t new_size = dst->max_capacity();
>> 141:   log_info(gc)("Transfer " SIZE_FORMAT " region(s) from %s to %s, yielding increased size: " PROPERFMT,
> 
> This should not be `log_info(gc)`. In fact, please look at other places where you print logging. Generally, `log_info(gc)` should provide _only_ the high-level GC info: which phases were running. Everything else should go under `log_debug(gc)`.

https://bugs.openjdk.org/browse/JDK-8344263

> src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp line 140:
> 
>> 138:       break;
>> 139:     }
>> 140:     case OLD:
> 
> Are we not doing OLD STW mark? Deserves a comment.

https://bugs.openjdk.org/browse/JDK-8344264

> src/hotspot/share/gc/shenandoah/shenandoahYoungGeneration.hpp line 54:
> 
>> 52:   void parallel_heap_region_iterate(ShenandoahHeapRegionClosure* cl) override;
>> 53: 
>> 54:   void parallel_region_iterate_free(ShenandoahHeapRegionClosure* cl) override;
> 
> Is this a sibling of `parallel_heap_region_iterate`? Shouldn't these be called `parallel_heap_region_iterate_free`?

https://bugs.openjdk.org/browse/JDK-8344264

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843073877
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843075157
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843075212

From wkemper at openjdk.org  Fri Nov 15 01:28:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 01:28:00 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <zvE0h9uweb8EUGvlN1IzBd588uanXbqLmlNlNP3AS-4=.4099afcb-3e2c-4a2a-a9e3-46aaf157496d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
 <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
 <O3oMAse5GA7hCUCsshJh3clb3lJsPRg6b2t9SUbdOi0=.dae0df8c-661e-452d-90ff-eeb1b740a392@github.com>
 <zvE0h9uweb8EUGvlN1IzBd588uanXbqLmlNlNP3AS-4=.4099afcb-3e2c-4a2a-a9e3-46aaf157496d@github.com>
Message-ID: <9UFMM5ji_G1XTowN-oY6n3JS5Enw_IFv9x8JShC1yhQ=.26f05071-7945-4268-8932-3486c319c139@github.com>

On Thu, 14 Nov 2024 19:16:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> See, with current code, default Shenandoah prints this:
>> 
>> 
>> [0.025s][info][gc] GC(0) Concurrent reset (NON-GENERATIONAL) (unload classes) 0.135ms
>> [0.025s][info][gc] GC(0) Pause Init Mark (NON-GENERATIONAL) (unload classes) 0.018ms
>> [0.025s][info][gc] GC(0) Concurrent marking roots 0.097ms
>> [0.029s][info][gc] GC(0) Concurrent marking (NON-GENERATIONAL) (unload classes) 4.050ms
>> [0.029s][info][gc] GC(0) Pause Final Mark (NON-GENERATIONAL) (unload classes) 0.123ms
>> [0.029s][info][gc] GC(0) Concurrent thread roots 0.205ms
>> [0.029s][info][gc] GC(0) Concurrent weak references (NON-GENERATIONAL) (unload classes) 0.014ms
>> [0.029s][info][gc] GC(0) Concurrent weak roots (NON-GENERATIONAL) (unload classes) 0.047ms
>> [0.029s][info][gc] GC(0) Concurrent cleanup (NON-GENERATIONAL) (unload classes) 40M->40M(2064M) 0.006ms
>> [0.029s][info][gc] GC(0) Concurrent class unloading 0.035ms
>> [0.029s][info][gc] GC(0) Concurrent strong roots 0.119ms
>> 
>> 
>> This is not great: experimental GenShen should not affect the logging for current product code. There is a risk we would be breaking some GC parsers if we change it.
>
> While we are at it, we should probably replace `YOUNG` -> `Young`, etc, and figure out if `Transition from`, `Heuristics request`, and ` Periodic Sample:` should be in `gc=debug`,  so logs are cleaner:
> 
> 
> [7.514s][info][gc] Transition from: normal to: idle
> [7.617s][info][gc] Trigger (YOUNG): Average GC time (123.52 ms) is above the time for average allocation rate (2868 MB/s) to deplete free headroom (350M) (margin of error = 1.79)
> [7.617s][info][gc] Transition from: idle to: normal
> [7.617s][info][gc] Heuristics request for young collection accepted
> [7.618s][info][gc] GC(31) Concurrent reset (YOUNG) 1.095ms
> [7.618s][info][gc] GC(31) Pause Init Mark (YOUNG) 0.026ms
> [7.618s][info][gc] GC(31) Concurrent remembered set scanning 0.148ms
> [7.618s][info][gc] GC(31) Concurrent marking roots 0.217ms
> [7.680s][info][gc] GC(31) Concurrent marking (YOUNG) 61.990ms
> [7.681s][info][gc] GC(31) Transfer 1 region(s) from OLD to YOUNG, yielding increased size: 1021M
> [7.681s][info][gc] GC(31) Transfer 1 region(s) from YOUNG to OLD, yielding increased size: 3584K
> [7.681s][info][gc] GC(31) Pause Final Mark (YOUNG) 0.246ms
> [7.681s][info][gc] GC(31) Concurrent thread roots 0.041ms
> [7.728s][info][gc] GC(31) Concurrent weak references (YOUNG) 47.278ms
> [7.728s][info][gc] GC(31) Concurrent weak roots (YOUNG) 0.114ms
> [7.728s][info][gc] GC(31) Concurrent cleanup (YOUNG) 674M->658M(1024M) 0.014ms
> [7.728s][info][gc] GC(31) Concurrent strong roots 0.217ms
> [7.730s][info][gc] GC(31) Concurrent evacuation 1.879ms
> [7.730s][info][gc] GC(31) Pause Init Update Refs 0.010ms
> [7.730s][info][gc] GC(31) Transfer 1 region(s) from OLD to YOUNG, yielding increased size: 1021M
> [7.739s][info][gc] GC(31) Concurrent update references 8.651ms
> [7.739s][info][gc] GC(31) Concurrent update thread roots 0.069ms
> [7.739s][info][gc] GC(31) Transfer 1 region(s) from YOUNG to OLD, yielding increased size: 3584K
> [7.739s][info][gc] GC(31) Pause Final Update Refs 0.076ms
> [7.739s][info][gc] GC(31) Concurrent cleanup (YOUNG) 690M->135M(1024M) 0.078ms

https://bugs.openjdk.org/browse/JDK-8344263

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843074126

From wkemper at openjdk.org  Fri Nov 15 01:30:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 01:30:38 GMT
Subject: RFR: 8344264: GenShen: Improve comments and method names
Message-ID: <9DLcYwQmX_oU0EhHjrhmTEW2iEauhr83ARyKEDR9O5Y=.a378906e-ec7a-4493-b3bd-5f9ad3c7f599@github.com>

More hygiene for GenShen.

-------------

Commit messages:
 - Improve method names and comments

Changes: https://git.openjdk.org/shenandoah/pull/539/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=539&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344264
  Stats: 15 lines in 7 files changed: 4 ins; 0 del; 11 mod
  Patch: https://git.openjdk.org/shenandoah/pull/539.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/539/head:pull/539

PR: https://git.openjdk.org/shenandoah/pull/539

From wkemper at openjdk.org  Fri Nov 15 01:32:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 01:32:23 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO level
Message-ID: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>

Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

-------------

Commit messages:
 - Reduce logging at info, use proper case for generation names

Changes: https://git.openjdk.org/shenandoah/pull/540/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344263
  Stats: 58 lines in 10 files changed: 0 ins; 1 del; 57 mod
  Patch: https://git.openjdk.org/shenandoah/pull/540.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/540/head:pull/540

PR: https://git.openjdk.org/shenandoah/pull/540

From ysr at openjdk.org  Fri Nov 15 01:57:53 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 15 Nov 2024 01:57:53 GMT
Subject: RFR: 8344050: Shenandoah: Retire GC LABs concurrently [v2]
In-Reply-To: <on_-E8AEbHEsRdG8qIP-PVvhY1lTI82opr2v40guElU=.a08f6178-58e1-423f-b2ff-69b9a455f45b@github.com>
References: <M8PzzuQu4nydYYd1B4YMFF8YjKeRuOewMEyXjhtx-I0=.77de1863-2c80-471b-965d-74e30c3aadc8@github.com>
 <on_-E8AEbHEsRdG8qIP-PVvhY1lTI82opr2v40guElU=.a08f6178-58e1-423f-b2ff-69b9a455f45b@github.com>
Message-ID: <z0J15SK5fdSRgzEzV484O0VeI3dfAUJtFyZOBkY9tOc=.6fcb37bb-24f1-4128-b0db-3fd67d8e8ec0@github.com>

On Wed, 13 Nov 2024 19:10:12 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Test results show a consistent reduction in the time stopped at `init-update-refs` (as expected). Other metrics and benchmark scores are not significantly affected. This change itself does not completely eliminate the safepoint.
>> 
>> Results comparing `-XX:ShenandoahGCMode=satb` to `shenandoah/master` on `aarch64` (results for generational mode and other platforms are similar):
>> 
>> -51.91% sunflow/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.020ms (+/-  0.00ms)         70
>>   Test:         0.013ms (+/-  0.00ms)         20
>> 
>> -50.70% luindex/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.019ms (+/-  0.00ms)         70
>>   Test:         0.013ms (+/-  0.00ms)         20
>> 
>> -50.50% avrora/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.017ms (+/-  0.00ms)        144
>>   Test:         0.012ms (+/-  0.00ms)         42
>> 
>> -49.79% fop/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.022ms (+/-  0.00ms)        572
>>   Test:         0.015ms (+/-  0.00ms)        164
>> 
>> -47.99% batik/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.024ms (+/-  0.01ms)        420
>>   Test:         0.016ms (+/-  0.00ms)        120
>> 
>> -42.54% graphchi/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.020ms (+/-  0.00ms)         70
>>   Test:         0.014ms (+/-  0.00ms)         20
>> 
>> -42.36% xalan/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.017ms (+/-  0.00ms)        213
>>   Test:         0.012ms (+/-  0.00ms)         60
>> 
>> -23.63% scimark.lu.large/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.045ms (+/-  0.00ms)         54
>>   Test:         0.037ms (+/-  0.00ms)         12
>> 
>> -20.66% serial/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.041ms (+/-  0.00ms)        140
>>   Test:         0.034ms (+/-  0.00ms)         40
>> 
>> -20.31% crypto.aes/shenandoahinitupdaterefs_stopped p=0.00000
>>   Control:      0.041ms (+/-  0.00ms)         69
>>   Test:         0.034ms (+/-  0.01ms)         20
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix builds

Changes look good. 

A couple of nits, not sure if they'll make any real difference though.

I also agree that this should go into 24 after the Genshen JEP PR is merged.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1225:

> 1223:     assert(gclab != nullptr, "GCLAB should be initialized for %s", thread->name());
> 1224:     gclab->retire();
> 1225:     if (_resize && ShenandoahThreadLocalData::gclab_size(thread) > 0) {

Nit: This isn't a change you made, but what's the point of the test? May be just set it to 0 if _resize, and save a load and test.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1229:

> 1227:     }
> 1228: 
> 1229:     if (ShenandoahHeap::heap()->mode()->is_generational()) {

Another nit: Just set this during construction in a field of the closure so you can test it cheaply here.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1237:

> 1235:       //  2. We need to establish a trustworthy UpdateWaterMark value within each old-gen heap region
> 1236:       ShenandoahGenerationalHeap::heap()->retire_plab(plab, thread);
> 1237:       if (_resize && ShenandoahThreadLocalData::plab_size(thread) > 0) {

Same as earlier: may be skip the second conjunct in the test.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/535#pullrequestreview-2437455680
PR Review Comment: https://git.openjdk.org/shenandoah/pull/535#discussion_r1843067447
PR Review Comment: https://git.openjdk.org/shenandoah/pull/535#discussion_r1843068264
PR Review Comment: https://git.openjdk.org/shenandoah/pull/535#discussion_r1843069407

From ysr at openjdk.org  Fri Nov 15 02:06:19 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 15 Nov 2024 02:06:19 GMT
Subject: RFR: 8344152: GenShen: Initialize RegionData in product builds
In-Reply-To: <GDLnpATgoVomDKfLCwxSf0gWUNaVheO_PbxYxAzFQhM=.c40ff124-eebf-4a27-aa83-a7f40b53b1ec@github.com>
References: <GDLnpATgoVomDKfLCwxSf0gWUNaVheO_PbxYxAzFQhM=.c40ff124-eebf-4a27-aa83-a7f40b53b1ec@github.com>
Message-ID: <VQ0HH0symvcdRda8d1AVQAiHK4-AYZwPdBdhhYUgbB4=.6f4b75ed-0cd0-4b8a-8801-49928f2e31eb@github.com>

On Wed, 13 Nov 2024 23:40:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Hygiene improvement.

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/537#pullrequestreview-2437522822

From ysr at openjdk.org  Fri Nov 15 02:11:56 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 15 Nov 2024 02:11:56 GMT
Subject: RFR: 8344260: GenShen: Excess unaffiliated regions reserved for
 old will not be transferred to young
In-Reply-To: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
References: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
Message-ID: <9ZiOdfBhEHXmdFWL6YPvn1K3f82NS-enH8JmwqIrEw0=.a351a270-8ba7-4692-b6f2-416d3d26ba5f@github.com>

On Fri, 15 Nov 2024 00:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Accidental assignment to new (and unread) variable.

Marked as reviewed by ysr (Committer).

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 455:

> 453:     // excess_old < unaffiliated old: we can give back MIN(excess_old/region_size_bytes, unaffiliated_old_regions)
> 454:     size_t excess_regions = excess_old / region_size_bytes;
> 455:     regions_to_xfer = MIN2(excess_regions, unaffiliated_old_regions);

Yikes! Thanks for finding this!

Did it make any difference to performance? Say with SPECjbb, e.g.?

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/538#pullrequestreview-2437527622
PR Review Comment: https://git.openjdk.org/shenandoah/pull/538#discussion_r1843100551

From ysr at openjdk.org  Fri Nov 15 03:53:56 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 15 Nov 2024 03:53:56 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <mfY9zvThGtuY-5EoPqEk3uvUufUKTRJEMPuDoUGPLcU=.7d413cfc-2fe8-48cc-9234-457c3608ff11@github.com>

On Fri, 15 Nov 2024 01:26:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

Can you share an example snippet (in the ticket or in the PR) of what we would see with a young and an old cycle with the reduced info-level logging?

The changes to convert from info to debug all look good to me.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/540#pullrequestreview-2437601109

From dholmes at openjdk.org  Fri Nov 15 04:52:04 2024
From: dholmes at openjdk.org (David Holmes)
Date: Fri, 15 Nov 2024 04:52:04 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <cI-IqlWyoHsFlZA2O6zhxhvSss5L2dbJeCydc-0MCLw=.bc364fb0-f4b2-46e3-b960-8426464e22d9@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

test/hotspot/jtreg/gtest/MetaspaceUtilsGtests.java line 1:


This file was reduced to empty but not actually deleted. Can you fix it please.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1843185719

From shade at openjdk.org  Fri Nov 15 10:27:33 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 15 Nov 2024 10:27:33 GMT
Subject: RFR: 8344152: GenShen: Initialize RegionData in product builds
In-Reply-To: <GDLnpATgoVomDKfLCwxSf0gWUNaVheO_PbxYxAzFQhM=.c40ff124-eebf-4a27-aa83-a7f40b53b1ec@github.com>
References: <GDLnpATgoVomDKfLCwxSf0gWUNaVheO_PbxYxAzFQhM=.c40ff124-eebf-4a27-aa83-a7f40b53b1ec@github.com>
Message-ID: <kP4DJ2jIPUPuNzujNBl-rbstWd6GUHToXLIqNSbSKkg=.0b95733d-f1e8-4033-9dbb-44a2962d4aa4@github.com>

On Wed, 13 Nov 2024 23:40:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Hygiene improvement.

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/537#pullrequestreview-2438229846

From shade at openjdk.org  Fri Nov 15 10:30:19 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 15 Nov 2024 10:30:19 GMT
Subject: RFR: 8344260: GenShen: Excess unaffiliated regions reserved for
 old will not be transferred to young
In-Reply-To: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
References: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
Message-ID: <rgbH9XzqX54egbIIP16TlgIzjbLsNi916Gp9VZ29smA=.c311ac67-1722-4eb1-a58e-9d23ea533eba@github.com>

On Fri, 15 Nov 2024 00:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Accidental assignment to new (and unread) variable.

Marked as reviewed by shade (Committer).

There are lots of crashes on related assert after this, I think.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/538#pullrequestreview-2438232892
Changes requested by shade (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/538#pullrequestreview-2438234191

From shade at openjdk.org  Fri Nov 15 11:29:11 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 15 Nov 2024 11:29:11 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <-4ElXqMU4hNKYEbkpsyURxSPEm3VNkUxY8BJLmnBDaI=.30ada198-7a05-46bb-bdf8-8e63c7288d62@github.com>

On Fri, 15 Nov 2024 01:26:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

Sorry, maybe I was too hard about `debug`. Maybe it should be `gc,ergo=info` instead, like the rest of heuristics code. "Want to see GC did? Enable -Xlog:gc. Want to see _why_ GC did it? Enable -Xlog:gc+ergo."

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/540#pullrequestreview-2438351304

From ihse at openjdk.org  Fri Nov 15 12:49:19 2024
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Fri, 15 Nov 2024 12:49:19 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <hE_1O5upcgRDnl5CnlKC-FyEkDyCx2k0EBSaEOupSuM=.51531484-da25-4221-9c49-f4d1bca667ac@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

test/hotspot/jtreg/runtime/FieldLayout/ArrayBaseOffsets.java line 1:

> 1: /*

This file too suffered the same fate; all contents were removed but the file was not deleted.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1843710074

From wkemper at openjdk.org  Fri Nov 15 14:17:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 14:17:41 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <nXVt5nih04r0qv1t6Z1IvkSlJtP2uuPST8kaWS70QLE=.51ae819a-bba6-4cf5-85a2-8cb7054b5d8b@github.com>

Merges tag jdk-24+24

-------------

Commit messages:
 - 8343875: Minor improvements of jpackage test library
 - 8344074: RISC-V: C1: More accurate _exception_handler_size and _deopt_handler_size
 - 8339288: Improve diagnostic logging runtime/cds/DeterministicDump.java
 - 8343984: Fix Unsafe address overflow
 - 8343958: Remove security manager impl in java.lang.Process and java.lang.Runtime.exec
 - 8344039: Remove security manager dependency in java.time
 - 8343219: Manual clientlibs test failures after SM removal
 - 8343416: CDS dump fails when unregistered class can also be loaded from system modules
 - 8344023: Unnecessary Hashtable usage in LdapClient.defaultBinaryAttrs
 - 8343981: Remove usage of security manager from Thread and related classes
 - ... and 113 more: https://git.openjdk.org/shenandoah/compare/c0e6c3b9...95a00f8a

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=541&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/541/files
  Stats: 107173 lines in 2834 files changed: 28069 ins; 70167 del; 8937 mod
  Patch: https://git.openjdk.org/shenandoah/pull/541.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/541/head:pull/541

PR: https://git.openjdk.org/shenandoah/pull/541

From shade at openjdk.org  Fri Nov 15 14:49:11 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 15 Nov 2024 14:49:11 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
Message-ID: <ODhnNeT_5i6LznhL5iHpqdms0B_SbFsiWIGcYujUh-c=.f5d2490d-90ab-447b-ba69-74b72e0138ee@github.com>

On Wed, 13 Nov 2024 19:32:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
> 
>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>  - Use new CompactHeader forwarding APIs in generational mode
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge
>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>    
>    Reviewed-by: wkemper
>  - Merge
>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>    
>    Reviewed-by: kdnilsen
>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8

Sonar findings:

src/hotspot/share/gc/shenandoah/shenandoahEvacTracker.cpp line 39:

> 37:   if (_use_age_table) {
> 38:     _age_table = new AgeTable(false);
> 39:   }

Sonar caught it: Initialize `_age_table` to `nullptr` for extra safety. Current uses seem to be gated by `_use_age_table`, though.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1505:

> 1503:   size_t transferred_regions = 0;
> 1504:   ShenandoahLeftRightIterator iterator(&_partitions, which_collector, true);
> 1505:   idx_t rightmost = _partitions.rightmost_empty(which_collector);

Sonar: `rightmost` is not used.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1567:

> 1565:                                  <= _partitions.rightmost(ShenandoahFreeSetPartitionId::Collector))) {
> 1566:     ShenandoahHeapLocker locker(_heap->lock());
> 1567:     max_xfer_regions -=

Sonar: Value stored to `max_xfer_regions` here is not used.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1856:

> 1854:     size_t consumed_old_collector = 0;
> 1855:     size_t consumed_mutator = 0;
> 1856:     size_t available_old = 0;

Sonar: `available_old` is not used.  `available_young` is not used.

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 380:

> 378: 
> 379:   size_t old_evacuated = collection_set->get_old_bytes_reserved_for_evacuation();
> 380:   size_t old_evacuated_committed = (size_t) (ShenandoahOldEvacWaste * old_evacuated);

Sonar: implicit conversion from 'size_t' (aka 'unsigned long') to 'double' may lose precision

Not sure if it breaks any math.

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 455:

> 453:     // excess_old < unaffiliated old: we can give back MIN(excess_old/region_size_bytes, unaffiliated_old_regions)
> 454:     size_t excess_regions = excess_old / region_size_bytes;
> 455:     size_t regions_to_xfer = MIN2(excess_regions, unaffiliated_old_regions);

Sonar: Value stored to 'regions_to_xfer' during its initialization is never read

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 549:

> 547:     }
> 548:     if (r->age() >= tenuring_threshold) {
> 549:       if ((r->garbage() < old_garbage_threshold)) {

Sonar: double parentheses.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.cpp line 276:

> 274:             // where abundance is defined as >= ShenGenHeap::plab_min_size().  In the former case, we try shrinking the
> 275:             // desired PLAB size to the minimum and retry PLAB allocation to avoid cascading of shared memory allocations.
> 276:             if (plab->words_remaining() < plab_min_size()) {

Sonar caught it: There is a `plab != nullptr` check above at L267, which implies `plab` can be `nullptr` here? This would SEGV.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.cpp line 656:

> 654:     // may not evacuate the entirety of unprocessed candidates in a single mixed evacuation.
> 655:     const size_t max_evac_need = (size_t)
> 656:             (old_generation()->unprocessed_collection_candidates_live_memory() * ShenandoahOldEvacWaste);

Sonar: implicit conversion from 'size_t' (aka 'unsigned long') to 'double' may lose precision

Since this is a heuristics calculation, should we be precise here?

src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.cpp line 1003:

> 1001: }
> 1002: 
> 1003: namespace ShenandoahCompositeRegionClosure {

We usually avoid `namespace`-s. See Hotspot style guide: https://github.com/openjdk/jdk/blob/master/doc/hotspot-style.md#namespaces

src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.inline.hpp line 89:

> 87: }
> 88: 
> 89: HeapWord* ShenandoahHeapRegion::allocate(size_t size, ShenandoahAllocRequest req) {

Sonar caught it: `ShenandoahAllocRequest` should be passed by reference here?

src/hotspot/share/gc/shenandoah/shenandoahHeapRegionCounters.hpp line 99:

> 97:                       size_t region_size, size_t protocolVersion);
> 98: 
> 99:   uint _count = 0;

Sonar caught it: this `_count` field does not seem to be used.

src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.cpp line 157:

> 155:   size_t card_at_start = _rs->card_index_for_addr(address);
> 156:   HeapWord* card_start_address = _rs->addr_for_card_index(card_at_start);
> 157:   uint8_t offset_in_card = address - card_start_address;

Sonar: implicit conversion loses integer precision: 'long' to 'uint8_t' (aka 'unsigned char').

This looks risky. Should probably be `checked_cast<uint8_t>(pointer_delta(address, card_start_address, 1))`?

src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.cpp line 889:

> 887:   size_t previous_group_span = _group_entries[0] * _group_chunk_size[0];
> 888:   for (size_t i = 1; i < _num_groups; i++) {
> 889:     size_t previous_group_entries = (i == 1)? _group_entries[0]: (_group_entries[i-1] - _group_entries[i-2]);

Sonar: Value stored to `previous_group_entries` during its initialization is never read

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2438690431
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843878467
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843904346
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843905277
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843912193
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843933121
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843906258
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843898116
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843857836
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843931737
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843872870
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843886835
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843892961
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843928920
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843907601

From mdoerr at openjdk.org  Fri Nov 15 15:18:09 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Fri, 15 Nov 2024 15:18:09 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <2m-YJkwNcpIEDdVElZOw2uqbTD6eOoBM0cRGoSFdADM=.86118c91-b8dd-4da9-b691-5ffe0643f51d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <2m-YJkwNcpIEDdVElZOw2uqbTD6eOoBM0cRGoSFdADM=.86118c91-b8dd-4da9-b691-5ffe0643f51d@github.com>
Message-ID: <tcyuhY4JtpqC91jPFVj5FCsoLCmO1DFW8JmA-K8-Jug=.f8e78c33-95dc-4b55-94bc-f9c2978af478@github.com>

On Thu, 14 Nov 2024 23:39:08 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> The new test TestAllocOutOfMemory.java#large failed on some PPC64 machines:
> 
> Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
> 
> java.lang.RuntimeException: 'java.lang.OutOfMemoryError: Java heap space' missing from stdout/stderr
> 
> Probably not a show-stopper. Should the test be adapted or disabled for this platform?

> @TheRealMDoerr - do you have the logs from the test failure? If so, could you open a ticket with them in JBS? Thank you!

Filed [JDK-8344312](https://bugs.openjdk.org/browse/JDK-8344312).

In addition, we see the following tests failing with "java.lang.OutOfMemoryError: Java heap space" on all Shenandoah platforms:
gc/stress/jfr/TestStressBigAllocationGCEventsWithShenandoah.java#default
gc/stress/jfr/TestStressBigAllocationGCEventsWithShenandoah.java#generational
Can you reproduce those?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2479110413

From shade at openjdk.org  Fri Nov 15 15:18:09 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 15 Nov 2024 15:18:09 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
Message-ID: <odQVMW4CmqeLWr09oTRydm7Y7ktWqFiZmlwRcEe4M1s=.6dd995ab-f6f2-40bc-8490-c343dc30fadf@github.com>

On Wed, 13 Nov 2024 19:32:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
> 
>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>  - Use new CompactHeader forwarding APIs in generational mode
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge
>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>    
>    Reviewed-by: wkemper
>  - Merge
>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>    
>    Reviewed-by: kdnilsen
>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8

src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.hpp line 361:

> 359:   // frequently than last byte.  This is true when number of clean cards is greater than number of dirty cards.
> 360:   static const uint16_t ObjectStartsInCardRegion = 0x80;
> 361:   static const uint16_t FirstStartBits           = 0x7f;

I see these are used to do operations against `uint8_t` (`....offsets.first`). Any reason why these should not be `uint8_t`?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1843980993

From kdnilsen at openjdk.org  Fri Nov 15 16:09:06 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 15 Nov 2024 16:09:06 GMT
Subject: RFR: 8344260: GenShen: Excess unaffiliated regions reserved for
 old will not be transferred to young
In-Reply-To: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
References: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
Message-ID: <1V60FIzFlrIh4QWfaFkuviS2Y_XwO4GM4Ehawn6dyAo=.62cf8150-8e5d-4bb4-bccf-b2ea98c0d1ea@github.com>

On Fri, 15 Nov 2024 00:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Accidental assignment to new (and unread) variable.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/538#pullrequestreview-2439049091

From wkemper at openjdk.org  Fri Nov 15 17:20:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 17:20:11 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <ODhnNeT_5i6LznhL5iHpqdms0B_SbFsiWIGcYujUh-c=.f5d2490d-90ab-447b-ba69-74b72e0138ee@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <ODhnNeT_5i6LznhL5iHpqdms0B_SbFsiWIGcYujUh-c=.f5d2490d-90ab-447b-ba69-74b72e0138ee@github.com>
Message-ID: <kbQcnEJxFzcdnS4YwM7lYZSHyVZwQGGnhaoe9khCSmo=.83bbf0c3-d047-42e3-9f65-987e7e5b7ef2@github.com>

On Fri, 15 Nov 2024 14:05:51 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.cpp line 276:
> 
>> 274:             // where abundance is defined as >= ShenGenHeap::plab_min_size().  In the former case, we try shrinking the
>> 275:             // desired PLAB size to the minimum and retry PLAB allocation to avoid cascading of shared memory allocations.
>> 276:             if (plab->words_remaining() < plab_min_size()) {
> 
> Sonar caught it: There is a `plab != nullptr` check above at L267, which implies `plab` can be `nullptr` here? This would SEGV.

https://bugs.openjdk.org/browse/JDK-8344320

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844209761

From wkemper at openjdk.org  Fri Nov 15 17:29:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 17:29:08 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <ODhnNeT_5i6LznhL5iHpqdms0B_SbFsiWIGcYujUh-c=.f5d2490d-90ab-447b-ba69-74b72e0138ee@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <ODhnNeT_5i6LznhL5iHpqdms0B_SbFsiWIGcYujUh-c=.f5d2490d-90ab-447b-ba69-74b72e0138ee@github.com>
Message-ID: <N-xP6yN0pecMpMVONkKs-l0v2iitP8WMMUcELyOqqWQ=.ea0b6744-c548-4634-a23e-c72e284d8d0a@github.com>

On Fri, 15 Nov 2024 14:11:10 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahEvacTracker.cpp line 39:
> 
>> 37:   if (_use_age_table) {
>> 38:     _age_table = new AgeTable(false);
>> 39:   }
> 
> Sonar caught it: Initialize `_age_table` to `nullptr` for extra safety. Current uses seem to be gated by `_use_age_table`, though.

https://bugs.openjdk.org/browse/JDK-8344321

> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1856:
> 
>> 1854:     size_t consumed_old_collector = 0;
>> 1855:     size_t consumed_mutator = 0;
>> 1856:     size_t available_old = 0;
> 
> Sonar: `available_old` is not used.  `available_young` is not used.

https://bugs.openjdk.org/browse/JDK-8344321

> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 380:
> 
>> 378: 
>> 379:   size_t old_evacuated = collection_set->get_old_bytes_reserved_for_evacuation();
>> 380:   size_t old_evacuated_committed = (size_t) (ShenandoahOldEvacWaste * old_evacuated);
> 
> Sonar: implicit conversion from 'size_t' (aka 'unsigned long') to 'double' may lose precision
> 
> Not sure if it breaks any math.

https://bugs.openjdk.org/browse/JDK-8344321

> src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.cpp line 656:
> 
>> 654:     // may not evacuate the entirety of unprocessed candidates in a single mixed evacuation.
>> 655:     const size_t max_evac_need = (size_t)
>> 656:             (old_generation()->unprocessed_collection_candidates_live_memory() * ShenandoahOldEvacWaste);
> 
> Sonar: implicit conversion from 'size_t' (aka 'unsigned long') to 'double' may lose precision
> 
> Since this is a heuristics calculation, should we be precise here?

https://bugs.openjdk.org/browse/JDK-8344321

> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.inline.hpp line 89:
> 
>> 87: }
>> 88: 
>> 89: HeapWord* ShenandoahHeapRegion::allocate(size_t size, ShenandoahAllocRequest req) {
> 
> Sonar caught it: `ShenandoahAllocRequest` should be passed by reference here?

https://bugs.openjdk.org/browse/JDK-8344321

> src/hotspot/share/gc/shenandoah/shenandoahHeapRegionCounters.hpp line 99:
> 
>> 97:                       size_t region_size, size_t protocolVersion);
>> 98: 
>> 99:   uint _count = 0;
> 
> Sonar caught it: this `_count` field does not seem to be used.

https://bugs.openjdk.org/browse/JDK-8344321

> src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.cpp line 157:
> 
>> 155:   size_t card_at_start = _rs->card_index_for_addr(address);
>> 156:   HeapWord* card_start_address = _rs->addr_for_card_index(card_at_start);
>> 157:   uint8_t offset_in_card = address - card_start_address;
> 
> Sonar: implicit conversion loses integer precision: 'long' to 'uint8_t' (aka 'unsigned char').
> 
> This looks risky. Should probably be `checked_cast<uint8_t>(pointer_delta(address, card_start_address, 1))`?

https://bugs.openjdk.org/browse/JDK-8344321

> src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.cpp line 889:
> 
>> 887:   size_t previous_group_span = _group_entries[0] * _group_chunk_size[0];
>> 888:   for (size_t i = 1; i < _num_groups; i++) {
>> 889:     size_t previous_group_entries = (i == 1)? _group_entries[0]: (_group_entries[i-1] - _group_entries[i-2]);
> 
> Sonar: Value stored to `previous_group_entries` during its initialization is never read

https://bugs.openjdk.org/browse/JDK-8344321

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844218002
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844219136
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844219821
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844219644
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844218123
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844218269
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844219351
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844219014

From wkemper at openjdk.org  Fri Nov 15 18:28:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 18:28:59 GMT
Subject: RFR: 8344260: GenShen: Excess unaffiliated regions reserved for
 old will not be transferred to young [v2]
In-Reply-To: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
References: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
Message-ID: <zqbNHSuWYfy7L0jza_q8v2ykrbOZzPWvb5Wx1UqmLw4=.cd1d9567-993d-4f4b-a3c4-5776709780ef@github.com>

> Accidental assignment to new (and unread) variable.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix assert

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/538/files
  - new: https://git.openjdk.org/shenandoah/pull/538/files/a5edfbdf..f1774d51

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=538&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=538&range=00-01

  Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/538.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/538/head:pull/538

PR: https://git.openjdk.org/shenandoah/pull/538

From wkemper at openjdk.org  Fri Nov 15 18:46:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 18:46:21 GMT
Subject: Integrated: 8344152: GenShen: Initialize RegionData in product builds
In-Reply-To: <GDLnpATgoVomDKfLCwxSf0gWUNaVheO_PbxYxAzFQhM=.c40ff124-eebf-4a27-aa83-a7f40b53b1ec@github.com>
References: <GDLnpATgoVomDKfLCwxSf0gWUNaVheO_PbxYxAzFQhM=.c40ff124-eebf-4a27-aa83-a7f40b53b1ec@github.com>
Message-ID: <H3X7oKvL6b1XodyshtFTc2H0cgxTiQGYKbhrGtzf6c4=.91915c96-3f42-425d-9332-3abc5208929f@github.com>

On Wed, 13 Nov 2024 23:40:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Hygiene improvement.

This pull request has now been integrated.

Changeset: 1e3b0810
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/1e3b08101c20a3903de068d5d19aa779dd99d379
Stats:     8 lines in 2 files changed: 4 ins; 4 del; 0 mod

8344152: GenShen: Initialize RegionData in product builds

Reviewed-by: ysr, shade

-------------

PR: https://git.openjdk.org/shenandoah/pull/537

From wkemper at openjdk.org  Fri Nov 15 19:24:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 19:24:20 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v2]
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <SZPpBVubYwN_lF9UcQGkCMkV5KgPdYbP5_7tWMkoOo8=.77d8fe4d-6a06-44d4-942b-c1f382a5ed9a@github.com>

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Log heuristics decisions at info with ergo tag

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/540/files
  - new: https://git.openjdk.org/shenandoah/pull/540/files/f02d0245..14a91864

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=00-01

  Stats: 28 lines in 3 files changed: 0 ins; 2 del; 26 mod
  Patch: https://git.openjdk.org/shenandoah/pull/540.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/540/head:pull/540

PR: https://git.openjdk.org/shenandoah/pull/540

From wkemper at openjdk.org  Fri Nov 15 19:37:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 19:37:54 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v2]
In-Reply-To: <-4ElXqMU4hNKYEbkpsyURxSPEm3VNkUxY8BJLmnBDaI=.30ada198-7a05-46bb-bdf8-8e63c7288d62@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
 <-4ElXqMU4hNKYEbkpsyURxSPEm3VNkUxY8BJLmnBDaI=.30ada198-7a05-46bb-bdf8-8e63c7288d62@github.com>
Message-ID: <jh1dc4o-67e35ZiXUp0OmR130MMuIcxVDSv2pMnUooc=.aa9432d1-086d-4ecc-ba84-6f3863567ed3@github.com>

On Fri, 15 Nov 2024 11:26:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Log heuristics decisions at info with ergo tag
>
> Sorry, maybe I was too hard about `debug`. Maybe it should be `gc,ergo=info` instead, like the rest of heuristics code. "Want to see GC did? Enable -Xlog:gc. Want to see _why_ GC did it? Enable -Xlog:gc+ergo."

@shipilev - meet in the middle? Log heuristic's decisions at `info` with tag `ergo`, the rest of the changes in this PR continue logging at `debug`.

@ysramakrishna - A typical young cycle looks like this:


[3.055s][info][gc] Trigger (YOUNG): Average GC time (22.07 ms) is above the time for average allocation rate (53621 KB/s) to deplete free headroom (0B) (margin of error = 1.91)
[3.055s][info][gc] GC(29) Concurrent reset (Young) 0.042ms
[3.055s][info][gc] GC(29) Pause Init Mark (Young) 0.037ms
[3.055s][info][gc] GC(29) Concurrent remembered set scanning 0.064ms
[3.056s][info][gc] GC(29) Concurrent marking roots 0.278ms
[3.057s][info][gc] GC(29) Concurrent marking (Young) 1.779ms
[3.058s][info][gc] GC(29) Pause Final Mark (Young) 0.116ms
[3.058s][info][gc] GC(29) Concurrent weak references (Young) 0.017ms
[3.058s][info][gc] GC(29) Concurrent weak roots (Young) 0.116ms
[3.058s][info][gc] GC(29) Concurrent cleanup (Young) 30M->30M(32M) 0.003ms


A typical cycle for the non-generational mode looks like this:


[2.729s][info][gc] Trigger (GLOBAL): Average GC time (52.17 ms) is above the time for average allocation rate (20698 KB/s) to deplete free headroom (0B) (margin of error = 3.29)
[2.729s][info][gc] GC(42) Concurrent reset  (unload classes) 0.068ms
[2.730s][info][gc] GC(42) Pause Init Mark  (unload classes) 0.018ms
[2.730s][info][gc] GC(42) Concurrent marking roots 0.390ms
[2.735s][info][gc] GC(42) Concurrent marking  (unload classes) 4.471ms
[2.735s][info][gc] GC(42) Pause Final Mark  (unload classes) 0.111ms
[2.735s][info][gc] GC(42) Concurrent thread roots 0.080ms
[2.735s][info][gc] GC(42) Concurrent weak references  (unload classes) 0.047ms
[2.736s][info][gc] GC(42) Concurrent weak roots  (unload classes) 0.740ms
[2.736s][info][gc] GC(42) Concurrent cleanup  (unload classes) 4M->4M(32M) 0.002ms
[2.744s][info][gc] GC(42) Concurrent class unloading 8.163ms
[2.744s][info][gc] GC(42) Concurrent strong roots 0.158ms
[2.745s][info][gc] GC(42) Concurrent evacuation 0.420ms
[2.745s][info][gc] GC(42) Pause Init Update Refs 0.008ms
[2.747s][info][gc] GC(42) Concurrent update references 2.494ms
[2.747s][info][gc] GC(42) Concurrent update thread roots 0.310ms
[2.748s][info][gc] GC(42) Pause Final Update Refs 0.044ms
[2.748s][info][gc] GC(42) Concurrent cleanup  (unload classes) 13M->12M(32M) 0.029ms

The `Trigger` still shows the generational name in non-generational modes. I'll see about that.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/540#issuecomment-2479781360

From wkemper at openjdk.org  Fri Nov 15 20:23:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 20:23:03 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v3]
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <oJgmZexsP-xE0H6L7lWypciM1OH6EfTeBrLbM--xu9g=.a4299cf7-8118-43df-8ce3-5739dea245c7@github.com>

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Make generation names proper case

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/540/files
  - new: https://git.openjdk.org/shenandoah/pull/540/files/14a91864..7743ceb9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=01-02

  Stats: 11 lines in 5 files changed: 0 ins; 1 del; 10 mod
  Patch: https://git.openjdk.org/shenandoah/pull/540.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/540/head:pull/540

PR: https://git.openjdk.org/shenandoah/pull/540

From wkemper at openjdk.org  Fri Nov 15 20:25:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 20:25:43 GMT
Subject: Integrated: 8344151: GenShen: Improve comments,
 clean up unnecessary changes
In-Reply-To: <wow-94HiLm0410FNuFD30qf13kdW6t-oKEDSDUzUGDo=.96708f45-d1df-450b-b9bc-0e90ed31da6f@github.com>
References: <wow-94HiLm0410FNuFD30qf13kdW6t-oKEDSDUzUGDo=.96708f45-d1df-450b-b9bc-0e90ed31da6f@github.com>
Message-ID: <Lev96K09xEn2e-tYqOl24GuWwIIK69Ts7iCBPjjL7ig=.5cba0a0a-5203-4029-8dfb-763286b5de8d@github.com>

On Wed, 13 Nov 2024 23:31:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Address a few style nits from: https://github.com/openjdk/jdk/pull/21273

This pull request has now been integrated.

Changeset: efb3a058
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/efb3a05827bd52ff9258d58eaf23bc0cdc471da9
Stats:     10 lines in 4 files changed: 1 ins; 6 del; 3 mod

8344151: GenShen: Improve comments, clean up unnecessary changes

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/shenandoah/pull/536

From shade at openjdk.org  Fri Nov 15 20:30:47 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 15 Nov 2024 20:30:47 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v3]
In-Reply-To: <-4ElXqMU4hNKYEbkpsyURxSPEm3VNkUxY8BJLmnBDaI=.30ada198-7a05-46bb-bdf8-8e63c7288d62@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
 <-4ElXqMU4hNKYEbkpsyURxSPEm3VNkUxY8BJLmnBDaI=.30ada198-7a05-46bb-bdf8-8e63c7288d62@github.com>
Message-ID: <wD9JPWkbBIkRxoJbPS-eUHB5GPxZDMZEAWoFGjujFv4=.0d54b147-3f94-4144-a7e2-5040ead9b860@github.com>

On Fri, 15 Nov 2024 11:26:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Make generation names proper case
>
> Sorry, maybe I was too hard about `debug`. Maybe it should be `gc,ergo=info` instead, like the rest of heuristics code. "Want to see GC did? Enable -Xlog:gc. Want to see _why_ GC did it? Enable -Xlog:gc+ergo."

> @shipilev - meet in the middle? Log heuristic's decisions at `info` with tag `ergo`, the rest of the changes in this PR continue logging at `debug`.

Sure, OK.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/540#issuecomment-2479863166

From shade at openjdk.org  Fri Nov 15 20:30:48 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 15 Nov 2024 20:30:48 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v3]
In-Reply-To: <oJgmZexsP-xE0H6L7lWypciM1OH6EfTeBrLbM--xu9g=.a4299cf7-8118-43df-8ce3-5739dea245c7@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
 <oJgmZexsP-xE0H6L7lWypciM1OH6EfTeBrLbM--xu9g=.a4299cf7-8118-43df-8ce3-5739dea245c7@github.com>
Message-ID: <rCbGbw3USzX-Lp3FP7IBBpIs11EFpfvAgsX3JiM0Jk8=.898e96bf-9d20-433b-8357-0361d5b32283@github.com>

On Fri, 15 Nov 2024 20:23:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Make generation names proper case

src/hotspot/share/gc/shenandoah/shenandoahUtils.hpp line 50:

> 48:   switch (generation_type) {                                              \
> 49:     case NON_GEN:                                                         \
> 50:       return prefix " " postfix;                                          \

I think this causes a double space, for example here between "roots" and "(unload classes)" here:


[2.736s][info][gc] GC(42) Concurrent weak roots  (unload classes) 0.740ms

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/540#discussion_r1844424401

From wkemper at openjdk.org  Fri Nov 15 21:04:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 21:04:00 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v4]
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <sWvHoBBt_YFMwHjZ2shkasI9P2i7bkN4vEXhhR2q9BU=.bb2fd85f-4ade-4802-aafe-ad4c4d44f104@github.com>

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix formatting for non-generational log messages

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/540/files
  - new: https://git.openjdk.org/shenandoah/pull/540/files/7743ceb9..1c7752c1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=02-03

  Stats: 16 lines in 3 files changed: 0 ins; 2 del; 14 mod
  Patch: https://git.openjdk.org/shenandoah/pull/540.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/540/head:pull/540

PR: https://git.openjdk.org/shenandoah/pull/540

From wkemper at openjdk.org  Fri Nov 15 21:08:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 21:08:12 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v4]
In-Reply-To: <mfY9zvThGtuY-5EoPqEk3uvUufUKTRJEMPuDoUGPLcU=.7d413cfc-2fe8-48cc-9234-457c3608ff11@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
 <mfY9zvThGtuY-5EoPqEk3uvUufUKTRJEMPuDoUGPLcU=.7d413cfc-2fe8-48cc-9234-457c3608ff11@github.com>
Message-ID: <J97QOh1HPQzq7-FPzQnngdq-UcaStIEUmm_83U_NeIc=.3cbdad84-be4f-4803-a9b3-e17ae959dd60@github.com>

On Fri, 15 Nov 2024 03:50:02 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix formatting for non-generational log messages
>
> Can you share an example snippet (in the ticket or in the PR) of what we would see with a young and an old cycle with the reduced info-level logging?
> 
> The changes to convert from info to debug all look good to me.

@ysramakrishna - I've changed the generation names to also be proper case for consistency with other usages. On second thought, I don't think we need the generation name in the trigger messages. It's pretty clear from context what's being collected (non-generational mode):

[2.644s][info][gc] Trigger: Average GC time (52.87 ms) is above the time for average allocation rate (20071 KB/s) to deplete free headroom (0B) (margin of error = 3.27)
[2.644s][info][gc] GC(40) Concurrent reset (unload classes) 0.077ms
[2.645s][info][gc] GC(40) Pause Init Mark (unload classes) 0.057ms

(generational mode):

[3.176s][info][gc] Trigger: Average GC time (14.74 ms) is above the time for average allocation rate (51928 KB/s) to deplete free headroom (542K) (margin of error = 2.01)
[3.176s][info][gc] GC(28) Concurrent reset (Young) 0.075ms
[3.176s][info][gc] GC(28) Pause Init Mark (Young) 0.074ms

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/540#issuecomment-2479922303

From wkemper at openjdk.org  Fri Nov 15 21:27:57 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 21:27:57 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v3]
In-Reply-To: <rCbGbw3USzX-Lp3FP7IBBpIs11EFpfvAgsX3JiM0Jk8=.898e96bf-9d20-433b-8357-0361d5b32283@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
 <oJgmZexsP-xE0H6L7lWypciM1OH6EfTeBrLbM--xu9g=.a4299cf7-8118-43df-8ce3-5739dea245c7@github.com>
 <rCbGbw3USzX-Lp3FP7IBBpIs11EFpfvAgsX3JiM0Jk8=.898e96bf-9d20-433b-8357-0361d5b32283@github.com>
Message-ID: <WTrSxxWp6Vlx44ibWY8Ox4n742f8KDHoh-IZaImcaSM=.481cb612-ecbc-49f8-88fc-8f65590265b0@github.com>

On Fri, 15 Nov 2024 20:26:56 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Make generation names proper case
>
> src/hotspot/share/gc/shenandoah/shenandoahUtils.hpp line 50:
> 
>> 48:   switch (generation_type) {                                              \
>> 49:     case NON_GEN:                                                         \
>> 50:       return prefix " " postfix;                                          \
> 
> I think this causes a double space, for example here between "roots" and "(unload classes)" here:
> 
> 
> [2.736s][info][gc] GC(42) Concurrent weak roots  (unload classes) 0.740ms

Yes, fixed here: https://github.com/openjdk/shenandoah/pull/540/commits/1c7752c121853c131a352025f3d87cc179549506

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/540#discussion_r1844491208

From wkemper at openjdk.org  Fri Nov 15 21:35:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 21:35:03 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v3]
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Prevent uncommit thread from running during GC

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22019/files
  - new: https://git.openjdk.org/jdk/pull/22019/files/7301871e..997360ac

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=01-02

  Stats: 105 lines in 5 files changed: 89 ins; 4 del; 12 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From wkemper at openjdk.org  Fri Nov 15 23:14:02 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 23:14:02 GMT
Subject: RFR: 8344339: GenShen: Avoid using namespaces
Message-ID: <VaonCsNTXKlOVSidiLA_vNEo79bVFsOFSMMW0oxVmuc=.00443a94-49f5-40c8-b587-66d17f687e68@github.com>

Namespaces are contrary to hotspot style guide: https://github.com/openjdk/jdk/blob/master/doc/hotspot-style.md#namespaces

-------------

Commit messages:
 - Avoid using namespaces

Changes: https://git.openjdk.org/shenandoah/pull/542/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=542&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344339
  Stats: 4 lines in 1 file changed: 0 ins; 1 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/542.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/542/head:pull/542

PR: https://git.openjdk.org/shenandoah/pull/542

From wkemper at openjdk.org  Fri Nov 15 23:55:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 15 Nov 2024 23:55:08 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <tcyuhY4JtpqC91jPFVj5FCsoLCmO1DFW8JmA-K8-Jug=.f8e78c33-95dc-4b55-94bc-f9c2978af478@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <2m-YJkwNcpIEDdVElZOw2uqbTD6eOoBM0cRGoSFdADM=.86118c91-b8dd-4da9-b691-5ffe0643f51d@github.com>
 <tcyuhY4JtpqC91jPFVj5FCsoLCmO1DFW8JmA-K8-Jug=.f8e78c33-95dc-4b55-94bc-f9c2978af478@github.com>
Message-ID: <Nmo8MtCYNmULLdaiFIUMgxSDi6lgaJnIAcpvTAsrv2o=.c28972b7-22fd-4b09-9cd2-9f13ecf88f93@github.com>

On Fri, 15 Nov 2024 15:12:30 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:

>> The new test TestAllocOutOfMemory.java#large failed on some PPC64 machines:
>> 
>> Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
>> 
>> java.lang.RuntimeException: 'java.lang.OutOfMemoryError: Java heap space' missing from stdout/stderr
>> 
>> Probably not a show-stopper. Should the test be adapted or disabled for this platform?
>
>> @TheRealMDoerr - do you have the logs from the test failure? If so, could you open a ticket with them in JBS? Thank you!
> 
> Filed [JDK-8344312](https://bugs.openjdk.org/browse/JDK-8344312).
> 
> In addition, we see the following tests failing with "java.lang.OutOfMemoryError: Java heap space" on all Shenandoah platforms:
> gc/stress/jfr/TestStressBigAllocationGCEventsWithShenandoah.java#default
> gc/stress/jfr/TestStressBigAllocationGCEventsWithShenandoah.java#generational
> Can you reproduce those?

@TheRealMDoerr - Yes, those tests failed for me. I'll look into them. I'm also working to get my hands on a PPC64 machine to investigate [JDK-8344312](https://bugs.openjdk.org/browse/JDK-8344312). We appreciate your help here!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2480152786

From gziemski at openjdk.org  Sat Nov 16 02:42:56 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Sat, 16 Nov 2024 02:42:56 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
Message-ID: <OamPeqcItZRJy99HX43OnnK7kLTZwJK-8D6mh3AadxM=.2fdbbfb6-bbdc-4b5b-a96e-ebede918c768@github.com>

On Mon, 4 Nov 2024 07:18:02 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> We use mtNone value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
>> 
>> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
>> 
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> src/hotspot/cpu/aarch64/compressedKlass_aarch64.cpp line 72:
> 
>> 70:     assert(immediate > 0 && Assembler::operand_valid_for_logical_immediate(/*is32*/false, immediate),
>> 71:            "Invalid immediate %d " UINT64_FORMAT, index, immediate);
>> 72:     result = os::attempt_reserve_memory_at((char*)immediate, size, false, mtMetaspace);
> 
> mtMetaspace is wrong. There is no right tag apart from mtNone, since this address range is a combination of mtClass+mtClassShared, depending on how class space is configured. mtNone signals "will be set later", which is the correct course here IHMO.
> 
> The tag is set later, see metaspace.cpp and metaspaceShared.cpp

Can we introduce `mtAllocated` and use it here instead of `mtNone` and keep `mtNone` only as the initial "unassigned" state?

> src/hotspot/os/bsd/gc/z/zPhysicalMemoryBacking_bsd.cpp line 82:
> 
>> 80: 
>> 81:   // Reserve address space for backing memory
>> 82:   _base = (uintptr_t)os::reserve_memory(max_capacity, false, mtGC);
> 
> This is Java heap.

Fixed.

> src/hotspot/share/cds/archiveBuilder.cpp line 330:
> 
>> 328:   size_t buffer_size = estimate_archive_size();
>> 329:   ReservedSpace rs(buffer_size, MetaspaceShared::core_region_alignment(),
>> 330:                    os::vm_page_size(), nullptr, mtMetaspace);
> 
> This is not Metaspace. Possibly mtClassShared, though that one is reserved for the runtime mapping of the CDS archive; this is the dumptime image. mtInternal would be the best fit, IMHO.

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844882191
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844882271
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844882386

From gziemski at openjdk.org  Sat Nov 16 02:57:54 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Sat, 16 Nov 2024 02:57:54 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
Message-ID: <n76XBMTnJSTCFSIiSuPfT6yYr6JL_qWT4_F2gy67nfs=.8043cf94-0fe3-4282-8342-5ccd431b7ac0@github.com>

On Mon, 4 Nov 2024 07:38:24 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> We use mtNone value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
>> 
>> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
>> 
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> src/hotspot/share/cds/metaspaceShared.cpp line 1387:
> 
>> 1385:       total_space_rs = ReservedSpace(total_range_size, base_address_alignment,
>> 1386:                                      os::vm_page_size(), (char*)base_address, mtClass);
>> 1387:     } else {
> 
> All the changes here in `MetaspaceShared::reserve_address_space_for_archives` are unnecessary; we tag the address region already. No need to do this multiple times, and in every case branch. The ReservedSpace objects here are not used for much and mostly ephemeral anyway - they are only used to reserve memory here, but after that have not much meaning, and apart from the class space rs don't even survive the invocation.

ReservedSpace() requires the client to pass in the `mem_tag` and I think it would be a mistake to allow a default `mtNone` tag here just to avoid setting it. It might be ephemeral, but technically we can and should set it if it's known.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 249:
> 
>> 247:             _bitmap_bytes_per_slice, bitmap_page_size);
>> 248: 
>> 249:   ReservedSpace bitmap(_bitmap_size, bitmap_page_size, mtGC);
> 
> remove nmt registration below?

Fixed.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 269:
> 
>> 267: 
>> 268:   if (ShenandoahVerify) {
>> 269:     ReservedSpace verify_bitmap(_bitmap_size, bitmap_page_size, mtGC);
> 
> remove nmt registration below?

Fixed.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 288:
> 
>> 286: 
>> 287:   ReservedSpace aux_bitmap(_bitmap_size, aux_bitmap_page_size, mtGC);
>> 288:   os::trace_page_sizes_for_requested_size("Aux Bitmap",
> 
> remove nmt registration below?

Fixed.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 306:
> 
>> 304: 
>> 305:   ReservedSpace region_storage(region_storage_size, region_page_size, mtGC);
>> 306:   os::trace_page_sizes_for_requested_size("Region Storage",
> 
> remove nmt registration below?

Fixed.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 340:
> 
>> 338: 
>> 339:     if (_collection_set == nullptr) {
>> 340:       cset_rs = ReservedSpace(cset_size, cset_align, os::vm_page_size(), nullptr, mtGC);
> 
> remove NMT registration in ShenandoahCollectionSet?

Fixed.

> src/hotspot/share/jfr/recorder/storage/jfrVirtualMemory.cpp line 107:
> 
>> 105:   _rs = ReservedSpace(reservation_size_request_bytes,
>> 106:                       os::vm_allocation_granularity(), os::vm_page_size(),
>> 107:                       nullptr, mtTracing);
> 
> remove NMT registraton?

Fixed.

> src/hotspot/share/memory/heap.cpp line 231:
> 
>> 229:   // reserve space for _segmap
>> 230:   ReservedSpace seg_rs(reserved_segments_size, mtCode);
>> 231:   if (!_segmap.initialize(seg_rs, committed_segments_size)) {
> 
> remove NMT registration

Fixed.

> src/hotspot/share/memory/metaspace.cpp line 605:
> 
>> 603:     assert(is_aligned(result, Metaspace::reserve_alignment()), "Alignment too small for metaspace");
>> 604:     rs = ReservedSpace::space_for_range(result, size, Metaspace::reserve_alignment(),
>> 605:                                                       os::vm_page_size(), false, false, mtMetaspace);
> 
> Please revert to mtNone; same reason as given in metaspaceShared.cpp. You can add a comment that tags are adjusted afterward, when the distribution of mtClass and mtClassShared is clear.

Can we use `mtAllocated` used here?

> src/hotspot/share/memory/metaspace.cpp line 746:
> 
>> 744:       }
>> 745:       rs = ReservedSpace(size, Metaspace::reserve_alignment(),
>> 746:                          os::vm_page_size() /* large */, (char*)base, mtMetaspace);
> 
> mtClass, but actually not needed, since we set mtClass below. Again, not necessary to do this for every branch individually we take here when reserving class space memory.

Fixed.

Since ReservedSpace requires `mem_tag` we need to pass it.

> src/hotspot/share/memory/metaspace.cpp line 775:
> 
>> 773: 
>> 774:     // Mark metaspace as such
>> 775:     MemTracker::record_virtual_memory_tag((address)rs.base(), mtMetaspace);
> 
> No, please revert. mtClass is correct here.

Removed `MemTracker::record_virtual_memory_tag()`

> src/hotspot/share/memory/metaspace/testHelpers.cpp line 82:
> 
>> 80:     // have reserve limit -> non-expandable context
>> 81:     _rs = ReservedSpace(reserve_limit * BytesPerWord, Metaspace::reserve_alignment(),
>> 82:                         os::vm_page_size(), nullptr, MemTag::mtMetaspace);
> 
> Give this mtTest

Fixed.

> src/hotspot/share/memory/metaspace/virtualSpaceNode.cpp line 258:
> 
>> 256:   ReservedSpace rs(word_size * BytesPerWord,
>> 257:                    Settings::virtual_space_node_reserve_alignment_words() * BytesPerWord,
>> 258:                    os::vm_page_size(), nullptr, mtMetaspace);
> 
> remove tracking below.
> 
> This is actually wrong, and only works out of accident correctly, since this constructor just happens to be called for mtMetaspace; more correct would be to hand in the tag. This is on my plate, there is a JBS issue somewhere.

Fixed.

> src/hotspot/share/memory/virtualspace.hpp line 50:
> 
>> 48:   // ReservedSpace
>> 49:   ReservedSpace(char* base, size_t size, size_t alignment,
>> 50:                 size_t page_size, bool special, bool executable, MemTag mem_tag);
> 
> The growth in complexity is unfortunate.

I think the cost is worth the fact that we mark the memory with a tag as soon as possible, so that we don't catch NMT with its trousers down.

> src/hotspot/share/memory/virtualspace.hpp line 67:
> 
>> 65: 
>> 66:   void reserve(size_t size, size_t alignment, size_t page_size,
>> 67:                char* requested_address, bool executable, MemTag mem_tag);
> 
> Why do the instance methods get MemTag? Why is MemTag not stored as a (const) attribute of ReservedSpace? 
> 
> But then, we would have double accounting - we'd store the tag both in NMT and in ReservedSpace instances. If we change them post-reservation, we would need to change both. Sigh.
> 
> So we pass in MemTag just for the ReservedSpace to pass it into os::reserve_memory. Seeing how often ReservedSpace is used in situations where the tag is not clear at reservation time, I wonder whether we are not better off letting the user of ReservedSpace register the NMT tag post reservation like they do today.

I like the simplicity of not having to deal with changing the tag afterwards personally. Does anyone else have an opinion here?

> src/hotspot/share/nmt/memReporter.cpp line 83:
> 
>> 81:   if (mem_tag != mtNone) {
>> 82:     out->print("(%s" SIZE_FORMAT "%s type=%u", alloc_type,
>> 83:       amount_in_current_scale(amount), scale, (unsigned)mem_tag);
> 
> No, please revert.

Why? Seeing it as a string makes it more human readable.

Are we thinking that this output is parsed by tools?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844883480
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844883500
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844883510
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844883526
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844883544
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844883562
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844883605
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844883613
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844884091
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844884520
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844884665
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844884842
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844885543
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844888407
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844888178
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844888695

From gziemski at openjdk.org  Sat Nov 16 03:07:45 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Sat, 16 Nov 2024 03:07:45 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
Message-ID: <9g7xAohk4P0MsFU2U319Jp00W-lH6B5BkHFCJwkudc4=.389a332d-64a1-47f8-adab-4373cf1c92cd@github.com>

On Mon, 4 Nov 2024 08:23:13 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> We use mtNone value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
>> 
>> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
>> 
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> src/hotspot/share/nmt/memReporter.cpp line 252:
> 
>> 250:    // report malloc'd memory
>> 251:   if (amount_in_current_scale(MAX2(malloc_memory->malloc_size(), pk_malloc)) > 0) {
>> 252:     print_malloc(malloc_memory->malloc_counter(), mtNMT);
> 
> Please revert, the original was correct. print_malloc is used for both summary and detail reports; for summary, we pass in mtNone deliberately since the report happens under a common tag.

I need to look into this further, but I think "mtAllocated" would be better.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844890055

From gziemski at openjdk.org  Sat Nov 16 03:29:27 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Sat, 16 Nov 2024 03:29:27 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <22YzSj5_JK4_OVNjoKOSD-kfuEOH78MzSMPJNpkDCaA=.e35389cc-47a4-4780-ab40-a44d681e3ba0@github.com>

On Fri, 1 Nov 2024 20:44:50 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

> We use mtNone value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
> 
> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
> 
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

I'm going to put it into draft, since there are things that I need to follow up on, but would like to push my partial changes.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2480362444

From gziemski at openjdk.org  Sat Nov 16 03:29:30 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Sat, 16 Nov 2024 03:29:30 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
Message-ID: <hr_KvfB_48dLeYyP0nLWl9NCA6WnZcm5gLQ3bTJVUV4=.dbde46e6-6142-4c70-b823-59d1be799025@github.com>

On Mon, 4 Nov 2024 08:28:29 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> We use mtNone value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by removing the default value and forcing, where possible, for the callers to declare NMT tag that reflects the usage.
>> 
>> Eventually the goal would be not to use mtNone anywhere, but we are not there quite yet.
>> 
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> src/hotspot/share/nmt/virtualMemoryTracker.cpp line 364:
> 
>> 362:       // thread does not detach from VM before exits, and leads to
>> 363:       // leak JavaThread object
>> 364:       if ((reserved_rgn->mem_tag() == mtThreadStack) && (reserved_rgn->mem_tag() == mem_tag)) {
> 
> Please explain the changes in this function, especially the changed conditions. The added conditions for tag equality seem wrong to me.

Sorry, I think this was just a debugging code that I left it by mistake - I was testing whether we can tighten up the logic of when we allow the tags to be allowed to change, but this looks indeed wrong, since nothing would change.

Reverted.

> src/hotspot/share/runtime/safepointMechanism.cpp line 61:
> 
>> 59:     const size_t page_size = os::vm_page_size();
>> 60:     const size_t allocation_size = 2 * page_size;
>> 61:     char* polling_page = os::reserve_memory(allocation_size, false, mtSafepoint);
> 
> remove registration

Fixed.

> src/hotspot/share/utilities/debug.cpp line 712:
> 
>> 710: 
>> 711: void initialize_assert_poison() {
>> 712:   char* page = os::reserve_memory(os::vm_page_size(), false, mtInternal);
> 
> remove registration

Fixed.

> test/hotspot/jtreg/runtime/NMT/MallocRoundingReportTest.java line 60:
> 
>> 58:                 NMTTestUtils.runJcmdSummaryReportAndCheckOutput(
>> 59:                         "Test (reserved=" + numKB + "KB, committed=" + numKB + "KB)",
>> 60:                         "(malloc=" + numKB + "KB type=12 #1) (at peak)"
> 
> Again, this is wrong. The memory had not been reserved for NMT usage, so it should not be tagged with mtNMT (12)

Looking into it...

> test/hotspot/jtreg/runtime/NMT/MallocRoundingReportTest.java line 68:
> 
>> 66:                 NMTTestUtils.runJcmdSummaryReportAndCheckOutput(
>> 67:                         "Test (reserved=0KB, committed=0KB)",
>> 68:                         "(malloc=0KB type=12) (peak=" + numKB + "KB #1)"
> 
> ditto

Looking into it...

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844890480
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844890575
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844890617
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844890676
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1844890710

From xpeng at openjdk.org  Sat Nov 16 08:22:17 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Sat, 16 Nov 2024 08:22:17 GMT
Subject: RFR: 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow on
 Windows after JDK-8340490
Message-ID: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>

Fixing the regression on Windows caused by JDK-8340490, it bug is actually caused by difference behavior in `os:: os::elapsed_counter()`, Windows doesn't really high nanoseconds hi-res support,  instead nanoseconds it returns the the current value of the performance counter([link](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter)).

Some additional changes are also included in the PR for better performance and throughput. 

### Tests
 - [x] Verify gc/shenandoah/oom/TestClassLoaderLeak.java on Windows, no regression at all(improved slightly by ~1s) 
 - [x] Run test suites hotspot_gc_shenandoah with linux-aarch64-server-fastdebug

-------------

Commit messages:
 - Use os::javaTimeNanos() instead of os::elapsed_counter()
 - format
 - os::elapsed_counter() is not always nanoseconds, e.g. Windows
 - Use os::elapsedTime()
 - Use os::javaTimeMillis instead of os::elapsed_counter() since we don't really need high-res time
 - Merge branch 'openjdk:master' into JDK-8342041
 - fix
 - Packer stop waiting whenever budget is replenished
 - Always take _wait_monitor lock at least once when claim_for_alloc fails
 - 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow on Windows after JDK-8340490

Changes: https://git.openjdk.org/jdk/pull/22172/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22172&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8342041
  Stats: 27 lines in 2 files changed: 9 ins; 9 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/22172.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22172/head:pull/22172

PR: https://git.openjdk.org/jdk/pull/22172

From xpeng at openjdk.org  Sat Nov 16 08:57:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Sat, 16 Nov 2024 08:57:20 GMT
Subject: RFR: 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow
 on Windows after JDK-8340490 [v2]
In-Reply-To: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
References: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
Message-ID: <YXGTYnfGiSuH0nLpIqIaEYOXvfBqy2NwyGq5v0QQkxU=.6b63e3db-e729-4a72-bc58-4eff1260fccd@github.com>

> Fixing the regression on Windows caused by JDK-8340490, the bug is actually caused by different behavior in `os:: os::elapsed_counter()` which I wasn't aware of.  Windows doesn't have nanosecond hi-res clock support, so instead of nanoseconds it returns the the current value of the performance counter([link](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter)).
> 
> Some additional changes are also included in the PR for better performance and throughput. 
> 
> ### Tests
>  - [x] Verify gc/shenandoah/oom/TestClassLoaderLeak.java on Windows, no regression at all(improved slightly by ~1s comparing to original impl) 
>  - [x] Run test suites hotspot_gc_shenandoah with linux-aarch64-server-fastdebug

Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:

  Revert "8342044: Increase timeout of gc/shenandoah/oom/TestClassLoaderLeak.java"
  
  This reverts commit 2c0c65353b2f67bdcd954b4d2c2ae3e9b24d1c22.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22172/files
  - new: https://git.openjdk.org/jdk/pull/22172/files/64751d8f..d83dfbd4

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22172&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22172&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/22172.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22172/head:pull/22172

PR: https://git.openjdk.org/jdk/pull/22172

From kdnilsen at openjdk.org  Sat Nov 16 17:53:53 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 16 Nov 2024 17:53:53 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v3]
In-Reply-To: <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
Message-ID: <Iyjch3jsxnHZR8KpiJG3RxyWHEcNuCeWdPHyYgygh6c=.5fbf2ebe-e98d-4464-9128-cd0b31fc503c@github.com>

On Fri, 15 Nov 2024 21:35:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Prevent uncommit thread from running during GC

I'm ok with this, but best to wait for @shipilev approval before integrating.

-------------

Marked as reviewed by kdnilsen (Author).

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2440779513

From kdnilsen at openjdk.org  Sat Nov 16 17:53:54 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 16 Nov 2024 17:53:54 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v3]
In-Reply-To: <TP6J3QB781DC0NGZsi2SBMgiqcSJzZO5dex5ros5SY0=.a00ee441-9a31-4bee-bd75-869ebd7cb9b5@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <m0nT9Z9rWxv035VXKArefrumnP31iQF8YAke5XLF1y8=.7b7b8328-5dce-40d1-9ea0-2badd8aeac3a@github.com>
 <seTWtGW6jGmYoZhVAChkayjal9FEEI-94YJC5IJF3Yg=.4978ecfc-4bad-4df5-b7c3-a3473a4dcf52@github.com>
 <dLO84w60iGOGN53ams38IG8QHsiS2LJv7kStQQ_2jhE=.e3177cfd-8e38-40b8-bc41-d49ef134943b@github.com>
 <Si5UjES5jyaglpGVCt4FhxxegutJ_abxY-iDOj8pj-U=.258887b1-ef26-440d-92c5-c931f4330e06@github.com>
 <TP6J3QB781DC0NGZsi2SBMgiqcSJzZO5dex5ros5SY0=.a00ee441-9a31-4bee-bd75-869ebd7cb9b5@github.com>
Message-ID: <eXCZ1KCnkhhgjLJpd_HXqMrF9Qx2QWnnJV3XeA1w5Gg=.ca790b24-f511-4494-b2a1-970bd6d9a60b@github.com>

On Tue, 12 Nov 2024 19:20:09 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> So, wait a sec. This code is in `ShenandoahResetBitmapTask`, so it can run in parallel. Putting a lock here inhibits parallelism. I understand the failure mode, but I think we should really be optimizing for the case when `ShenandoahUncommit` is not enabled (e.g. `-Xmx` == `-Xms`).
>> 
>> Sounds like there is a hassle in allowing concurrent uncommit to overlap with the GC cycle. In addition to this particular problem, we might be stealing cycles from the GC threads and take additional TTSP lag to park the uncommitter for the in-cycle GC pauses. I have no clear solution for this yet, but I think we need to explore if we can suspend the uncommit before going into GC cycle...
>
> We could have the control and uncommit threads coordinate their efforts. In the worst case, it could mean delaying concurrent reset while the control thread waits for the uncommit thread to yield.
> 
> We could also try a more targeted lock only for the region's bitmap slice, but it doesn't seem right that one thread would be trying to clear a bitmap, while the other is trying to uncommit it. A lock could preserve technical correctness, but contention here would just mean that one thread would have wasted its time (either clearing a bitmap that is then uncommitted, or attempting to clear a bitmap that was first uncommitted (in this case, we would need the control thread to detect this and skip the region)).

Can we open a ticket to consider future improved concurrency by moving clear_bitmap(region) outside the global heap lock?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1845144702

From kdnilsen at openjdk.org  Sat Nov 16 17:57:01 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 16 Nov 2024 17:57:01 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
 <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
Message-ID: <AKEbtx-2XVnBn851w_lG7X1hH2Za0LykrXP10h1_AsM=.9871d6f9-6d58-4b9d-87af-3f03781147a6@github.com>

On Wed, 13 Nov 2024 23:21:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahVMOperations.cpp line 83:
>> 
>>> 81: void VM_ShenandoahInitMark::doit() {
>>> 82:   ShenandoahGCPauseMark mark(_gc_id, "Init Mark", SvcGCMarker::CONCURRENT);
>>> 83:   set_active_generation();
>> 
>> Why is it here, and not down in `entry_*` methods, probably in helper classes?
>
> I think it's here for the same reason `propagate_gc_state_to_java_threads` is here. Having it here makes it easier to see that this critical synchronization step happens for every safepoint.

I'm inclined to leave as is, not wanting to disrupt stability of code that is known to be working.  But if there is strong preference otherwise, we can make a change...

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1845149976

From xpeng at openjdk.org  Sat Nov 16 19:49:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Sat, 16 Nov 2024 19:49:20 GMT
Subject: RFR: 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow
 on Windows after JDK-8340490 [v3]
In-Reply-To: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
References: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
Message-ID: <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>

> Fixing the regression on Windows caused by JDK-8340490, the bug is actually caused by different behavior in `os:: os::elapsed_counter()` which I wasn't aware of.  Windows doesn't have nanosecond hi-res clock support, so instead of nanoseconds it returns the the current value of the performance counter([link](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter)).
> 
> ### Tests
>  - [x] Verify gc/shenandoah/oom/TestClassLoaderLeak.java on Windows, no regression at all(improved slightly by ~1s comparing to original impl) 
>  - [x] Run test suites hotspot_gc_shenandoah with linux-aarch64-server-fastdebug

Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:

 - Revert all the changes not related to the bug fix
 - Simplify pace_for_alloc

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22172/files
  - new: https://git.openjdk.org/jdk/pull/22172/files/d83dfbd4..19e782c6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22172&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22172&range=01-02

  Stats: 27 lines in 2 files changed: 9 ins; 9 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/22172.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22172/head:pull/22172

PR: https://git.openjdk.org/jdk/pull/22172

From fweimer at redhat.com  Mon Nov 18 09:05:18 2024
From: fweimer at redhat.com (Florian Weimer)
Date: Mon, 18 Nov 2024 10:05:18 +0100
Subject: CAS in load barrier self-healing
Message-ID: <87bjycx617.fsf@oldenburg.str.redhat.com>

How performance-sensitive is this code path?  I wonder if this could be
a regular store if the field is final, final fields are trusted, and the
class has only constructors that write to each final field once.

In this case, the data race is benign, I think: Outside the barrier, the
mutator may write a non-null pointer, but then the previous pointer must
have been null.  Several Two mutator threads may update the same field
in the barrier, but new object address is the same on all threads.

Thanks,
Florian


From shade at openjdk.org  Mon Nov 18 10:24:16 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 18 Nov 2024 10:24:16 GMT
Subject: RFR: 8344260: GenShen: Excess unaffiliated regions reserved for
 old will not be transferred to young [v2]
In-Reply-To: <zqbNHSuWYfy7L0jza_q8v2ykrbOZzPWvb5Wx1UqmLw4=.cd1d9567-993d-4f4b-a3c4-5776709780ef@github.com>
References: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
 <zqbNHSuWYfy7L0jza_q8v2ykrbOZzPWvb5Wx1UqmLw4=.cd1d9567-993d-4f4b-a3c4-5776709780ef@github.com>
Message-ID: <feIGYACKwvQLZr1FBl7r_bS9I3XoZ7TWuo5HL_LdpI4=.23bd0f72-2415-49bb-bb22-bce50e2a967a@github.com>

On Fri, 15 Nov 2024 18:28:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Accidental assignment to new (and unread) variable.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix assert

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/538#pullrequestreview-2442176817

From shade at openjdk.org  Mon Nov 18 10:24:27 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 18 Nov 2024 10:24:27 GMT
Subject: RFR: 8344339: GenShen: Avoid using namespaces
In-Reply-To: <VaonCsNTXKlOVSidiLA_vNEo79bVFsOFSMMW0oxVmuc=.00443a94-49f5-40c8-b587-66d17f687e68@github.com>
References: <VaonCsNTXKlOVSidiLA_vNEo79bVFsOFSMMW0oxVmuc=.00443a94-49f5-40c8-b587-66d17f687e68@github.com>
Message-ID: <L7fGqgCCydR5FHWMGdPPss29qYIqd5HXLvc4PoHjD1s=.71cbe1ec-0069-4fab-866f-27cfdb38b828@github.com>

On Fri, 15 Nov 2024 23:08:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Namespaces are contrary to hotspot style guide: https://github.com/openjdk/jdk/blob/master/doc/hotspot-style.md#namespaces

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/542#pullrequestreview-2442173932

From shade at openjdk.org  Mon Nov 18 12:34:48 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 18 Nov 2024 12:34:48 GMT
Subject: RFR: 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow
 on Windows after JDK-8340490 [v3]
In-Reply-To: <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>
References: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
 <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>
Message-ID: <GxpxYc8FL2j5QGg0ISCaJ_HfIUNsoGasMTbITIns4Y4=.5e0946a6-6fd8-43ce-a6a1-4144a55f1fea@github.com>

On Sat, 16 Nov 2024 19:49:20 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Fixing the regression on Windows caused by JDK-8340490, the bug is actually caused by different behavior in `os:: os::elapsed_counter()` which I wasn't aware of.  Windows doesn't have nanosecond hi-res clock support, so instead of nanoseconds it returns the the current value of the performance counter([link](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter)).
>> 
>> ### Tests
>>  - [x] Verify gc/shenandoah/oom/TestClassLoaderLeak.java on Windows, no regression at all(improved slightly by ~1s comparing to original impl) 
>>  - [x] Run test suites hotspot_gc_shenandoah with linux-aarch64-server-fastdebug
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Revert all the changes not related to the bug fix
>  - Simplify pace_for_alloc

Ouch. So `elapsed_counter` is nanoseconds only on POSIX! My bad for not catching this during the original review in [JDK-8340490](https://bugs.openjdk.org/browse/JDK-8340490). The fix looks good.

Attn @kdnilsen, @earthling-amzn.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22172#pullrequestreview-2442489636
PR Comment: https://git.openjdk.org/jdk/pull/22172#issuecomment-2482912074

From shipilev at amazon.de  Mon Nov 18 13:20:38 2024
From: shipilev at amazon.de (Aleksey Shipilev)
Date: Mon, 18 Nov 2024 14:20:38 +0100
Subject: CAS in load barrier self-healing
In-Reply-To: <87bjycx617.fsf@oldenburg.str.redhat.com>
References: <87bjycx617.fsf@oldenburg.str.redhat.com>
Message-ID: <51e2f192-6828-4b2e-9fc0-486569426798@amazon.de>

Hi,

On 18.11.24 10:05, Florian Weimer wrote:
> How performance-sensitive is this code path? 

Self-healing CAS is likely not a performance sensitive path. IIRC, in the overwhelming majority of 
experiments we have done, we rarely entered LRB slowpath. The bulk of the work is completed by the 
GC itself.

> I wonder if this could be a regular store if the field is final, final fields are trusted, and the
> class has only constructors that write to each final field once.

Java final fields are awkward in the sense that write-once guarantee does not actually exist for 
them. There is a weak constraint that language rules would refuse up-to-spec Java code to write to 
final fields, but you can still do it with Unsafe, Reflection, etc.

We used to have the final-field optimizations back in the days when we had read barriers, but 
ultimately we backed off doing smarter things there, because real-world Java code often does awkward 
things that break the properties you would have thought are guarantees, but actually are not.


> In this case, the data race is benign, I think: Outside the barrier, the
> mutator may write a non-null pointer, but then the previous pointer must
> have been null.  Several Two mutator threads may update the same field
> in the barrier, but new object address is the same on all threads.

Speaking of data races, there are secondary considerations around memory ordering, see the commit 
from here: https://bugs.openjdk.org/browse/JDK-8261495. *That one* is a large can of worms we would 
like to eventually figure out.

-Aleksey



Amazon Web Services Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597

From epeter at openjdk.org  Mon Nov 18 14:08:18 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:08:18 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
Message-ID: <_8-9ZYtVnjHTl3zce1wjZUCJZ6j1I5LgVfmUT4VKkm8=.74799b71-4c26-4c6c-8299-2efd02292548@github.com>

On Fri, 8 Nov 2024 17:42:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Could you please cherry pick https://github.com/mur47x111/jdk/commit/c45ebc2a89d0b25a3dd8cc46386e37a635ff9af2 for the JVMCI support?
>
> @mur47x111 it's now intergrated in jdk24. do your magic in Graal ;-)

@rkennke I have now looked more into the SuperWord collateral damage:
[JDK-8340010](https://bugs.openjdk.org/browse/JDK-8340010): Fix vectorization tests with compact headers

Do we care about `AlignVector` and `UseCompactObjectHeaders` enabled together? If so, we have a serious issue with mixed type examples. There are actually now some failing cases:


Failed IR Rules (3) of Methods (3)
----------------------------------
1) Method "public char[] compiler.vectorization.runner.ArrayTypeConvertTest.convertFloatToChar()" - [Failed IR rules: 1]:
   * @IR rule 1: "@compiler.lib.ir_framework.IR(phase={DEFAULT}, applyIfPlatformAnd={}, applyIfCPUFeatureOr={"asimd", "true", "avx2", "true"}, counts={"_#V#VECTOR_CAST_F2S#_", "_ at min(max_float, max_char)", ">0"}, applyIfPlatform={}, applyIfPlatformOr={}, failOn={}, applyIfOr={}, applyIfCPUFeatureAnd={}, applyIf={}, applyIfCPUFeature={}, applyIfAnd={}, applyIfNot={})"
     > Phase "PrintIdeal":
       - counts: Graph contains wrong number of nodes:
         * Constraint 1: "(\\d+(\\s){2}(VectorCastF2X.*)+(\\s){2}===.*vector[A-Za-z]<S,16>)"
           - Failed comparison: [found] 0 > 0 [given]
           - No nodes matched!

2) Method "public short[] compiler.vectorization.runner.ArrayTypeConvertTest.convertFloatToShort()" - [Failed IR rules: 1]:
   * @IR rule 1: "@compiler.lib.ir_framework.IR(phase={DEFAULT}, applyIfPlatformAnd={}, applyIfCPUFeatureOr={"asimd", "true", "avx2", "true"}, counts={"_#V#VECTOR_CAST_F2S#_", "_ at min(max_float, max_short)", ">0"}, applyIfPlatform={}, applyIfPlatformOr={}, failOn={}, applyIfOr={}, applyIfCPUFeatureAnd={}, applyIf={}, applyIfCPUFeature={}, applyIfAnd={}, applyIfNot={})"
     > Phase "PrintIdeal":
       - counts: Graph contains wrong number of nodes:
         * Constraint 1: "(\\d+(\\s){2}(VectorCastF2X.*)+(\\s){2}===.*vector[A-Za-z]<S,16>)"
           - Failed comparison: [found] 0 > 0 [given]
           - No nodes matched!

3) Method "public float[] compiler.vectorization.runner.ArrayTypeConvertTest.convertShortToFloat()" - [Failed IR rules: 1]:
   * @IR rule 1: "@compiler.lib.ir_framework.IR(phase={DEFAULT}, applyIfPlatformAnd={}, applyIfCPUFeatureOr={"asimd", "true", "avx2", "true"}, counts={"_#V#VECTOR_CAST_S2F#_", "_ at min(max_short, max_float)", ">0"}, applyIfPlatform={}, applyIfPlatformOr={}, failOn={}, applyIfOr={}, applyIfCPUFeatureAnd={}, applyIf={}, applyIfCPUFeature={}, applyIfAnd={}, applyIfNot={})"
     > Phase "PrintIdeal":
       - counts: Graph contains wrong number of nodes:
         * Constraint 1: "(\\d+(\\s){2}(VectorCastS2X.*)+(\\s){2}===.*vector[A-Za-z]<F,16>)"
           - Failed comparison: [found] 0 > 0 [given]
           - No nodes matched!


Let me explain:
If we enable AlignVector, we need 8-byte alignment.

As long as `UseCompactObjectHeaders` is disabled, all of these are `=16`:

UNSAFE.ARRAY_BYTE_BASE_OFFSET
UNSAFE.ARRAY_SHORT_BASE_OFFSET
UNSAFE.ARRAY_CHAR_BASE_OFFSET
UNSAFE.ARRAY_INT_BASE_OFFSET
UNSAFE.ARRAY_LONG_BASE_OFFSET
UNSAFE.ARRAY_FLOAT_BASE_OFFSET
UNSAFE.ARRAY_DOUBLE_BASE_OFFSET


However, with `UseCompactObjectHeaders` endabled, these are now 12:

UNSAFE.ARRAY_BYTE_BASE_OFFSET
UNSAFE.ARRAY_SHORT_BASE_OFFSET
UNSAFE.ARRAY_CHAR_BASE_OFFSET
UNSAFE.ARRAY_INT_BASE_OFFSET
UNSAFE.ARRAY_FLOAT_BASE_OFFSET

And these still 16:

UNSAFE.ARRAY_LONG_BASE_OFFSET
UNSAFE.ARRAY_DOUBLE_BASE_OFFSET


Now let's try to get that 8-byte alignment in some example, one from the above:

    public short[] convertFloatToShort() {
        short[] res = new short[SIZE];
        for (int i = 0; i < SIZE; i++) {
            res[i] = (short) floats[i];
        }
        return res;
    }


Let's look at the two addresses with `UseCompactObjectHeaders=false`, where we **can** vectorize:

F_adr = base + 16 + 4 * i
-> aligned for: i % 2 = 0
S_adr = base + 16 + 2 * i
-> aligned for: i % 4 = 0

-> solution for both: i % 4 = 0, i.e. we have alignment for both vector accesses every 4th iteration.


Let's look at the two addresses with `UseCompactObjectHeaders=true`, where we **cannot** vectorize:

F_adr = base + 12 + 4 * i
-> aligned for: i % 2 = 1
S_adr = base + 12 + 2 * i
-> aligned for: i % 4 = 2

-> There is no solution to satisfy both alignment constraints!


It's a little sad that I only just realized this now... but oh well. The issue is that we apparently did not run testing for these examples, so I did not see the impact immediately.

So my question: do we care about `UseCompactObjectHeaders` and `AlignVector` enabled at the same time? If so, we have to accept that some examples with mixed types simply will not vectorize.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483138198

From epeter at openjdk.org  Mon Nov 18 14:12:21 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:12:21 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
Message-ID: <UW9OLlZ6f-Pa-GT0eqjaOgCuroVZJj7meg18HNMqPYc=.0a2a4768-ab01-47f9-8d9d-688ee45416ea@github.com>

On Fri, 8 Nov 2024 17:42:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Could you please cherry pick https://github.com/mur47x111/jdk/commit/c45ebc2a89d0b25a3dd8cc46386e37a635ff9af2 for the JVMCI support?
>
> @mur47x111 it's now intergrated in jdk24. do your magic in Graal ;-)

@rkennke How important is the 4-byte saving on `byte, char, short, int, float` arrays? I'd assume they are not generally that small, at least a few elements? So could we make an exception, and have a `16-byte` offset to the payload of all these primitive (and maybe all) arrays, at least under `AlignVector`?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483153279

From rkennke at openjdk.org  Mon Nov 18 14:16:26 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 18 Nov 2024 14:16:26 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
Message-ID: <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>

On Fri, 8 Nov 2024 17:42:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Could you please cherry pick https://github.com/mur47x111/jdk/commit/c45ebc2a89d0b25a3dd8cc46386e37a635ff9af2 for the JVMCI support?
>
> @mur47x111 it's now intergrated in jdk24. do your magic in Graal ;-)

> @rkennke How important is the 4-byte saving on `byte, char, short, int, float` arrays? I'd assume they are not generally that small, at least a few elements? So could we make an exception, and have a `16-byte` offset to the payload of all these primitive (and maybe all) arrays, at least under `AlignVector`?

For byte[] and to some extend for char[] it is quite important, because those are the backing types for String and related classes, and Java apps often have *many* of them, and also quite small. I would not want to to sacrifize them for vectorization, especially not for the relatively uncommon (I think) case of mixed type access.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483162512

From epeter at openjdk.org  Mon Nov 18 14:20:20 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:20:20 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
Message-ID: <55hlCTAhtpoZT9LDQUkHwPQ5UUTylLzfNDYiFaBTXes=.9d9d6874-2f59-4833-9226-9e7f6410ca8d@github.com>

On Mon, 18 Nov 2024 14:13:17 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> @mur47x111 it's now intergrated in jdk24. do your magic in Graal ;-)
>
>> @rkennke How important is the 4-byte saving on `byte, char, short, int, float` arrays? I'd assume they are not generally that small, at least a few elements? So could we make an exception, and have a `16-byte` offset to the payload of all these primitive (and maybe all) arrays, at least under `AlignVector`?
> 
> For byte[] and to some extend for char[] it is quite important, because those are the backing types for String and related classes, and Java apps often have *many* of them, and also quite small. I would not want to to sacrifize them for vectorization, especially not for the relatively uncommon (I think) case of mixed type access.

@rkennke Ok, fair enough. As far as I know, we at Oracle do not super care about strict alignment `AlignVector`. But maybe other people care, and have to make that tradeoff between vectorization and small object headers.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483170957

From epeter at openjdk.org  Mon Nov 18 14:28:20 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:28:20 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
Message-ID: <wvQrqWpNxhvaKRqkaHCDjywoShedHn8wdRmuFHCpCDg=.63d2a0eb-9e03-40b4-b770-a799bd67021a@github.com>

On Mon, 18 Nov 2024 14:23:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> @rkennke How important is the 4-byte saving on `byte, char, short, int, float` arrays? I'd assume they are not generally that small, at least a few elements? So could we make an exception, and have a `16-byte` offset to the payload of all these primitive (and maybe all) arrays, at least under `AlignVector`?
>> 
>> For byte[] and to some extend for char[] it is quite important, because those are the backing types for String and related classes, and Java apps often have *many* of them, and also quite small. I would not want to to sacrifize them for vectorization, especially not for the relatively uncommon (I think) case of mixed type access.
>
>> @rkennke Ok, fair enough. As far as I know, we at Oracle do not super care about strict alignment `AlignVector`. But maybe other people care, and have to make that tradeoff between vectorization and small object headers.
> 
> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.
> 
> What is the failure mode, though? When running with -UCOH and +AlignVector, would it crash or misbehave? Or would it (silently?) not vectorize? I think we could live with the latter, but not with the former.

@rkennke It just will (silently) not vectorize, thus running slower but still correct.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483202341

From rkennke at openjdk.org  Mon Nov 18 14:28:20 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 18 Nov 2024 14:28:20 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
Message-ID: <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>

On Mon, 18 Nov 2024 14:13:17 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> @mur47x111 it's now intergrated in jdk24. do your magic in Graal ;-)
>
>> @rkennke How important is the 4-byte saving on `byte, char, short, int, float` arrays? I'd assume they are not generally that small, at least a few elements? So could we make an exception, and have a `16-byte` offset to the payload of all these primitive (and maybe all) arrays, at least under `AlignVector`?
> 
> For byte[] and to some extend for char[] it is quite important, because those are the backing types for String and related classes, and Java apps often have *many* of them, and also quite small. I would not want to to sacrifize them for vectorization, especially not for the relatively uncommon (I think) case of mixed type access.

> @rkennke Ok, fair enough. As far as I know, we at Oracle do not super care about strict alignment `AlignVector`. But maybe other people care, and have to make that tradeoff between vectorization and small object headers.

BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.

What is the failure mode, though? When running with -UCOH and +AlignVector, would it crash or misbehave? Or would it (silently?) not vectorize? I think we could live with the latter, but not with the former.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483195304

From epeter at openjdk.org  Mon Nov 18 14:41:21 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:41:21 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
Message-ID: <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>

On Mon, 18 Nov 2024 14:23:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> @rkennke How important is the 4-byte saving on `byte, char, short, int, float` arrays? I'd assume they are not generally that small, at least a few elements? So could we make an exception, and have a `16-byte` offset to the payload of all these primitive (and maybe all) arrays, at least under `AlignVector`?
>> 
>> For byte[] and to some extend for char[] it is quite important, because those are the backing types for String and related classes, and Java apps often have *many* of them, and also quite small. I would not want to to sacrifize them for vectorization, especially not for the relatively uncommon (I think) case of mixed type access.
>
>> @rkennke Ok, fair enough. As far as I know, we at Oracle do not super care about strict alignment `AlignVector`. But maybe other people care, and have to make that tradeoff between vectorization and small object headers.
> 
> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.
> 
> What is the failure mode, though? When running with -UCOH and +AlignVector, would it crash or misbehave? Or would it (silently?) not vectorize? I think we could live with the latter, but not with the former.

@rkennke
> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.

Sure. But I guess some people will want to run both `AlignVector` and `UseCompactObjectHeaders` in the future. Some machines simply do require strict alignment. So they will have to live with that tradeoff.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483225393

From qamai at openjdk.org  Mon Nov 18 14:41:21 2024
From: qamai at openjdk.org (Quan Anh Mai)
Date: Mon, 18 Nov 2024 14:41:21 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
Message-ID: <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>

On Mon, 18 Nov 2024 14:31:52 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>>> @rkennke Ok, fair enough. As far as I know, we at Oracle do not super care about strict alignment `AlignVector`. But maybe other people care, and have to make that tradeoff between vectorization and small object headers.
>> 
>> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.
>> 
>> What is the failure mode, though? When running with -UCOH and +AlignVector, would it crash or misbehave? Or would it (silently?) not vectorize? I think we could live with the latter, but not with the former.
>
> @rkennke
>> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.
> 
> Sure. But I guess some people will want to run both `AlignVector` and `UseCompactObjectHeaders` in the future. Some machines simply do require strict alignment. So they will have to live with that tradeoff.

@eme64 Tbh I don't see how `AlignVector` can mitigate the issue if strict alignment is required unless the object base is guaranteed to be aligned at least as much as the vector length.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483230986

From rkennke at openjdk.org  Mon Nov 18 14:41:21 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 18 Nov 2024 14:41:21 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
Message-ID: <sHj9-RwLVlCLoaaIl1PbYHwX5PPMqIhoL6ek2b4BVnM=.6151b583-36bc-4642-87ed-59de5e3bfbf1@github.com>

On Mon, 18 Nov 2024 14:23:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> @rkennke How important is the 4-byte saving on `byte, char, short, int, float` arrays? I'd assume they are not generally that small, at least a few elements? So could we make an exception, and have a `16-byte` offset to the payload of all these primitive (and maybe all) arrays, at least under `AlignVector`?
>> 
>> For byte[] and to some extend for char[] it is quite important, because those are the backing types for String and related classes, and Java apps often have *many* of them, and also quite small. I would not want to to sacrifize them for vectorization, especially not for the relatively uncommon (I think) case of mixed type access.
>
>> @rkennke Ok, fair enough. As far as I know, we at Oracle do not super care about strict alignment `AlignVector`. But maybe other people care, and have to make that tradeoff between vectorization and small object headers.
> 
> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.
> 
> What is the failure mode, though? When running with -UCOH and +AlignVector, would it crash or misbehave? Or would it (silently?) not vectorize? I think we could live with the latter, but not with the former.

> @rkennke It just will (silently) not vectorize, thus running slower but still correct.

Ok, I think we can live with that for now.

As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.

The tests need fixing, though.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483234723

From epeter at openjdk.org  Mon Nov 18 14:41:21 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:41:21 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
Message-ID: <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>

On Mon, 18 Nov 2024 14:34:13 GMT, Quan Anh Mai <qamai at openjdk.org> wrote:

>> @rkennke
>>> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.
>> 
>> Sure. But I guess some people will want to run both `AlignVector` and `UseCompactObjectHeaders` in the future. Some machines simply do require strict alignment. So they will have to live with that tradeoff.
>
> @eme64 Tbh I don't see how `AlignVector` can mitigate the issue if strict alignment is required unless the object base is guaranteed to be aligned at least as much as the vector length.

@merykitty the object base is always at least `8-byte` aligned, see `ObjectAlignmentInBytes` - this also holds for all arrays. But the issue is the offset from the object base to the array payload.

@rkennke yes, working on fixing the tests :)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483236250

From qamai at openjdk.org  Mon Nov 18 14:41:21 2024
From: qamai at openjdk.org (Quan Anh Mai)
Date: Mon, 18 Nov 2024 14:41:21 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
 <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
Message-ID: <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>

On Mon, 18 Nov 2024 14:36:17 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> @eme64 Tbh I don't see how `AlignVector` can mitigate the issue if strict alignment is required unless the object base is guaranteed to be aligned at least as much as the vector length.
>
> @merykitty the object base is always at least `8-byte` aligned, see `ObjectAlignmentInBytes` - this also holds for all arrays. But the issue is the offset from the object base to the array payload.
> 
> @rkennke yes, working on fixing the tests :)

@eme64 Please correct me if I'm wrong but the issue is you need the base to be aligned at 32 bytes on AVX2 machines for any alignment for vector instruction to be meaningful, so I don't see the value of vector alignment at all.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483241445

From epeter at openjdk.org  Mon Nov 18 14:41:22 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:41:22 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <sHj9-RwLVlCLoaaIl1PbYHwX5PPMqIhoL6ek2b4BVnM=.6151b583-36bc-4642-87ed-59de5e3bfbf1@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <sHj9-RwLVlCLoaaIl1PbYHwX5PPMqIhoL6ek2b4BVnM=.6151b583-36bc-4642-87ed-59de5e3bfbf1@github.com>
Message-ID: <LWcfTkWFr_EtYmJ_5af6dxZ93u4oV4i-9H9Eb3UIeYw=.000bf232-8346-4786-bb5d-078d2a308fea@github.com>

On Mon, 18 Nov 2024 14:35:41 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> @rkennke Ok, fair enough. As far as I know, we at Oracle do not super care about strict alignment `AlignVector`. But maybe other people care, and have to make that tradeoff between vectorization and small object headers.
>> 
>> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.
>> 
>> What is the failure mode, though? When running with -UCOH and +AlignVector, would it crash or misbehave? Or would it (silently?) not vectorize? I think we could live with the latter, but not with the former.
>
>> @rkennke It just will (silently) not vectorize, thus running slower but still correct.
> 
> Ok, I think we can live with that for now.
> 
> As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.
> 
> The tests need fixing, though.

@rkennke
> As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.

Ah. So we would eventually have not a `12-byte` but `8-byte` offset from base to payload? Would that happen in all cases? And could that happen before `UseCompactObjectHeaders` leaves experimental status? Because it is going to be a little annoying to adjust all vectorization tests for the special case of `UseCompactObjectHeaders + AlignVector`. Though I can surely do it.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483242899

From epeter at openjdk.org  Mon Nov 18 14:44:19 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:44:19 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
 <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
 <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>
Message-ID: <T6QHiE1n9zwjXcADCVCoep71pvccwaxF6mO3DkM4GMU=.7de33596-658c-4e4e-b472-d2bc1313bdb7@github.com>

On Mon, 18 Nov 2024 14:38:20 GMT, Quan Anh Mai <qamai at openjdk.org> wrote:

>> @merykitty the object base is always at least `8-byte` aligned, see `ObjectAlignmentInBytes` - this also holds for all arrays. But the issue is the offset from the object base to the array payload.
>> 
>> @rkennke yes, working on fixing the tests :)
>
> @eme64 Please correct me if I'm wrong but the issue is you need the base to be aligned at 32 bytes on AVX2 machines for any alignment for vector instruction to be meaningful, so I don't see the value of vector alignment at all.

@merykitty 
> Please correct me if I'm wrong but the issue is you need the base to be aligned at 32 bytes on AVX2 machines for any alignment for vector instruction to be meaningful, so I don't see the value of vector alignment at all.

First: without `AlignVector`, the vector instructions can have completely free alignment. On x64 and aarch64 generally I think most machines do not need alignment at all. And as far as I know there is also no performance penalty on modern CPUs for misalignment. I could be wrong here. On older CPUs alignment was important for performance though.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483249163

From qamai at openjdk.org  Mon Nov 18 14:56:28 2024
From: qamai at openjdk.org (Quan Anh Mai)
Date: Mon, 18 Nov 2024 14:56:28 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <T6QHiE1n9zwjXcADCVCoep71pvccwaxF6mO3DkM4GMU=.7de33596-658c-4e4e-b472-d2bc1313bdb7@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
 <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
 <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>
 <T6QHiE1n9zwjXcADCVCoep71pvccwaxF6mO
 3DkM4GMU=.7de33596-658c-4e4e-b472-d2bc1313bdb7@github.com>
Message-ID: <RyUA-340_zPYsg0k1sYOhuLmqMU2P0gDnHaASsQJOOo=.1bb26bb4-23c4-447b-8dfb-dbb7c351848c@github.com>

On Mon, 18 Nov 2024 14:41:25 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> @eme64 Please correct me if I'm wrong but the issue is you need the base to be aligned at 32 bytes on AVX2 machines for any alignment for vector instruction to be meaningful, so I don't see the value of vector alignment at all.
>
> @merykitty 
>> Please correct me if I'm wrong but the issue is you need the base to be aligned at 32 bytes on AVX2 machines for any alignment for vector instruction to be meaningful, so I don't see the value of vector alignment at all.
> 
> First: without `AlignVector`, the vector instructions can have completely free alignment. On x64 and aarch64 generally I think most machines do not need alignment at all. And as far as I know there is also no performance penalty on modern CPUs for misalignment. I could be wrong here. On older CPUs alignment was important for performance though.

@eme64 You will need the alignment for the whole vector (which means 32 bytes for a `ymm` load), not alignment only on its elements. Vector element is the artefact of ALU units, not the load/store units that actually care about alignment.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483255086

From epeter at openjdk.org  Mon Nov 18 14:56:28 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 14:56:28 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <RyUA-340_zPYsg0k1sYOhuLmqMU2P0gDnHaASsQJOOo=.1bb26bb4-23c4-447b-8dfb-dbb7c351848c@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
 <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
 <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>
 <T6QHiE1n9zwjXcADCVCoep71pvccwaxF6mO
 3DkM4GMU=.7de33596-658c-4e4e-b472-d2bc1313bdb7@github.com>
 <RyUA-340_zPYsg0k1sYOhuLmqMU2P0gDnHaASsQJOOo=.1bb26bb4-23c4-447b-8dfb-dbb7c351848c@github.com>
Message-ID: <7h8Il7V3a1tbo_U2y2GyUY2tH8UPXtKc3we3ZZi47d4=.4a4cbe88-92a5-43cf-a6a9-48d0bed41cf7@github.com>

On Mon, 18 Nov 2024 14:43:48 GMT, Quan Anh Mai <qamai at openjdk.org> wrote:

>> @merykitty 
>>> Please correct me if I'm wrong but the issue is you need the base to be aligned at 32 bytes on AVX2 machines for any alignment for vector instruction to be meaningful, so I don't see the value of vector alignment at all.
>> 
>> First: without `AlignVector`, the vector instructions can have completely free alignment. On x64 and aarch64 generally I think most machines do not need alignment at all. And as far as I know there is also no performance penalty on modern CPUs for misalignment. I could be wrong here. On older CPUs alignment was important for performance though.
>
> @eme64 You will need the alignment for the whole vector (which means 32 bytes for a `ymm` load), not alignment only on its elements. Vector element is the artefact of ALU units, not the load/store units that actually care about alignment.

@merykitty I don't think I understand. When and for what do I need the full 32-byte alignment?

@merykitty In `AlignmentSolver::solve` / `src/hotspot/share/opto/vectorization.cpp` you can see how I compute if vectors can be aligned.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483261148
PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483266962

From qamai at openjdk.org  Mon Nov 18 14:56:28 2024
From: qamai at openjdk.org (Quan Anh Mai)
Date: Mon, 18 Nov 2024 14:56:28 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <7h8Il7V3a1tbo_U2y2GyUY2tH8UPXtKc3we3ZZi47d4=.4a4cbe88-92a5-43cf-a6a9-48d0bed41cf7@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
 <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
 <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>
 <T6QHiE1n9zwjXcADCVCoep71pvccwaxF6mO
 3DkM4GMU=.7de33596-658c-4e4e-b472-d2bc1313bdb7@github.com>
 <RyUA-340_zPYsg0k1sYOhuLmqMU2P0gDnHaASsQJOOo=.1bb26bb4-23c4-447b-8dfb-dbb7c351848c@github.com>
 <7h8Il7V3a1tbo_U2y2GyUY2tH8UPXtKc3we3ZZi47d4=.4a4cbe88-92a5-43cf-a6a9-48d0bed41cf7@github.com>
Message-ID: <6rwCNBLV4-VemVsKR8KWYEgSIKfHQxS_RuxsPwX7TZo=.5fe167a3-1f97-408d-9d41-23d4d0fb42df@github.com>

On Mon, 18 Nov 2024 14:48:22 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> @eme64 You will need the alignment for the whole vector (which means 32 bytes for a `ymm` load), not alignment only on its elements. Vector element is the artefact of ALU units, not the load/store units that actually care about alignment.
>
> @merykitty In `AlignmentSolver::solve` / `src/hotspot/share/opto/vectorization.cpp` you can see how I compute if vectors can be aligned.

@eme64 If you load a 32-byte (256-bit) vector, then the load is aligned if the address is divisible by 32, otherwise the load is misaligned. That's why [`vmovdqua`](https://www.felixcloutier.com/x86/movdqa:vmovdqa32:vmovdqa64) requires 16-byte alignment for 16-byte loads/stores, 32-byte alignment for 32-byte loads/stores, 64-byte alignment for 64-byte loads/stores.

As a result, I don't see how you can align a vector load/store if the object base is only guaranteed to align at 8-byte boundaries. I mean there is no use trying to align an access if you cannot align it at the access size, the access is going to be misaligned anyway.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483275575

From rkennke at openjdk.org  Mon Nov 18 15:04:25 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 18 Nov 2024 15:04:25 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <sHj9-RwLVlCLoaaIl1PbYHwX5PPMqIhoL6ek2b4BVnM=.6151b583-36bc-4642-87ed-59de5e3bfbf1@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <sHj9-RwLVlCLoaaIl1PbYHwX5PPMqIhoL6ek2b4BVnM=.6151b583-36bc-4642-87ed-59de5e3bfbf1@github.com>
Message-ID: <K2YIK4uxtVVVaw9dZL4JF5B-byoZS7SQZmV8aNge9VQ=.e590d32b-1a64-4880-8b92-a070e952f9f4@github.com>

On Mon, 18 Nov 2024 14:35:41 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> @rkennke Ok, fair enough. As far as I know, we at Oracle do not super care about strict alignment `AlignVector`. But maybe other people care, and have to make that tradeoff between vectorization and small object headers.
>> 
>> BTW, this problem is not specific to UseCompactObjectHeaders - I think the same problem would happen with -UseCompressedClassPointers. With uncompressed class-pointers, byte[] would start at offset 20, while long[] start at offset 24. But nobody cares about -UCCP I think.
>> 
>> What is the failure mode, though? When running with -UCOH and +AlignVector, would it crash or misbehave? Or would it (silently?) not vectorize? I think we could live with the latter, but not with the former.
>
>> @rkennke It just will (silently) not vectorize, thus running slower but still correct.
> 
> Ok, I think we can live with that for now.
> 
> As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.
> 
> The tests need fixing, though.

> @rkennke
> 
> > As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.
> 
> Ah. So we would eventually have not a `12-byte` but `8-byte` offset from base to payload? Would that happen in all cases? And could that happen before `UseCompactObjectHeaders` leaves experimental status? Because it is going to be a little annoying to adjust all vectorization tests for the special case of `UseCompactObjectHeaders + AlignVector`. Though I can surely do it.

I am not sure if and when this is going to happen. When I presented the idea at JVMLS, I got some resistance. I am also not sure if we first leave experimental status for UCOH, and then introduce 4-byte headers under a new flag (or no flag?), or if we first do 4-byte headers and only leave experimental status once that is done. The latter sounds more reasonable to me.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483304257

From epeter at openjdk.org  Mon Nov 18 15:04:25 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 15:04:25 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <6rwCNBLV4-VemVsKR8KWYEgSIKfHQxS_RuxsPwX7TZo=.5fe167a3-1f97-408d-9d41-23d4d0fb42df@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
 <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
 <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>
 <T6QHiE1n9zwjXcADCVCoep71pvccwaxF6mO
 3DkM4GMU=.7de33596-658c-4e4e-b472-d2bc1313bdb7@github.com>
 <RyUA-340_zPYsg0k1sYOhuLmqMU2P0gDnHaASsQJOOo=.1bb26bb4-23c4-447b-8dfb-dbb7c351848c@github.com>
 <7h8Il7V3a1tbo_U2y2GyUY2tH8UPXtKc3we3ZZi47d4=.4a4cbe88-92a5-43cf-a6a9-48d0bed41cf7@github.com>
 <6rwCNBLV4-VemVsKR8KWYEgSIKfHQxS_RuxsPwX7TZo=.5fe167a3-1f97-408d-9d41-23d4d0fb42df@github.com>
Message-ID: <fXmxNXIiQx5OdHoSCzZgjaKwmEJprMevnwUuyOMYfXQ=.ee2e7590-2143-40bb-b514-6ef20f00823f@github.com>

On Mon, 18 Nov 2024 14:50:51 GMT, Quan Anh Mai <qamai at openjdk.org> wrote:

>> @merykitty In `AlignmentSolver::solve` / `src/hotspot/share/opto/vectorization.cpp` you can see how I compute if vectors can be aligned.
>
> @eme64 If you load a 32-byte (256-bit) vector, then the load is aligned if the address is divisible by 32, otherwise the load is misaligned. That's why [`vmovdqua`](https://www.felixcloutier.com/x86/movdqa:vmovdqa32:vmovdqa64) requires 16-byte alignment for 16-byte loads/stores, 32-byte alignment for 32-byte loads/stores, 64-byte alignment for 64-byte loads/stores.
> 
> As a result, I don't see how you can align a vector load/store if the object base is only guaranteed to align at 8-byte boundaries. I mean there is no use trying to align an access if you cannot align it at the access size, the access is going to be misaligned anyway.

@merykitty I guess we can always use [vmovdqu](https://www.felixcloutier.com/x86/movdqu:vmovdqu8:vmovdqu16:vmovdqu32:vmovdqu64).

And in fact that is exactly what we do:

public class Test {
    static int RANGE = 1024*1024;

    public static void main(String[] args) {
        byte[] aB = new byte[RANGE];
        byte[] bB = new byte[RANGE];
        for (int i = 0; i < 100_000; i++) {
            test1(aB, bB);
        }
    }

    static void test1(byte[] a, byte[] b) {                                    
        for (int i = 0; i < RANGE; i++) {                                                    
            a[i] = b[i];
        }                                                                                     
    }
}

`../java -XX:CompileCommand=compileonly,Test::test* -XX:CompileCommand=printcompilation,Test::test* -XX:+TraceLoopOpts -XX:-TraceSuperWord -XX:+TraceNewVectors -Xbatch -XX:+AlignVector -XX:CompileCommand=compileonly,Test::test* -XX:CompileCommand=printassembly,Test::test* Test.java`


 ;; B20: #      out( B20 B21 ) <- in( B19 B20 ) Loop( B20-B20 inner main of N178 strip mined) Freq: 8.13586e+09
  0x00007fc3a4bb0780:   movslq %ebx,%rdi
  0x00007fc3a4bb0783:   movslq %ebx,%r14
  0x00007fc3a4bb0786:   vmovdqu32 0x10(%r13,%r14,1),%zmm1
  0x00007fc3a4bb0791:   vmovdqu32 %zmm1,0x10(%r9,%r14,1)
  0x00007fc3a4bb079c:   vmovdqu32 0x50(%r13,%rdi,1),%zmm1
  0x00007fc3a4bb07a7:   vmovdqu32 %zmm1,0x50(%r9,%rdi,1)
  0x00007fc3a4bb07b2:   vmovdqu32 0x90(%r13,%rdi,1),%zmm1
  0x00007fc3a4bb07bd:   vmovdqu32 %zmm1,0x90(%r9,%rdi,1)
  0x00007fc3a4bb07c8:   vmovdqu32 0xd0(%r13,%rdi,1),%zmm1
  0x00007fc3a4bb07d3:   vmovdqu32 %zmm1,0xd0(%r9,%rdi,1)
  0x00007fc3a4bb07de:   vmovdqu32 0x110(%r13,%rdi,1),%zmm1
  0x00007fc3a4bb07e9:   vmovdqu32 %zmm1,0x110(%r9,%rdi,1)
  0x00007fc3a4bb07f4:   vmovdqu32 0x150(%r13,%rdi,1),%zmm1
  0x00007fc3a4bb07ff:   vmovdqu32 %zmm1,0x150(%r9,%rdi,1)
  0x00007fc3a4bb080a:   vmovdqu32 0x190(%r13,%rdi,1),%zmm1
  0x00007fc3a4bb0815:   vmovdqu32 %zmm1,0x190(%r9,%rdi,1)
  0x00007fc3a4bb0820:   vmovdqu32 0x1d0(%r13,%rdi,1),%zmm1
  0x00007fc3a4bb082b:   vmovdqu32 %zmm1,0x1d0(%r9,%rdi,1)   ;*bastore {reexecute=0 rethrow=0 return_oop=0}
                                                            ; - Test::test1 at 14 (line 14)
  0x00007fc3a4bb0836:   add    $0x200,%ebx                  ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                            ; - Test::test1 at 15 (line 13)
  0x00007fc3a4bb083c:   cmp    %r11d,%ebx
  0x00007fc3a4bb083f:   jl     0x00007fc3a4bb0780

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483305049

From qamai at openjdk.org  Mon Nov 18 15:23:34 2024
From: qamai at openjdk.org (Quan Anh Mai)
Date: Mon, 18 Nov 2024 15:23:34 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <fXmxNXIiQx5OdHoSCzZgjaKwmEJprMevnwUuyOMYfXQ=.ee2e7590-2143-40bb-b514-6ef20f00823f@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
 <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
 <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>
 <T6QHiE1n9zwjXcADCVCoep71pvccwaxF6mO
 3DkM4GMU=.7de33596-658c-4e4e-b472-d2bc1313bdb7@github.com>
 <RyUA-340_zPYsg0k1sYOhuLmqMU2P0gDnHaASsQJOOo=.1bb26bb4-23c4-447b-8dfb-dbb7c351848c@github.com>
 <7h8Il7V3a1tbo_U2y2GyUY2tH8UPXtKc3we3ZZi47d4=.4a4cbe88-92a5-43cf-a6a9-48d0bed41cf7@github.com>
 <6rwCNBLV4-VemVsKR8KWYEgSIKfHQxS_RuxsPwX7TZo=.5fe167a3-1f97-408d-9d41-23d4d0fb42df@github.com>
 <fXmxNXIiQx5OdHoSCzZgjaKwmEJprMevnwUuyOMYfXQ=.ee2e7590-2143-40bb-b514-6ef20f00823f@github.com>
Message-ID: <CQhgHm_cZPPVxn63q-tfdIxp6mye95vzwlmB4YAKB4A=.b44a79a7-6590-4e93-9586-84428a88f076@github.com>

On Mon, 18 Nov 2024 15:01:09 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> @eme64 If you load a 32-byte (256-bit) vector, then the load is aligned if the address is divisible by 32, otherwise the load is misaligned. That's why [`vmovdqua`](https://www.felixcloutier.com/x86/movdqa:vmovdqa32:vmovdqa64) requires 16-byte alignment for 16-byte loads/stores, 32-byte alignment for 32-byte loads/stores, 64-byte alignment for 64-byte loads/stores.
>> 
>> As a result, I don't see how you can align a vector load/store if the object base is only guaranteed to align at 8-byte boundaries. I mean there is no use trying to align an access if you cannot align it at the access size, the access is going to be misaligned anyway.
>
> @merykitty I guess we can always use [vmovdqu](https://www.felixcloutier.com/x86/movdqu:vmovdqu8:vmovdqu16:vmovdqu32:vmovdqu64).
> 
> And in fact that is exactly what we do:
> 
> public class Test {
>     static int RANGE = 1024*1024;
> 
>     public static void main(String[] args) {
>         byte[] aB = new byte[RANGE];
>         byte[] bB = new byte[RANGE];
>         for (int i = 0; i < 100_000; i++) {
>             test1(aB, bB);
>         }
>     }
> 
>     static void test1(byte[] a, byte[] b) {                                    
>         for (int i = 0; i < RANGE; i++) {                                                    
>             a[i] = b[i];
>         }                                                                                     
>     }
> }
> 
> `../java -XX:CompileCommand=compileonly,Test::test* -XX:CompileCommand=printcompilation,Test::test* -XX:+TraceLoopOpts -XX:-TraceSuperWord -XX:+TraceNewVectors -Xbatch -XX:+AlignVector -XX:CompileCommand=compileonly,Test::test* -XX:CompileCommand=printassembly,Test::test* Test.java`
> 
> 
>  ;; B20: #      out( B20 B21 ) <- in( B19 B20 ) Loop( B20-B20 inner main of N178 strip mined) Freq: 8.13586e+09
>   0x00007fc3a4bb0780:   movslq %ebx,%rdi
>   0x00007fc3a4bb0783:   movslq %ebx,%r14
>   0x00007fc3a4bb0786:   vmovdqu32 0x10(%r13,%r14,1),%zmm1
>   0x00007fc3a4bb0791:   vmovdqu32 %zmm1,0x10(%r9,%r14,1)
>   0x00007fc3a4bb079c:   vmovdqu32 0x50(%r13,%rdi,1),%zmm1
>   0x00007fc3a4bb07a7:   vmovdqu32 %zmm1,0x50(%r9,%rdi,1)
>   0x00007fc3a4bb07b2:   vmovdqu32 0x90(%r13,%rdi,1),%zmm1
>   0x00007fc3a4bb07bd:   vmovdqu32 %zmm1,0x90(%r9,%rdi,1)
>   0x00007fc3a4bb07c8:   vmovdqu32 0xd0(%r13,%rdi,1),%zmm1
>   0x00007fc3a4bb07d3:   vmovdqu32 %zmm1,0xd0(%r9,%rdi,1)
>   0x00007fc3a4bb07de:   vmovdqu32 0x110(%r13,%rdi,1),%zmm1
>   0x00007fc3a4bb07e9:   vmovdqu32 %zmm1,0x110(%r9,%rdi,1)
>   0x00007fc3a4bb07f4:   vmovdqu32 0x150(%r13,%rdi,1),%zmm1
>   0x00007fc3a4bb07ff:   vmovdqu32 %zmm1,0x150(%r9,%rdi,1)
>   0x00007fc3a4bb080a:   vmovdqu32 0x190(%r13,%rdi,1),%zmm1
>   0x00007fc3a4bb0815:   vmovdqu32 %zmm1,0x190(%r9,%rdi,1)
>   0x00007fc3a4bb0820:   vmovdqu32 0x1d0(%r13,%rdi,1),%zmm1
>   0x00007fc3a4bb082b:   vmovdqu32 %zmm1,0x1d0(%r9,%rdi,1)   ;*bastore {reexecute=0 rethrow=0 return_oop=0}
>                                                             ; - Test::test1 at 14 (line 14)
>   0x00007fc3a4bb0836:   add    $0x200,%ebx                  ;*iinc {reexecute=0 rethrow=0 return_oop=0}
>                                                             ; - Test::test1 at 15 (line 13)
>   0x00007fc3a4bb083c:   c...

@eme64 What I mean here is that `AlignVector` seems useless because the accesses are going to be misaligned either way.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483356306

From ihse at openjdk.org  Mon Nov 18 15:32:21 2024
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Mon, 18 Nov 2024 15:32:21 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <cI-IqlWyoHsFlZA2O6zhxhvSss5L2dbJeCydc-0MCLw=.bc364fb0-f4b2-46e3-b960-8426464e22d9@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
 <cI-IqlWyoHsFlZA2O6zhxhvSss5L2dbJeCydc-0MCLw=.bc364fb0-f4b2-46e3-b960-8426464e22d9@github.com>
Message-ID: <LDw4_Y_9UiuntX87ZcQG87bS-3alE4QMuBta2tJkWpM=.92a95e0f-6665-4610-93da-32fd86ab4dd5@github.com>

On Fri, 15 Nov 2024 04:49:51 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
>> 
>>  - Merge branch 'master' into JDK-8305895-v4
>>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>>    
>>    Added tag jdk-24+23 for changeset c0e6c3b9
>>  - Fix gen-ZGC removal
>>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>>    
>>    Added tag jdk-24+22 for changeset 388d44fb
>>  - Enable riscv in CompressedClassPointersEncodingScheme test
>>  - s390 port
>>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>>  - Update copyright
>>  - Avoid assert/endless-loop in JFR code
>>  - Update copyright headers
>>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b
>
> test/hotspot/jtreg/gtest/MetaspaceUtilsGtests.java line 1:
> 
> 
> This file was reduced to empty but not actually deleted. Can you fix it please.

@rkennke Just making sure this is not being missed. Can you please open a JBS issue to correct this and the file below?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1846790097

From epeter at openjdk.org  Mon Nov 18 16:20:34 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 16:20:34 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <CQhgHm_cZPPVxn63q-tfdIxp6mye95vzwlmB4YAKB4A=.b44a79a7-6590-4e93-9586-84428a88f076@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <3rKTyNEnmn0CsKA-GlyyzcxyD6hu9lulWO8N0GYO4vA=.8bfdde20-62a7-467d-8b79-dc3d3bb625f2@github.com>
 <G4-v2M7izsxIGrkgDyiZiLY5IrNrg1k5X0XGx7ZtW9g=.4e88f9aa-973d-4a16-8ebf-65304172b877@github.com>
 <ba-yFtsGlQe6vmFPEs3sK4dgh4yhaRdjQgydfzVCwuM=.3203a85c-41cc-44a3-881d-16c9bb87bc14@github.com>
 <qAraWaXtPiJVxtM5-d78a1tB2MImDtggAu8Rht8yGp0=.59622f2b-fa2f-4a6d-8667-a918ea4383b1@github.com>
 <T6QHiE1n9zwjXcADCVCoep71pvccwaxF6mO
 3DkM4GMU=.7de33596-658c-4e4e-b472-d2bc1313bdb7@github.com>
 <RyUA-340_zPYsg0k1sYOhuLmqMU2P0gDnHaASsQJOOo=.1bb26bb4-23c4-447b-8dfb-dbb7c351848c@github.com>
 <7h8Il7V3a1tbo_U2y2GyUY2tH8UPXtKc3we3ZZi47d4=.4a4cbe88-92a5-43cf-a6a9-48d0bed41cf7@github.com>
 <6rwCNBLV4-VemVsKR8KWYEgSIKfHQxS_RuxsPwX7TZo=.5fe167a3-1f97-408d-9d41-23d4d0fb42df@github.com>
 <fXmxNXIiQx5OdHoSCzZgjaKwmEJprMevnwUuyOMYfXQ=.ee2e7590-2143-40bb-b514-6ef20f00823f@github.com>
 <CQhgHm_cZPPVxn63q-tfdIxp6mye95vzwlmB4YAKB4A=.b44a79a7-6590-4e93-9586-84428a88f076@github.com>
Message-ID: <-uhyD7i_oXhrCIMqAvFf7nt6DsjM6OY-_erP6UDAitg=.bb94ed2c-75f1-4d7a-b45a-113a5886a268@github.com>

On Mon, 18 Nov 2024 15:20:17 GMT, Quan Anh Mai <qamai at openjdk.org> wrote:

>> @merykitty I guess we can always use [vmovdqu](https://www.felixcloutier.com/x86/movdqu:vmovdqu8:vmovdqu16:vmovdqu32:vmovdqu64).
>> 
>> And in fact that is exactly what we do:
>> 
>> public class Test {
>>     static int RANGE = 1024*1024;
>> 
>>     public static void main(String[] args) {
>>         byte[] aB = new byte[RANGE];
>>         byte[] bB = new byte[RANGE];
>>         for (int i = 0; i < 100_000; i++) {
>>             test1(aB, bB);
>>         }
>>     }
>> 
>>     static void test1(byte[] a, byte[] b) {                                    
>>         for (int i = 0; i < RANGE; i++) {                                                    
>>             a[i] = b[i];
>>         }                                                                                     
>>     }
>> }
>> 
>> `../java -XX:CompileCommand=compileonly,Test::test* -XX:CompileCommand=printcompilation,Test::test* -XX:+TraceLoopOpts -XX:-TraceSuperWord -XX:+TraceNewVectors -Xbatch -XX:+AlignVector -XX:CompileCommand=compileonly,Test::test* -XX:CompileCommand=printassembly,Test::test* Test.java`
>> 
>> 
>>  ;; B20: #      out( B20 B21 ) <- in( B19 B20 ) Loop( B20-B20 inner main of N178 strip mined) Freq: 8.13586e+09
>>   0x00007fc3a4bb0780:   movslq %ebx,%rdi
>>   0x00007fc3a4bb0783:   movslq %ebx,%r14
>>   0x00007fc3a4bb0786:   vmovdqu32 0x10(%r13,%r14,1),%zmm1
>>   0x00007fc3a4bb0791:   vmovdqu32 %zmm1,0x10(%r9,%r14,1)
>>   0x00007fc3a4bb079c:   vmovdqu32 0x50(%r13,%rdi,1),%zmm1
>>   0x00007fc3a4bb07a7:   vmovdqu32 %zmm1,0x50(%r9,%rdi,1)
>>   0x00007fc3a4bb07b2:   vmovdqu32 0x90(%r13,%rdi,1),%zmm1
>>   0x00007fc3a4bb07bd:   vmovdqu32 %zmm1,0x90(%r9,%rdi,1)
>>   0x00007fc3a4bb07c8:   vmovdqu32 0xd0(%r13,%rdi,1),%zmm1
>>   0x00007fc3a4bb07d3:   vmovdqu32 %zmm1,0xd0(%r9,%rdi,1)
>>   0x00007fc3a4bb07de:   vmovdqu32 0x110(%r13,%rdi,1),%zmm1
>>   0x00007fc3a4bb07e9:   vmovdqu32 %zmm1,0x110(%r9,%rdi,1)
>>   0x00007fc3a4bb07f4:   vmovdqu32 0x150(%r13,%rdi,1),%zmm1
>>   0x00007fc3a4bb07ff:   vmovdqu32 %zmm1,0x150(%r9,%rdi,1)
>>   0x00007fc3a4bb080a:   vmovdqu32 0x190(%r13,%rdi,1),%zmm1
>>   0x00007fc3a4bb0815:   vmovdqu32 %zmm1,0x190(%r9,%rdi,1)
>>   0x00007fc3a4bb0820:   vmovdqu32 0x1d0(%r13,%rdi,1),%zmm1
>>   0x00007fc3a4bb082b:   vmovdqu32 %zmm1,0x1d0(%r9,%rdi,1)   ;*bastore {reexecute=0 rethrow=0 return_oop=0}
>>                                                             ; - Test::test1 at 14 (line 14)
>>   0x00007fc3a4bb0836:   add    $0x200,%ebx                  ;*iinc {reexecute=0 rethrow=0 return_oop=0}
>>                     ...
>
> @eme64 What I mean here is that `AlignVector` seems useless because the accesses are going to be misaligned either way.

@merykitty FYI:

`src/hotspot/share/opto/vectorization.hpp:  static bool vectors_should_be_aligned() { return !Matcher::misaligned_vectors_ok() || AlignVector; }`


The relevant code:

src/hotspot/cpu/x86/matcher_x86.hpp:  static constexpr bool misaligned_vectors_ok() {
  // x86 supports misaligned vectors store/load.
  static constexpr bool misaligned_vectors_ok() {
    return true;
  }


src/hotspot/cpu/ppc/matcher_ppc.hpp:  static constexpr bool misaligned_vectors_ok() {
  // PPC implementation uses VSX load/store instructions (if
  // SuperwordUseVSX) which support 4 byte but not arbitrary alignment
  static constexpr bool misaligned_vectors_ok() {
    return false;
  }

src/hotspot/cpu/aarch64/matcher_aarch64.hpp:  static constexpr bool misaligned_vectors_ok() {
  // aarch64 supports misaligned vectors store/load.
  static constexpr bool misaligned_vectors_ok() {
    return true;
  }

src/hotspot/cpu/s390/matcher_s390.hpp:  static constexpr bool misaligned_vectors_ok() {
  // z/Architecture does support misaligned store/load at minimal extra cost.
  static constexpr bool misaligned_vectors_ok() {
    return true;
  }

src/hotspot/cpu/arm/matcher_arm.hpp:  static constexpr bool misaligned_vectors_ok() {
  // ARM doesn't support misaligned vectors store/load.
  static constexpr bool misaligned_vectors_ok() {
    return false;
  }

src/hotspot/cpu/riscv/matcher_riscv.hpp:  static constexpr bool misaligned_vectors_ok() {
  // riscv supports misaligned vectors store/load.
  static constexpr bool misaligned_vectors_ok() {
    return true;
  }


We can see that only PPC and ARM32 have such strict alignment requirements. And it turns out that PPC only needs 4-byte alignment, and ARM32 is fine with 8-byte alignment. So all of our platforms do not necessarily need full vector-width alignment.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483505834

From epeter at openjdk.org  Mon Nov 18 16:32:28 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 16:32:28 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <ZsR40NNlKPWgfZsKgZ0Mf1NUab67xV7ZZlThTnHvx2Q=.68fea8c3-78a6-4893-8f46-64af1af40573@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

Ah there are some exceptions:

x86:
`src/hotspot/cpu/x86/vm_version_x86.cpp:    AlignVector = !UseUnalignedLoadStores;`


      if (supports_sse4_2()) { // new ZX cpus
        if (FLAG_IS_DEFAULT(UseUnalignedLoadStores)) {
          UseUnalignedLoadStores = true; // use movdqu on newest ZX cpus
        }
      }

So I suppose some older platforms may be affected, though I have not seen one yet. They would have to be missing the unaligned `movdqu` instructions.

aarch64:
`src/hotspot/cpu/aarch64/vm_version_aarch64.cpp:    AlignVector = AvoidUnalignedAccesses;`


  // Ampere eMAG
  if (_cpu == CPU_AMCC && (_model == CPU_MODEL_EMAG) && (_variant == 0x3)) {
    if (FLAG_IS_DEFAULT(AvoidUnalignedAccesses)) {
      FLAG_SET_DEFAULT(AvoidUnalignedAccesses, true);
    }

and

  // ThunderX
  if (_cpu == CPU_CAVIUM && (_model == 0xA1)) {
    guarantee(_variant != 0, "Pre-release hardware no longer supported.");
    if (FLAG_IS_DEFAULT(AvoidUnalignedAccesses)) {
      FLAG_SET_DEFAULT(AvoidUnalignedAccesses, true);
    }

and

  // ThunderX2
  if ((_cpu == CPU_CAVIUM && (_model == 0xAF)) ||
      (_cpu == CPU_BROADCOM && (_model == 0x516))) {
    if (FLAG_IS_DEFAULT(AvoidUnalignedAccesses)) {
      FLAG_SET_DEFAULT(AvoidUnalignedAccesses, true);
    }

and

  // HiSilicon TSV110
  if (_cpu == CPU_HISILICON && _model == 0xd01) {
    if (FLAG_IS_DEFAULT(AvoidUnalignedAccesses)) {
      FLAG_SET_DEFAULT(AvoidUnalignedAccesses, true);
    }


So yes, some platforms are affected. But they seem to be the exception.

And again: we have only had `ObjectAlignmentInBytes=8` alignment for vectors since forever - and no platform vendor has ever complained about that. Arrays never had a stronger alignment guarantee than that.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483528037
PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483531916

From wkemper at openjdk.org  Mon Nov 18 16:46:09 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Nov 2024 16:46:09 GMT
Subject: Integrated: 8344339: GenShen: Avoid using namespaces
In-Reply-To: <VaonCsNTXKlOVSidiLA_vNEo79bVFsOFSMMW0oxVmuc=.00443a94-49f5-40c8-b587-66d17f687e68@github.com>
References: <VaonCsNTXKlOVSidiLA_vNEo79bVFsOFSMMW0oxVmuc=.00443a94-49f5-40c8-b587-66d17f687e68@github.com>
Message-ID: <uuYJg-67y9g1fZi57XtHvmEC-GIyhMWTvpuYg1enwzQ=.9a58e6e8-8245-435a-adb2-3462a1c94b54@github.com>

On Fri, 15 Nov 2024 23:08:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Namespaces are contrary to hotspot style guide: https://github.com/openjdk/jdk/blob/master/doc/hotspot-style.md#namespaces

This pull request has now been integrated.

Changeset: f851bcd3
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/f851bcd34057f5eec13ddf20dffa3e8763b63bc3
Stats:     4 lines in 1 file changed: 0 ins; 1 del; 3 mod

8344339: GenShen: Avoid using namespaces

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/shenandoah/pull/542

From epeter at openjdk.org  Mon Nov 18 16:52:22 2024
From: epeter at openjdk.org (Emanuel Peter)
Date: Mon, 18 Nov 2024 16:52:22 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <K2YIK4uxtVVVaw9dZL4JF5B-byoZS7SQZmV8aNge9VQ=.e590d32b-1a64-4880-8b92-a070e952f9f4@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <sHj9-RwLVlCLoaaIl1PbYHwX5PPMqIhoL6ek2b4BVnM=.6151b583-36bc-4642-87ed-59de5e3bfbf1@github.com>
 <K2YIK4uxtVVVaw9dZL4JF5B-byoZS7SQZmV8aNge9VQ=.e590d32b-1a64-4880-8b92-a070e952f9f4@github.com>
Message-ID: <DCDH1z-FZR6G6pMA2syspQuNtBYqWpdEioQiRyWA-z8=.c3c6f7cc-2fed-44a9-bd78-2426e28a2c34@github.com>

On Mon, 18 Nov 2024 15:00:51 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> @rkennke It just will (silently) not vectorize, thus running slower but still correct.
>> 
>> Ok, I think we can live with that for now.
>> 
>> As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.
>> 
>> The tests need fixing, though.
>
>> @rkennke
>> 
>> > As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.
>> 
>> Ah. So we would eventually have not a `12-byte` but `8-byte` offset from base to payload? Would that happen in all cases? And could that happen before `UseCompactObjectHeaders` leaves experimental status? Because it is going to be a little annoying to adjust all vectorization tests for the special case of `UseCompactObjectHeaders + AlignVector`. Though I can surely do it.
> 
> I am not sure if and when this is going to happen. When I presented the idea at JVMLS, I got some resistance. I am also not sure if we first leave experimental status for UCOH, and then introduce 4-byte headers under a new flag (or no flag?), or if we first do 4-byte headers and only leave experimental status once that is done. The latter sounds more reasonable to me.

@rkennke Filed a bug to track this (we may close it as NotAnIssue, but this way people are aware / can find the analysis):
[JDK-8344424](https://bugs.openjdk.org/browse/JDK-8344424): C2 SuperWord: mixed type loops do not vectorize with UseCompactObjectHeaders and AlignVector

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483579571

From wkemper at openjdk.org  Mon Nov 18 16:52:53 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Nov 2024 16:52:53 GMT
Subject: Integrated: 8344260: GenShen: Excess unaffiliated regions reserved for
 old will not be transferred to young
In-Reply-To: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
References: <r4Gtorazjp701mMIP-exfinjocVJ8xafTEmrpnKzXGM=.eb1ed970-5b6b-462a-bde3-9fd5a833b109@github.com>
Message-ID: <52FUr2W2DhgoDKtjnevgrnRYF0_T3BbTE21nJ6raNog=.f7934df7-06c8-41d6-84e6-7bfa0a2621e3@github.com>

On Fri, 15 Nov 2024 00:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Accidental assignment to new (and unread) variable.

This pull request has now been integrated.

Changeset: b5842a73
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/b5842a738ce181a26c12708ea58c11d9f0a759b1
Stats:     4 lines in 1 file changed: 2 ins; 0 del; 2 mod

8344260: GenShen: Excess unaffiliated regions reserved for old will not be transferred to young

Reviewed-by: ysr, shade, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/538

From rkennke at openjdk.org  Mon Nov 18 17:00:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 18 Nov 2024 17:00:24 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <LDw4_Y_9UiuntX87ZcQG87bS-3alE4QMuBta2tJkWpM=.92a95e0f-6665-4610-93da-32fd86ab4dd5@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
 <cI-IqlWyoHsFlZA2O6zhxhvSss5L2dbJeCydc-0MCLw=.bc364fb0-f4b2-46e3-b960-8426464e22d9@github.com>
 <LDw4_Y_9UiuntX87ZcQG87bS-3alE4QMuBta2tJkWpM=.92a95e0f-6665-4610-93da-32fd86ab4dd5@github.com>
Message-ID: <HvYJCbEPHich6bY-vODY9r83xySJMnicTt89_IymmYA=.7c198e02-ef29-454d-8530-a02fc1a07c10@github.com>

On Mon, 18 Nov 2024 15:30:14 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

>> test/hotspot/jtreg/gtest/MetaspaceUtilsGtests.java line 1:
>> 
>> 
>> This file was reduced to empty but not actually deleted. Can you fix it please.
>
> @rkennke Just making sure this is not being missed. Can you please open a JBS issue to correct this and the file below?

I filed:
https://bugs.openjdk.org/browse/JDK-8344425
@tstuefe is working on it (mostly checking that nothing important has been removed)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20677#discussion_r1846945329

From wkemper at openjdk.org  Mon Nov 18 17:01:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Nov 2024 17:01:39 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <fCQhKSyRmZIprJNkE78hBD3RTOrmVEb-bYviw3Y4tp8=.e0f8aa9f-507c-4c94-a124-7abfd825be5d@github.com>
References: <fCQhKSyRmZIprJNkE78hBD3RTOrmVEb-bYviw3Y4tp8=.e0f8aa9f-507c-4c94-a124-7abfd825be5d@github.com>
Message-ID: <lnI9MV-M2YhgKVzJNeZ5H-4YEy4wwieJMZs4Yg_cUmI=.8764dadb-a166-4316-a16e-b1248b2f1fae@github.com>

> Merges tag jdk-21.0.6+3

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/137/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/137/files/9bdf2a61..9bdf2a61

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=137&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=137&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/137.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/137/head:pull/137

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/137

From wkemper at openjdk.org  Mon Nov 18 17:01:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Nov 2024 17:01:40 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <fCQhKSyRmZIprJNkE78hBD3RTOrmVEb-bYviw3Y4tp8=.e0f8aa9f-507c-4c94-a124-7abfd825be5d@github.com>
References: <fCQhKSyRmZIprJNkE78hBD3RTOrmVEb-bYviw3Y4tp8=.e0f8aa9f-507c-4c94-a124-7abfd825be5d@github.com>
Message-ID: <7r40YccadpVKnHCn7Pr85Rp50KyhGqPkhKcom77cvio=.dadbf12a-47af-4c01-82ce-b97811a0cc79@github.com>

On Thu, 14 Nov 2024 14:19:50 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.6+3

This pull request has now been integrated.

Changeset: 0f8bf026
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/0f8bf026d3341ae0fb6891fa7dcdb44290c9487c
Stats:     1634 lines in 274 files changed: 696 ins; 179 del; 759 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/137

From rkennke at openjdk.org  Mon Nov 18 17:09:24 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 18 Nov 2024 17:09:24 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v19]
In-Reply-To: <K2YIK4uxtVVVaw9dZL4JF5B-byoZS7SQZmV8aNge9VQ=.e590d32b-1a64-4880-8b92-a070e952f9f4@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <FENW_mCsovAzl3Jpaf-3Sz4Lm8PWbCYlDHEqrDT4qjM=.0f01c195-d868-4ba0-a3f6-2bdf8e95e510@github.com>
 <RfVmBzOwNMSLLg29JMF6QNg_5I_VBr4sYTepEVylQc8=.02dbd983-ddb2-4602-914f-1d0e13c05e29@github.com>
 <zzFZKTS9H1YIiuSMeyp6zhjC3_4JpwROSllDTrjCWf8=.afcbbe98-cf83-4966-b9ce-e3abd601847d@github.com>
 <Fo5eAnOZQpOQ4Z6J6QfjVxHHNVUFb1tipGTdxZpkUj4=.595732e0-875c-44a1-a324-5a963388e88b@github.com>
 <mMpZPw_YwSO54z9YRL1tG5zroYYz6g_fVH3H4ASVcd0=.db2a3014-79e0-4ba1-9093-6d8dc435ca23@github.com>
 <sHj9-RwLVlCLoaaIl1PbYHwX5PPMqIhoL6ek2b4BVnM=.6151b583-36bc-4642-87ed-59de5e3bfbf1@github.com>
 <K2YIK4uxtVVVaw9dZL4JF5B-byoZS7SQZmV8aNge9VQ=.e590d32b-1a64-4880-8b92-a070e952f9f4@github.com>
Message-ID: <mvz9A9NHoGCVW04AE1H9Bt2yHTLE8IGq7GjzV_27ouc=.5d3677b8-c07e-4bd8-be55-7e5d5fb1dab1@github.com>

On Mon, 18 Nov 2024 15:00:51 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> @rkennke It just will (silently) not vectorize, thus running slower but still correct.
>> 
>> Ok, I think we can live with that for now.
>> 
>> As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.
>> 
>> The tests need fixing, though.
>
>> @rkennke
>> 
>> > As said elsewhere, we are currently working on 4-byte-headers, which would make that problem go away.
>> 
>> Ah. So we would eventually have not a `12-byte` but `8-byte` offset from base to payload? Would that happen in all cases? And could that happen before `UseCompactObjectHeaders` leaves experimental status? Because it is going to be a little annoying to adjust all vectorization tests for the special case of `UseCompactObjectHeaders + AlignVector`. Though I can surely do it.
> 
> I am not sure if and when this is going to happen. When I presented the idea at JVMLS, I got some resistance. I am also not sure if we first leave experimental status for UCOH, and then introduce 4-byte headers under a new flag (or no flag?), or if we first do 4-byte headers and only leave experimental status once that is done. The latter sounds more reasonable to me.

> @rkennke Filed a bug to track this (we may close it as NotAnIssue, but this way people are aware / can find the analysis): [JDK-8344424](https://bugs.openjdk.org/browse/JDK-8344424): C2 SuperWord: mixed type loops do not vectorize with UseCompactObjectHeaders and AlignVector

Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2483619681

From shade at openjdk.org  Mon Nov 18 17:43:47 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 18 Nov 2024 17:43:47 GMT
Subject: RFR: 8344264: GenShen: Improve comments and method names
In-Reply-To: <9DLcYwQmX_oU0EhHjrhmTEW2iEauhr83ARyKEDR9O5Y=.a378906e-ec7a-4493-b3bd-5f9ad3c7f599@github.com>
References: <9DLcYwQmX_oU0EhHjrhmTEW2iEauhr83ARyKEDR9O5Y=.a378906e-ec7a-4493-b3bd-5f9ad3c7f599@github.com>
Message-ID: <FaSc4a6Thc3LOcGiGMW_LJB6MtwTUP0pPCJF6XKD-cE=.da3441aa-d6ce-4c75-a135-8fe7a4d0a339@github.com>

On Fri, 15 Nov 2024 01:26:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

> More hygiene for GenShen.

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/539#pullrequestreview-2443296474

From wkemper at openjdk.org  Mon Nov 18 18:15:47 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Nov 2024 18:15:47 GMT
Subject: Integrated: 8344264: GenShen: Improve comments and method names
In-Reply-To: <9DLcYwQmX_oU0EhHjrhmTEW2iEauhr83ARyKEDR9O5Y=.a378906e-ec7a-4493-b3bd-5f9ad3c7f599@github.com>
References: <9DLcYwQmX_oU0EhHjrhmTEW2iEauhr83ARyKEDR9O5Y=.a378906e-ec7a-4493-b3bd-5f9ad3c7f599@github.com>
Message-ID: <hZcF69FUgKSf9N26MUD5wY2vxuMO7wju0otBAK1KHTc=.55cedb7e-bc79-468b-a81c-e81b072f2b6d@github.com>

On Fri, 15 Nov 2024 01:26:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

> More hygiene for GenShen.

This pull request has now been integrated.

Changeset: 8bc2dff6
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/8bc2dff6803ff053a0b498266c85f78df958e5c0
Stats:     15 lines in 7 files changed: 4 ins; 0 del; 11 mod

8344264: GenShen: Improve comments and method names

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/shenandoah/pull/539

From ysr at openjdk.org  Mon Nov 18 18:29:17 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 18 Nov 2024 18:29:17 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v4]
In-Reply-To: <sWvHoBBt_YFMwHjZ2shkasI9P2i7bkN4vEXhhR2q9BU=.bb2fd85f-4ade-4802-aafe-ad4c4d44f104@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
 <sWvHoBBt_YFMwHjZ2shkasI9P2i7bkN4vEXhhR2q9BU=.bb2fd85f-4ade-4802-aafe-ad4c4d44f104@github.com>
Message-ID: <YfbOPlfJoKYYu3YXcfBt4jDkoNkRDiqulJ-PHnAdS4E=.5076fc36-c85a-48d7-a65a-af829eed2b55@github.com>

On Fri, 15 Nov 2024 21:04:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix formatting for non-generational log messages

One question re:

> [3.055s][info][gc] Trigger: Average GC time (22.07 ms) is above the time for average allocation rate (53621 KB/s) to deplete free headroom (0B) (margin of error = 1.91)
[3.055s][info][gc] GC(29) Concurrent reset (Young) 0.042ms
[3.055s][info][gc] GC(29) Pause Init Mark (Young) 0.037ms
[3.055s][info][gc] GC(29) Concurrent remembered set scanning 0.064ms
[3.056s][info][gc] GC(29) Concurrent marking roots 0.278ms
[3.057s][info][gc] GC(29) Concurrent marking (Young) 1.779ms
[3.058s][info][gc] GC(29) Pause Final Mark (Young) 0.116ms
[3.058s][info][gc] GC(29) Concurrent weak references (Young) 0.017ms
[3.058s][info][gc] GC(29) Concurrent weak roots (Young) 0.116ms
[3.058s][info][gc] GC(29) Concurrent cleanup (Young) 30M->30M(32M) 0.003ms

Is there a reason not to use "(Young)" against the remainder of the phases as well for clarity, when appropriate? Also wondering if the same would be true of the Trigger.

I imagine there are Triggers for Old. How are they logged?

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/540#issuecomment-2483807263

From wkemper at openjdk.org  Mon Nov 18 21:56:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Nov 2024 21:56:01 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v5]
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <7gEJ-mbo8YcMW7tDkET_Gz0tP5Dh1V34YVMwsCx5OJw=.d226bea8-7357-4a83-869f-d5a91fb21280@github.com>

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Update test, remove unused header, use proper case for old gen phases

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/540/files
  - new: https://git.openjdk.org/shenandoah/pull/540/files/1c7752c1..df0748ff

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=04
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=03-04

  Stats: 10 lines in 4 files changed: 1 ins; 2 del; 7 mod
  Patch: https://git.openjdk.org/shenandoah/pull/540.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/540/head:pull/540

PR: https://git.openjdk.org/shenandoah/pull/540

From wkemper at openjdk.org  Mon Nov 18 22:32:02 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 18 Nov 2024 22:32:02 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <tcyuhY4JtpqC91jPFVj5FCsoLCmO1DFW8JmA-K8-Jug=.f8e78c33-95dc-4b55-94bc-f9c2978af478@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <2m-YJkwNcpIEDdVElZOw2uqbTD6eOoBM0cRGoSFdADM=.86118c91-b8dd-4da9-b691-5ffe0643f51d@github.com>
 <tcyuhY4JtpqC91jPFVj5FCsoLCmO1DFW8JmA-K8-Jug=.f8e78c33-95dc-4b55-94bc-f9c2978af478@github.com>
Message-ID: <YqFdsk9sVxVXXbMyv3OX4saxbBTDgg6il_eAofgH6X0=.0da584a9-2599-4952-8002-b1d654f150e8@github.com>

On Fri, 15 Nov 2024 15:12:30 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:

>> The new test TestAllocOutOfMemory.java#large failed on some PPC64 machines:
>> 
>> Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
>> 
>> java.lang.RuntimeException: 'java.lang.OutOfMemoryError: Java heap space' missing from stdout/stderr
>> 
>> Probably not a show-stopper. Should the test be adapted or disabled for this platform?
>
>> @TheRealMDoerr - do you have the logs from the test failure? If so, could you open a ticket with them in JBS? Thank you!
> 
> Filed [JDK-8344312](https://bugs.openjdk.org/browse/JDK-8344312).
> 
> In addition, we see the following tests failing with "java.lang.OutOfMemoryError: Java heap space" on all Shenandoah platforms:
> gc/stress/jfr/TestStressBigAllocationGCEventsWithShenandoah.java#default
> gc/stress/jfr/TestStressBigAllocationGCEventsWithShenandoah.java#generational
> Can you reproduce those?

@TheRealMDoerr , it looks like @ysramakrishna has [filed a ticket already](https://bugs.openjdk.org/browse/JDK-8342786) for the `TestStressBigAllocationGCEventsWithShenandoah` failures.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2484266111

From kdnilsen at openjdk.org  Mon Nov 18 23:21:54 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 18 Nov 2024 23:21:54 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v3]
In-Reply-To: <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
Message-ID: <OxYrUbenEc2WjyaB1SeDptsns5cvd3dvY8d9mJ_YTMU=.d3d763eb-c773-4b19-bcd4-2bf46822fcef@github.com>

On Fri, 15 Nov 2024 21:35:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Prevent uncommit thread from running during GC

Have read through the latest version of the code.  Thanks.

-------------

Marked as reviewed by kdnilsen (Author).

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2443969990

From ysr at openjdk.org  Tue Nov 19 02:05:55 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 19 Nov 2024 02:05:55 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v3]
In-Reply-To: <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
Message-ID: <2aAK-4vQxs3-P-cscvtJqOwn40B2aduezOZkQIKK-BY=.f1b9a066-63ef-4fe3-bb3d-2efcfac2b6af@github.com>

On Fri, 15 Nov 2024 21:35:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Prevent uncommit thread from running during GC

Looks good to me. A few documentation comment requests.

Also please share performance data in this PR or in the ticket, especially from the perf/benchmark that may have precipitated this change.

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 214:

> 212: //
> 213: public:
> 214:   void notify_heap_changed();

Let's place a single line of documentation comment for all the public and private APIs at lines that we touch in a PR where documentation is missing. (I realize you merely changed the method from private to public.)

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 406:

> 404:   void notify_soft_max_changed();
> 405:   void notify_explicit_gc_requested();
> 406: 

1-line documentation of API.

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 411:

> 409: private:
> 410:   ShenandoahControlThread*   _control_thread;
> 411:   ShenandoahUncommitThread*  _uncommit_thread;

Role of thread, e.g.

.... // a thread to uncommit selected free regions of the heap

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 628:

> 626:   bool is_uncommit_in_progress();
> 627: #endif
> 628: 

1-line API documentation each.

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 87:

> 85:   // Determine if there is work to do. This avoids taking heap lock if there is
> 86:   // no work available, avoids spamming logs with superfluous logging messages,
> 87:   // and minimises the amount of work while locks are taken.

last word: taken -> held.

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.hpp line 40:

> 38:   ShenandoahSharedFlag _uncommit_in_progress;
> 39:   Monitor _stop_lock;
> 40:   Monitor _uncommit_lock;

A 1-line comment on role of each field, e.g.

ShenandoahSharedFlag _soft_max_changed;    // the heap's soft max target has changed recently

etc.

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.hpp line 46:

> 44:   void uncommit(double shrink_before, size_t shrink_until);
> 45: 
> 46:   bool is_uncommit_allowed();

Would be nice to document these private methods as well.

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2444049683
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1847475867
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1847477020
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1847479562
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1847480153
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1847516208
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1847519410
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1847508283

From aph at openjdk.org  Tue Nov 19 09:45:24 2024
From: aph at openjdk.org (Andrew Haley)
Date: Tue, 19 Nov 2024 09:45:24 GMT
Subject: RFR: 8305895: Implement JEP 450: Compact Object Headers
 (Experimental) [v57]
In-Reply-To: <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
References: <TMFZyyalInj2UamEkFjV62vkXfP3zRF_fZq1NR4Msw4=.abb047d2-8eaa-447c-854e-ff5fe40a7169@github.com>
 <Zi_70fXsIZwV9Fzkz59YFmhAMfvgmjU0xuhKkQQKKsQ=.4a629d71-5cef-4c62-b9dd-6baf4ca5b9f0@github.com>
Message-ID: <jOwPpU5tpDnF44ewV5LASE1pOIR5ZQN2qEcbEpgeFuM=.24abf6b1-4d83-4d19-9e85-9d55a43e82c7@github.com>

On Thu, 7 Nov 2024 17:25:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is the main body of the JEP 450: Compact Object Headers (Experimental).
>> 
>> It is also a follow-up to #20640, which now also includes (and supersedes) #20603 and #20605, plus the Tiny Class-Pointers parts that have been previously missing.
>> 
>> Main changes:
>>  - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. The purpose of the flag is to provide a fallback, in case that users unexpectedly observe problems with the new implementation. The intention is that this flag will remain experimental and opt-in for at least one release, then make it on-by-default and diagnostic (?), and eventually deprecate and obsolete it. However, there are a few unknowns in that plan, specifically, we may want to further improve compact headers to 4 bytes, we are planning to enhance the Klass* encoding to support virtually unlimited number of Klasses, at which point we could also obsolete UseCompressedClassPointers.
>>  - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are add some changes to GC forwarding (see below) to protect the relevant (upper 22) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded.
>>  - Self-forwarding in GCs (which is used to deal with promotion failure) now uses a bit to indicate 'self-forwarding'. This is needed to preserve the crucial Klass* bits in the header. This also allows to get rid of preserved-header machinery in SerialGC and G1 (Parallel GC abuses preserved-marks to also find all other relevant oops).
>>  - Full GC forwarding now uses an encoding similar to compressed-oops. We have 40 bits for that, and can encode up to 8TB of heap. When exceeding 8TB, we turn off UseCompressedClassPointers (except in ZGC, which doesn't use the GC forwarding at all).
>>  - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16).
>>  - Arrays will now store their length at offset 8.
>>  - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. Some build machinery is added so that _co...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 107 commits:
> 
>  - Merge branch 'master' into JDK-8305895-v4
>  - Merge tag 'jdk-25+23' into JDK-8305895-v4
>    
>    Added tag jdk-24+23 for changeset c0e6c3b9
>  - Fix gen-ZGC removal
>  - Merge tag 'jdk-24+22' into JDK-8305895-v4
>    
>    Added tag jdk-24+22 for changeset 388d44fb
>  - Enable riscv in CompressedClassPointersEncodingScheme test
>  - s390 port
>  - Conditionalize platform specific parts of CompressedClassPointersEncodingScheme test
>  - Update copyright
>  - Avoid assert/endless-loop in JFR code
>  - Update copyright headers
>  - ... and 97 more: https://git.openjdk.org/jdk/compare/d3c042f9...c1a6323b

> So yes, some platforms [have alignment requirements for vectors]. But they seem to be the exception.

All AArch64 implementations work with unaligned vectors ? that's in the architecture spec ? but some designs thaht were made years ago performed badly. It's not a problem with new designs.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20677#issuecomment-2485185002

From shade at openjdk.org  Tue Nov 19 11:37:58 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 19 Nov 2024 11:37:58 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v3]
In-Reply-To: <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
Message-ID: <pLTbLXqQhiClbu5C9OkQpMExCYBnNji_ZOJC_a3Fark=.da72bf7c-4a6e-485b-8f0e-64cd0b3d875e@github.com>

On Fri, 15 Nov 2024 21:35:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Prevent uncommit thread from running during GC

I like it, thanks!

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 253:

> 251:     }
> 252: 
> 253:     heap->allow_uncommit();

This looks to be happening on every iteration, even if no GC happened. Should this `allow_uncommit()` go into the same block where `allow_commit()` is? Maybe it would be cleaner to make a `StackObj` mark object to manage this state -- up to you.

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 189:

> 187: void ShenandoahUncommitThread::allow_uncommit() {
> 188:   _uncommit_allowed.set();
> 189: }

New line at the end of file here.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2445178952
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1848181197
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1848183019

From kdnilsen at openjdk.org  Tue Nov 19 16:05:58 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 19 Nov 2024 16:05:58 GMT
Subject: RFR: 8344320: GenShen: Possible null pointer usage in
 shGenerationalHeapFix plab nullptr
Message-ID: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>

Certain threads, including the vm thread and Shenandoah's control thread, do not have PLABs.  There are scenarios where each may need to allocate old-gen memory on the slow-path of the load reference barrier.  This PR fixes an assumption in the allocator that assumed all threads have plabs.

-------------

Commit messages:
 - Only dereference plab if plab != nullptr
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - Merge branch 'openjdk:master' into master
 - ... and 17 more: https://git.openjdk.org/shenandoah/compare/8bc2dff6...b21c47e3

Changes: https://git.openjdk.org/shenandoah/pull/543/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=543&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344320
  Stats: 17 lines in 1 file changed: 1 ins; 1 del; 15 mod
  Patch: https://git.openjdk.org/shenandoah/pull/543.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/543/head:pull/543

PR: https://git.openjdk.org/shenandoah/pull/543

From kdnilsen at openjdk.org  Tue Nov 19 16:05:58 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 19 Nov 2024 16:05:58 GMT
Subject: RFR: 8344320: GenShen: Possible null pointer usage in
 shGenerationalHeapFix plab nullptr
In-Reply-To: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
References: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
Message-ID: <z1shj6ER0U8H56LB8zvIr-Mp5c0li6ZFqIha3Hk0_U0=.baf82d63-eb47-42dc-9dde-0e499db56744@github.com>

On Tue, 19 Nov 2024 16:01:10 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Certain threads, including the vm thread and Shenandoah's control thread, do not have PLABs.  There are scenarios where each may need to allocate old-gen memory on the slow-path of the load reference barrier.  This PR fixes an assumption in the allocator that assumed all threads have plabs.

Currently running regression tests.  Will not integrate until all tests succeed.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/543#issuecomment-2486121575

From shade at openjdk.org  Tue Nov 19 16:08:32 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 19 Nov 2024 16:08:32 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
Message-ID: <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>

On Wed, 13 Nov 2024 19:32:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
> 
>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>  - Use new CompactHeader forwarding APIs in generational mode
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge
>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>    
>    Reviewed-by: wkemper
>  - Merge
>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>    
>    Reviewed-by: kdnilsen
>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8

Some more stuff. 180/230 files done %)

src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 420:

> 418:   }
> 419:   int gc_state = _heap->gc_state();
> 420:   if ((gc_state & ShenandoahHeap::YOUNG_MARKING) != 0) {

It is not very clear this works fine with single-gen mode. Does `MARKING` imply `YOUNG_MARKING` in single-gen mode? I think we better make it abundantly clear with `is_generational()` checks.

src/hotspot/share/gc/shenandoah/shenandoahCollectionSetPreselector.hpp line 34:

> 32:   ShenandoahCollectionSet* _cset;
> 33:   bool* _pset;
> 34:   ResourceMark _rm;

Not necessarily for this PR, but something to rectify going forward. It is generally not safe to hide `ResourceMark`-s like this in the objects. Sometimes the callers return stuff allocated in resource area, and if the object like this leaves the scope, it would corrupt the memory. Normally, when we develop resource-area-returning paths, we look around near `ResourceMark`-s to see is any thing fishy is going on. It does not help when `RM` is hidden in the object like this.

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 276:

> 274:     log_info(gc)("GC cancellation took %.3fs", cancel_time);
> 275:     _cancel_requested_time = 0;
> 276:   }

Do we need this? Is this useful?

src/hotspot/share/gc/shenandoah/shenandoahHeapRegionCounters.cpp line 49:

> 47:     const char* cns = PerfDataManager::name_space("shenandoah", "regions");
> 48:     _name_space = NEW_C_HEAP_ARRAY(char, strlen(cns)+1, mtGC);
> 49:     strcpy(_name_space, cns); // copy cns into _name_space

Suggestion:

    strcpy(_name_space, cns);


(The comment is self-obvious)

src/hotspot/share/gc/shenandoah/shenandoahHeapRegionCounters.cpp line 57:

> 55:     PerfDataManager::create_constant(SUN_GC, cname, PerfData::U_None, num_regions, CHECK);
> 56: 
> 57:     cname = PerfDataManager::counter_name(_name_space, "protocol_version"); //creating new protocol_version

Suggestion:

    cname = PerfDataManager::counter_name(_name_space, "protocol_version");

src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.cpp line 193:

> 191:     Phase phase = Phase(pi);
> 192:     if (is_worker_phase(phase)) {
> 193:       double sum = uninitialized();

I think you can avoid changing this method to limit the churn.

src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 455:

> 453:     // Note: would be sufficient to mark only the card that holds the start of this Reference object.
> 454:     heap->old_generation()->card_scan()->mark_range_as_dirty(cast_from_oop<HeapWord*>(reference), reference->size());
> 455:   }

Two things:
 a) This sounds like `card_table_barrier(reference, raw_referent)`? Since this code is getting called for every dropped reference, just checking a `ShenandoahCardBarrier` flag sounds more efficient. 
 b) Is there a point in dirtying up to `reference->size()`?

src/hotspot/share/gc/shenandoah/shenandoahSharedVariables.hpp line 125:

> 123:       ShenandoahSharedValue ov = Atomic::load_acquire(&value);
> 124:       // We require all bits of mask_val to be set
> 125:       if ((ov & mask_val) == mask_val) {

This looks like a pre-existing bug, right? I assume this does not affect current Shenandoah, since we do not set multiple bits. I am fine with doing this in this PR, but it would probably be good to backport this fix in earlier releases too.

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2439178281
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1848286637
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1846888999
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844183614
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844187355
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1844186936
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1846892659
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1846923725
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1848264678

From shade at openjdk.org  Tue Nov 19 17:07:19 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 19 Nov 2024 17:07:19 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v5]
In-Reply-To: <7gEJ-mbo8YcMW7tDkET_Gz0tP5Dh1V34YVMwsCx5OJw=.d226bea8-7357-4a83-869f-d5a91fb21280@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
 <7gEJ-mbo8YcMW7tDkET_Gz0tP5Dh1V34YVMwsCx5OJw=.d226bea8-7357-4a83-869f-d5a91fb21280@github.com>
Message-ID: <55Yi5FEMBFgKqIopPDefdKLu68WqMg43sr1gmt8gMR4=.fc08a8c8-d8e2-42a2-9151-dce4bf526ebb@github.com>

On Mon, 18 Nov 2024 21:56:01 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update test, remove unused header, use proper case for old gen phases

I looked at `-Xlog:gc` with this patch, and it looks good. I say we do this.

-------------

Marked as reviewed by shade (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/540#pullrequestreview-2446093205

From wkemper at openjdk.org  Tue Nov 19 18:30:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 18:30:11 GMT
Subject: RFR: 8344321: GenShen: Fix various sonar scan warnings
Message-ID: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>

Various sonar warnings are fixed: 
* unused variables removed
* implicit casts made explicit
* members are all initialized

-------------

Commit messages:
 - Merge remote-tracking branch 'shenandoah/master' into fix-sonar-findings
 - Use checked_cast in more places, fix usage error in another
 - Bundle of sonar fixes

Changes: https://git.openjdk.org/shenandoah/pull/544/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=544&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344321
  Stats: 53 lines in 9 files changed: 1 ins; 31 del; 21 mod
  Patch: https://git.openjdk.org/shenandoah/pull/544.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/544/head:pull/544

PR: https://git.openjdk.org/shenandoah/pull/544

From kdnilsen at openjdk.org  Tue Nov 19 18:30:11 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 19 Nov 2024 18:30:11 GMT
Subject: RFR: 8344321: GenShen: Fix various sonar scan warnings
In-Reply-To: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>
References: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>
Message-ID: <jlGojXlxOQjN2ZlOQpOPx7Q-IF4VZ_FJvtVXLNhR5UM=.2e9fdead-e70b-45fa-94ec-ce8fc98512ae@github.com>

On Tue, 19 Nov 2024 18:19:47 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Various sonar warnings are fixed: 
> * unused variables removed
> * implicit casts made explicit
> * members are all initialized

Thanks.  Looks good.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/544#pullrequestreview-2446276313

From shade at openjdk.org  Tue Nov 19 18:54:17 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 19 Nov 2024 18:54:17 GMT
Subject: RFR: 8344321: GenShen: Fix various sonar scan warnings
In-Reply-To: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>
References: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>
Message-ID: <qnVPgtfouQtsl-ZZdoGfrF7_Z_7M2wE3XeaV61aimlk=.56baa01d-0961-47ee-bc5a-907cb5a01c6e@github.com>

On Tue, 19 Nov 2024 18:19:47 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Various sonar warnings are fixed: 
> * unused variables removed
> * implicit casts made explicit
> * members are all initialized

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/544#pullrequestreview-2446325003

From wkemper at openjdk.org  Tue Nov 19 19:26:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 19:26:06 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v3]
In-Reply-To: <pLTbLXqQhiClbu5C9OkQpMExCYBnNji_ZOJC_a3Fark=.da72bf7c-4a6e-485b-8f0e-64cd0b3d875e@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
 <pLTbLXqQhiClbu5C9OkQpMExCYBnNji_ZOJC_a3Fark=.da72bf7c-4a6e-485b-8f0e-64cd0b3d875e@github.com>
Message-ID: <AqqoeeZSa4CLnTbl0e8pqWcIwyqZWHyYY4JJydwJj_c=.a4eb613d-bcf9-4f0a-b47e-0ca2b64128ae@github.com>

On Tue, 19 Nov 2024 11:33:51 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Prevent uncommit thread from running during GC
>
> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 253:
> 
>> 251:     }
>> 252: 
>> 253:     heap->allow_uncommit();
> 
> This looks to be happening on every iteration, even if no GC happened. Should this `allow_uncommit()` go into the same block where `allow_commit()` is? Maybe it would be cleaner to make a `StackObj` mark object to manage this state -- up to you.

Hmm, good catch. I think it's a little worse than this even. The code that is meant to trigger an uncommit:

    if (ShenandoahUncommit) {
      if (heap->check_soft_max_changed()) {
        heap->notify_soft_max_changed();
      } else if (is_gc_requested) {
        heap->notify_explicit_gc_requested();
      }
    }

Will only happen when uncommit is forbidden. The notification will fall on deaf ears, as it were. I'll fix this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1848942547

From wkemper at openjdk.org  Tue Nov 19 19:44:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 19:44:36 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v4]
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <ycS5eL563F0kpmJopE_KWCEbFWnaWa6psZddeCXtqFs=.f5b4b2a1-b655-4b13-81f6-bca602c1c82d@github.com>

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Improve comments
 - Do not notify uncommit thread when uncommit is forbidden

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22019/files
  - new: https://git.openjdk.org/jdk/pull/22019/files/997360ac..05db9558

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=02-03

  Stats: 69 lines in 4 files changed: 58 ins; 1 del; 10 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From wkemper at openjdk.org  Tue Nov 19 19:54:07 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 19:54:07 GMT
Subject: Integrated: 8344321: GenShen: Fix various sonar scan warnings
In-Reply-To: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>
References: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>
Message-ID: <FSOjlYfqGeVmll81C4rEQvR6eRgcydtKZajBMrH_jAA=.07f8c35f-d769-415b-99e5-da2d43340b4b@github.com>

On Tue, 19 Nov 2024 18:19:47 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Various sonar warnings are fixed: 
> * unused variables removed
> * implicit casts made explicit
> * members are all initialized

This pull request has now been integrated.

Changeset: 6df9c536
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/6df9c536f1938520b180be3d5c4966112f4b74d9
Stats:     53 lines in 9 files changed: 1 ins; 31 del; 21 mod

8344321: GenShen: Fix various sonar scan warnings

Reviewed-by: kdnilsen, shade

-------------

PR: https://git.openjdk.org/shenandoah/pull/544

From wkemper at openjdk.org  Tue Nov 19 19:57:07 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 19:57:07 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
Message-ID: <yY__QZ-cqMGOphPPU6JkdLm6oNFPDhcP6jYlnK7OSz0=.8759b8ec-fe3d-4b98-ac61-04cd5f9fab20@github.com>

On Fri, 15 Nov 2024 17:01:29 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 276:
> 
>> 274:     log_info(gc)("GC cancellation took %.3fs", cancel_time);
>> 275:     _cancel_requested_time = 0;
>> 276:   }
> 
> Do we need this? Is this useful?

We could log this at debug? I believe it is useful because this time (currently) is not accounted for in the heuristic's notion of 'cycle time'.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1848979381

From wkemper at openjdk.org  Tue Nov 19 20:03:52 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 20:03:52 GMT
Subject: RFR: 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow
 on Windows after JDK-8340490 [v3]
In-Reply-To: <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>
References: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
 <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>
Message-ID: <5nf28UIt0ZW_9Df6LutJLLEaLUoBsJEZJhbpn2usRJA=.61835f96-5b3f-43ff-9a4b-d7446a9f3146@github.com>

On Sat, 16 Nov 2024 19:49:20 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Fixing the regression on Windows caused by JDK-8340490, the bug is actually caused by different behavior in `os:: os::elapsed_counter()` which I wasn't aware of.  Windows doesn't have nanosecond hi-res clock support, so instead of nanoseconds it returns the the current value of the performance counter([link](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter)).
>> 
>> ### Tests
>>  - [x] Verify gc/shenandoah/oom/TestClassLoaderLeak.java on Windows, no regression at all 
>>  - [x] Run test suites hotspot_gc_shenandoah with linux-aarch64-server-fastdebug
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Revert all the changes not related to the bug fix
>  - Simplify pace_for_alloc

LGTM

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/jdk/pull/22172#pullrequestreview-2446541678

From xpeng at openjdk.org  Tue Nov 19 20:12:49 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 19 Nov 2024 20:12:49 GMT
Subject: RFR: 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow
 on Windows after JDK-8340490 [v3]
In-Reply-To: <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>
References: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
 <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>
Message-ID: <5FLMEa2WpCyG2gJfES1wcimSOjcbpSUwZUObT_GWyh0=.7ce51686-b4f7-414e-85e4-45a51f0ead62@github.com>

On Sat, 16 Nov 2024 19:49:20 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Fixing the regression on Windows caused by JDK-8340490, the bug is actually caused by different behavior in `os:: os::elapsed_counter()` which I wasn't aware of.  Windows doesn't have nanosecond hi-res clock support, so instead of nanoseconds it returns the the current value of the performance counter([link](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter)).
>> 
>> ### Tests
>>  - [x] Verify gc/shenandoah/oom/TestClassLoaderLeak.java on Windows, no regression at all 
>>  - [x] Run test suites hotspot_gc_shenandoah with linux-aarch64-server-fastdebug
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Revert all the changes not related to the bug fix
>  - Simplify pace_for_alloc

Thank you all for the reviews, I'll start integration for the PR.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22172#issuecomment-2486652926

From duke at openjdk.org  Tue Nov 19 20:12:49 2024
From: duke at openjdk.org (duke)
Date: Tue, 19 Nov 2024 20:12:49 GMT
Subject: RFR: 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow
 on Windows after JDK-8340490 [v3]
In-Reply-To: <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>
References: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
 <DwExuZ4ZZfsMOInYYzsAdNzrwj7MmK_upD0P2wgqWWQ=.33566a58-3067-4d76-961c-b68ef52c05bd@github.com>
Message-ID: <-w_0ZvyS9ZlGmjuwDmrkgsO3lefaTDbq2FQoT81BLb4=.11c51082-f96e-490e-9ec6-f9f730523132@github.com>

On Sat, 16 Nov 2024 19:49:20 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> Fixing the regression on Windows caused by JDK-8340490, the bug is actually caused by different behavior in `os:: os::elapsed_counter()` which I wasn't aware of.  Windows doesn't have nanosecond hi-res clock support, so instead of nanoseconds it returns the the current value of the performance counter([link](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter)).
>> 
>> ### Tests
>>  - [x] Verify gc/shenandoah/oom/TestClassLoaderLeak.java on Windows, no regression at all 
>>  - [x] Run test suites hotspot_gc_shenandoah with linux-aarch64-server-fastdebug
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Revert all the changes not related to the bug fix
>  - Simplify pace_for_alloc

@pengxiaolong 
Your change (at version 19e782c6189e3df68c9a3ebfc6bd8ee5934ee67e) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22172#issuecomment-2486653685

From shade at openjdk.org  Tue Nov 19 20:18:20 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 19 Nov 2024 20:18:20 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
Message-ID: <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>

On Wed, 13 Nov 2024 19:32:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
> 
>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>  - Use new CompactHeader forwarding APIs in generational mode
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge
>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>    
>    Reviewed-by: wkemper
>  - Merge
>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>    
>    Reviewed-by: kdnilsen
>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8

Dropping the rest of my comments from today's read before I sign off. I have reviewed 200/230 files, will continue tomorrow.

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.hpp line 76:

> 74:   void vmop_entry_final_roots();
> 75: 
> 76: private:

Feel free to just make all these `protected`. It is more fuzz to try to provide the minimal possible visibility, especially if it gives the ragged visibility blocks in source. The real reason these are `private` is to protect them from accidental external use. `protected` is also good for this.

src/hotspot/share/gc/shenandoah/shenandoahMarkClosures.hpp line 34:

> 32: class ShenandoahHeapRegion;
> 33: 
> 34: class ShenandoahFinalMarkUpdateRegionStateClosure : public ShenandoahHeapRegionClosure {

There is the `shenandoahHeapRegionClosures.hpp` for these, no?

src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.cpp line 44:

> 42:   for (size_t idx = 0; idx < num_regions; idx++) {
> 43:     ShenandoahHeapRegion* r = heap->get_region(idx);
> 44:     if (r->is_affiliated() && heap->is_bitmap_slice_committed(r) && !is_bitmap_clear_range(r->bottom(), r->end())) {

I don't understand this for single gen mode. In that mode `is_affiliated() == false` always, right? So this check never passes, and `is_bitmap_clear` always returns `true`?

src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 657:

> 655:           _generation(nullptr) {
> 656:     if (_options._verify_marked == ShenandoahVerifier::_verify_marked_complete_satb_empty) {
> 657:       Threads::change_thread_claim_token();

It is fairly odd to see Verifier touching the claim token, since the bug _may be_ somewhere in parallel thread oop iteration infra. I think it is fine to just use `Threads::threads_do` (non-parallel version), which AFAIU does not require token modifications.

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2445988182
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1848675048
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1848786419
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1848781508
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1848711178

From xpeng at openjdk.org  Tue Nov 19 20:21:52 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 19 Nov 2024 20:21:52 GMT
Subject: Integrated: 8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java
 slow on Windows after JDK-8340490
In-Reply-To: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
References: <jqSF0qZLtKEao_wltNptf8bpitoQcyCmEfP4l071tY4=.38118a45-b66e-4cf5-babf-486883f25a68@github.com>
Message-ID: <loWAlnduJ37Nefst4Bh6X38ANC29iVpORUbonlo1VVE=.8cf6ee78-3560-4546-aa0d-9bd17f414784@github.com>

On Sat, 16 Nov 2024 00:11:39 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

> Fixing the regression on Windows caused by JDK-8340490, the bug is actually caused by different behavior in `os:: os::elapsed_counter()` which I wasn't aware of.  Windows doesn't have nanosecond hi-res clock support, so instead of nanoseconds it returns the the current value of the performance counter([link](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter)).
> 
> ### Tests
>  - [x] Verify gc/shenandoah/oom/TestClassLoaderLeak.java on Windows, no regression at all 
>  - [x] Run test suites hotspot_gc_shenandoah with linux-aarch64-server-fastdebug

This pull request has now been integrated.

Changeset: cd45ba32
Author:    Xiaolong Peng <xpeng at openjdk.org>
Committer: Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/cd45ba32f026ba3827d18836cab37a73f59346ed
Stats:     6 lines in 2 files changed: 0 ins; 0 del; 6 mod

8342041: Test gc/shenandoah/oom/TestClassLoaderLeak.java slow on Windows after JDK-8340490

Reviewed-by: shade, wkemper

-------------

PR: https://git.openjdk.org/jdk/pull/22172

From shade at openjdk.org  Tue Nov 19 20:26:56 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 19 Nov 2024 20:26:56 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v4]
In-Reply-To: <ycS5eL563F0kpmJopE_KWCEbFWnaWa6psZddeCXtqFs=.f5b4b2a1-b655-4b13-81f6-bca602c1c82d@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <ycS5eL563F0kpmJopE_KWCEbFWnaWa6psZddeCXtqFs=.f5b4b2a1-b655-4b13-81f6-bca602c1c82d@github.com>
Message-ID: <r9eH1RrD9UQsGFb7kcQO92lR8S3SPsEFN0kNu3wxEoE=.b1914970-c264-4216-a86e-0e8edb5e8f8d@github.com>

On Tue, 19 Nov 2024 19:44:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Improve comments
>  - Do not notify uncommit thread when uncommit is forbidden

I'll approve again, with the following nits:

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 45:

> 43: // Regions cannot be uncommitted when concurrent reset is zeroing out the bitmaps.
> 44: // This CADR class enforces this by forbidding region uncommits while it is in scope.
> 45: struct ShenandoahForbidRegionUncommit : public StackObj {

This is `class`, not `struct`, right? A common name for these in Hotspot are `*No*Mark`, so `ShenandoahNoUncommitMark`?

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2446555609
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1848995584

From shade at openjdk.org  Tue Nov 19 20:26:58 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 19 Nov 2024 20:26:58 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v4]
In-Reply-To: <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
 <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>
Message-ID: <2ctRsJ6JhReYC9yjcPV4eRljht3bMAZ9B3urifpvQXQ=.470aa336-f334-4345-863b-557250aa3416@github.com>

On Tue, 12 Nov 2024 19:22:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1932:
>> 
>>> 1930:   if (_uncommit_thread != nullptr) {
>>> 1931:     _uncommit_thread->stop();
>>> 1932:   }
>> 
>> Are there limits on proper sequencing here? Can we shutdown uncommit thread before cancelling the GC and waiting for control thread to exit? This would save end-to-end time for short commands, as we would hide the uncommit thread shutdown in the shadow of control thread exiting.
>
> I'm not sure the order matters here. `ConcurrentGCThread::stop` waits until the target thread sets `_has_terminated`.

OK, nevermind. We can fix it later if it becomes a problem.

>> src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 139:
>> 
>>> 137:     _heap->notify_heap_changed();
>>> 138:     double elapsed = os::elapsedTime() - start;
>>> 139:     log_info(gc)("Uncommitted " SIZE_FORMAT " regions, in %.3fs", count, elapsed);
>> 
>> If we can, can we match the current log format? E.g. print `Concurrent uncommit`, with appropriately formatted timestamp? I think we also want `log_info(gc,start)` at the beginning of the method. I think `ShenandoahConcurrentPhase` helper did all that, can we still use it?
>
> We can restore the log messages, but I don't think `ShenandoahConcurrentPhase` and friends will like being used outside of a cycle. I'll look into it.

Yeah, at least restore the log format and add `gc+start` log as well.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1849006729
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1849011585

From ysr at openjdk.org  Tue Nov 19 21:08:55 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 19 Nov 2024 21:08:55 GMT
Subject: RFR: 8344321: GenShen: Fix various sonar scan warnings
In-Reply-To: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>
References: <wsg8WkDsVakE9Vtcy3aJ03eoZ_50reiAwhSnKjPhSWk=.dfb9f0f3-cd43-4c9c-9045-c9bc602365c0@github.com>
Message-ID: <yLsenqComMlx6bAGYROkN3m4vpdexDhkJEV5ybwx4nc=.94c121a2-c532-4e2e-8bb2-e883d432044f@github.com>

On Tue, 19 Nov 2024 18:19:47 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Various sonar warnings are fixed: 
> * unused variables removed
> * implicit casts made explicit
> * members are all initialized

src/hotspot/share/gc/shenandoah/shenandoahEvacTracker.cpp line 37:

> 35:     _evacuations_attempted(0), _bytes_attempted(0),
> 36:     _use_age_table(ShenandoahGenerationalCensusAtEvac || !ShenandoahGenerationalAdaptiveTenuring),
> 37:     _age_table(nullptr) {

Good catch; thanks!

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/544#discussion_r1849054720

From gziemski at openjdk.org  Tue Nov 19 22:01:13 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Tue, 19 Nov 2024 22:01:13 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>

> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by avoiding using `mtNone`.
> 
> Eventually the goal would be to eliminate using `mtNone` whenever possible.
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:

 - avoid using mtNone
 - Merge remote-tracking branch 'upstream/master' into JDK-8328944
 - revert, we will re-do with a smaller change
 - remove more mtNone
 - remove API that allows to change the mem_tag for virtual memory, feedback
 - do not allow default parameter for mtNone

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21843/files
  - new: https://git.openjdk.org/jdk/pull/21843/files/483e08c2..33dbafdc

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21843&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21843&range=00-01

  Stats: 317732 lines in 4373 files changed: 167679 ins; 118575 del; 31478 mod
  Patch: https://git.openjdk.org/jdk/pull/21843.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21843/head:pull/21843

PR: https://git.openjdk.org/jdk/pull/21843

From gziemski at openjdk.org  Tue Nov 19 22:01:14 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Tue, 19 Nov 2024 22:01:14 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <9xoUh9CZ5_j-7NY_SEyCdBNRKoa2sh0Kjvvc2GqkPlM=.51706e78-822d-4c9e-a486-d4a839366854@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
 <9xoUh9CZ5_j-7NY_SEyCdBNRKoa2sh0Kjvvc2GqkPlM=.51706e78-822d-4c9e-a486-d4a839366854@github.com>
Message-ID: <VLsOBHdTMm-imD1vswh6aY0jxytsWEjFP9TPH0-lN2g=.2a1b3a39-73af-427c-88b1-2b396270debf@github.com>

On Mon, 18 Nov 2024 10:42:59 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> Why should we supplement `mtNone` with `mtAllocated`? Is one indicative of a failure mode?

That is the general idea, if we see `mtNone` in a report we know we missed something. `mtAllocated` would be the one indicating that we are "in transition".

But I put it back into draft mode to experiment and get a better picture on how to proceed.

> src/hotspot/share/nmt/virtualMemoryTracker.hpp line 389:
> 
>> 387:   static bool remove_released_region    (address base_addr, size_t size);
>> 388:   static bool remove_released_region    (ReservedMemoryRegion* rgn);
>> 389:   static void set_reserved_region_type  (address addr, MemTag mem_tag);
> 
> I don't think that we should remove this API. It may very well need to be used, even if its usage should be minimized.

If there is currently no-one using it, then we should remove it. We can add it back, once we need it.

> src/hotspot/share/runtime/os.hpp line 460:
> 
>> 458:   // Attempts to reserve the virtual memory at [addr, addr + bytes).
>> 459:   // Does not overwrite existing mappings.
>> 460:   static char*  attempt_reserve_memory_at(char* addr, size_t bytes, bool executable, MemTag mem_tag);
> 
> Is it worth adding an overload: `reserve_memory(bytes, mem_tag)` where `executable` is implicitly `false`?

I would want that myself too, but that requires that we put it at the end, which I did in fact had at one point, but didn't like the parameters call order - the mem_tag is usually the very last one.

> test/hotspot/gtest/memory/test_virtualspace.cpp line 374:
> 
>> 372:                      alignment,       // alignment
>> 373:                      page_size,       // page size
>> 374:                      (char *)nullptr, // requested_address
> 
> Style fix: Remove space between `char` and `*`

Fixed.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2483404110
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1847064817
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1846811525
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1847065123

From gziemski at openjdk.org  Tue Nov 19 22:01:14 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Tue, 19 Nov 2024 22:01:14 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <jw9dxvd8xR8w6kdkHuzT209N0i4kPTyH0AWtzA7hdIk=.e2d95bf9-756a-4645-8dc2-686d726ea74e@github.com>

On Fri, 1 Nov 2024 20:44:50 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by avoiding using `mtNone`.
> 
> Eventually the goal would be to eliminate using `mtNone` whenever possible.
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

I re-did this fix from scratch, making sure that we do not loose Thomas'es expert knowledge, which he shared in his feedback.

This fix focuses on fixing the cases in the issue.

The more general approach to limit `mtNone` usage is left for a followup.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2486834505

From jsjolen at openjdk.org  Tue Nov 19 22:01:14 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Tue, 19 Nov 2024 22:01:14 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
Message-ID: <9xoUh9CZ5_j-7NY_SEyCdBNRKoa2sh0Kjvvc2GqkPlM=.51706e78-822d-4c9e-a486-d4a839366854@github.com>

On Tue, 19 Nov 2024 21:57:52 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by avoiding using `mtNone`.
>> 
>> Eventually the goal would be to eliminate using `mtNone` whenever possible.
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - avoid using mtNone
>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>  - revert, we will re-do with a smaller change
>  - remove more mtNone
>  - remove API that allows to change the mem_tag for virtual memory, feedback
>  - do not allow default parameter for mtNone

Why should we supplement `mtNone` with `mtAllocated`? Is one indicative of a failure mode?

src/hotspot/share/nmt/virtualMemoryTracker.hpp line 389:

> 387:   static bool remove_released_region    (address base_addr, size_t size);
> 388:   static bool remove_released_region    (ReservedMemoryRegion* rgn);
> 389:   static void set_reserved_region_type  (address addr, MemTag mem_tag);

I don't think that we should remove this API. It may very well need to be used, even if its usage should be minimized.

src/hotspot/share/runtime/os.hpp line 460:

> 458:   // Attempts to reserve the virtual memory at [addr, addr + bytes).
> 459:   // Does not overwrite existing mappings.
> 460:   static char*  attempt_reserve_memory_at(char* addr, size_t bytes, bool executable, MemTag mem_tag);

Is it worth adding an overload: `reserve_memory(bytes, mem_tag)` where `executable` is implicitly `false`?

test/hotspot/gtest/memory/test_virtualspace.cpp line 374:

> 372:                      alignment,       // alignment
> 373:                      page_size,       // page size
> 374:                      (char *)nullptr, // requested_address

Style fix: Remove space between `char` and `*`

-------------

PR Review: https://git.openjdk.org/jdk/pull/21843#pullrequestreview-2442231730
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1846345086
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1846346929
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1846350091

From wkemper at openjdk.org  Tue Nov 19 22:14:10 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 22:14:10 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v5]
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <E-5kTVHd45LMmlgD2Qc7PNwc4zvb2yLxVrdnCO0gQxA=.d45968c6-01ac-4b8c-a440-621c53eed996@github.com>

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Use idiomatic name for CADR class

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22019/files
  - new: https://git.openjdk.org/jdk/pull/22019/files/05db9558..7007415a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=03-04

  Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From wkemper at openjdk.org  Tue Nov 19 22:50:50 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 22:50:50 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v6]
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <Ntwy8_hAd5tvbJ_J9RsIVUK3odLUc4lop7GLIm73k4k=.f000f8bd-fcf0-47b8-842a-52b4c20f0035@github.com>

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Log all heuristic triggers with consistent formatting

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/540/files
  - new: https://git.openjdk.org/shenandoah/pull/540/files/df0748ff..c05feb34

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=05
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=04-05

  Stats: 64 lines in 11 files changed: 24 ins; 7 del; 33 mod
  Patch: https://git.openjdk.org/shenandoah/pull/540.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/540/head:pull/540

PR: https://git.openjdk.org/shenandoah/pull/540

From wkemper at openjdk.org  Tue Nov 19 23:02:22 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 23:02:22 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
Message-ID: <bZ8e8s6cvCCal957VpuKvcE5QBMN20zqxlbp0iEscXI=.53206bc0-2d93-4e8c-bcfc-3ea50142201c@github.com>

On Mon, 18 Nov 2024 16:22:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.cpp line 193:
> 
>> 191:     Phase phase = Phase(pi);
>> 192:     if (is_worker_phase(phase)) {
>> 193:       double sum = uninitialized();
> 
> I think you can avoid changing this method to limit the churn.

Hmm, I find the version in the PR much more readable. How about we upstream these changes separately?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849160336

From wkemper at openjdk.org  Tue Nov 19 23:10:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 23:10:34 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
Message-ID: <fVT6LGlgaT1etS3mg7cUvO5IT2EsfxP_ilN13TM77XY=.1fcd85a0-e957-4834-a788-ae4bd62356b5@github.com>

On Tue, 19 Nov 2024 17:31:15 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.cpp line 44:
> 
>> 42:   for (size_t idx = 0; idx < num_regions; idx++) {
>> 43:     ShenandoahHeapRegion* r = heap->get_region(idx);
>> 44:     if (r->is_affiliated() && heap->is_bitmap_slice_committed(r) && !is_bitmap_clear_range(r->bottom(), r->end())) {
> 
> I don't understand this for single gen mode. In that mode `is_affiliated() == false` always, right? So this check never passes, and `is_bitmap_clear` always returns `true`?

No, in single gen mode regions that are not `FREE` are affiliated with `YOUNG`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849165775

From wkemper at openjdk.org  Tue Nov 19 23:20:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 23:20:39 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
Message-ID: <gBRZJiSA8NPZMAVo5Hl8rld6r3b90934rhufvajDQgY=.edd5ab35-170a-4afc-a1cb-1f6d7ea29783@github.com>

On Tue, 19 Nov 2024 17:35:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahMarkClosures.hpp line 34:
> 
>> 32: class ShenandoahHeapRegion;
>> 33: 
>> 34: class ShenandoahFinalMarkUpdateRegionStateClosure : public ShenandoahHeapRegionClosure {
> 
> There is the `shenandoahHeapRegionClosures.hpp` for these, no?

It wasn't the intention that `shenandoahHeapRegionClosures.hpp` should become home to all of the `*HeapRegionClosures`. `ShenandoahFinalMarkUpdateRegionStateClosure` is shared in a couple of implementation files, but not the same as the ones using `shenandoahHeapRegionClosures.hpp`. I think `ShenandoahUpdateCensusZeroCohortClosure` should be moved into the only implementation file that uses it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849213506

From wkemper at openjdk.org  Tue Nov 19 23:50:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 23:50:38 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
Message-ID: <9VBIb_uybg8Or00efuFMAfa2JbZ3RVzE6avnR5u3mBs=.bcd6bf33-b37c-4c9f-aa4b-5d540c258fdd@github.com>

On Mon, 18 Nov 2024 16:20:09 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahCollectionSetPreselector.hpp line 34:
> 
>> 32:   ShenandoahCollectionSet* _cset;
>> 33:   bool* _pset;
>> 34:   ResourceMark _rm;
> 
> Not necessarily for this PR, but something to rectify going forward. It is generally not safe to hide `ResourceMark`-s like this in the objects. Sometimes the callers return stuff allocated in resource area, and if the object like this leaves the scope, it would corrupt the memory. Normally, when we develop resource-area-returning paths, we look around near `ResourceMark`-s to see is any thing fishy is going on. It does not help when `RM` is hidden in the object like this.

Understood. This change was requested earlier in the PR.

> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.hpp line 76:
> 
>> 74:   void vmop_entry_final_roots();
>> 75: 
>> 76: private:
> 
> Feel free to just make all these `protected`. It is more fuzz to try to provide the minimal possible visibility, especially if it gives the ragged visibility blocks in source. The real reason these are `private` is to protect them from accidental external use. `protected` is also good for this.

https://bugs.openjdk.org/browse/JDK-8344592

> src/hotspot/share/gc/shenandoah/shenandoahHeapRegionCounters.cpp line 49:
> 
>> 47:     const char* cns = PerfDataManager::name_space("shenandoah", "regions");
>> 48:     _name_space = NEW_C_HEAP_ARRAY(char, strlen(cns)+1, mtGC);
>> 49:     strcpy(_name_space, cns); // copy cns into _name_space
> 
> Suggestion:
> 
>     strcpy(_name_space, cns);
> 
> 
> (The comment is self-obvious)

https://bugs.openjdk.org/browse/JDK-8344592

> src/hotspot/share/gc/shenandoah/shenandoahHeapRegionCounters.cpp line 57:
> 
>> 55:     PerfDataManager::create_constant(SUN_GC, cname, PerfData::U_None, num_regions, CHECK);
>> 56: 
>> 57:     cname = PerfDataManager::counter_name(_name_space, "protocol_version"); //creating new protocol_version
> 
> Suggestion:
> 
>     cname = PerfDataManager::counter_name(_name_space, "protocol_version");

https://bugs.openjdk.org/browse/JDK-8344592

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849309543
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849308822
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849309104
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849309202

From wkemper at openjdk.org  Tue Nov 19 23:51:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 23:51:03 GMT
Subject: RFR: 8344592: GenShen: Remove unnecessary comments and changes
Message-ID: <fRCVFh33mzdYWCeg0y02S5zliVPskSn4KrNzufSwIDc=.33a78f41-e509-4d5b-9a10-c83e3951a8e7@github.com>

Remove unnecessary comments and reduce granularity of access controls in shConcurrentGC.hpp.

-------------

Commit messages:
 - Decrease granularity of access modifiers
 - Remove unnecessary comments

Changes: https://git.openjdk.org/shenandoah/pull/545/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=545&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344592
  Stats: 32 lines in 3 files changed: 4 ins; 21 del; 7 mod
  Patch: https://git.openjdk.org/shenandoah/pull/545.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/545/head:pull/545

PR: https://git.openjdk.org/shenandoah/pull/545

From wkemper at openjdk.org  Tue Nov 19 23:59:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 19 Nov 2024 23:59:36 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
Message-ID: <bKR7YS16f4lnW2CvJZ0PmyOGqhwn88MyxCva9kS7jno=.543f2586-6baa-4cb5-9679-e8a760f5569a@github.com>

On Mon, 18 Nov 2024 16:42:50 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 455:
> 
>> 453:     // Note: would be sufficient to mark only the card that holds the start of this Reference object.
>> 454:     heap->old_generation()->card_scan()->mark_range_as_dirty(cast_from_oop<HeapWord*>(reference), reference->size());
>> 455:   }
> 
> Two things:
>  a) This sounds like `card_table_barrier(reference, raw_referent)`? Since this code is getting called for every dropped reference, just checking a `ShenandoahCardBarrier` flag sounds more efficient. 
>  b) Is there a point in dirtying up to `reference->size()`?

Will test this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849315332

From ysr at openjdk.org  Wed Nov 20 00:04:40 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 20 Nov 2024 00:04:40 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
Message-ID: <0FH7N5Xa3DLDGZcWdxwWQHivNSTxGj9w3p-hWFSgMIs=.654c7bd9-90bb-436d-aedd-08d9fa77efc8@github.com>

On Wed, 13 Nov 2024 10:51:48 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 503 commits:
>> 
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8342861: GenShen: Old generation in unexpected state when abandoning mixed gc candidates
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342734: GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
>>    
>>    Reviewed-by: ysr
>>  - 8342919: GenShen: Fix whitespace
>>    
>>    Reviewed-by: xpeng, kdnilsen
>>  - 8342927: GenShen: Guarantee slices of time for coalesce and filling
>>    
>>    Reviewed-by: kdnilsen
>>  - 8342924: GenShen: Problem list gc/shenandoah/TestReferenceRefersToShenandoah.java
>>    
>>    Reviewed-by: kdnilsen, ysr
>>  - 8342848: Shenandoah: Marking bitmap may not be completely cleared in generational mode
>>    
>>    Reviewed-by: wkemper
>>  - ... and 493 more: https://git.openjdk.org/jdk/compare/1c448347...19b25bc3
>
> src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 474:
> 
>> 472:     // elision safe.
>> 473:     return;
>> 474:   }
> 
> Why is this safe for Shenandoah? I suspect it needs `CardTableBarrierSet::on_slowpath_allocation_exit` to work. `G1BarrierSetC2` gets it by subclassing `CardTableBarrierSetC2`. But `ShenandoahBarrierSetC2` does not. Should it?

Good question. I'm tracking at https://bugs.openjdk.org/browse/JDK-8344593 , and will follow up there.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849318244

From wkemper at openjdk.org  Wed Nov 20 00:08:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 00:08:41 GMT
Subject: RFR: 8344320: GenShen: Possible null pointer usage in
 shGenerationalHeap
In-Reply-To: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
References: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
Message-ID: <AHQMP_aAFcBJI0KTPPUPjlzfG5rg5FqWYokNFRJrmD0=.9b575299-8bf6-48c3-8451-655f37eb4e13@github.com>

On Tue, 19 Nov 2024 16:01:10 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Certain threads, including the vm thread and Shenandoah's control thread, do not have PLABs.  There are scenarios where each may need to allocate old-gen memory on the slow-path of the load reference barrier.  This PR fixes an assumption in the allocator that assumed all threads have plabs.

Looks good to me.

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/543#pullrequestreview-2447062878

From wkemper at openjdk.org  Wed Nov 20 00:13:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 00:13:31 GMT
Subject: RFR: 8344320: GenShen: Possible null pointer usage in
 shGenerationalHeap
In-Reply-To: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
References: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
Message-ID: <4rLuCNz_XZG6q5jTZGF95a2l0m3CaM595n6QQJvZ5gQ=.a937c355-cac1-45fe-88a4-4960d82f8fd8@github.com>

On Tue, 19 Nov 2024 16:01:10 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Certain threads, including the vm thread and Shenandoah's control thread, do not have PLABs.  There are scenarios where each may need to allocate old-gen memory on the slow-path of the load reference barrier.  This PR fixes an assumption in the allocator that assumed all threads have plabs.

Not sure why this PR is pulling in 27 commits, I don't think it matters, but in the future try creating new branches with:

% git fetch <shenandoah>
% git switch -c <new-branch-name> <shenandoah>/master

Replace `<shenandoah>` with your remote name for `https://github.com/openjdk/shenandoah`.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/543#issuecomment-2487021218

From wkemper at openjdk.org  Wed Nov 20 00:16:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 00:16:36 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
Message-ID: <i-MvUaoe8n9apI8y-4hQ9ohIGHwSPaufWIkdMLyBdLM=.3bcf0c60-9b68-4552-b425-45a8ff946664@github.com>

On Tue, 19 Nov 2024 12:38:59 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 420:
> 
>> 418:   }
>> 419:   int gc_state = _heap->gc_state();
>> 420:   if ((gc_state & ShenandoahHeap::YOUNG_MARKING) != 0) {
> 
> It is not very clear this works fine with single-gen mode. Does `MARKING` imply `YOUNG_MARKING` in single-gen mode? I think we better make it abundantly clear with `is_generational()` checks.

Yes, in single gen mode we set `MARKING` and `YOUNG_MARKING` together. See https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahGlobalGeneration.cpp#L81

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849325345

From wkemper at openjdk.org  Wed Nov 20 00:22:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 00:22:40 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
Message-ID: <o3cfHdWibPFrzuN4WGIJJpUXNklEFCfyP65OjEMx3aU=.1cd3f803-ce24-4162-a426-1ff858ee9f3c@github.com>

On Tue, 19 Nov 2024 16:46:36 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 657:
> 
>> 655:           _generation(nullptr) {
>> 656:     if (_options._verify_marked == ShenandoahVerifier::_verify_marked_complete_satb_empty) {
>> 657:       Threads::change_thread_claim_token();
> 
> It is fairly odd to see Verifier touching the claim token, since the bug _may be_ somewhere in parallel thread oop iteration infra. I think it is fine to just use `Threads::threads_do` (non-parallel version), which AFAIU does not require token modifications.

Will test this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1849329578

From wkemper at openjdk.org  Wed Nov 20 00:32:15 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 00:32:15 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v7]
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <nISAw1bbIF6SqG2AesZnUBxaZPL2mviklmxGEJaxd3c=.f03abe84-bb69-4b00-8c82-7d4dde8d51c5@github.com>

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Fix old growth triggers test

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/540/files
  - new: https://git.openjdk.org/shenandoah/pull/540/files/c05feb34..02d3d24d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=06
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=05-06

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/540.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/540/head:pull/540

PR: https://git.openjdk.org/shenandoah/pull/540

From wkemper at openjdk.org  Wed Nov 20 00:47:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 00:47:12 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <nXVt5nih04r0qv1t6Z1IvkSlJtP2uuPST8kaWS70QLE=.51ae819a-bba6-4cf5-85a2-8cb7054b5d8b@github.com>
References: <nXVt5nih04r0qv1t6Z1IvkSlJtP2uuPST8kaWS70QLE=.51ae819a-bba6-4cf5-85a2-8cb7054b5d8b@github.com>
Message-ID: <lqLj0aRlruB-xAguajXZiScgpz1yJ6j999B5fFA-zyI=.614ef3a7-b849-4712-8424-0c6f6e9dccd7@github.com>

> Merges tag jdk-24+24

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 124 commits:

 - Merge tag 'jdk-24+24' into merge-jdk-24+24
   
   Added tag jdk-24+24 for changeset 95a00f8a
 - 8343875: Minor improvements of jpackage test library
   
   Reviewed-by: almatvee
 - 8344074: RISC-V: C1: More accurate _exception_handler_size and _deopt_handler_size
   
   Reviewed-by: mli, fjiang
 - 8339288: Improve diagnostic logging runtime/cds/DeterministicDump.java
   
   Reviewed-by: ccheung, iklam
 - 8343984: Fix Unsafe address overflow
   
   Reviewed-by: pminborg, alanb
 - 8343958: Remove security manager impl in java.lang.Process and java.lang.Runtime.exec
   
   Reviewed-by: jpai, mullan, alanb
 - 8344039: Remove security manager dependency in java.time
   
   Reviewed-by: naoto, mullan, lancea
 - 8343219: Manual clientlibs test failures after SM removal
   
   Reviewed-by: azvegint, aivanov, prr
 - 8343416: CDS dump fails when unregistered class can also be loaded from system modules
   
   Reviewed-by: iklam, matsaave
 - 8344023: Unnecessary Hashtable usage in LdapClient.defaultBinaryAttrs
   
   Reviewed-by: dfuchs, aefimov
 - ... and 114 more: https://git.openjdk.org/shenandoah/compare/6df9c536...f8f8525e

-------------

Changes: https://git.openjdk.org/shenandoah/pull/541/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=541&range=01
  Stats: 107177 lines in 2836 files changed: 28070 ins; 70167 del; 8940 mod
  Patch: https://git.openjdk.org/shenandoah/pull/541.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/541/head:pull/541

PR: https://git.openjdk.org/shenandoah/pull/541

From wkemper at openjdk.org  Wed Nov 20 01:44:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 01:44:28 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v6]
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <iNt4LObx10MItKRhAOLdW0sMoBDqSQVsfMNgiD9CqYk=.1382cbb6-951f-4a5a-aead-e035744ca653@github.com>

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Allow commits initially

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22019/files
  - new: https://git.openjdk.org/jdk/pull/22019/files/7007415a..a00945c2

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=04-05

  Stats: 9 lines in 1 file changed: 5 ins; 4 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From ysr at openjdk.org  Wed Nov 20 08:25:42 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 20 Nov 2024 08:25:42 GMT
Subject: RFR: 8344320: GenShen: Possible null pointer usage in
 shGenerationalHeap
In-Reply-To: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
References: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
Message-ID: <P1INJY1-irmvjQsUidQZ68vkP67YP2j7wOLxQXYSaWg=.01da5af4-8e50-4bf3-a9fb-a2f68bc84331@github.com>

On Tue, 19 Nov 2024 16:01:10 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Certain threads, including the vm thread and Shenandoah's control thread, do not have PLABs.  There are scenarios where each may need to allocate old-gen memory on the slow-path of the load reference barrier.  This PR fixes an assumption in the allocator that assumed all threads have plabs.

Marked as reviewed by ysr (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/543#pullrequestreview-2447839579

From shade at openjdk.org  Wed Nov 20 09:25:41 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 20 Nov 2024 09:25:41 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v7]
In-Reply-To: <nISAw1bbIF6SqG2AesZnUBxaZPL2mviklmxGEJaxd3c=.f03abe84-bb69-4b00-8c82-7d4dde8d51c5@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
 <nISAw1bbIF6SqG2AesZnUBxaZPL2mviklmxGEJaxd3c=.f03abe84-bb69-4b00-8c82-7d4dde8d51c5@github.com>
Message-ID: <QCbHg_KoZMoH5f8Zx4psc5kfM9GUPYZ7jPTDDLhED44=.2098141e-efa5-4a35-a61b-1e6f5fe92e7b@github.com>

On Wed, 20 Nov 2024 00:32:15 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix old growth triggers test

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/540#pullrequestreview-2448007688

From shade at openjdk.org  Wed Nov 20 09:33:17 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 20 Nov 2024 09:33:17 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v6]
In-Reply-To: <iNt4LObx10MItKRhAOLdW0sMoBDqSQVsfMNgiD9CqYk=.1382cbb6-951f-4a5a-aead-e035744ca653@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <iNt4LObx10MItKRhAOLdW0sMoBDqSQVsfMNgiD9CqYk=.1382cbb6-951f-4a5a-aead-e035744ca653@github.com>
Message-ID: <ULR8MPTzpz9lnoGvexQSF8hMkURlGmdSJuF3UMnan1I=.ae39fb1a-ffa6-46eb-9b08-dee1360272ea@github.com>

On Wed, 20 Nov 2024 01:44:28 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Allow commits initially

Almost there, modulo restoring the logging.

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 267:

> 265:       last_sleep_adjust_time = current;
> 266:     }
> 267: 

Nit: No need for this newline, the sleep logically relates to this block.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2448019848
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1849943600

From shade at openjdk.org  Wed Nov 20 09:33:18 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 20 Nov 2024 09:33:18 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v6]
In-Reply-To: <2ctRsJ6JhReYC9yjcPV4eRljht3bMAZ9B3urifpvQXQ=.470aa336-f334-4345-863b-557250aa3416@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
 <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>
 <2ctRsJ6JhReYC9yjcPV4eRljht3bMAZ9B3urifpvQXQ=.470aa336-f334-4345-863b-557250aa3416@github.com>
Message-ID: <I0GzgbIQG1aGxmJ4c1wh0N5uYDDO-5fIrnEtT4t675s=.b21ca11b-1061-4bbe-97c3-79edccdabab4@github.com>

On Tue, 19 Nov 2024 20:23:57 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We can restore the log messages, but I don't think `ShenandoahConcurrentPhase` and friends will like being used outside of a cycle. I'll look into it.
>
> Yeah, at least restore the log format and add `gc+start` log as well.

This one is still not addressed, unfortunately ^

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1849936299

From azafari at openjdk.org  Wed Nov 20 10:00:28 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 20 Nov 2024 10:00:28 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
Message-ID: <jSPfSoOor2d_WfTRe8Zs8TZKmSfqxggbSmpP_P5_iOM=.fc738f7f-0425-4357-8479-fc08f3246841@github.com>

On Tue, 19 Nov 2024 22:01:13 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by avoiding using `mtNone`.
>> 
>> Eventually the goal would be to eliminate using `mtNone` whenever possible.
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - avoid using mtNone
>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>  - revert, we will re-do with a smaller change
>  - remove more mtNone
>  - remove API that allows to change the mem_tag for virtual memory, feedback
>  - do not allow default parameter for mtNone

src/hotspot/os/bsd/gc/z/zPhysicalMemoryBacking_bsd.cpp line 1:

> 1: /*

Copyright year to be updated.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1849994608

From azafari at openjdk.org  Wed Nov 20 10:06:25 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 20 Nov 2024 10:06:25 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
Message-ID: <PA7QHXA2Mt29C7SYevcLXekhj1GDax0YP-cJo73i5xQ=.4f9821f9-1580-4844-8dbf-23fa1c73b7cd@github.com>

On Tue, 19 Nov 2024 22:01:13 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by avoiding using `mtNone`.
>> 
>> Eventually the goal would be to eliminate using `mtNone` whenever possible.
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - avoid using mtNone
>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>  - revert, we will re-do with a smaller change
>  - remove more mtNone
>  - remove API that allows to change the mem_tag for virtual memory, feedback
>  - do not allow default parameter for mtNone

src/hotspot/share/memory/virtualspace.hpp line 65:

> 63: 
> 64:   void initialize(size_t size, size_t alignment, size_t page_size,
> 65:                   char* requested_address, bool executable, MemTag mem_tag = mtNone);

When `mtNone` is default, it is still possible to introduce/create regions of Unknown types. The MemTag arg should be mandatory.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1850003383

From azafari at openjdk.org  Wed Nov 20 10:06:26 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 20 Nov 2024 10:06:26 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <n76XBMTnJSTCFSIiSuPfT6yYr6JL_qWT4_F2gy67nfs=.8043cf94-0fe3-4282-8342-5ccd431b7ac0@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <fy4GFfvINDvYH99jnhery1Xg5Z1EotnJukw4kJEM1xU=.737a9712-1f93-488c-9436-c429b850a5da@github.com>
 <n76XBMTnJSTCFSIiSuPfT6yYr6JL_qWT4_F2gy67nfs=.8043cf94-0fe3-4282-8342-5ccd431b7ac0@github.com>
Message-ID: <hP1hwRq4_mMAsPIOFoZ7VlO_8ry7Ch8DAombbvbb0rU=.394e45ae-0dd8-4fe4-809a-f95fb7ee02be@github.com>

On Sat, 16 Nov 2024 02:52:18 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> src/hotspot/share/memory/virtualspace.hpp line 67:
>> 
>>> 65: 
>>> 66:   void reserve(size_t size, size_t alignment, size_t page_size,
>>> 67:                char* requested_address, bool executable, MemTag mem_tag);
>> 
>> Why do the instance methods get MemTag? Why is MemTag not stored as a (const) attribute of ReservedSpace? 
>> 
>> But then, we would have double accounting - we'd store the tag both in NMT and in ReservedSpace instances. If we change them post-reservation, we would need to change both. Sigh.
>> 
>> So we pass in MemTag just for the ReservedSpace to pass it into os::reserve_memory. Seeing how often ReservedSpace is used in situations where the tag is not clear at reservation time, I wonder whether we are not better off letting the user of ReservedSpace register the NMT tag post reservation like they do today.
>
> I like the simplicity of not having to deal with changing the tag afterwards personally. Does anyone else have an opinion here?

We need to keep a member MemTag for ReservedSpace. There are cases in the code where a new copy of a ReservedSpace is to be created. Then, we need to know the original MemTag and pass it again down to the other functions.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1850001690

From rkennke at openjdk.org  Wed Nov 20 11:30:09 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 20 Nov 2024 11:30:09 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize
Message-ID: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>

>From the bug description:
ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().

There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.

The fix to both issues is:
- First disable UCOH
- and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
- then enable any flags required by compact headers
- Initialize ObjLayout after all flags are done (it's in the correct place, already)

Testing:
 - [x] tier1 -UCOH (default)
 - [ ] tier1 +UCOH
 - [x] Manual testing several flag combinations

-------------

Commit messages:
 - 8344363: FullGCForwarding::initialize_flags is called after ObjLayout::initialize

Changes: https://git.openjdk.org/jdk/pull/22255/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344363
  Stats: 35 lines in 9 files changed: 14 ins; 16 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/22255.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22255/head:pull/22255

PR: https://git.openjdk.org/jdk/pull/22255

From azafari at openjdk.org  Wed Nov 20 11:35:27 2024
From: azafari at openjdk.org (Afshin Zafari)
Date: Wed, 20 Nov 2024 11:35:27 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
Message-ID: <gOgY8WianvtvdUezeU0iMihioxoN4JEhdscPeMIkAvk=.badfee4f-f4d6-4835-aa6d-b4bfa3ae41b6@github.com>

On Tue, 19 Nov 2024 22:01:13 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by avoiding using `mtNone`.
>> 
>> Eventually the goal would be to eliminate using `mtNone` whenever possible.
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - avoid using mtNone
>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>  - revert, we will re-do with a smaller change
>  - remove more mtNone
>  - remove API that allows to change the mem_tag for virtual memory, feedback
>  - do not allow default parameter for mtNone

Thanks for taking this issue. I add my points here:

1) The `ReservedSpace` objects in these files have to have extra MemTag (`mtTest`) parameter:
  - test_freeRegionList.cpp
  - test_virtualspace.cpp
  - test_virtualMemoryTracker.cpp
 
2) The `mtNone` as default value for the parameter should be removed from the following functions, otherwise Unknown memory types can still be reported:



Source root ? src/hotspot/os/windows/os_windows.cpp:
    3239: static char* map_or_reserve_memory_aligned(size_t size, size_t alignment, int file_desc, MemTag mem_tag = mtNone) {
  3239    assert(is_aligned(alignment, os::vm_allocation_granularity()),
  3240    assert(is_aligned(alignment, os::vm_allocation_granularity()),

Source root ? src/hotspot/share/cds/filemap.cpp:
  1773                          char *addr, size_t bytes, bool read_only,
  1774:                         bool allow_exec, MemTag mem_tag = mtNone) {
  1775    char* mem = os::map_memory(fd, file_name, file_offset, addr, bytes,

Source root ? src/hotspot/share/memory/virtualspace.hpp:
  63    void initialize(size_t size, size_t alignment, size_t page_size,
  64    void initialize(size_t size, size_t alignment, size_t page_size,
  65:                   char* requested_address, bool executable, MemTag mem_tag = mtNone);
  65  
  66  

Source root ? src/hotspot/share/nmt/memReporter.hpp:
  110    void print_total(size_t reserved, size_t committed, size_t peak = 0) const;
  111:   void print_malloc(const MemoryCounter* c, MemTag mem_tag = mtNone) const;
  112    void print_virtual_memory(size_t reserved, size_t committed, size_t peak) const;

Source root ? src/hotspot/share/nmt/memTracker.hpp:
  123    static inline void record_virtual_memory_reserve(void* addr, size_t size, const NativeCallStack& stack,
  124:     MemTag mem_tag = mtNone) {
  125      assert_post_init();

  149    static inline void record_virtual_memory_reserve_and_commit(void* addr, size_t size,
  150:     const NativeCallStack& stack, MemTag mem_tag = mtNone) {

Source root ? src/hotspot/share/nmt/virtualMemoryTracker.hpp:
  299    ReservedMemoryRegion(address base, size_t size, const NativeCallStack& stack,
  300:     MemTag mem_tag = mtNone) :

  383:   static bool add_reserved_region (address base_addr, size_t size, const NativeCallStack& stack, MemTag mem_tag = mtNone);
  384  

Source root ? src/hotspot/share/runtime/os.hpp:
  453:   static char*  reserve_memory(size_t bytes, bool executable = false, MemTag mem_tag = mtNone);

  460:   static char*  attempt_reserve_memory_at(char* addr, size_t bytes, bool executable = false, MemTag mem_tag = mtNone);

  512:   static char* map_memory_to_file(size_t size, int fd, MemTag mem_tag = mtNone);
  513:   static char* map_memory_to_file_aligned(size_t size, size_t alignment, int fd, MemTag mem_tag = mtNone);

  515:   static char* attempt_map_memory_to_file_at(char* base, size_t size, int fd, MemTag mem_tag = mtNone);

  519  static char*  map_memory(int fd, const char* file_name, size_t file_offset,
                          char *addr, size_t bytes, bool read_only = false,
  520                             char *addr, size_t bytes, bool read_only = false,
  521:                            bool allow_exec = false, MemTag mem_tag = mtNone);



3) In virtualspace.cpp, the `result` in these functions has `mtNone` tag while the parent may have another tag:


ReservedSpace ReservedSpace::first_part(size_t partition_size, size_t alignment) {
  assert(partition_size <= size(), "partition failed");
  ReservedSpace result(base(), partition_size, alignment, page_size(), special(), executable());
  return result;
}

ReservedSpace ReservedSpace::last_part(size_t partition_size, size_t alignment) {
  assert(partition_size <= size(), "partition failed");
  ReservedSpace result(base() + partition_size, size() - partition_size,
                       alignment, page_size(), special(), executable());
  return result;
}

ReservedSpace ReservedSpace::partition(size_t offset, size_t partition_size, size_t alignment) {
  assert(offset + partition_size <= size(), "partition failed");
  ReservedSpace result(base() + offset, partition_size, alignment, page_size(), special(), executable());
  return result;
}

4) In virtualspace.cpp, the `space` has `mtNone` tag at this function:

ReservedSpace ReservedSpace::space_for_range(char* base, size_t size, size_t alignment,
                                             size_t page_size, bool special, bool executable) {
  assert(is_aligned(base, os::vm_allocation_granularity()), "Unaligned base");
  assert(is_aligned(size, os::vm_page_size()), "Unaligned size");
  assert(os::page_sizes().contains(page_size), "Invalid pagesize");
  ReservedSpace space;
  space.initialize_members(base, size, alignment, page_size, special, executable);
  return space;
}

-------------

PR Review: https://git.openjdk.org/jdk/pull/21843#pullrequestreview-2448359154

From gziemski at openjdk.org  Wed Nov 20 14:53:25 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Wed, 20 Nov 2024 14:53:25 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <PA7QHXA2Mt29C7SYevcLXekhj1GDax0YP-cJo73i5xQ=.4f9821f9-1580-4844-8dbf-23fa1c73b7cd@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
 <PA7QHXA2Mt29C7SYevcLXekhj1GDax0YP-cJo73i5xQ=.4f9821f9-1580-4844-8dbf-23fa1c73b7cd@github.com>
Message-ID: <kDZC4TmszLzpwFrm7pY_a2Q66Y8lFZTfofT2r8i0CK4=.79c68298-bbc8-4e1b-b0df-99255a1b8dc8@github.com>

On Wed, 20 Nov 2024 10:03:08 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - avoid using mtNone
>>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>>  - revert, we will re-do with a smaller change
>>  - remove more mtNone
>>  - remove API that allows to change the mem_tag for virtual memory, feedback
>>  - do not allow default parameter for mtNone
>
> src/hotspot/share/memory/virtualspace.hpp line 65:
> 
>> 63: 
>> 64:   void initialize(size_t size, size_t alignment, size_t page_size,
>> 65:                   char* requested_address, bool executable, MemTag mem_tag = mtNone);
> 
> When `mtNone` is default, it is still possible to introduce/create regions of Unknown types. The MemTag arg should be mandatory.

Sorry Afshin, I didn't mention it in my summary at the top, but the goal in this fix is only to address the cases covered by the issue. I tried fixing all of them in my 1st try, but Thomas had some reservations. The "missing" cases will be addressed in a follow up(s).

However, I will copy all your feedback, so we don't loose it.

Thank you very much for taking detailed look and providing feedback!

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1850456338

From stefank at openjdk.org  Wed Nov 20 15:30:19 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 20 Nov 2024 15:30:19 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize
In-Reply-To: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
Message-ID: <6x_KtDdLE-YqtTKg9GGTnOsF6Supzmiu2VSG7UYuckU=.4b001e60-aa92-48ef-8eb8-fb7a3cc2fcaa@github.com>

On Tue, 19 Nov 2024 20:07:38 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> From the bug description:
> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
> 
> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
> 
> The fix to both issues is:
> - First disable UCOH
> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
> - then enable any flags required by compact headers
> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
> 
> Testing:
>  - [x] tier1 -UCOH (default)
>  - [ ] tier1 +UCOH
>  - [x] Manual testing several flag combinations

I think this looks good. I ran it through gdb with awatch to see that I there was no obvious reads of the variables before they were set.

A couple of small nits that would be nice fix.

src/hotspot/share/gc/serial/serialArguments.cpp line 32:

> 30: 
> 31: void SerialArguments::initialize() {
> 32:   GenArguments::initialize();

Suggestion:

  GCArguments::initialize();

There's no `GenArguments::initialize()` so I think it would be clearer to call the function with the parent class instead. Alternatively, make `ParallelGC::initialize` also call `GenArguments::initialize`.

src/hotspot/share/gc/serial/serialArguments.hpp line 35:

> 33: private:
> 34:   virtual CollectedHeap* create_heap();
> 35:   virtual void initialize();

All other GC's put the `initialize` function before the `create_heap`. It might be nice to keep that consistent.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2448960364
PR Review Comment: https://git.openjdk.org/jdk/pull/22255#discussion_r1850523493
PR Review Comment: https://git.openjdk.org/jdk/pull/22255#discussion_r1850529411

From stefank at openjdk.org  Wed Nov 20 15:38:24 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 20 Nov 2024 15:38:24 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
Message-ID: <7UxSMg_or6Avy_wuFlkkuDO0pBpyR5DiKeuY0N4uhVs=.d2c7835f-6e68-4f87-857a-e4fd6eb5f12b@github.com>

On Tue, 19 Nov 2024 22:01:13 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by avoiding using `mtNone`.
>> 
>> This fix only addresses the cases covered by the issue. I am not trying to replace every single `mtNone` here, but eventually the goal would be to do just that.
>> 
>> TODO: update copyrights.
>> 
>> Testing: undergoing MARCH5 tier1-5 ...
>
> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - avoid using mtNone
>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>  - revert, we will re-do with a smaller change
>  - remove more mtNone
>  - remove API that allows to change the mem_tag for virtual memory, feedback
>  - do not allow default parameter for mtNone

src/hotspot/os/linux/os_linux.cpp line 4587:

> 4585:   char* hint = (char*)(os::Linux::initial_thread_stack_bottom() -
> 4586:                        (StackOverflow::stack_guard_zone_size() + page_size));
> 4587:   char* codebuf = os::attempt_reserve_memory_at(hint, page_size, false, mtInternal);

Could this be mtThread instead?

src/hotspot/os/linux/os_linux.cpp line 4595:

> 4593:     // map our codebuf, try again at an address one megabyte lower.
> 4594:     hint -= 1 * M;
> 4595:     codebuf = os::attempt_reserve_memory_at(hint, page_size, false, mtInternal);

Could this be mtThread instead?

src/hotspot/share/memory/virtualspace.hpp line 74:

> 72:   // Initialize the reserved space with the given size. Depending on the size
> 73:   // a suitable page size and alignment will be used.
> 74:   explicit ReservedSpace(size_t size, MemTag mem_tag);

Suggestion:

  ReservedSpace(size_t size, MemTag mem_tag);

No need for `explicit` when you've added a parameter.

src/hotspot/share/runtime/safepointMechanism.cpp line 61:

> 59:     const size_t page_size = os::vm_page_size();
> 60:     const size_t allocation_size = 2 * page_size;
> 61:     char* polling_page = os::reserve_memory(allocation_size, false, mtSafepoint);

Suggestion:

    char* polling_page = os::reserve_memory(allocation_size, !ExecMem, mtSafepoint);

src/hotspot/share/utilities/debug.cpp line 712:

> 710: 
> 711: void initialize_assert_poison() {
> 712:   char* page = os::reserve_memory(os::vm_page_size(), false, mtInternal);

Suggestion:

  char* page = os::reserve_memory(os::vm_page_size(), !ExecMem, mtTest);

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1850535276
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1850535378
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1850543822
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1850541711
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1850541996

From shade at openjdk.org  Wed Nov 20 15:45:18 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 20 Nov 2024 15:45:18 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize
In-Reply-To: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
Message-ID: <fvjsdTn8wenlkq8OFYxQ2yIE6Wg-7821NN-Mq-ceaOQ=.5e764404-f9a3-4e7f-8f62-0ab6b4d79ba4@github.com>

On Tue, 19 Nov 2024 20:07:38 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> From the bug description:
> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
> 
> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
> 
> The fix to both issues is:
> - First disable UCOH
> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
> - then enable any flags required by compact headers
> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
> 
> Testing:
>  - [x] tier1 -UCOH (default)
>  - [ ] tier1 +UCOH
>  - [x] Manual testing several flag combinations

So we are inlining `initialize_heap_flags_and_sizes` into `initialize`, right? Is `ParallelArguments::initialize_heap_flags_and_sizes()` dangling now?

It sounds like there is a little bit of symmetry between `CompressedKlassPointers::initialize` that is called in `universe_init`, and thus after argument adjustments. I don't know if it makes sense to move `ObjLayout::initialize` to `universe_init`?

src/hotspot/share/runtime/arguments.hpp line 267:

> 265:   static void set_use_compressed_oops();
> 266:   static jint set_ergonomics_flags();
> 267:   static void setup_compact_headers();

Consistency with adjacent methods: should it be called `set_compact_headers`?

-------------

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2448920615
PR Review Comment: https://git.openjdk.org/jdk/pull/22255#discussion_r1850499811

From kdnilsen at openjdk.org  Wed Nov 20 18:33:17 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 20 Nov 2024 18:33:17 GMT
Subject: Integrated: 8344320: GenShen: Possible null pointer usage in
 shGenerationalHeap
In-Reply-To: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
References: <44dVRMMqe10lTOoeCUacRFwIo2gMuXOItULgfDE3y4g=.54b4322f-3a56-4950-9d02-f2d51bf20af3@github.com>
Message-ID: <QuYI8KoPQ81YrI3rJyFlYGOKyXV5pyvCWIma2mgE_YM=.595b077a-8315-4899-9371-ff1680319ca7@github.com>

On Tue, 19 Nov 2024 16:01:10 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Certain threads, including the vm thread and Shenandoah's control thread, do not have PLABs.  There are scenarios where each may need to allocate old-gen memory on the slow-path of the load reference barrier.  This PR fixes an assumption in the allocator that assumed all threads have plabs.

This pull request has now been integrated.

Changeset: bf938e57
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/bf938e57c53be07011552d35398cbe7ec2366b22
Stats:     17 lines in 1 file changed: 1 ins; 1 del; 15 mod

8344320: GenShen: Possible null pointer usage in shGenerationalHeap

Reviewed-by: wkemper, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/543

From wkemper at openjdk.org  Wed Nov 20 19:00:50 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 19:00:50 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <nXVt5nih04r0qv1t6Z1IvkSlJtP2uuPST8kaWS70QLE=.51ae819a-bba6-4cf5-85a2-8cb7054b5d8b@github.com>
References: <nXVt5nih04r0qv1t6Z1IvkSlJtP2uuPST8kaWS70QLE=.51ae819a-bba6-4cf5-85a2-8cb7054b5d8b@github.com>
Message-ID: <8GfCpyvJ0PV4YAUBIerijNJv7BBJ8DfCQtV5dvut71k=.ab16c4a8-c02d-4c5d-8d46-cd0f2801e988@github.com>

On Fri, 15 Nov 2024 14:13:14 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-24+24

This pull request has now been integrated.

Changeset: cc46aa7e
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/cc46aa7e96d530a217f3075faa85f53f9a1b64e0
Stats:     107177 lines in 2836 files changed: 28070 ins; 70167 del; 8940 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/541

From wkemper at openjdk.org  Wed Nov 20 19:07:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 19:07:59 GMT
Subject: RFR: 8344263: GenShen: Reduce extraneous log messages at INFO
 level [v8]
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <a1Ci-MZdW29ldxVd7WXeaeIk5suhofm3WA-0Hy1tEt0=.2e90f685-6eb7-4c46-9afc-a2a7bff222f9@github.com>

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:

 - Merge remote-tracking branch 'shenandoah/master' into logging-cleanup
 - Fix old growth triggers test
 - Log all heuristic triggers with consistent formatting
 - Update test, remove unused header, use proper case for old gen phases
 - Fix formatting for non-generational log messages
 - Make generation names proper case
 - Log heuristics decisions at info with ergo tag
 - Reduce logging at info, use proper case for generation names

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/540/files
  - new: https://git.openjdk.org/shenandoah/pull/540/files/02d3d24d..da70b543

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=07
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=540&range=06-07

  Stats: 107288 lines in 2854 files changed: 28083 ins; 70210 del; 8995 mod
  Patch: https://git.openjdk.org/shenandoah/pull/540.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/540/head:pull/540

PR: https://git.openjdk.org/shenandoah/pull/540

From wkemper at openjdk.org  Wed Nov 20 19:23:22 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 19:23:22 GMT
Subject: RFR: 8344638: GenShen: Verifier should not touch claim token
Message-ID: <WnkoviUrBdAerh6IIru3WW-B2sFg-gs_AhTHBNEAl3A=.06688d27-1f0c-4d9a-8772-ef5e6b25387f@github.com>

GenShen is using parallel_threads_do for a verification step, but this uses a 'claim token' which is shared by normal gc activity. The verifier should not touch this shared state, lest it interfere with (or be interfered with by) the gc cycle.

# Testing

Ran this through internal pipelines: Dacapo, Extremem, SpecJBB2015, SpecJVM2008, Heapothesys (with and without GC stress flags).

-------------

Commit messages:
 - Verifier should not touch claim token (lest it unwittingly obscure a defect)

Changes: https://git.openjdk.org/shenandoah/pull/546/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=546&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344638
  Stats: 5 lines in 1 file changed: 0 ins; 4 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/546.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/546/head:pull/546

PR: https://git.openjdk.org/shenandoah/pull/546

From shade at openjdk.org  Wed Nov 20 19:29:33 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 20 Nov 2024 19:29:33 GMT
Subject: RFR: 8344638: GenShen: Verifier should not touch claim token
In-Reply-To: <WnkoviUrBdAerh6IIru3WW-B2sFg-gs_AhTHBNEAl3A=.06688d27-1f0c-4d9a-8772-ef5e6b25387f@github.com>
References: <WnkoviUrBdAerh6IIru3WW-B2sFg-gs_AhTHBNEAl3A=.06688d27-1f0c-4d9a-8772-ef5e6b25387f@github.com>
Message-ID: <UyP6VGbw7hV9fFAWTOgtcp15_2x60_-oa_kYkI6_9Kk=.ff15e66b-e205-465b-a38d-c75e474a1b2d@github.com>

On Wed, 20 Nov 2024 19:17:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

> GenShen is using parallel_threads_do for a verification step, but this uses a 'claim token' which is shared by normal gc activity. The verifier should not touch this shared state, lest it interfere with (or be interfered with by) the gc cycle.
> 
> # Testing
> 
> Ran this through internal pipelines: Dacapo, Extremem, SpecJBB2015, SpecJVM2008, Heapothesys (with and without GC stress flags).

Yup, this is a right thing to do.

-------------

Marked as reviewed by shade (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/546#pullrequestreview-2449517446

From wkemper at openjdk.org  Wed Nov 20 19:33:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 19:33:06 GMT
Subject: RFR: 8344640: GenShen: Reuse existing card mark barrier function when
 dropping references
Message-ID: <S-3nB6wFEsHxy4jBfSN5mh7LcfXywlfo5s6f2BI1Mfk=.7de6eec5-bf55-4f02-af33-1c73f6a478b2@github.com>

When GenShen drops references during concurrent weak reference processing, it manipulates pointers in a list that is threaded through java objects and so must go through the card mark barrier. This change has it reuse an existing function for this purpose.

## Testing

Ran this through internal pipelines: Dacapo, Extremem, SpecJBB2015, SpecJVM2008, Heapothesys (with and without GC stress flags).

-------------

Commit messages:
 - Merge remote-tracking branch 'shenandoah/master' into drop-with-card-barrier
 - Use card barrier when dropping reference

Changes: https://git.openjdk.org/shenandoah/pull/547/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=547&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344640
  Stats: 5 lines in 1 file changed: 0 ins; 3 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/547.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/547/head:pull/547

PR: https://git.openjdk.org/shenandoah/pull/547

From shade at openjdk.org  Wed Nov 20 19:33:07 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 20 Nov 2024 19:33:07 GMT
Subject: RFR: 8344640: GenShen: Reuse existing card mark barrier function
 when dropping references
In-Reply-To: <S-3nB6wFEsHxy4jBfSN5mh7LcfXywlfo5s6f2BI1Mfk=.7de6eec5-bf55-4f02-af33-1c73f6a478b2@github.com>
References: <S-3nB6wFEsHxy4jBfSN5mh7LcfXywlfo5s6f2BI1Mfk=.7de6eec5-bf55-4f02-af33-1c73f6a478b2@github.com>
Message-ID: <J9N7w-F4Bw46Qzk_AFhf2_UaDuZAmDX_U1zAaiVMGVg=.b72b666a-5611-4866-8ce9-1e0912988721@github.com>

On Wed, 20 Nov 2024 19:24:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When GenShen drops references during concurrent weak reference processing, it manipulates pointers in a list that is threaded through java objects and so must go through the card mark barrier. This change has it reuse an existing function for this purpose.
> 
> ## Testing
> 
> Ran this through internal pipelines: Dacapo, Extremem, SpecJBB2015, SpecJVM2008, Heapothesys (with and without GC stress flags).

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/547#pullrequestreview-2449521452

From rkennke at openjdk.org  Wed Nov 20 19:51:04 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 20 Nov 2024 19:51:04 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v2]
In-Reply-To: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
Message-ID: <OCt-NBfG1eA1aqaWcPmS-QrYC2k_IscXiWLCvtMa2-s=.0fc6f83d-7865-44df-8134-c0c4c9faa4eb@github.com>

> From the bug description:
> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
> 
> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
> 
> The fix to both issues is:
> - First disable UCOH
> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
> - then enable any flags required by compact headers
> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
> 
> Testing:
>  - [x] tier1 -UCOH (default)
>  - [ ] tier1 +UCOH
>  - [x] Manual testing several flag combinations

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Address review comments

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22255/files
  - new: https://git.openjdk.org/jdk/pull/22255/files/26c3680e..6f8b5adf

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=00-01

  Stats: 10 lines in 4 files changed: 4 ins; 4 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/22255.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22255/head:pull/22255

PR: https://git.openjdk.org/jdk/pull/22255

From rkennke at openjdk.org  Wed Nov 20 19:51:04 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 20 Nov 2024 19:51:04 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v2]
In-Reply-To: <fvjsdTn8wenlkq8OFYxQ2yIE6Wg-7821NN-Mq-ceaOQ=.5e764404-f9a3-4e7f-8f62-0ab6b4d79ba4@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <fvjsdTn8wenlkq8OFYxQ2yIE6Wg-7821NN-Mq-ceaOQ=.5e764404-f9a3-4e7f-8f62-0ab6b4d79ba4@github.com>
Message-ID: <yPrHfuThnt72UcYBxvPagfaMjN9J4g8Ntuc-FoPlhqE=.c627d0e0-f904-4273-973f-c174644f5a84@github.com>

On Wed, 20 Nov 2024 15:42:46 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> So we are inlining `initialize_heap_flags_and_sizes` into `initialize`, right?

No, not really. `initialize_heap_flags_and_sizes` is still called from `initialize_heap_sizes` which is called from `universe_init`.

> Is `ParallelArguments::initialize_heap_flags_and_sizes()` dangling now?

No.
 
> It sounds like there is a little bit of symmetry between `CompressedKlassPointers::initialize` that is called from `universe_init`, and thus after argument adjustments. I don't know if it makes sense to move `ObjLayout::initialize` to `universe_init`? 

Hmm yeah, that makes sense. I'll do that.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22255#issuecomment-2489412764

From wkemper at openjdk.org  Wed Nov 20 19:59:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 19:59:42 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <bKR7YS16f4lnW2CvJZ0PmyOGqhwn88MyxCva9kS7jno=.543f2586-6baa-4cb5-9679-e8a760f5569a@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
 <bKR7YS16f4lnW2CvJZ0PmyOGqhwn88MyxCva9kS7jno=.543f2586-6baa-4cb5-9679-e8a760f5569a@github.com>
Message-ID: <aMApW8Zrx34CZH9mm730zgziuuSKg97Gnw-4DahXChU=.f8217423-0dca-4a54-92e7-15e0f9c545ca@github.com>

On Tue, 19 Nov 2024 23:56:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 455:
>> 
>>> 453:     // Note: would be sufficient to mark only the card that holds the start of this Reference object.
>>> 454:     heap->old_generation()->card_scan()->mark_range_as_dirty(cast_from_oop<HeapWord*>(reference), reference->size());
>>> 455:   }
>> 
>> Two things:
>>  a) This sounds like `card_table_barrier(reference, raw_referent)`? Since this code is getting called for every dropped reference, just checking a `ShenandoahCardBarrier` flag sounds more efficient. 
>>  b) Is there a point in dirtying up to `reference->size()`?
>
> Will test this.

https://bugs.openjdk.org/browse/JDK-8344640

>> src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 657:
>> 
>>> 655:           _generation(nullptr) {
>>> 656:     if (_options._verify_marked == ShenandoahVerifier::_verify_marked_complete_satb_empty) {
>>> 657:       Threads::change_thread_claim_token();
>> 
>> It is fairly odd to see Verifier touching the claim token, since the bug _may be_ somewhere in parallel thread oop iteration infra. I think it is fine to just use `Threads::threads_do` (non-parallel version), which AFAIU does not require token modifications.
>
> Will test this.

https://bugs.openjdk.org/browse/JDK-8344638

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1850900559
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1850901137

From shade at openjdk.org  Wed Nov 20 20:22:27 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 20 Nov 2024 20:22:27 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
Message-ID: <_9t4lSh7P6yPIZtllm4hVRuZ7kqyhWcyCfrKh37B8fA=.fb510cc5-c9db-4c31-ba90-6189789f10b8@github.com>

On Wed, 13 Nov 2024 19:32:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
> 
>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>  - Use new CompactHeader forwarding APIs in generational mode
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge
>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>    
>    Reviewed-by: wkemper
>  - Merge
>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>    
>    Reviewed-by: kdnilsen
>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8

Almost there, 220/230 reviewed.

src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.cpp line 62:

> 60:   void work(uint worker_id) {
> 61:     ShenandoahHeap* heap = ShenandoahHeap::heap();
> 62:     ShenandoahParallelWorkerSession worker_session(worker_id);

Why this needs to be `ShenandoahParallelWorkerSession`, not `ShenandoahConcurrentWorkerSession`? This is a concurrent step, is it?

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2449189030
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1850899803

From shade at openjdk.org  Wed Nov 20 20:22:28 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 20 Nov 2024 20:22:28 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <fVT6LGlgaT1etS3mg7cUvO5IT2EsfxP_ilN13TM77XY=.1fcd85a0-e957-4834-a788-ae4bd62356b5@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
 <fVT6LGlgaT1etS3mg7cUvO5IT2EsfxP_ilN13TM77XY=.1fcd85a0-e957-4834-a788-ae4bd62356b5@github.com>
Message-ID: <86FIc0saJkWuxy228AuIYA6VKDYhwQCd5bH31sBK6Ws=.aa86fa8d-6c88-4edb-ac89-c89b68e65f29@github.com>

On Tue, 19 Nov 2024 23:07:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.cpp line 44:
>> 
>>> 42:   for (size_t idx = 0; idx < num_regions; idx++) {
>>> 43:     ShenandoahHeapRegion* r = heap->get_region(idx);
>>> 44:     if (r->is_affiliated() && heap->is_bitmap_slice_committed(r) && !is_bitmap_clear_range(r->bottom(), r->end())) {
>> 
>> I don't understand this for single gen mode. In that mode `is_affiliated() == false` always, right? So this check never passes, and `is_bitmap_clear` always returns `true`?
>
> No, in single gen mode regions that are not `FREE` are affiliated with `YOUNG`.

That's weird. We have `Young`, `Old` and `Global` generations. Why affiliation is not `Free`, `Young`, `Old`, and `Global` then?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1850665665

From wkemper at openjdk.org  Wed Nov 20 20:22:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 20:22:29 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <86FIc0saJkWuxy228AuIYA6VKDYhwQCd5bH31sBK6Ws=.aa86fa8d-6c88-4edb-ac89-c89b68e65f29@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
 <fVT6LGlgaT1etS3mg7cUvO5IT2EsfxP_ilN13TM77XY=.1fcd85a0-e957-4834-a788-ae4bd62356b5@github.com>
 <86FIc0saJkWuxy228AuIYA6VKDYhwQCd5bH31sBK6Ws=.aa86fa8d-6c88-4edb-ac89-c89b68e65f29@github.com>
Message-ID: <cLbNu0S8S4ANigA6LkDXLOQS2nbpIGt1IlDPuiIpa1g=.17652f7f-3ce5-421f-9d08-305910bfd56c@github.com>

On Wed, 20 Nov 2024 16:47:20 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> No, in single gen mode regions that are not `FREE` are affiliated with `YOUNG`.
>
> That's weird. We have `Young`, `Old` and `Global` generations. Why affiliation is not `Free`, `Young`, `Old`, and `Global` then?

Yes, it's a little weird. It didn't seem necessary to have a `Global` affiliation.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1850927526

From wkemper at openjdk.org  Wed Nov 20 20:31:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 20:31:29 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v7]
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <fJF30IcUBcbu84Aq7kLn9OeygdPHM_Nk58dhjdSs2Eo=.8e2599f6-d875-444a-9a1a-a8ff93332a26@github.com>

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Restore logging format, show change in committed heap, rather than usage

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22019/files
  - new: https://git.openjdk.org/jdk/pull/22019/files/a00945c2..2cb71140

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=06
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=05-06

  Stats: 14 lines in 2 files changed: 10 ins; 3 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From wkemper at openjdk.org  Wed Nov 20 20:31:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 20:31:29 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v7]
In-Reply-To: <I0GzgbIQG1aGxmJ4c1wh0N5uYDDO-5fIrnEtT4t675s=.b21ca11b-1061-4bbe-97c3-79edccdabab4@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
 <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>
 <2ctRsJ6JhReYC9yjcPV4eRljht3bMAZ9B3urifpvQXQ=.470aa336-f334-4345-863b-557250aa3416@github.com>
 <I0GzgbIQG1aGxmJ4c1wh0N5uYDDO-5fIrnEtT4t675s=.b21ca11b-1061-4bbe-97c3-79edccdabab4@github.com>
Message-ID: <TygSXGP4yj46QlEcRv-m5AfZTb6rreI9SGdjbZpziiM=.fbe4a0be-b601-4219-bc88-40d27f1787f4@github.com>

On Wed, 20 Nov 2024 09:26:19 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Yeah, at least restore the log format and add `gc+start` log as well.
>
> This one is still not addressed, unfortunately ^

Yes, I spent some time trying to resurrect `ShenandoahConcurrentPhase` for uncommit here, but it really doesn't want to be used outside of a gc cycle. Also, previously it was logging heap _usage_, which isn't quite what we want here (this may actually increase during this phase, which makes it seem as though nothing is being uncommitted).

I've restored the original logging format, but instead of logging heap usage it is now logging heap committed before and after. Here is an excerpt from specjbb2015 with `-Xms5g -Xmx10g`:


[2024-11-20T20:02:25.056+0000][97.396s][22293][info][gc,start       ] Concurrent uncommit
[2024-11-20T20:02:25.072+0000][97.412s][22293][info][gc             ] Concurrent uncommit 5424M->5120M(5120M) 15.988ms
[2024-11-20T20:05:17.916+0000][270.255s][22293][info][gc,start       ] Concurrent uncommit
[2024-11-20T20:05:18.169+0000][270.508s][22293][info][gc             ] Concurrent uncommit 10240M->5120M(5120M) 253.048ms
[2024-11-20T20:06:45.329+0000][357.668s][22293][info][gc,start       ] Concurrent uncommit
[2024-11-20T20:06:45.596+0000][357.935s][22293][info][gc             ] Concurrent uncommit 10240M->5120M(5120M) 267.144ms
[2024-11-20T20:06:57.147+0000][369.486s][22293][info][gc,start       ] Concurrent uncommit
[2024-11-20T20:06:57.148+0000][369.487s][22293][info][gc             ] Concurrent uncommit 5456M->5440M(5440M) 1.189ms

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1850939713

From wkemper at openjdk.org  Wed Nov 20 20:33:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 20:33:38 GMT
Subject: Integrated: 8344638: GenShen: Verifier should not touch claim token
In-Reply-To: <WnkoviUrBdAerh6IIru3WW-B2sFg-gs_AhTHBNEAl3A=.06688d27-1f0c-4d9a-8772-ef5e6b25387f@github.com>
References: <WnkoviUrBdAerh6IIru3WW-B2sFg-gs_AhTHBNEAl3A=.06688d27-1f0c-4d9a-8772-ef5e6b25387f@github.com>
Message-ID: <3VW22qzjwyAxEjaPIeJQNL61_hjpvrc39fLgXBErpOs=.89a023ee-77bd-4317-8ba2-198ef42ad9a4@github.com>

On Wed, 20 Nov 2024 19:17:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

> GenShen is using parallel_threads_do for a verification step, but this uses a 'claim token' which is shared by normal gc activity. The verifier should not touch this shared state, lest it interfere with (or be interfered with by) the gc cycle.
> 
> # Testing
> 
> Ran this through internal pipelines: Dacapo, Extremem, SpecJBB2015, SpecJVM2008, Heapothesys (with and without GC stress flags).

This pull request has now been integrated.

Changeset: 034d5859
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/034d58599ef7350def3f3b152b50650407368b78
Stats:     5 lines in 1 file changed: 0 ins; 4 del; 1 mod

8344638: GenShen: Verifier should not touch claim token

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/shenandoah/pull/546

From wkemper at openjdk.org  Wed Nov 20 20:34:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 20:34:33 GMT
Subject: Integrated: 8344263: GenShen: Reduce extraneous log messages at INFO
 level
In-Reply-To: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
References: <7YFs3mkanxIx_NcVWqhFfwSWsjUfRQyZSwBXLnx_Vsk=.6526d1d7-1e31-4734-80bb-f066aab05c1a@github.com>
Message-ID: <gfKon0bffrDiJS9FpAr9iMyFjYLAZHosIbHbUBpqcRE=.c75f4908-83ad-4637-9d41-d38f7af7fbea@github.com>

On Fri, 15 Nov 2024 01:26:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Genshen logs a lot of fine level detail at `gc=info`. This category should be reserved for major phase changes in the gc cycle.

This pull request has now been integrated.

Changeset: 2e516c9e
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/2e516c9e553be9c65a7dfddb6c8e84ec4f62f66e
Stats:     146 lines in 25 files changed: 23 ins; 13 del; 110 mod

8344263: GenShen: Reduce extraneous log messages at INFO level

Reviewed-by: ysr, shade

-------------

PR: https://git.openjdk.org/shenandoah/pull/540

From kdnilsen at openjdk.org  Wed Nov 20 20:53:17 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 20 Nov 2024 20:53:17 GMT
Subject: RFR: 8344592: GenShen: Remove unnecessary comments and changes
In-Reply-To: <fRCVFh33mzdYWCeg0y02S5zliVPskSn4KrNzufSwIDc=.33a78f41-e509-4d5b-9a10-c83e3951a8e7@github.com>
References: <fRCVFh33mzdYWCeg0y02S5zliVPskSn4KrNzufSwIDc=.33a78f41-e509-4d5b-9a10-c83e3951a8e7@github.com>
Message-ID: <DRQvYnz-P5Qyj47HMyQbUDyBzg8KDlqGwMWlPzvPrsc=.191d70b0-b8a2-4a2a-9f88-88b45829032d@github.com>

On Tue, 19 Nov 2024 23:46:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Remove unnecessary comments and reduce granularity of access controls in shConcurrentGC.hpp.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/545#pullrequestreview-2449670345

From wkemper at openjdk.org  Wed Nov 20 20:56:50 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 20 Nov 2024 20:56:50 GMT
Subject: Integrated: 8344592: GenShen: Remove unnecessary comments and changes
In-Reply-To: <fRCVFh33mzdYWCeg0y02S5zliVPskSn4KrNzufSwIDc=.33a78f41-e509-4d5b-9a10-c83e3951a8e7@github.com>
References: <fRCVFh33mzdYWCeg0y02S5zliVPskSn4KrNzufSwIDc=.33a78f41-e509-4d5b-9a10-c83e3951a8e7@github.com>
Message-ID: <zrLIKrsdttwtEBQrCdDJBSyrs5O4OQ_Vsd-gomeHBTs=.69bf4cbc-9b71-4ff5-a871-ece886677a85@github.com>

On Tue, 19 Nov 2024 23:46:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Remove unnecessary comments and reduce granularity of access controls in shConcurrentGC.hpp.

This pull request has now been integrated.

Changeset: 49c7a784
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/49c7a78468456f79531424086fd1b4afdbbd8e42
Stats:     32 lines in 3 files changed: 4 ins; 21 del; 7 mod

8344592: GenShen: Remove unnecessary comments and changes

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/545

From ysr at openjdk.org  Wed Nov 20 23:57:38 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 20 Nov 2024 23:57:38 GMT
Subject: RFR: 8344640: GenShen: Reuse existing card mark barrier function
 when dropping references
In-Reply-To: <S-3nB6wFEsHxy4jBfSN5mh7LcfXywlfo5s6f2BI1Mfk=.7de6eec5-bf55-4f02-af33-1c73f6a478b2@github.com>
References: <S-3nB6wFEsHxy4jBfSN5mh7LcfXywlfo5s6f2BI1Mfk=.7de6eec5-bf55-4f02-af33-1c73f6a478b2@github.com>
Message-ID: <JqI6DEgFWCwvvMz80PGL05N4-OCxd9O2FEyOXwnjf-s=.3122807f-5115-41c4-94b0-ec8441e7cd4e@github.com>

On Wed, 20 Nov 2024 19:24:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When GenShen drops references during concurrent weak reference processing, it manipulates pointers in a list that is threaded through java objects and so must go through the card mark barrier. This change has it reuse an existing function for this purpose.
> 
> ## Testing
> 
> Ran this through internal pipelines: Dacapo, Extremem, SpecJBB2015, SpecJVM2008, Heapothesys (with and without GC stress flags).

good catch and cleanup.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/547#pullrequestreview-2449947029

From wkemper at openjdk.org  Thu Nov 21 00:35:02 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 00:35:02 GMT
Subject: RFR: 8344670: GenShen: Use concurrent worker session for concurrent
 mark phase
Message-ID: <XOVlCS229U01vQ1ZFUUcQTefDv7MJqEfcTqpX-BdeIo=.95bda441-b9aa-4f82-a0a2-632b3063c699@github.com>

Concurrent/parallel worker sessions emit different JFR events. Mark is concurrent, so should emit `EventGCPhaseConcurrent`.

-------------

Commit messages:
 - Use concurrent worker session for concurrent phase

Changes: https://git.openjdk.org/shenandoah/pull/548/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=548&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344670
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/548.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/548/head:pull/548

PR: https://git.openjdk.org/shenandoah/pull/548

From wkemper at openjdk.org  Thu Nov 21 00:36:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 00:36:41 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <_9t4lSh7P6yPIZtllm4hVRuZ7kqyhWcyCfrKh37B8fA=.fb510cc5-c9db-4c31-ba90-6189789f10b8@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <_9t4lSh7P6yPIZtllm4hVRuZ7kqyhWcyCfrKh37B8fA=.fb510cc5-c9db-4c31-ba90-6189789f10b8@github.com>
Message-ID: <Yfa1kc7wc574qkAWQnA9TWmwSyKlTuMEF4BO5iiBp8s=.a2ee3b3d-89f1-4644-a701-332559cb80a3@github.com>

On Wed, 20 Nov 2024 19:55:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 510 commits:
>> 
>>  - Merge branch 'merge-latest' into great-genshen-pr-redux
>>  - Use new CompactHeader forwarding APIs in generational mode
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge
>>  - 8343649: Shenandoah: ShenandoahEvacInfo event does not follow JFR guidelines
>>    
>>    Reviewed-by: wkemper
>>  - Merge
>>  - 8343227: GenShen: Fold resource mark into management of preselected regions
>>    
>>    Reviewed-by: kdnilsen
>>  - Merge openjdk/jdk tip into great-genshen-pr-redux
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - Merge remote-tracking branch 'jdk/master' into merge-latest
>>  - ... and 500 more: https://git.openjdk.org/jdk/compare/889f9062...5e02b5d8
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.cpp line 62:
> 
>> 60:   void work(uint worker_id) {
>> 61:     ShenandoahHeap* heap = ShenandoahHeap::heap();
>> 62:     ShenandoahParallelWorkerSession worker_session(worker_id);
> 
> Why this needs to be `ShenandoahParallelWorkerSession`, not `ShenandoahConcurrentWorkerSession`? This is a concurrent step, is it?

Yes, https://bugs.openjdk.org/browse/JDK-8344670

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1851158343

From wkemper at openjdk.org  Thu Nov 21 00:38:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 00:38:36 GMT
Subject: Integrated: 8344640: GenShen: Reuse existing card mark barrier
 function when dropping references
In-Reply-To: <S-3nB6wFEsHxy4jBfSN5mh7LcfXywlfo5s6f2BI1Mfk=.7de6eec5-bf55-4f02-af33-1c73f6a478b2@github.com>
References: <S-3nB6wFEsHxy4jBfSN5mh7LcfXywlfo5s6f2BI1Mfk=.7de6eec5-bf55-4f02-af33-1c73f6a478b2@github.com>
Message-ID: <QJUO4ff79CRQ9tk0fruqVEF0eiTnsxP_72ifAAU6zC8=.dc79dfb4-dec8-4fad-80e2-2f154eb446d2@github.com>

On Wed, 20 Nov 2024 19:24:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When GenShen drops references during concurrent weak reference processing, it manipulates pointers in a list that is threaded through java objects and so must go through the card mark barrier. This change has it reuse an existing function for this purpose.
> 
> ## Testing
> 
> Ran this through internal pipelines: Dacapo, Extremem, SpecJBB2015, SpecJVM2008, Heapothesys (with and without GC stress flags).

This pull request has now been integrated.

Changeset: 73cab701
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/73cab701bc41f66c43b10f54aacae14e7720a04b
Stats:     5 lines in 1 file changed: 0 ins; 3 del; 2 mod

8344640: GenShen: Reuse existing card mark barrier function when dropping references

Reviewed-by: shade, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/547

From kdnilsen at openjdk.org  Thu Nov 21 00:40:38 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 21 Nov 2024 00:40:38 GMT
Subject: RFR: 8344670: GenShen: Use concurrent worker session for
 concurrent mark phase
In-Reply-To: <XOVlCS229U01vQ1ZFUUcQTefDv7MJqEfcTqpX-BdeIo=.95bda441-b9aa-4f82-a0a2-632b3063c699@github.com>
References: <XOVlCS229U01vQ1ZFUUcQTefDv7MJqEfcTqpX-BdeIo=.95bda441-b9aa-4f82-a0a2-632b3063c699@github.com>
Message-ID: <wLJcJ6bJr8x2aUHPqD6Gmr5jo0tn0vDR7ERj2LZvAmo=.87ffcd47-6bcf-4f5f-b58d-5810bf9dd46e@github.com>

On Thu, 21 Nov 2024 00:31:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Concurrent/parallel worker sessions emit different JFR events. Mark is concurrent, so should emit `EventGCPhaseConcurrent`.

Thanks for catching this.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/548#pullrequestreview-2449991235

From wkemper at openjdk.org  Thu Nov 21 00:53:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 00:53:38 GMT
Subject: RFR: 8344670: GenShen: Use concurrent worker session for
 concurrent mark phase
In-Reply-To: <XOVlCS229U01vQ1ZFUUcQTefDv7MJqEfcTqpX-BdeIo=.95bda441-b9aa-4f82-a0a2-632b3063c699@github.com>
References: <XOVlCS229U01vQ1ZFUUcQTefDv7MJqEfcTqpX-BdeIo=.95bda441-b9aa-4f82-a0a2-632b3063c699@github.com>
Message-ID: <JLGrUks3Nyp-LmzeqdHnYE1tZSYfEDAoFSh1d6CwFV8=.2d421ca0-eb82-45ce-a71a-b641cbebd84b@github.com>

On Thu, 21 Nov 2024 00:31:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Concurrent/parallel worker sessions emit different JFR events. Mark is concurrent, so should emit `EventGCPhaseConcurrent`.

@shipilev caught it!

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/548#issuecomment-2489839358

From wkemper at openjdk.org  Thu Nov 21 00:53:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 00:53:38 GMT
Subject: Integrated: 8344670: GenShen: Use concurrent worker session for
 concurrent mark phase
In-Reply-To: <XOVlCS229U01vQ1ZFUUcQTefDv7MJqEfcTqpX-BdeIo=.95bda441-b9aa-4f82-a0a2-632b3063c699@github.com>
References: <XOVlCS229U01vQ1ZFUUcQTefDv7MJqEfcTqpX-BdeIo=.95bda441-b9aa-4f82-a0a2-632b3063c699@github.com>
Message-ID: <sLAnFmu6pIN_-FngdyDMbYbMAf3vVAKgpcmATWt4NmM=.c1f49936-0c15-4c21-b26c-173d9c6f4038@github.com>

On Thu, 21 Nov 2024 00:31:24 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Concurrent/parallel worker sessions emit different JFR events. Mark is concurrent, so should emit `EventGCPhaseConcurrent`.

This pull request has now been integrated.

Changeset: 88a784a5
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/88a784a582c896762f1a982b61bf9e4b725004e9
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8344670: GenShen: Use concurrent worker session for concurrent mark phase

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/548

From wkemper at openjdk.org  Thu Nov 21 00:59:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 00:59:27 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 524 commits:

 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
   
   Reviewed-by: kdnilsen
 - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
   
   Reviewed-by: shade, ysr
 - 8344592: GenShen: Remove unnecessary comments and changes
   
   Reviewed-by: kdnilsen
 - 8344263: GenShen: Reduce extraneous log messages at INFO level
   
   Reviewed-by: ysr, shade
 - 8344638: GenShen: Verifier should not touch claim token
   
   Reviewed-by: shade
 - Merge
 - 8344320: GenShen: Possible null pointer usage in shGenerationalHeap
   
   Reviewed-by: wkemper, ysr
 - 8344321: GenShen: Fix various sonar scan warnings
   
   Reviewed-by: kdnilsen, shade
 - 8344264: GenShen: Improve comments and method names
   
   Reviewed-by: shade
 - ... and 514 more: https://git.openjdk.org/jdk/compare/95a00f8a...7ab16403

-------------

Changes: https://git.openjdk.org/jdk/pull/21273/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=07
  Stats: 22960 lines in 230 files changed: 21196 ins; 892 del; 872 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From shade at openjdk.org  Thu Nov 21 08:20:17 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Nov 2024 08:20:17 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v7]
In-Reply-To: <fJF30IcUBcbu84Aq7kLn9OeygdPHM_Nk58dhjdSs2Eo=.8e2599f6-d875-444a-9a1a-a8ff93332a26@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <fJF30IcUBcbu84Aq7kLn9OeygdPHM_Nk58dhjdSs2Eo=.8e2599f6-d875-444a-9a1a-a8ff93332a26@github.com>
Message-ID: <IVlOgfwwCeqv7NnYNxjkmyw_94lOSrdwQpPX25tISys=.d9aaa52c-5de6-4c5b-bda8-bca165407522@github.com>

On Wed, 20 Nov 2024 20:31:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Restore logging format, show change in committed heap, rather than usage

I think log message is still confusing a bit...

-------------

Changes requested by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2450572312

From shade at openjdk.org  Thu Nov 21 08:20:18 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Nov 2024 08:20:18 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v7]
In-Reply-To: <TygSXGP4yj46QlEcRv-m5AfZTb6rreI9SGdjbZpziiM=.fbe4a0be-b601-4219-bc88-40d27f1787f4@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
 <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>
 <2ctRsJ6JhReYC9yjcPV4eRljht3bMAZ9B3urifpvQXQ=.470aa336-f334-4345-863b-557250aa3416@github.com>
 <I0GzgbIQG1aGxmJ4c1wh0N5uYDDO-5fIrnEtT4t675s=.b21ca11b-1061-4bbe-97c3-79edccdabab4@github.com>
 <TygSXGP4yj46QlEcRv-m5AfZTb6rreI9SGdjbZpziiM=.fbe4a0be-b601-4219-bc88-40d27f1787f4@github.com>
Message-ID: <9MFkD3YNuSJIBPOsHYEKFX98XkMGKgMYa_7j-7usxC0=.0a99aabd-6272-40db-a5aa-6022023c1e4f@github.com>

On Wed, 20 Nov 2024 20:26:15 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This one is still not addressed, unfortunately ^
>
> Yes, I spent some time trying to resurrect `ShenandoahConcurrentPhase` for uncommit here, but it really doesn't want to be used outside of a gc cycle. Also, previously it was logging heap _usage_, which isn't quite what we want here (this may actually increase during this phase, which makes it seem as though nothing is being uncommitted).
> 
> I've restored the original logging format, but instead of logging heap usage it is now logging heap committed before and after. Here is an excerpt from specjbb2015 with `-Xms5g -Xmx10g`:
> 
> 
> [2024-11-20T20:02:25.056+0000][97.396s][22293][info][gc,start       ] Concurrent uncommit
> [2024-11-20T20:02:25.072+0000][97.412s][22293][info][gc             ] Concurrent uncommit 5424M->5120M(5120M) 15.988ms
> [2024-11-20T20:05:17.916+0000][270.255s][22293][info][gc,start       ] Concurrent uncommit
> [2024-11-20T20:05:18.169+0000][270.508s][22293][info][gc             ] Concurrent uncommit 10240M->5120M(5120M) 253.048ms
> [2024-11-20T20:06:45.329+0000][357.668s][22293][info][gc,start       ] Concurrent uncommit
> [2024-11-20T20:06:45.596+0000][357.935s][22293][info][gc             ] Concurrent uncommit 10240M->5120M(5120M) 267.144ms
> [2024-11-20T20:06:57.147+0000][369.486s][22293][info][gc,start       ] Concurrent uncommit
> [2024-11-20T20:06:57.148+0000][369.487s][22293][info][gc             ] Concurrent uncommit 5456M->5440M(5440M) 1.189ms

If we are emitting a log line that looks like a properly formatted GC log line, but the numbers there mean something else for `Concurrent uncommit`, we are bound to confuse users and automatic tools. Uncommit should affect `capacity`, this is how we know how deep we have uncommitted. So, I suggest we emit:


Concurrent uncommit XXXXM->XXXXM (YYYYM) z.zzzms


...where `XXXX` is the heap used at the end of uncommit (note before and after are the same) and YYYY is capacity. This will not expose users to thinking uncommit grows the heap usage, and would give us instantaneous view on heap usage and capacity.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1851556763

From rkennke at openjdk.org  Thu Nov 21 11:15:55 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 21 Nov 2024 11:15:55 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v3]
In-Reply-To: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
Message-ID: <V2OdkJPssdAYP20z_B9Sl6N2d5NtmxMy2Fe5QqAKCR8=.f89ba298-8660-4112-a2ef-88201a8e927b@github.com>

> From the bug description:
> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
> 
> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
> 
> The fix to both issues is:
> - First disable UCOH
> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
> - then enable any flags required by compact headers
> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
> 
> Testing:
>  - [x] tier1 -UCOH (default)
>  - [ ] tier1 +UCOH
>  - [x] Manual testing several flag combinations

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Rename setup_compact_headers

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22255/files
  - new: https://git.openjdk.org/jdk/pull/22255/files/6f8b5adf..a42d4900

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=01-02

  Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/22255.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22255/head:pull/22255

PR: https://git.openjdk.org/jdk/pull/22255

From stefank at openjdk.org  Thu Nov 21 11:54:18 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 21 Nov 2024 11:54:18 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v3]
In-Reply-To: <V2OdkJPssdAYP20z_B9Sl6N2d5NtmxMy2Fe5QqAKCR8=.f89ba298-8660-4112-a2ef-88201a8e927b@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <V2OdkJPssdAYP20z_B9Sl6N2d5NtmxMy2Fe5QqAKCR8=.f89ba298-8660-4112-a2ef-88201a8e927b@github.com>
Message-ID: <_UZfXCkvLXjIqY7uVzpdLC6IFH-Jz3F7QNbtBgIaUFw=.80ef87ef-c534-4b57-b4bb-affaee6cdc30@github.com>

On Thu, 21 Nov 2024 11:15:55 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename setup_compact_headers

Looks good.

I would prefer though if ObjLayout::initialize moved up to above adjust_heap_sizes_for_dumping, so that adjust_heap_sizes_for_dumping and initialize_heap_sizes continue to be kept together.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2451141037

From rkennke at openjdk.org  Thu Nov 21 12:44:58 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 21 Nov 2024 12:44:58 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v4]
In-Reply-To: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
Message-ID: <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>

> From the bug description:
> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
> 
> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
> 
> The fix to both issues is:
> - First disable UCOH
> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
> - then enable any flags required by compact headers
> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
> 
> Testing:
>  - [x] tier1 -UCOH (default)
>  - [ ] tier1 +UCOH
>  - [x] Manual testing several flag combinations

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Move ObjLayout::initialize() up a little

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22255/files
  - new: https://git.openjdk.org/jdk/pull/22255/files/a42d4900..43c7668e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=02-03

  Stats: 4 lines in 1 file changed: 2 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/22255.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22255/head:pull/22255

PR: https://git.openjdk.org/jdk/pull/22255

From shade at openjdk.org  Thu Nov 21 13:57:46 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Nov 2024 13:57:46 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
Message-ID: <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>

On Thu, 21 Nov 2024 00:59:27 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 524 commits:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
>    
>    Reviewed-by: kdnilsen
>  - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
>    
>    Reviewed-by: shade, ysr
>  - 8344592: GenShen: Remove unnecessary comments and changes
>    
>    Reviewed-by: kdnilsen
>  - 8344263: GenShen: Reduce extraneous log messages at INFO level
>    
>    Reviewed-by: ysr, shade
>  - 8344638: GenShen: Verifier should not touch claim token
>    
>    Reviewed-by: shade
>  - Merge
>  - 8344320: GenShen: Possible null pointer usage in shGenerationalHeap
>    
>    Reviewed-by: wkemper, ysr
>  - 8344321: GenShen: Fix various sonar scan warnings
>    
>    Reviewed-by: kdnilsen, shade
>  - 8344264: GenShen: Improve comments and method names
>    
>    Reviewed-by: shade
>  - ... and 514 more: https://git.openjdk.org/jdk/compare/95a00f8a...7ab16403

229/229 files read! This is an impressive, monumental piece of work, kudos. 

More comments from this read follow. I am going to circle back to some open threads in this PR. I am also running local tests, and in my ad-hoc performance tests generational mode performs admirably well.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 881:

> 879:   // update costs on slow path.
> 880:   monitoring_support()->notify_heap_changed();
> 881:   _heap_changed.set();

Why not leave `try_set` intact? This alleviates contention on the shared counter, AFAICS.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1972:

> 1970:   // Check that if concurrent weak root is set then active_gen isn't null
> 1971:   assert(!is_concurrent_weak_root_in_progress() || active_generation() != nullptr, "Error");
> 1972:   shenandoah_assert_generations_reconciled();

Why all of this is checked here? I would have thought `gc_state` machinery should only check things related to gc-state.

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 134:

> 132:   bool is_thread_safe() override { return true; }
> 133: };
> 134: 

This looks like belonging in one of the `*closures.hpp` header.

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 514:

> 512: 
> 513: public:
> 514:   ShenandoahController*   control_thread() { return _control_thread; }

This method and field is probably `controller` then, right?

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 663:

> 661: // ---------- CDS archive support
> 662: 
> 663:   bool can_load_archived_objects() const override { return !ShenandoahCardBarrier; }

This means CDS heap loading is not yet working with generational Shenandoah? This looks OK for the experimental code. Please submit a bug for this and assign it to me. I will take a look at it some time later.

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 754:

> 752:   oop try_evacuate_object(oop src, Thread* thread, ShenandoahHeapRegion* from_region, ShenandoahAffiliation target_gen);
> 753: public:
> 754: 

I think this new line should be before `public:`

src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.cpp line 64:

> 62:     }
> 63:   }
> 64:   return _mark_bit_map.is_bitmap_clear_range(start, end);

Comprehension check: this seems to bail out the moment it discovers an uncommitted slice. Does it really happen? More worryingly, if there is a mix of committed and uncommitted chunks, this method returns `true`, even if committed chunks are not actually clean?

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2450709063
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852054187
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852075850
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1851646208
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1851763351
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1851761345
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1851765188
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852139996

From wkemper at openjdk.org  Thu Nov 21 14:24:53 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 14:24:53 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <LjuLZWkU5S7aRlYwd5wpdMY-iD_XFRSIISKfCyuCaNs=.41ae42ab-dcac-4528-8a36-6c9cc99b8a2a@github.com>

Merges tag jdk-21.0.6+4

-------------

Commit messages:
 - 8343923: GHA: Switch to Xcode 15 on MacOS AArch64 runners
 - 8338389: [JFR] Long strings should be added to the string pool
 - 8342823: Ubsan: ciEnv.cpp:1614:65: runtime error: member call on null pointer of type 'struct CompileTask'
 - 8338449: ubsan: division by zero in sharedRuntimeTrans.cpp
 - 8336911: ZGC: Division by zero in heuristics after JDK-8332717
 - 8344164: [s390x] ProblemList hotspot/jtreg/runtime/NMT/VirtualAllocCommitMerge.java
 - 8343724: [PPC64] Disallow OptoScheduling
 - 8342607: Enhance register printing on x86_64 platforms
 - 8343506: [s390x] multiple test failures with ubsan
 - 8342183: Update tests to use stronger algorithms and keys
 - ... and 10 more: https://git.openjdk.org/shenandoah-jdk21u/compare/9bdf2a61...4b1365ea

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/138/files
  Stats: 3648 lines in 121 files changed: 1674 ins; 439 del; 1535 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/138.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/138/head:pull/138

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/138

From stefank at openjdk.org  Thu Nov 21 15:04:14 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 21 Nov 2024 15:04:14 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v4]
In-Reply-To: <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>
Message-ID: <Dm1KeB-OWzdu74y-2lZwDfdUGsPZekFFgu24Ft9u2Cw=.90e266f3-ef37-4763-af1c-37c958ee71b4@github.com>

On Thu, 21 Nov 2024 12:44:58 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Move ObjLayout::initialize() up a little

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2451761570

From gziemski at openjdk.org  Thu Nov 21 16:42:30 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Thu, 21 Nov 2024 16:42:30 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <jSPfSoOor2d_WfTRe8Zs8TZKmSfqxggbSmpP_P5_iOM=.fc738f7f-0425-4357-8479-fc08f3246841@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
 <jSPfSoOor2d_WfTRe8Zs8TZKmSfqxggbSmpP_P5_iOM=.fc738f7f-0425-4357-8479-fc08f3246841@github.com>
Message-ID: <yyggcPR1b37Gvt1re6RJrlMWLY5IknPI7nh4OpyolKQ=.c6846cc9-74b2-4d90-8269-a39457ca867c@github.com>

On Wed, 20 Nov 2024 09:58:02 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - avoid using mtNone
>>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>>  - revert, we will re-do with a smaller change
>>  - remove more mtNone
>>  - remove API that allows to change the mem_tag for virtual memory, feedback
>>  - do not allow default parameter for mtNone
>
> src/hotspot/os/bsd/gc/z/zPhysicalMemoryBacking_bsd.cpp line 1:
> 
>> 1: /*
> 
> Copyright year to be updated.

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1852490749

From shade at openjdk.org  Thu Nov 21 17:02:37 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Nov 2024 17:02:37 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <AKEbtx-2XVnBn851w_lG7X1hH2Za0LykrXP10h1_AsM=.9871d6f9-6d58-4b9d-87af-3f03781147a6@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
 <a-7kRBhDFfXqQRwhDv4fwDPoSX-2x4QdBKMMFyOnM0Q=.5bafe80e-77e6-4654-a59a-1308c05b20bd@github.com>
 <AKEbtx-2XVnBn851w_lG7X1hH2Za0LykrXP10h1_AsM=.9871d6f9-6d58-4b9d-87af-3f03781147a6@github.com>
Message-ID: <J60mFlVFP2mykH8TTtjDyQjiuazOV72N1yHTHyyq9Oc=.cca1c3f4-d255-4e6d-bf9a-035374b28180@github.com>

On Sat, 16 Nov 2024 17:54:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> I think it's here for the same reason `propagate_gc_state_to_java_threads` is here. Having it here makes it easier to see that this critical synchronization step happens for every safepoint.
>
> I'm inclined to leave as is, not wanting to disrupt stability of code that is known to be working.  But if there is strong preference otherwise, we can make a change...

OK, but file a bug for it to clean up.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852524191

From shade at openjdk.org  Thu Nov 21 17:05:44 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Nov 2024 17:05:44 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <yY__QZ-cqMGOphPPU6JkdLm6oNFPDhcP6jYlnK7OSz0=.8759b8ec-fe3d-4b98-ac61-04cd5f9fab20@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
 <yY__QZ-cqMGOphPPU6JkdLm6oNFPDhcP6jYlnK7OSz0=.8759b8ec-fe3d-4b98-ac61-04cd5f9fab20@github.com>
Message-ID: <8p63u9MBsujLClN-XXtWMs5OOIUKV4E-foooFIqdyYA=.442aee6c-8733-488e-a050-1c2459d30631@github.com>

On Tue, 19 Nov 2024 19:54:45 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 276:
>> 
>>> 274:     log_info(gc)("GC cancellation took %.3fs", cancel_time);
>>> 275:     _cancel_requested_time = 0;
>>> 276:   }
>> 
>> Do we need this? Is this useful?
>
> We could log this at debug? I believe it is useful because this time (currently) is not accounted for in the heuristic's notion of 'cycle time'.

Yeah, `gc=debug` would be okay for this.

>> src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.cpp line 193:
>> 
>>> 191:     Phase phase = Phase(pi);
>>> 192:     if (is_worker_phase(phase)) {
>>> 193:       double sum = uninitialized();
>> 
>> I think you can avoid changing this method to limit the churn.
>
> Hmm, I find the version in the PR much more readable. How about we upstream these changes separately?

Yes, let's do that, if you have cycles.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852533117
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852533548

From gziemski at openjdk.org  Thu Nov 21 17:22:55 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Thu, 21 Nov 2024 17:22:55 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v3]
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <HOKKzy-cmfiajUaUpqMnlQQaEl__pX0i6oOkUelV3GE=.38aae647-c058-40ac-8f43-202a4c1b622d@github.com>

> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by avoiding using `mtNone`.
> 
> This fix only addresses the cases covered by the issue. I am not trying to replace every single `mtNone` here, but eventually the goal would be to do just that.
> 
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

Gerard Ziemski has updated the pull request incrementally with one additional commit since the last revision:

  copyright update

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21843/files
  - new: https://git.openjdk.org/jdk/pull/21843/files/33dbafdc..62a98865

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21843&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21843&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21843.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21843/head:pull/21843

PR: https://git.openjdk.org/jdk/pull/21843

From gziemski at openjdk.org  Thu Nov 21 18:04:40 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Thu, 21 Nov 2024 18:04:40 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <7UxSMg_or6Avy_wuFlkkuDO0pBpyR5DiKeuY0N4uhVs=.d2c7835f-6e68-4f87-857a-e4fd6eb5f12b@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
 <7UxSMg_or6Avy_wuFlkkuDO0pBpyR5DiKeuY0N4uhVs=.d2c7835f-6e68-4f87-857a-e4fd6eb5f12b@github.com>
Message-ID: <YmsFwtyhk8UvkRMkRx59bTUeJK2q1LcyzCb6F9pgZZk=.02f1c4cf-b235-4ddd-8245-b47b42da352d@github.com>

On Wed, 20 Nov 2024 15:34:42 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - avoid using mtNone
>>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>>  - revert, we will re-do with a smaller change
>>  - remove more mtNone
>>  - remove API that allows to change the mem_tag for virtual memory, feedback
>>  - do not allow default parameter for mtNone
>
> src/hotspot/share/memory/virtualspace.hpp line 74:
> 
>> 72:   // Initialize the reserved space with the given size. Depending on the size
>> 73:   // a suitable page size and alignment will be used.
>> 74:   explicit ReservedSpace(size_t size, MemTag mem_tag);
> 
> Suggestion:
> 
>   ReservedSpace(size_t size, MemTag mem_tag);
> 
> No need for `explicit` when you've added a parameter.

Fixed.

> src/hotspot/share/runtime/safepointMechanism.cpp line 61:
> 
>> 59:     const size_t page_size = os::vm_page_size();
>> 60:     const size_t allocation_size = 2 * page_size;
>> 61:     char* polling_page = os::reserve_memory(allocation_size, false, mtSafepoint);
> 
> Suggestion:
> 
>     char* polling_page = os::reserve_memory(allocation_size, !ExecMem, mtSafepoint);

Fixed.

> src/hotspot/share/utilities/debug.cpp line 712:
> 
>> 710: 
>> 711: void initialize_assert_poison() {
>> 712:   char* page = os::reserve_memory(os::vm_page_size(), false, mtInternal);
> 
> Suggestion:
> 
>   char* page = os::reserve_memory(os::vm_page_size(), !ExecMem, mtTest);

Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1852621576
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1852622931
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1852622511

From wkemper at openjdk.org  Thu Nov 21 18:12:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 18:12:45 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
Message-ID: <jUc37YU3fT64MIcY7tlO1W5KSukJCpts_SRxzf7uG-s=.46525617-de1e-422c-b4f8-099731bb5cfc@github.com>

On Thu, 21 Nov 2024 10:15:27 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 524 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
>>    
>>    Reviewed-by: shade, ysr
>>  - 8344592: GenShen: Remove unnecessary comments and changes
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344263: GenShen: Reduce extraneous log messages at INFO level
>>    
>>    Reviewed-by: ysr, shade
>>  - 8344638: GenShen: Verifier should not touch claim token
>>    
>>    Reviewed-by: shade
>>  - Merge
>>  - 8344320: GenShen: Possible null pointer usage in shGenerationalHeap
>>    
>>    Reviewed-by: wkemper, ysr
>>  - 8344321: GenShen: Fix various sonar scan warnings
>>    
>>    Reviewed-by: kdnilsen, shade
>>  - 8344264: GenShen: Improve comments and method names
>>    
>>    Reviewed-by: shade
>>  - ... and 514 more: https://git.openjdk.org/jdk/compare/95a00f8a...7ab16403
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 663:
> 
>> 661: // ---------- CDS archive support
>> 662: 
>> 663:   bool can_load_archived_objects() const override { return !ShenandoahCardBarrier; }
> 
> This means CDS heap loading is not yet working with generational Shenandoah? This looks OK for the experimental code. Please submit a bug for this and assign it to me. I will take a look at it some time later.

Yes, we have: https://bugs.openjdk.org/browse/JDK-8339182

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852633232

From gziemski at openjdk.org  Thu Nov 21 18:13:36 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Thu, 21 Nov 2024 18:13:36 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v2]
In-Reply-To: <7UxSMg_or6Avy_wuFlkkuDO0pBpyR5DiKeuY0N4uhVs=.d2c7835f-6e68-4f87-857a-e4fd6eb5f12b@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <6Up1fGCFJl2HAUGl6IBKEDRodItQLn5TigHJEdzZpbU=.5f5ae17c-edc4-4dd8-ae09-8e8ae8567354@github.com>
 <7UxSMg_or6Avy_wuFlkkuDO0pBpyR5DiKeuY0N4uhVs=.d2c7835f-6e68-4f87-857a-e4fd6eb5f12b@github.com>
Message-ID: <5_ysL6UzUTxpXuDqgrEIsxGZbc5LzbOS8HACNE28LfA=.3ebd5b9d-ca74-4d32-a657-ee64739b0103@github.com>

On Wed, 20 Nov 2024 15:29:45 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Gerard Ziemski has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - avoid using mtNone
>>  - Merge remote-tracking branch 'upstream/master' into JDK-8328944
>>  - revert, we will re-do with a smaller change
>>  - remove more mtNone
>>  - remove API that allows to change the mem_tag for virtual memory, feedback
>>  - do not allow default parameter for mtNone
>
> src/hotspot/os/linux/os_linux.cpp line 4587:
> 
>> 4585:   char* hint = (char*)(os::Linux::initial_thread_stack_bottom() -
>> 4586:                        (StackOverflow::stack_guard_zone_size() + page_size));
>> 4587:   char* codebuf = os::attempt_reserve_memory_at(hint, page_size, false, mtInternal);
> 
> Could this be mtThread instead?

Makes sense, fixed.

> src/hotspot/os/linux/os_linux.cpp line 4595:
> 
>> 4593:     // map our codebuf, try again at an address one megabyte lower.
>> 4594:     hint -= 1 * M;
>> 4595:     codebuf = os::attempt_reserve_memory_at(hint, page_size, false, mtInternal);
> 
> Could this be mtThread instead?

Makes sense, fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1852634508
PR Review Comment: https://git.openjdk.org/jdk/pull/21843#discussion_r1852634648

From wkemper at openjdk.org  Thu Nov 21 18:15:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 18:15:42 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
Message-ID: <41m15WV5CgJ9hVyFzomp3dsrutwUTkoWsTu95JuYUiI=.6fd92984-1d04-4c78-858b-c16a98fcc382@github.com>

On Thu, 21 Nov 2024 10:16:16 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 524 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
>>    
>>    Reviewed-by: shade, ysr
>>  - 8344592: GenShen: Remove unnecessary comments and changes
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344263: GenShen: Reduce extraneous log messages at INFO level
>>    
>>    Reviewed-by: ysr, shade
>>  - 8344638: GenShen: Verifier should not touch claim token
>>    
>>    Reviewed-by: shade
>>  - Merge
>>  - 8344320: GenShen: Possible null pointer usage in shGenerationalHeap
>>    
>>    Reviewed-by: wkemper, ysr
>>  - 8344321: GenShen: Fix various sonar scan warnings
>>    
>>    Reviewed-by: kdnilsen, shade
>>  - 8344264: GenShen: Improve comments and method names
>>    
>>    Reviewed-by: shade
>>  - ... and 514 more: https://git.openjdk.org/jdk/compare/95a00f8a...7ab16403
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 514:
> 
>> 512: 
>> 513: public:
>> 514:   ShenandoahController*   control_thread() { return _control_thread; }
> 
> This method and field is probably `controller` then, right?

Left the accessor and field with the same name to reduce churn. Clients of this method don't really need to change because in most (all?) cases, the public API of `ShenandoahControlThread` is declared in `ShenandoahController`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852636871

From shade at openjdk.org  Thu Nov 21 18:33:42 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Nov 2024 18:33:42 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <i-MvUaoe8n9apI8y-4hQ9ohIGHwSPaufWIkdMLyBdLM=.3bcf0c60-9b68-4552-b425-45a8ff946664@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
 <i-MvUaoe8n9apI8y-4hQ9ohIGHwSPaufWIkdMLyBdLM=.3bcf0c60-9b68-4552-b425-45a8ff946664@github.com>
Message-ID: <idpksYdcncdiU3JIH_SR7gPS0tyz-dNp_4uRYHP33tE=.328cd779-dda1-4e4d-a60f-63e6500d94d8@github.com>

On Wed, 20 Nov 2024 00:13:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 420:
>> 
>>> 418:   }
>>> 419:   int gc_state = _heap->gc_state();
>>> 420:   if ((gc_state & ShenandoahHeap::YOUNG_MARKING) != 0) {
>> 
>> It is not very clear this works fine with single-gen mode. Does `MARKING` imply `YOUNG_MARKING` in single-gen mode? I think we better make it abundantly clear with `is_generational()` checks.
>
> Yes, in single gen mode we set `MARKING` and `YOUNG_MARKING` together. See https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahGlobalGeneration.cpp#L81

We should really either predicate it on generational mode (we already poll it a few lines below) and check for `MARKING` specifically, or assert it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852659199

From wkemper at openjdk.org  Thu Nov 21 19:14:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 19:14:39 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
Message-ID: <emb6wjdHu5VEzvGrWp3hIt0yu_glDWUFhTwikTBa9hI=.dfa328ab-bc3b-4dca-94c6-77658dcfd943@github.com>

On Thu, 21 Nov 2024 09:14:57 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 524 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
>>    
>>    Reviewed-by: shade, ysr
>>  - 8344592: GenShen: Remove unnecessary comments and changes
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344263: GenShen: Reduce extraneous log messages at INFO level
>>    
>>    Reviewed-by: ysr, shade
>>  - 8344638: GenShen: Verifier should not touch claim token
>>    
>>    Reviewed-by: shade
>>  - Merge
>>  - 8344320: GenShen: Possible null pointer usage in shGenerationalHeap
>>    
>>    Reviewed-by: wkemper, ysr
>>  - 8344321: GenShen: Fix various sonar scan warnings
>>    
>>    Reviewed-by: kdnilsen, shade
>>  - 8344264: GenShen: Improve comments and method names
>>    
>>    Reviewed-by: shade
>>  - ... and 514 more: https://git.openjdk.org/jdk/compare/95a00f8a...7ab16403
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 134:
> 
>> 132:   bool is_thread_safe() override { return true; }
>> 133: };
>> 134: 
> 
> This looks like belonging in one of the `*closures.hpp` header.

https://bugs.openjdk.org/browse/JDK-8344779

> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 754:
> 
>> 752:   oop try_evacuate_object(oop src, Thread* thread, ShenandoahHeapRegion* from_region, ShenandoahAffiliation target_gen);
>> 753: public:
>> 754: 
> 
> I think this new line should be before `public:`

Sneaking this into: https://bugs.openjdk.org/browse/JDK-8344779

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852709552
PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852711083

From wkemper at openjdk.org  Thu Nov 21 19:14:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 19:14:40 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <gBRZJiSA8NPZMAVo5Hl8rld6r3b90934rhufvajDQgY=.edd5ab35-170a-4afc-a1cb-1f6d7ea29783@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <zU-DeWcD1mFlROkHshzqgKnvfC1CqVJUygEh2P2TNJ4=.473860af-d8a6-4dd7-80a0-5eaba70daa09@github.com>
 <gBRZJiSA8NPZMAVo5Hl8rld6r3b90934rhufvajDQgY=.edd5ab35-170a-4afc-a1cb-1f6d7ea29783@github.com>
Message-ID: <fCps2wKxWeOzYQhuSx0QMSryRBicwwXzmG5cTJL9Smg=.233f0ac2-97e8-4073-be84-5bc6899bef19@github.com>

On Tue, 19 Nov 2024 23:17:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahMarkClosures.hpp line 34:
>> 
>>> 32: class ShenandoahHeapRegion;
>>> 33: 
>>> 34: class ShenandoahFinalMarkUpdateRegionStateClosure : public ShenandoahHeapRegionClosure {
>> 
>> There is the `shenandoahHeapRegionClosures.hpp` for these, no?
>
> It wasn't the intention that `shenandoahHeapRegionClosures.hpp` should become home to all of the `*HeapRegionClosures`. `ShenandoahFinalMarkUpdateRegionStateClosure` is shared in a couple of implementation files, but not the same as the ones using `shenandoahHeapRegionClosures.hpp`. I think `ShenandoahUpdateCensusZeroCohortClosure` should be moved into the only implementation file that uses it.

https://bugs.openjdk.org/browse/JDK-8344779

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852709824

From wkemper at openjdk.org  Thu Nov 21 19:30:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 19:30:03 GMT
Subject: RFR: 8344779: GenShen: Consolidate shared heap region closures into
 one header
Message-ID: <IWSLnyraAkVWjIudIQZjUhq0l2qHfqpSxlQLJBHoI-w=.60b32529-db5c-414a-b3e3-9490028a3ee7@github.com>

Move shared heap region closure implementations into one header, move non-shared implementations closer to their only usage.

-------------

Commit messages:
 - Consolidate shared heap region closures into one header

Changes: https://git.openjdk.org/shenandoah/pull/549/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=549&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344779
  Stats: 351 lines in 9 files changed: 152 ins; 195 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/549.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/549/head:pull/549

PR: https://git.openjdk.org/shenandoah/pull/549

From shade at openjdk.org  Thu Nov 21 19:33:42 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Nov 2024 19:33:42 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <41m15WV5CgJ9hVyFzomp3dsrutwUTkoWsTu95JuYUiI=.6fd92984-1d04-4c78-858b-c16a98fcc382@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
 <41m15WV5CgJ9hVyFzomp3dsrutwUTkoWsTu95JuYUiI=.6fd92984-1d04-4c78-858b-c16a98fcc382@github.com>
Message-ID: <87AIGvZXmJftw8Sj68WTINanxXFtPVb7ITiRQLv1LsU=.68baafe0-87ce-4e4c-b02e-63ac460d7434@github.com>

On Thu, 21 Nov 2024 18:12:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 514:
>> 
>>> 512: 
>>> 513: public:
>>> 514:   ShenandoahController*   control_thread() { return _control_thread; }
>> 
>> This method and field is probably `controller` then, right?
>
> Left the accessor and field with the same name to reduce churn. Clients of this method don't really need to change because in most (all?) cases, the public API of `ShenandoahControlThread` is declared in `ShenandoahController`.

OK.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852731729

From shade at openjdk.org  Thu Nov 21 19:43:19 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 21 Nov 2024 19:43:19 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v4]
In-Reply-To: <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>
Message-ID: <o8kZv3n8sZOmx_FMj8uLufs42CZuoHorgSElQ7TsNX0=.53dae6c2-d365-4e18-8336-49e1f0bd9b92@github.com>

On Thu, 21 Nov 2024 12:44:58 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Move ObjLayout::initialize() up a little

OK, this makes sense. But it is hairy, so maybe we need someone else to take a look.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2452465992

From wkemper at openjdk.org  Thu Nov 21 20:13:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 20:13:44 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
Message-ID: <Q818Jvivypq7WqmKgq4eJ9Cj4-y4FxN8GHMzEd-jgpg=.66236db5-eae3-45a2-a69f-269efd9e73bd@github.com>

On Thu, 21 Nov 2024 13:50:42 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 524 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
>>    
>>    Reviewed-by: shade, ysr
>>  - 8344592: GenShen: Remove unnecessary comments and changes
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344263: GenShen: Reduce extraneous log messages at INFO level
>>    
>>    Reviewed-by: ysr, shade
>>  - 8344638: GenShen: Verifier should not touch claim token
>>    
>>    Reviewed-by: shade
>>  - Merge
>>  - 8344320: GenShen: Possible null pointer usage in shGenerationalHeap
>>    
>>    Reviewed-by: wkemper, ysr
>>  - 8344321: GenShen: Fix various sonar scan warnings
>>    
>>    Reviewed-by: kdnilsen, shade
>>  - 8344264: GenShen: Improve comments and method names
>>    
>>    Reviewed-by: shade
>>  - ... and 514 more: https://git.openjdk.org/jdk/compare/95a00f8a...7ab16403
>
> src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.cpp line 64:
> 
>> 62:     }
>> 63:   }
>> 64:   return _mark_bit_map.is_bitmap_clear_range(start, end);
> 
> Comprehension check: this seems to bail out the moment it discovers an uncommitted slice. Does it really happen? More worryingly, if there is a mix of committed and uncommitted chunks, this method returns `true`, even if committed chunks are not actually clean?

In all (3) usages of this method, the `start` and `end` are within the same region, so it isn't really iterating across regions. I'll rewrite it to handle this case more clearly.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852802615

From ysr at openjdk.org  Thu Nov 21 20:23:40 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 21 Nov 2024 20:23:40 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
Message-ID: <Jzr7Xmrznds1YBhn3hGO-cdeliiZPZdXcd_DRVCvFEk=.537b5312-76b2-4371-aad7-d2e5b6a21d0c@github.com>

On Thu, 21 Nov 2024 13:34:51 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 524 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
>>    
>>    Reviewed-by: shade, ysr
>>  - 8344592: GenShen: Remove unnecessary comments and changes
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344263: GenShen: Reduce extraneous log messages at INFO level
>>    
>>    Reviewed-by: ysr, shade
>>  - 8344638: GenShen: Verifier should not touch claim token
>>    
>>    Reviewed-by: shade
>>  - Merge
>>  - 8344320: GenShen: Possible null pointer usage in shGenerationalHeap
>>    
>>    Reviewed-by: wkemper, ysr
>>  - 8344321: GenShen: Fix various sonar scan warnings
>>    
>>    Reviewed-by: kdnilsen, shade
>>  - 8344264: GenShen: Improve comments and method names
>>    
>>    Reviewed-by: shade
>>  - ... and 514 more: https://git.openjdk.org/jdk/compare/95a00f8a...7ab16403
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1972:
> 
>> 1970:   // Check that if concurrent weak root is set then active_gen isn't null
>> 1971:   assert(!is_concurrent_weak_root_in_progress() || active_generation() != nullptr, "Error");
>> 1972:   shenandoah_assert_generations_reconciled();
> 
> Why all of this is checked here? I would have thought `gc_state` machinery should only check things related to gc-state.

Likely debugging detritus that I unintentionally left behind. Feel free to delete.
I think the `shenandoah_assert_generations_reconciled()` is already checked in a most spots where it's needed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852820087

From gziemski at openjdk.org  Thu Nov 21 21:09:33 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Thu, 21 Nov 2024 21:09:33 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v4]
In-Reply-To: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
Message-ID: <WE3g3ujzzV9COo98K9kSBuhjku1qQBsh7Lp3ajsZsDg=.53795dab-550d-47b2-96b1-55cdfe82d111@github.com>

> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
> 
> We address this, by avoiding using `mtNone`.
> 
> This fix only addresses the cases covered by the issue. I am not trying to replace every single `mtNone` here, but eventually the goal would be to do just that.
> 
> TODO: update copyrights.
> 
> Testing: undergoing MARCH5 tier1-5 ...

Gerard Ziemski has updated the pull request incrementally with one additional commit since the last revision:

  Stefan's feedback

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21843/files
  - new: https://git.openjdk.org/jdk/pull/21843/files/62a98865..42793a9a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21843&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21843&range=02-03

  Stats: 16 lines in 7 files changed: 0 ins; 0 del; 16 mod
  Patch: https://git.openjdk.org/jdk/pull/21843.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21843/head:pull/21843

PR: https://git.openjdk.org/jdk/pull/21843

From wkemper at openjdk.org  Thu Nov 21 21:50:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 21:50:39 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
Message-ID: <JGKNZfGKL5EKPI3A5dEsfD4nHhrqjFIK2PePbdmu2gk=.607eb5ad-1123-469a-80dc-d851a132e560@github.com>

On Thu, 21 Nov 2024 13:22:42 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 524 commits:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>>  - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
>>    
>>    Reviewed-by: shade, ysr
>>  - 8344592: GenShen: Remove unnecessary comments and changes
>>    
>>    Reviewed-by: kdnilsen
>>  - 8344263: GenShen: Reduce extraneous log messages at INFO level
>>    
>>    Reviewed-by: ysr, shade
>>  - 8344638: GenShen: Verifier should not touch claim token
>>    
>>    Reviewed-by: shade
>>  - Merge
>>  - 8344320: GenShen: Possible null pointer usage in shGenerationalHeap
>>    
>>    Reviewed-by: wkemper, ysr
>>  - 8344321: GenShen: Fix various sonar scan warnings
>>    
>>    Reviewed-by: kdnilsen, shade
>>  - 8344264: GenShen: Improve comments and method names
>>    
>>    Reviewed-by: shade
>>  - ... and 514 more: https://git.openjdk.org/jdk/compare/95a00f8a...7ab16403
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 881:
> 
>> 879:   // update costs on slow path.
>> 880:   monitoring_support()->notify_heap_changed();
>> 881:   _heap_changed.set();
> 
> Why not leave `try_set` intact? This alleviates contention on the shared counter, AFAICS.

https://bugs.openjdk.org/browse/JDK-8344797

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852952949

From wkemper at openjdk.org  Thu Nov 21 21:50:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 21:50:39 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <Jzr7Xmrznds1YBhn3hGO-cdeliiZPZdXcd_DRVCvFEk=.537b5312-76b2-4371-aad7-d2e5b6a21d0c@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
 <Jzr7Xmrznds1YBhn3hGO-cdeliiZPZdXcd_DRVCvFEk=.537b5312-76b2-4371-aad7-d2e5b6a21d0c@github.com>
Message-ID: <ZWesnM5XEPjZNcAf83Z-1x6y8g3gDNWMbpUokED1mTc=.962f432f-138a-457d-8a60-bdd66d6332f8@github.com>

On Thu, 21 Nov 2024 20:20:12 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1972:
>> 
>>> 1970:   // Check that if concurrent weak root is set then active_gen isn't null
>>> 1971:   assert(!is_concurrent_weak_root_in_progress() || active_generation() != nullptr, "Error");
>>> 1972:   shenandoah_assert_generations_reconciled();
>> 
>> Why all of this is checked here? I would have thought `gc_state` machinery should only check things related to gc-state.
>
> Likely debugging detritus that I unintentionally left behind. Feel free to delete.
> I think the `shenandoah_assert_generations_reconciled()` is already checked in a most spots where it's needed.

https://bugs.openjdk.org/browse/JDK-8344797

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852949534

From wkemper at openjdk.org  Thu Nov 21 21:50:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 21:50:40 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v8]
In-Reply-To: <Q818Jvivypq7WqmKgq4eJ9Cj4-y4FxN8GHMzEd-jgpg=.66236db5-eae3-45a2-a69f-269efd9e73bd@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <myfaIiznLjLfHaS5W9PAbo2y5cVdcyjwjwPD8pcb4r8=.8c409cfd-12ab-439b-92f3-93969da21999@github.com>
 <5hEAJIG9hrrYUlCc2bSeh67dMfE56wt8JcFfzdpF8aw=.4f6a03e8-f5fe-4a91-9003-e04c3abbec0f@github.com>
 <Q818Jvivypq7WqmKgq4eJ9Cj4-y4FxN8GHMzEd-jgpg=.66236db5-eae3-45a2-a69f-269efd9e73bd@github.com>
Message-ID: <ySEZ25fDfpFAX9h1Bm5SrNswKwfn0L9rmjfG4mACS2s=.e0300d23-6929-4f55-9594-32902848264d@github.com>

On Thu, 21 Nov 2024 20:10:20 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.cpp line 64:
>> 
>>> 62:     }
>>> 63:   }
>>> 64:   return _mark_bit_map.is_bitmap_clear_range(start, end);
>> 
>> Comprehension check: this seems to bail out the moment it discovers an uncommitted slice. Does it really happen? More worryingly, if there is a mix of committed and uncommitted chunks, this method returns `true`, even if committed chunks are not actually clean?
>
> In all (3) usages of this method, the `start` and `end` are within the same region, so it isn't really iterating across regions. I'll rewrite it to handle this case more clearly.

https://bugs.openjdk.org/browse/JDK-8344797

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852949662

From wkemper at openjdk.org  Thu Nov 21 21:54:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 21:54:13 GMT
Subject: RFR: 8344797: GenShen: Update and rename confusing method
Message-ID: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>

A few riders here:
* [Remove vestigial debugging assertions](https://github.com/openjdk/shenandoah/commit/9062113458bd1909409967abb743e11982f351ab)
* [Log cancellation time at debug level](https://github.com/openjdk/shenandoah/commit/e3dd23efcb179bc61565d25f6b46f3690a50caa4)
* [Use try_set instead of set](https://github.com/openjdk/shenandoah/pull/550/commits/49cb66a0fa7d0d660f5b986eea9d71035c157310)

-------------

Commit messages:
 - Use try_set instead of set
 - Update and rename confusing method to match intended usage
 - Log cancellation time at debug level
 - Remove vestigial debugging assertions

Changes: https://git.openjdk.org/shenandoah/pull/550/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=550&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344797
  Stats: 23 lines in 8 files changed: 3 ins; 4 del; 16 mod
  Patch: https://git.openjdk.org/shenandoah/pull/550.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/550/head:pull/550

PR: https://git.openjdk.org/shenandoah/pull/550

From wkemper at openjdk.org  Thu Nov 21 22:14:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 22:14:37 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <8p63u9MBsujLClN-XXtWMs5OOIUKV4E-foooFIqdyYA=.442aee6c-8733-488e-a050-1c2459d30631@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
 <yY__QZ-cqMGOphPPU6JkdLm6oNFPDhcP6jYlnK7OSz0=.8759b8ec-fe3d-4b98-ac61-04cd5f9fab20@github.com>
 <8p63u9MBsujLClN-XXtWMs5OOIUKV4E-foooFIqdyYA=.442aee6c-8733-488e-a050-1c2459d30631@github.com>
Message-ID: <FKUc88TO0WBr8rfsaPJSevTSA0MKAb53ICjuWBzvn8A=.d72667a7-70f5-4d1c-91ef-8f6d58a009cd@github.com>

On Thu, 21 Nov 2024 17:02:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Hmm, I find the version in the PR much more readable. How about we upstream these changes separately?
>
> Yes, let's do that, if you have cycles.

https://bugs.openjdk.org/browse/JDK-8344797

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1852976446

From wkemper at openjdk.org  Thu Nov 21 22:17:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 22:17:23 GMT
Subject: RFR: 8344798: Shenandoah: Use more descriptive variable names in
 shPhaseTimings.cpp
Message-ID: <ctsTiB5ScgGnTMO859N_j0ykm7wvwMzS0JDaCGeNd1U=.d5e34f3e-544f-4478-816c-0074feb30fef@github.com>

The single letter variable names make some of this code harder to read.

-------------

Commit messages:
 - Use more descriptive variable names

Changes: https://git.openjdk.org/jdk/pull/22310/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22310&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344798
  Stats: 14 lines in 1 file changed: 0 ins; 0 del; 14 mod
  Patch: https://git.openjdk.org/jdk/pull/22310.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22310/head:pull/22310

PR: https://git.openjdk.org/jdk/pull/22310

From wkemper at openjdk.org  Thu Nov 21 22:41:09 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 22:41:09 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <LjuLZWkU5S7aRlYwd5wpdMY-iD_XFRSIISKfCyuCaNs=.41ae42ab-dcac-4528-8a36-6c9cc99b8a2a@github.com>
References: <LjuLZWkU5S7aRlYwd5wpdMY-iD_XFRSIISKfCyuCaNs=.41ae42ab-dcac-4528-8a36-6c9cc99b8a2a@github.com>
Message-ID: <C-WHfqBbJXkRb600iaN3QNoxo6cMeTlCSmzLoAroIUY=.997de688-ab72-4a66-a8a8-c6c4fc0aaaeb@github.com>

> Merges tag jdk-21.0.6+4

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah-jdk21u/pull/138/files
  - new: https://git.openjdk.org/shenandoah-jdk21u/pull/138/files/4b1365ea..4b1365ea

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=138&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=138&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/138.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/138/head:pull/138

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/138

From wkemper at openjdk.org  Thu Nov 21 22:41:11 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 22:41:11 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <LjuLZWkU5S7aRlYwd5wpdMY-iD_XFRSIISKfCyuCaNs=.41ae42ab-dcac-4528-8a36-6c9cc99b8a2a@github.com>
References: <LjuLZWkU5S7aRlYwd5wpdMY-iD_XFRSIISKfCyuCaNs=.41ae42ab-dcac-4528-8a36-6c9cc99b8a2a@github.com>
Message-ID: <mMine8KxboAFD2_n2dWIWWYP8UHWXZscnOYMhC090P4=.98921d19-6808-4182-9c2a-0aea2a251157@github.com>

On Thu, 21 Nov 2024 14:20:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-21.0.6+4

This pull request has now been integrated.

Changeset: ab8e8c34
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah-jdk21u/commit/ab8e8c34ff78b6fc400aff6ed5612ecf89675c81
Stats:     3648 lines in 121 files changed: 1674 ins; 439 del; 1535 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/138

From ysr at openjdk.org  Thu Nov 21 23:24:19 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 21 Nov 2024 23:24:19 GMT
Subject: RFR: 8344798: Shenandoah: Use more descriptive variable names in
 shPhaseTimings.cpp
In-Reply-To: <ctsTiB5ScgGnTMO859N_j0ykm7wvwMzS0JDaCGeNd1U=.d5e34f3e-544f-4478-816c-0074feb30fef@github.com>
References: <ctsTiB5ScgGnTMO859N_j0ykm7wvwMzS0JDaCGeNd1U=.d5e34f3e-544f-4478-816c-0074feb30fef@github.com>
Message-ID: <THeiNcurYb2PHQDURVMRM8FUnXli9pVscRetZ_iuS2s=.a1e62312-9ee9-4ab5-bd23-d478baec9440@github.com>

On Thu, 21 Nov 2024 22:12:40 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The single letter variable names make some of this code harder to read.

Marked as reviewed by ysr (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/22310#pullrequestreview-2452995376

From wkemper at openjdk.org  Thu Nov 21 23:28:19 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 21 Nov 2024 23:28:19 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v7]
In-Reply-To: <9MFkD3YNuSJIBPOsHYEKFX98XkMGKgMYa_7j-7usxC0=.0a99aabd-6272-40db-a5aa-6022023c1e4f@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
 <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>
 <2ctRsJ6JhReYC9yjcPV4eRljht3bMAZ9B3urifpvQXQ=.470aa336-f334-4345-863b-557250aa3416@github.com>
 <I0GzgbIQG1aGxmJ4c1wh0N5uYDDO-5fIrnEtT4t675s=.b21ca11b-1061-4bbe-97c3-79edccdabab4@github.com>
 <TygSXGP4yj46QlEcRv-m5AfZTb6rreI9SGdjbZpziiM=.fbe4a0be-b601-4219-bc88-40d27f1787f4@github.com>
 <9MFkD3YNuSJIBPOsHYEKFX98XkMGKgMYa_7j-7usxC0=.0a99aabd-6272-40db-a5aa-6022023c1e4f@github.com>
Message-ID: <rYJmY3cIclpM5IVaxSLmnWJvFKXmj1nfU9PBT31KSns=.9c62ca5e-9b87-4106-9d79-d7829733018b@github.com>

On Thu, 21 Nov 2024 08:17:02 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Yes, I spent some time trying to resurrect `ShenandoahConcurrentPhase` for uncommit here, but it really doesn't want to be used outside of a gc cycle. Also, previously it was logging heap _usage_, which isn't quite what we want here (this may actually increase during this phase, which makes it seem as though nothing is being uncommitted).
>> 
>> I've restored the original logging format, but instead of logging heap usage it is now logging heap committed before and after. Here is an excerpt from specjbb2015 with `-Xms5g -Xmx10g`:
>> 
>> 
>> [2024-11-20T20:02:25.056+0000][97.396s][22293][info][gc,start       ] Concurrent uncommit
>> [2024-11-20T20:02:25.072+0000][97.412s][22293][info][gc             ] Concurrent uncommit 5424M->5120M(5120M) 15.988ms
>> [2024-11-20T20:05:17.916+0000][270.255s][22293][info][gc,start       ] Concurrent uncommit
>> [2024-11-20T20:05:18.169+0000][270.508s][22293][info][gc             ] Concurrent uncommit 10240M->5120M(5120M) 253.048ms
>> [2024-11-20T20:06:45.329+0000][357.668s][22293][info][gc,start       ] Concurrent uncommit
>> [2024-11-20T20:06:45.596+0000][357.935s][22293][info][gc             ] Concurrent uncommit 10240M->5120M(5120M) 267.144ms
>> [2024-11-20T20:06:57.147+0000][369.486s][22293][info][gc,start       ] Concurrent uncommit
>> [2024-11-20T20:06:57.148+0000][369.487s][22293][info][gc             ] Concurrent uncommit 5456M->5440M(5440M) 1.189ms
>
> If we are emitting a log line that looks like a properly formatted GC log line, but the numbers there mean something else for `Concurrent uncommit`, we are bound to confuse users and automatic tools. Uncommit should affect `capacity`, this is how we know how deep we have uncommitted. So, I suggest we emit:
> 
> 
> Concurrent uncommit XXXXM->XXXXM (YYYYM) z.zzzms
> 
> 
> ...where `XXXX` is the heap used at the end of uncommit (note before and after are the same) and YYYY is capacity. This will not expose users to thinking uncommit grows the heap usage, and would give us instantaneous view on heap usage and capacity.

Hmm, the numbers are preceded by `Concurrent uncommit`, with that context it's not much of a stretch to think these numbers represent the change in _committed_ memory. The original log message (in which heap usage may increase during uncommit) was _not_ helpful. A message with the same format in which heap usage also appears to _not change at all_ during an uncommit is also perplexing. Are we trying too hard to preserve the original, not useful message? Maybe we just want a new message that plainly says: 


Concurrently uncommitted XXXXM in z.zzzms

or

Concurrent uncommit: time z.zzzms, committed before XXXXM, committed after YYYYM, capacity ZZZZM

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1853077616

From wkemper at openjdk.org  Fri Nov 22 00:08:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 00:08:23 GMT
Subject: Integrated: 8344798: Shenandoah: Use more descriptive variable names
 in shPhaseTimings.cpp
In-Reply-To: <ctsTiB5ScgGnTMO859N_j0ykm7wvwMzS0JDaCGeNd1U=.d5e34f3e-544f-4478-816c-0074feb30fef@github.com>
References: <ctsTiB5ScgGnTMO859N_j0ykm7wvwMzS0JDaCGeNd1U=.d5e34f3e-544f-4478-816c-0074feb30fef@github.com>
Message-ID: <ZoxoVaZAqNKJYun5xuJOys75o7FRaIc5_AHTYv0OFNA=.b1f39387-bf03-41d6-b453-da53be740aa5@github.com>

On Thu, 21 Nov 2024 22:12:40 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The single letter variable names make some of this code harder to read.

This pull request has now been integrated.

Changeset: db44e97c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/db44e97c5dfd286a58985be9b091fd43f5ad03be
Stats:     14 lines in 1 file changed: 0 ins; 0 del; 14 mod

8344798: Shenandoah: Use more descriptive variable names in shPhaseTimings.cpp

Reviewed-by: ysr

-------------

PR: https://git.openjdk.org/jdk/pull/22310

From kdnilsen at openjdk.org  Fri Nov 22 00:11:38 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 22 Nov 2024 00:11:38 GMT
Subject: RFR: 8344779: GenShen: Consolidate shared heap region closures
 into one header
In-Reply-To: <IWSLnyraAkVWjIudIQZjUhq0l2qHfqpSxlQLJBHoI-w=.60b32529-db5c-414a-b3e3-9490028a3ee7@github.com>
References: <IWSLnyraAkVWjIudIQZjUhq0l2qHfqpSxlQLJBHoI-w=.60b32529-db5c-414a-b3e3-9490028a3ee7@github.com>
Message-ID: <USwTzn1bcTvFqa2bQfVryilY_cqnsg0ssl2nASHST9s=.a1f00a79-0626-467e-b483-ac25fbc09355@github.com>

On Thu, 21 Nov 2024 19:25:27 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Move shared heap region closure implementations into one header, move non-shared implementations closer to their only usage.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/549#pullrequestreview-2453040529

From ysr at openjdk.org  Fri Nov 22 00:20:34 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 22 Nov 2024 00:20:34 GMT
Subject: RFR: 8344797: GenShen: Update and rename confusing method
In-Reply-To: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
References: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
Message-ID: <-y9CzdK1h2kHuDa2H4CrjHMpo07U7s1K1WVdO3bKEkg=.03f9c023-1322-45c4-b232-7c26a5ac2795@github.com>

On Thu, 21 Nov 2024 21:49:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

> A few riders here:
> * [Remove vestigial debugging assertions](https://github.com/openjdk/shenandoah/commit/9062113458bd1909409967abb743e11982f351ab)
> * [Log cancellation time at debug level](https://github.com/openjdk/shenandoah/commit/e3dd23efcb179bc61565d25f6b46f3690a50caa4)
> * [Use try_set instead of set](https://github.com/openjdk/shenandoah/pull/550/commits/49cb66a0fa7d0d660f5b986eea9d71035c157310)

LGTM (including riders :-)

src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.cpp line 61:

> 59:         return true;
> 60:       }
> 61:       start_idx++;

Thanks for fixing this and emphasizing that the test is ever only done within a single region.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/550#pullrequestreview-2453042394
PR Review Comment: https://git.openjdk.org/shenandoah/pull/550#discussion_r1853106837

From ysr at openjdk.org  Fri Nov 22 00:42:34 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 22 Nov 2024 00:42:34 GMT
Subject: RFR: 8344779: GenShen: Consolidate shared heap region closures
 into one header
In-Reply-To: <IWSLnyraAkVWjIudIQZjUhq0l2qHfqpSxlQLJBHoI-w=.60b32529-db5c-414a-b3e3-9490028a3ee7@github.com>
References: <IWSLnyraAkVWjIudIQZjUhq0l2qHfqpSxlQLJBHoI-w=.60b32529-db5c-414a-b3e3-9490028a3ee7@github.com>
Message-ID: <jZtsz-0FCZQ661CM0hrVPNcYvJn_73aYgJXArQHw9N4=.77f5e2a2-e1b8-4316-9231-1b6e3c2246d6@github.com>

On Thu, 21 Nov 2024 19:25:27 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Move shared heap region closure implementations into one header, move non-shared implementations closer to their only usage.

LGTM

src/hotspot/share/gc/shenandoah/shenandoahMarkClosures.cpp line 2:

> 1: /*
> 2:  * Copyright (c) 2021, Red Hat, Inc. All rights reserved.

This had me confused, but I verified that the deletion of the Copyright line is indeed correct based on what I can tell from `git annotate`.

Also the way the rename shows up in the diffs here is pretty subtle.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/549#pullrequestreview-2453066219
PR Review Comment: https://git.openjdk.org/shenandoah/pull/549#discussion_r1853122849

From kdnilsen at openjdk.org  Fri Nov 22 00:46:30 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 22 Nov 2024 00:46:30 GMT
Subject: RFR: 8344797: GenShen: Update and rename confusing method
In-Reply-To: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
References: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
Message-ID: <O1nGNR7pX7xY-S2MQmxkxcJGeT2qLvcrBrkgTIXGQ_c=.287ca91b-ab57-4738-90fc-0f9efea8cc46@github.com>

On Thu, 21 Nov 2024 21:49:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

> A few riders here:
> * [Remove vestigial debugging assertions](https://github.com/openjdk/shenandoah/commit/9062113458bd1909409967abb743e11982f351ab)
> * [Log cancellation time at debug level](https://github.com/openjdk/shenandoah/commit/e3dd23efcb179bc61565d25f6b46f3690a50caa4)
> * [Use try_set instead of set](https://github.com/openjdk/shenandoah/pull/550/commits/49cb66a0fa7d0d660f5b986eea9d71035c157310)

Marked as reviewed by kdnilsen (Committer).

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 881:

> 879:   // update costs on slow path.
> 880:   monitoring_support()->notify_heap_changed();
> 881:   _heap_changed.try_set();

I'm not sure I understand the significance of this change from .set() to .try_set().

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 274:

> 272:   if (_cancel_requested_time > 0) {
> 273:     double cancel_time = os::elapsedTime() - _cancel_requested_time;
> 274:     log_debug(gc)("GC cancellation took %.3fs", cancel_time);

may not be significant, but would prefer to avoid os::elapsedTime() invocation if we're not logging debug.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/550#pullrequestreview-2453043393
PR Review Comment: https://git.openjdk.org/shenandoah/pull/550#discussion_r1853124695
PR Review Comment: https://git.openjdk.org/shenandoah/pull/550#discussion_r1853107542

From kdnilsen at openjdk.org  Fri Nov 22 00:46:30 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 22 Nov 2024 00:46:30 GMT
Subject: RFR: 8344797: GenShen: Update and rename confusing method
In-Reply-To: <O1nGNR7pX7xY-S2MQmxkxcJGeT2qLvcrBrkgTIXGQ_c=.287ca91b-ab57-4738-90fc-0f9efea8cc46@github.com>
References: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
 <O1nGNR7pX7xY-S2MQmxkxcJGeT2qLvcrBrkgTIXGQ_c=.287ca91b-ab57-4738-90fc-0f9efea8cc46@github.com>
Message-ID: <C4B0SQ2L-8dGKVl0ejWa9uhYGt42w3gnzbdZJC6Bj-k=.0f788b51-e844-45ad-8719-07076a0fcaf9@github.com>

On Fri, 22 Nov 2024 00:41:26 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> A few riders here:
>> * [Remove vestigial debugging assertions](https://github.com/openjdk/shenandoah/commit/9062113458bd1909409967abb743e11982f351ab)
>> * [Log cancellation time at debug level](https://github.com/openjdk/shenandoah/commit/e3dd23efcb179bc61565d25f6b46f3690a50caa4)
>> * [Use try_set instead of set](https://github.com/openjdk/shenandoah/pull/550/commits/49cb66a0fa7d0d660f5b986eea9d71035c157310)
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 881:
> 
>> 879:   // update costs on slow path.
>> 880:   monitoring_support()->notify_heap_changed();
>> 881:   _heap_changed.try_set();
> 
> I'm not sure I understand the significance of this change from .set() to .try_set().

Would a comment help explain why it is ok for set() to fail here?

> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 274:
> 
>> 272:   if (_cancel_requested_time > 0) {
>> 273:     double cancel_time = os::elapsedTime() - _cancel_requested_time;
>> 274:     log_debug(gc)("GC cancellation took %.3fs", cancel_time);
> 
> may not be significant, but would prefer to avoid os::elapsedTime() invocation if we're not logging debug.

maybe write the argument to log_debug() as os::elapsedTime() - _cancel_requested_time.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/550#discussion_r1853125010
PR Review Comment: https://git.openjdk.org/shenandoah/pull/550#discussion_r1853125940

From stefank at openjdk.org  Fri Nov 22 11:22:19 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 22 Nov 2024 11:22:19 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v4]
In-Reply-To: <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>
Message-ID: <ILGpyi7zH4ePp3V7FPNGPyczcn3dZjpBXLd7jq_EoVA=.04a2061d-14c8-4e94-9013-f8264b0baff3@github.com>

On Thu, 21 Nov 2024 12:44:58 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Move ObjLayout::initialize() up a little

Ping hotspot-gc.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22255#issuecomment-2493521567

From wkemper at openjdk.org  Fri Nov 22 14:16:44 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 14:16:44 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <dnFM8zsxrQQfFGipeGUQ6vulQfibss0xpD5W6NjMjSU=.da917aa5-dc73-4b61-959f-f09bb8fa0982@github.com>

Merges tag jdk-24+25

-------------

Commit messages:
 - 8344526: RISC-V: implement -XX:+VerifyActivationFrameSize
 - 8340334: Update jcmd VM.events max parameter to be INT
 - 8344524: Remove SecurityManager related code from jdk.jlink module
 - 8344471: Remove SecurityManager related code from java.compiler module
 - 8344057: Remove doPrivileged calls from unix platform sources in the java.desktop module
 - 8344595: State transitions in internal VirtualThread comment needs to be updated
 - 8344064: Remove doPrivileged calls from print/imageio/media classes in the java.desktop module
 - 8344578: Clean up left over java.security and sun.security imports in Swing classes
 - 8289771: jpackage: ResourceEditor error when path is overly long on Windows
 - 8344562: Remove security manager dependency from module jdk.jdi
 - ... and 164 more: https://git.openjdk.org/shenandoah/compare/95a00f8a...4fbf2720

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/552/files
  Stats: 82439 lines in 1443 files changed: 37422 ins; 38857 del; 6160 mod
  Patch: https://git.openjdk.org/shenandoah/pull/552.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/552/head:pull/552

PR: https://git.openjdk.org/shenandoah/pull/552

From ayang at openjdk.org  Fri Nov 22 17:22:18 2024
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Fri, 22 Nov 2024 17:22:18 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v4]
In-Reply-To: <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <yMrvzA0HdErvHelVo0TP6jB8hFdCWXS2oWR6RPi-0-Y=.03943bfd-2689-49c5-b572-ee1f10909137@github.com>
Message-ID: <BjOHfRz17iTpqeXTtenvDh_dXVLpNwnv2Ijqv8b6vxI=.efbffebe-b10c-442c-a3a7-b9ae59d542a2@github.com>

On Thu, 21 Nov 2024 12:44:58 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Move ObjLayout::initialize() up a little

Marked as reviewed by ayang (Reviewer).

src/hotspot/share/runtime/arguments.cpp line 3660:

> 3658: }
> 3659: 
> 3660: void Arguments::set_compact_headers() {

Since this method does more than setting the compact-headers flag, could it be renamed to `set_compact_headers_flags`, following the convention from its neighbor `set_ergonomics_flags`?

-------------

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2455137935
PR Review Comment: https://git.openjdk.org/jdk/pull/22255#discussion_r1854326306

From wkemper at openjdk.org  Fri Nov 22 17:22:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 17:22:42 GMT
Subject: RFR: 8344797: GenShen: Update and rename confusing method
In-Reply-To: <C4B0SQ2L-8dGKVl0ejWa9uhYGt42w3gnzbdZJC6Bj-k=.0f788b51-e844-45ad-8719-07076a0fcaf9@github.com>
References: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
 <O1nGNR7pX7xY-S2MQmxkxcJGeT2qLvcrBrkgTIXGQ_c=.287ca91b-ab57-4738-90fc-0f9efea8cc46@github.com>
 <C4B0SQ2L-8dGKVl0ejWa9uhYGt42w3gnzbdZJC6Bj-k=.0f788b51-e844-45ad-8719-07076a0fcaf9@github.com>
Message-ID: <VDrD0IhggciRCAepY_eiTiOHuxMhGEbKQtfftM4Uuio=.ed292381-f3e5-4751-8ecb-f6db673b84fb@github.com>

On Fri, 22 Nov 2024 00:43:31 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 274:
>> 
>>> 272:   if (_cancel_requested_time > 0) {
>>> 273:     double cancel_time = os::elapsedTime() - _cancel_requested_time;
>>> 274:     log_debug(gc)("GC cancellation took %.3fs", cancel_time);
>> 
>> may not be significant, but would prefer to avoid os::elapsedTime() invocation if we're not logging debug.
>
> maybe write the argument to log_debug() as os::elapsedTime() - _cancel_requested_time.

That's a good catch and an easy change.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/550#discussion_r1854336421

From wkemper at openjdk.org  Fri Nov 22 17:33:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 17:33:20 GMT
Subject: RFR: 8344797: GenShen: Update and rename confusing method [v2]
In-Reply-To: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
References: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
Message-ID: <lTaH_QWQsBJJ7fyVY41sgJtgaKjFNAYdV63V4NP_IDQ=.48922a4b-6abc-489c-bcb5-3cb712cb6312@github.com>

> A few riders here:
> * [Remove vestigial debugging assertions](https://github.com/openjdk/shenandoah/commit/9062113458bd1909409967abb743e11982f351ab)
> * [Log cancellation time at debug level](https://github.com/openjdk/shenandoah/commit/e3dd23efcb179bc61565d25f6b46f3690a50caa4)
> * [Use try_set instead of set](https://github.com/openjdk/shenandoah/pull/550/commits/49cb66a0fa7d0d660f5b986eea9d71035c157310)

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Only get elapsedTime for debug log message when debug logging is enabled

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/550/files
  - new: https://git.openjdk.org/shenandoah/pull/550/files/49cb66a0..978283fa

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=550&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=550&range=00-01

  Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/550.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/550/head:pull/550

PR: https://git.openjdk.org/shenandoah/pull/550

From wkemper at openjdk.org  Fri Nov 22 17:33:21 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 17:33:21 GMT
Subject: RFR: 8344797: GenShen: Update and rename confusing method [v2]
In-Reply-To: <C4B0SQ2L-8dGKVl0ejWa9uhYGt42w3gnzbdZJC6Bj-k=.0f788b51-e844-45ad-8719-07076a0fcaf9@github.com>
References: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
 <O1nGNR7pX7xY-S2MQmxkxcJGeT2qLvcrBrkgTIXGQ_c=.287ca91b-ab57-4738-90fc-0f9efea8cc46@github.com>
 <C4B0SQ2L-8dGKVl0ejWa9uhYGt42w3gnzbdZJC6Bj-k=.0f788b51-e844-45ad-8719-07076a0fcaf9@github.com>
Message-ID: <thW4nmpC0Ym_RhOn8VS64NSWRKUVHuKvGIsI30cXGcQ=.97a679a0-7034-48b6-bf4b-bdafd8adcdd2@github.com>

On Fri, 22 Nov 2024 00:41:58 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 881:
>> 
>>> 879:   // update costs on slow path.
>>> 880:   monitoring_support()->notify_heap_changed();
>>> 881:   _heap_changed.try_set();
>> 
>> I'm not sure I understand the significance of this change from .set() to .try_set().
>
> Would a comment help explain why it is ok for set() to fail here?

It was an unnecessary change. Upstream has `try_set`. We just need one thread to set it. `try_set` avoid modifying the shared variable if it is already set.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/550#discussion_r1854349553

From wkemper at openjdk.org  Fri Nov 22 17:33:22 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 17:33:22 GMT
Subject: Integrated: 8344797: GenShen: Update and rename confusing method
In-Reply-To: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
References: <wXEmsPNB0708N85B0VmVtiJvEH_BtuN5DAwDqcu4LEw=.b7e0e8bc-b372-43a5-b21a-1261a9d4f55c@github.com>
Message-ID: <AuwyNd9DZ1WZuvuudL7S7aIVM0u-npLoPLiIM0waCMI=.22dcc0e5-fc62-4158-9370-376ffbb7350d@github.com>

On Thu, 21 Nov 2024 21:49:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

> A few riders here:
> * [Remove vestigial debugging assertions](https://github.com/openjdk/shenandoah/commit/9062113458bd1909409967abb743e11982f351ab)
> * [Log cancellation time at debug level](https://github.com/openjdk/shenandoah/commit/e3dd23efcb179bc61565d25f6b46f3690a50caa4)
> * [Use try_set instead of set](https://github.com/openjdk/shenandoah/pull/550/commits/49cb66a0fa7d0d660f5b986eea9d71035c157310)

This pull request has now been integrated.

Changeset: 9b7c86a5
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/9b7c86a528d13cdfcda2ea6cff6c29c251ec24b4
Stats:     24 lines in 8 files changed: 3 ins; 5 del; 16 mod

8344797: GenShen: Update and rename confusing method

Reviewed-by: ysr, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/550

From wkemper at openjdk.org  Fri Nov 22 17:35:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 17:35:35 GMT
Subject: RFR: 8344779: GenShen: Consolidate shared heap region closures
 into one header
In-Reply-To: <jZtsz-0FCZQ661CM0hrVPNcYvJn_73aYgJXArQHw9N4=.77f5e2a2-e1b8-4316-9231-1b6e3c2246d6@github.com>
References: <IWSLnyraAkVWjIudIQZjUhq0l2qHfqpSxlQLJBHoI-w=.60b32529-db5c-414a-b3e3-9490028a3ee7@github.com>
 <jZtsz-0FCZQ661CM0hrVPNcYvJn_73aYgJXArQHw9N4=.77f5e2a2-e1b8-4316-9231-1b6e3c2246d6@github.com>
Message-ID: <umoEYgXN3YY-NU7TpiXkSsBxY4OT3Kjto0utdKfmBlo=.21e44eb8-4ec2-4903-954c-71b5b6baa3a4@github.com>

On Fri, 22 Nov 2024 00:38:15 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Move shared heap region closure implementations into one header, move non-shared implementations closer to their only usage.
>
> src/hotspot/share/gc/shenandoah/shenandoahMarkClosures.cpp line 2:
> 
>> 1: /*
>> 2:  * Copyright (c) 2021, Red Hat, Inc. All rights reserved.
> 
> This had me confused, but I verified that the deletion of the Copyright line is indeed correct based on what I can tell from `git annotate`.
> 
> Also the way the rename shows up in the diffs here is pretty subtle.

Yes, git has a mind of its own when it comes to renames.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/549#discussion_r1854356528

From wkemper at openjdk.org  Fri Nov 22 17:35:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 17:35:36 GMT
Subject: Integrated: 8344779: GenShen: Consolidate shared heap region closures
 into one header
In-Reply-To: <IWSLnyraAkVWjIudIQZjUhq0l2qHfqpSxlQLJBHoI-w=.60b32529-db5c-414a-b3e3-9490028a3ee7@github.com>
References: <IWSLnyraAkVWjIudIQZjUhq0l2qHfqpSxlQLJBHoI-w=.60b32529-db5c-414a-b3e3-9490028a3ee7@github.com>
Message-ID: <fu0Sg3lFnuAdth20bxdW0YEIC3GTXNmduA0QelytE-E=.ba693d1d-7f82-4036-8fa1-f8b394356112@github.com>

On Thu, 21 Nov 2024 19:25:27 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Move shared heap region closure implementations into one header, move non-shared implementations closer to their only usage.

This pull request has now been integrated.

Changeset: 996b27e0
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/996b27e02d35f3a69d473d4648c18e84a4241213
Stats:     351 lines in 9 files changed: 152 ins; 195 del; 4 mod

8344779: GenShen: Consolidate shared heap region closures into one header

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/549

From wkemper at openjdk.org  Fri Nov 22 17:37:57 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 17:37:57 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <dnFM8zsxrQQfFGipeGUQ6vulQfibss0xpD5W6NjMjSU=.da917aa5-dc73-4b61-959f-f09bb8fa0982@github.com>
References: <dnFM8zsxrQQfFGipeGUQ6vulQfibss0xpD5W6NjMjSU=.da917aa5-dc73-4b61-959f-f09bb8fa0982@github.com>
Message-ID: <39Jt8D1QTEPntAP7lPVGaRgtJ2zaZPDgv7uiuITB2cE=.b2340fcb-b194-4818-8779-b19b5879af41@github.com>

On Fri, 22 Nov 2024 14:13:00 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-24+25

This pull request has now been integrated.

Changeset: e22e8d56
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/e22e8d565c4ca85462feba05de36d79ae953ac1b
Stats:     82439 lines in 1443 files changed: 37422 ins; 38857 del; 6160 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/552

From wkemper at openjdk.org  Fri Nov 22 17:37:56 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 17:37:56 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <dnFM8zsxrQQfFGipeGUQ6vulQfibss0xpD5W6NjMjSU=.da917aa5-dc73-4b61-959f-f09bb8fa0982@github.com>
References: <dnFM8zsxrQQfFGipeGUQ6vulQfibss0xpD5W6NjMjSU=.da917aa5-dc73-4b61-959f-f09bb8fa0982@github.com>
Message-ID: <MXyeKSr8Aamo0Zkzm6ugc2iId7bFf-_MaiHL15lPVn4=.5a3632e4-f4e8-4906-a11b-1c7d35f4ddd0@github.com>

> Merges tag jdk-24+25

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/552/files
  - new: https://git.openjdk.org/shenandoah/pull/552/files/4fbf2720..4fbf2720

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=552&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=552&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/552.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/552/head:pull/552

PR: https://git.openjdk.org/shenandoah/pull/552

From rkennke at openjdk.org  Fri Nov 22 18:08:32 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 22 Nov 2024 18:08:32 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v5]
In-Reply-To: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
Message-ID: <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>

> From the bug description:
> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
> 
> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
> 
> The fix to both issues is:
> - First disable UCOH
> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
> - then enable any flags required by compact headers
> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
> 
> Testing:
>  - [x] tier1 -UCOH (default)
>  - [ ] tier1 +UCOH
>  - [x] Manual testing several flag combinations

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Rename set_compact_headers() -> set_compact_headers_flags()

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22255/files
  - new: https://git.openjdk.org/jdk/pull/22255/files/43c7668e..28a1fa3d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22255&range=03-04

  Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/22255.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22255/head:pull/22255

PR: https://git.openjdk.org/jdk/pull/22255

From gziemski at openjdk.org  Fri Nov 22 19:01:27 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Fri, 22 Nov 2024 19:01:27 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v4]
In-Reply-To: <WE3g3ujzzV9COo98K9kSBuhjku1qQBsh7Lp3ajsZsDg=.53795dab-550d-47b2-96b1-55cdfe82d111@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <WE3g3ujzzV9COo98K9kSBuhjku1qQBsh7Lp3ajsZsDg=.53795dab-550d-47b2-96b1-55cdfe82d111@github.com>
Message-ID: <iltY2xoaPDWygwrENXifa33GTI9eyel1dIaO1Z1BkAI=.49b8b0e6-e856-448e-a8d5-f326f14312e0@github.com>

On Thu, 21 Nov 2024 21:09:33 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by avoiding using `mtNone`.
>> 
>> This fix only addresses the cases covered by the issue. I am not trying to replace every single `mtNone` here, but eventually the goal would be to do just that.
>> 
>> Testing: passes MARCH5 tier1-5
>
> Gerard Ziemski has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Stefan's feedback

Filed [Do not use mtNone if we know the tag type](https://bugs.openjdk.org/browse/JDK-8344883) as a followup to fix the remaining `mtNone` tags.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2494552118

From stefank at openjdk.org  Fri Nov 22 21:08:41 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 22 Nov 2024 21:08:41 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v5]
In-Reply-To: <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>
Message-ID: <zNQ3mn58tWBWlX8eGQgfHytfZ1Jsx1bx9qZVQmfjBR8=.7cb97de1-ea17-41ec-82a4-db5417e08808@github.com>

On Fri, 22 Nov 2024 18:08:32 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename set_compact_headers() -> set_compact_headers_flags()

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2455645151

From wkemper at openjdk.org  Fri Nov 22 22:38:02 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 22 Nov 2024 22:38:02 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v9]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <4AQkQIp1NjzLOj76FbXQ8HRcLhkjATlIHV4122VHjSE=.1fd8a480-070c-4133-acfd-0e051e9bdbce@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 528 commits:

 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - Merge
 - 8344779: GenShen: Consolidate shared heap region closures into one header
   
   Reviewed-by: kdnilsen, ysr
 - 8344797: GenShen: Update and rename confusing method
   
   Reviewed-by: ysr, kdnilsen
 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - 8344670: GenShen: Use concurrent worker session for concurrent mark phase
   
   Reviewed-by: kdnilsen
 - 8344640: GenShen: Reuse existing card mark barrier function when dropping references
   
   Reviewed-by: shade, ysr
 - 8344592: GenShen: Remove unnecessary comments and changes
   
   Reviewed-by: kdnilsen
 - 8344263: GenShen: Reduce extraneous log messages at INFO level
   
   Reviewed-by: ysr, shade
 - 8344638: GenShen: Verifier should not touch claim token
   
   Reviewed-by: shade
 - ... and 518 more: https://git.openjdk.org/jdk/compare/4fbf2720...7eea1d5c

-------------

Changes: https://git.openjdk.org/jdk/pull/21273/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=08
  Stats: 22940 lines in 229 files changed: 21170 ins; 911 del; 859 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From shade at openjdk.org  Mon Nov 25 08:42:15 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 25 Nov 2024 08:42:15 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v5]
In-Reply-To: <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>
Message-ID: <54Zkl4v43F0p-3jP5zqgTFPS30GeydTUkeTViZ6EBNg=.78612187-33a3-46d7-9657-cce715d01935@github.com>

On Fri, 22 Nov 2024 18:08:32 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename set_compact_headers() -> set_compact_headers_flags()

Still good.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2457684037

From ayang at openjdk.org  Mon Nov 25 12:54:16 2024
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Mon, 25 Nov 2024 12:54:16 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v5]
In-Reply-To: <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>
Message-ID: <TYFyKuvSs77A0FVeXb02cgIBEBkRJez3atq-fAnnObo=.8e59b787-7857-46c7-9105-c7e1c131a4a3@github.com>

On Fri, 22 Nov 2024 18:08:32 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename set_compact_headers() -> set_compact_headers_flags()

Marked as reviewed by ayang (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/22255#pullrequestreview-2458352531

From rkennke at openjdk.org  Mon Nov 25 13:50:27 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 25 Nov 2024 13:50:27 GMT
Subject: RFR: 8344363: FullGCForwarding::initialize_flags is called after
 ObjLayout::initialize [v5]
In-Reply-To: <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
 <4S2_JFKMsJ5XkPN5Lg_LLl0cRpAYTDiX0rqRIL3S2rM=.6fdba7be-ef33-40f8-b682-3bf83132cdb1@github.com>
Message-ID: <5xykYi1ZaYCjlUi1BzV8HlFD7byikWpvDVx_Xnwwkl4=.74728d39-c273-41f9-9e79-589b5de06c85@github.com>

On Fri, 22 Nov 2024 18:08:32 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> From the bug description:
>> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
>> 
>> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
>> 
>> The fix to both issues is:
>> - First disable UCOH
>> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
>> - then enable any flags required by compact headers
>> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
>> 
>> Testing:
>>  - [x] tier1 -UCOH (default)
>>  - [ ] tier1 +UCOH
>>  - [x] Manual testing several flag combinations
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename set_compact_headers() -> set_compact_headers_flags()

Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22255#issuecomment-2498068162

From rkennke at openjdk.org  Mon Nov 25 13:50:28 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 25 Nov 2024 13:50:28 GMT
Subject: Integrated: 8344363: FullGCForwarding::initialize_flags is called
 after ObjLayout::initialize
In-Reply-To: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
References: <IObXwH1gwDjh7WaXl2nDPkOkha4dm5Oq14ohRXEMZwE=.a317b2b9-46ba-438a-ab7b-e27667f49539@github.com>
Message-ID: <-z5Ym7oqEjeKU3m-UNr6Rk1SS8fzWGGAap0Ycw0fEnA=.2d74a477-535a-4175-ab49-eb881a39f89c@github.com>

On Tue, 19 Nov 2024 20:07:38 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> From the bug description:
> ObjLayout::initialize() is called in Arguments::parse(const JavaVMInitArgs*) which sets ObjLayout::_klass_mode. FullGCForwarding::initialize_flags(size_t) is called in init_globals() which seems to be later in the Threads::create_vm(JavaVMInitArgs*, bool*) routine. The latter, however, can unset the UseCompactObjectHeaders flag, which leads to a potential mismatch with ObjLayout::_klass_mode, firing the asserts in ObjLayout::klass_mode().
> 
> There is also a related (and somewhat minor) problem: In Arguments::parse() we enable a bunch of stuff when UseCompactObjectHeaders is on (LW locking, but that's the default anyway, and object-monitor-tables, which are otherwise off), but then may disable UseCompactObjectHeaders, e.g. in the GC or when -UseCompressedClassPointers is requested. This then leaves the odd situation that we run without compact headers, but have object monitor tables turned on.
> 
> The fix to both issues is:
> - First disable UCOH
> - and do that in apply_ergo() (e.g. GCs should do it in GCArguments::initialize())
> - then enable any flags required by compact headers
> - Initialize ObjLayout after all flags are done (it's in the correct place, already)
> 
> Testing:
>  - [x] tier1 -UCOH (default)
>  - [ ] tier1 +UCOH
>  - [x] Manual testing several flag combinations

This pull request has now been integrated.

Changeset: cb1c7366
Author:    Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/cb1c73663e91e632d643c23e6c5acc1c5118ac8b
Stats:     43 lines in 11 files changed: 18 ins; 20 del; 5 mod

8344363: FullGCForwarding::initialize_flags is called after ObjLayout::initialize

Reviewed-by: stefank, shade, ayang

-------------

PR: https://git.openjdk.org/jdk/pull/22255

From gziemski at openjdk.org  Mon Nov 25 16:11:20 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Mon, 25 Nov 2024 16:11:20 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v4]
In-Reply-To: <WE3g3ujzzV9COo98K9kSBuhjku1qQBsh7Lp3ajsZsDg=.53795dab-550d-47b2-96b1-55cdfe82d111@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <WE3g3ujzzV9COo98K9kSBuhjku1qQBsh7Lp3ajsZsDg=.53795dab-550d-47b2-96b1-55cdfe82d111@github.com>
Message-ID: <ECuUrd3nSdZSl2x6a-uUnLPTI8myKK3Iio1kW0ekwJU=.6e713668-7866-4464-9e77-7d628e9b98a0@github.com>

On Thu, 21 Nov 2024 21:09:33 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by avoiding using `mtNone`.
>> 
>> This fix only addresses the cases covered by the issue. I am not trying to replace every single `mtNone` here, but eventually the goal would be to do just that.
>> 
>> Testing: passes MARCH5 tier1-5
>
> Gerard Ziemski has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Stefan's feedback

ping

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2498436409

From shade at openjdk.org  Mon Nov 25 17:31:17 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 25 Nov 2024 17:31:17 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v7]
In-Reply-To: <rYJmY3cIclpM5IVaxSLmnWJvFKXmj1nfU9PBT31KSns=.9c62ca5e-9b87-4106-9d79-d7829733018b@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
 <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>
 <2ctRsJ6JhReYC9yjcPV4eRljht3bMAZ9B3urifpvQXQ=.470aa336-f334-4345-863b-557250aa3416@github.com>
 <I0GzgbIQG1aGxmJ4c1wh0N5uYDDO-5fIrnEtT4t675s=.b21ca11b-1061-4bbe-97c3-79edccdabab4@github.com>
 <TygSXGP4yj46QlEcRv-m5AfZTb6rreI9SGdjbZpziiM=.fbe4a0be-b601-4219-bc88-40d27f1787f4@github.com>
 <9MFkD3YNuSJIBPOsHYEKFX98XkMGKgMYa_7j-7usxC0=.0a99aabd-6272-40db-a5aa-6022023c1e4f@github.com>
 <rYJmY3cIclpM5IVaxSLmnWJvFKXmj1nfU9PBT31KSns=.9c62ca5e-9b87-4106-9d79-d7829733018b@github.com>
Message-ID: <9HyWt8dTl_6N6S0YhLw22cBJz8PMP1SVGXgB_ELdYF8=.4f59fd3a-fa0a-4fcf-84c8-30616b463a0c@github.com>

On Thu, 21 Nov 2024 23:26:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> If we are emitting a log line that looks like a properly formatted GC log line, but the numbers there mean something else for `Concurrent uncommit`, we are bound to confuse users and automatic tools. Uncommit should affect `capacity`, this is how we know how deep we have uncommitted. So, I suggest we emit:
>> 
>> 
>> Concurrent uncommit XXXXM->XXXXM (YYYYM) z.zzzms
>> 
>> 
>> ...where `XXXX` is the heap used at the end of uncommit (note before and after are the same) and YYYY is capacity. This will not expose users to thinking uncommit grows the heap usage, and would give us instantaneous view on heap usage and capacity.
>
> Hmm, the numbers are preceded by `Concurrent uncommit`, with that context it's not much of a stretch to think these numbers represent the change in _committed_ memory. The original log message (in which heap usage may increase during uncommit) was _not_ helpful. A message with the same format in which heap usage also appears to _not change at all_ during an uncommit is also perplexing. Are we trying too hard to preserve the original, not useful message? Maybe we just want a new message that plainly says: 
> 
> 
> Concurrently uncommitted XXXXM in z.zzzms
> 
> or
> 
> Concurrent uncommit: time z.zzzms, committed before XXXXM, committed after YYYYM, capacity ZZZZM

Yes, I don't want to emit something that looks like a heap usage GC log line, if it is not. Unfortunately, `X->Y (Z) T.TTTTms` is a common format for X and Y as heap use. I agree posting X == Y would be only marginally better. So, maybe this goes as middle ground:


Concurrent uncommit XXXXM (YYYYM) z.zzzms


...where XXXX is the amount uncommitted, YYYY is the final heap capacity?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1857007496

From wkemper at openjdk.org  Mon Nov 25 17:59:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Nov 2024 17:59:26 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v7]
In-Reply-To: <9HyWt8dTl_6N6S0YhLw22cBJz8PMP1SVGXgB_ELdYF8=.4f59fd3a-fa0a-4fcf-84c8-30616b463a0c@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <tJdzLF1v9E_x3YoN15SGmf943sqXDEPw2tUO7Ze90UY=.3111a44e-ebca-4cb6-828f-66b0d84fb3ad@github.com>
 <VyJdWIZIW-oB1LYcxbjHcVt0tk1MI0IhBmm-oygCVBY=.5e458bea-b417-4ae3-88f9-d0aa0bf97c3b@github.com>
 <2ctRsJ6JhReYC9yjcPV4eRljht3bMAZ9B3urifpvQXQ=.470aa336-f334-4345-863b-557250aa3416@github.com>
 <I0GzgbIQG1aGxmJ4c1wh0N5uYDDO-5fIrnEtT4t675s=.b21ca11b-1061-4bbe-97c3-79edccdabab4@github.com>
 <TygSXGP4yj46QlEcRv-m5AfZTb6rreI9SGdjbZpziiM=.fbe4a0be-b601-4219-bc88-40d27f1787f4@github.com>
 <9MFkD3YNuSJIBPOsHYEKFX98XkMGKgMYa_7j-7usxC0=.0a99aabd-6272-40db-a5aa-6022023c1e4f@github.com>
 <rYJmY3cIclpM5IVaxSLmnWJvFKXmj1nfU9PBT31KSns=.9c62ca5e-9b87-4106-9d79-d7829733018b@github.com>
 <9HyWt8dTl_6N6S0YhLw22cBJz8PMP1SVGXgB_ELdYF8=.4f59fd3a-fa0a-4fcf-84c8-30616b463a0c@github.com>
Message-ID: <cDkoM6VHKo3tuWbxFM7Fl4kUP0Pi4iP7c48oxq0bPgU=.7fc3f14a-71bf-481a-a606-24e7da83328e@github.com>

On Mon, 25 Nov 2024 17:28:19 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Hmm, the numbers are preceded by `Concurrent uncommit`, with that context it's not much of a stretch to think these numbers represent the change in _committed_ memory. The original log message (in which heap usage may increase during uncommit) was _not_ helpful. A message with the same format in which heap usage also appears to _not change at all_ during an uncommit is also perplexing. Are we trying too hard to preserve the original, not useful message? Maybe we just want a new message that plainly says: 
>> 
>> 
>> Concurrently uncommitted XXXXM in z.zzzms
>> 
>> or
>> 
>> Concurrent uncommit: time z.zzzms, committed before XXXXM, committed after YYYYM, capacity ZZZZM
>
> Yes, I don't want to emit something that looks like a heap usage GC log line, if it is not. Unfortunately, `X->Y (Z) T.TTTTms` is a common format for X and Y as heap use. I agree posting X == Y would be only marginally better. So, maybe this goes as middle ground:
> 
> 
> Concurrent uncommit XXXXM (YYYYM) z.zzzms
> 
> 
> ...where XXXX is the amount uncommitted, YYYY is the final heap capacity?

Okay, this looks good.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1857092060

From wkemper at openjdk.org  Mon Nov 25 18:01:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Nov 2024 18:01:40 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <idpksYdcncdiU3JIH_SR7gPS0tyz-dNp_4uRYHP33tE=.328cd779-dda1-4e4d-a60f-63e6500d94d8@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
 <i-MvUaoe8n9apI8y-4hQ9ohIGHwSPaufWIkdMLyBdLM=.3bcf0c60-9b68-4552-b425-45a8ff946664@github.com>
 <idpksYdcncdiU3JIH_SR7gPS0tyz-dNp_4uRYHP33tE=.328cd779-dda1-4e4d-a60f-63e6500d94d8@github.com>
Message-ID: <APW2rDug_Eyva5EePCc_Z5H2APJbXTuIqHObJ1ZEafU=.be7edeef-e18c-4ee7-8cd5-8beff3c77bb1@github.com>

On Thu, 21 Nov 2024 18:30:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Yes, in single gen mode we set `MARKING` and `YOUNG_MARKING` together. See https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahGlobalGeneration.cpp#L81
>
> We should really either predicate it on generational mode (we already poll it a few lines below) and check for `MARKING` specifically, or assert it.

I'll make some adjustments here. Note that `OLD_MARKING` also implies (and sets) `MARKING`, and - unlike `YOUNG_MARKING` - is not exclusive with `EVACUTION` and `UPDATEREFS`. Perhaps an explanatory comment is the best option here?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1857098111

From shade at openjdk.org  Mon Nov 25 18:25:39 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 25 Nov 2024 18:25:39 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <APW2rDug_Eyva5EePCc_Z5H2APJbXTuIqHObJ1ZEafU=.be7edeef-e18c-4ee7-8cd5-8beff3c77bb1@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
 <i-MvUaoe8n9apI8y-4hQ9ohIGHwSPaufWIkdMLyBdLM=.3bcf0c60-9b68-4552-b425-45a8ff946664@github.com>
 <idpksYdcncdiU3JIH_SR7gPS0tyz-dNp_4uRYHP33tE=.328cd779-dda1-4e4d-a60f-63e6500d94d8@github.com>
 <APW2rDug_Eyva5EePCc_Z5H2APJbXTuIqHObJ1ZEafU=.be7edeef-e18c-4ee7-8cd5-8beff3c77bb1@github.com>
Message-ID: <jARwQdGF0AwLvyOr3vTGCyx3ADrvH05BtmJWMzSJWC4=.086fb71c-e489-42b9-b30b-7af976288456@github.com>

On Mon, 25 Nov 2024 17:59:05 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> We should really either predicate it on generational mode (we already poll it a few lines below) and check for `MARKING` specifically, or assert it.
>
> I'll make some adjustments here. Note that `OLD_MARKING` also implies (and sets) `MARKING`, and - unlike `YOUNG_MARKING` - is not exclusive with `EVACUTION` and `UPDATEREFS`. Perhaps an explanatory comment is the best option here?

Thing is, the description for `MARKING` says "marking is in progress" (for any mode), while `YOUNG_MARKING` says that it is only true for yougn marking. So for a casual reader, checking for `YOUNG_MARKING` in non-generational code looks weird. I don't know the best way out of this, really. I think this is the only place when we check `YOUNG_MARKING` for non-gen mode? If so, we should probably fix it right here?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1857151960

From jsjolen at openjdk.org  Mon Nov 25 19:08:19 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Mon, 25 Nov 2024 19:08:19 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v4]
In-Reply-To: <WE3g3ujzzV9COo98K9kSBuhjku1qQBsh7Lp3ajsZsDg=.53795dab-550d-47b2-96b1-55cdfe82d111@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <WE3g3ujzzV9COo98K9kSBuhjku1qQBsh7Lp3ajsZsDg=.53795dab-550d-47b2-96b1-55cdfe82d111@github.com>
Message-ID: <OWT0yvNSRVILU5oqnU28W9iTXGWvOjZtxVzDGwXmxE0=.ee837d76-1901-47c7-b849-9526d373e5fc@github.com>

On Thu, 21 Nov 2024 21:09:33 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

>> We use `mtNone` value in several functions default parameters, which may show up in NMT reports.
>> 
>> We address this, by avoiding using `mtNone`.
>> 
>> This fix only addresses the cases covered by the issue. I am not trying to replace every single `mtNone` here, but eventually the goal would be to do just that.
>> 
>> Testing: passes MARCH5 tier1-5
>
> Gerard Ziemski has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Stefan's feedback

This LGTM and the chosen categories seem to make sense.

-------------

Marked as reviewed by jsjolen (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21843#pullrequestreview-2459413170

From wkemper at openjdk.org  Mon Nov 25 19:16:56 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Nov 2024 19:16:56 GMT
Subject: RFR: 8344985: GenShen: Refactor arraycopy barrier for generational
 mode
Message-ID: <9mhJhHK2sJXcIcxudHmf5rCE36eeo2cIMJD5jw17sUc=.3724a0ed-4863-4258-882c-48c8250104ca@github.com>

Make the difference in behavior for generational and other modes more clear.

-------------

Commit messages:
 - Make the separation between generational and other modes more clear

Changes: https://git.openjdk.org/shenandoah/pull/553/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=553&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8344985
  Stats: 14 lines in 1 file changed: 6 ins; 7 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/553.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/553/head:pull/553

PR: https://git.openjdk.org/shenandoah/pull/553

From shade at openjdk.org  Mon Nov 25 19:36:51 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 25 Nov 2024 19:36:51 GMT
Subject: RFR: 8344985: GenShen: Refactor arraycopy barrier for generational
 mode
In-Reply-To: <9mhJhHK2sJXcIcxudHmf5rCE36eeo2cIMJD5jw17sUc=.3724a0ed-4863-4258-882c-48c8250104ca@github.com>
References: <9mhJhHK2sJXcIcxudHmf5rCE36eeo2cIMJD5jw17sUc=.3724a0ed-4863-4258-882c-48c8250104ca@github.com>
Message-ID: <hgwBrwHK5eBpahw86AuSoK3Di6_3L45A64m1VrZ0axU=.89f43763-9fff-4878-b7ab-1a30592b54f0@github.com>

On Mon, 25 Nov 2024 19:12:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Make the difference in behavior for generational and other modes more clear.

Marked as reviewed by shade (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/553#pullrequestreview-2459463181

From wkemper at openjdk.org  Mon Nov 25 19:40:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Nov 2024 19:40:25 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v10]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <kx9OWGRZvHr9ioNzMwhwKNKGec0Nh66eAuUJze3T7VU=.b6e23f5f-668e-4e05-90bf-9f14c81cbd4d@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Remove incomplete test case

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21273/files
  - new: https://git.openjdk.org/jdk/pull/21273/files/7eea1d5c..381af0cd

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=09
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=08-09

  Stats: 50 lines in 1 file changed: 0 ins; 50 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From wkemper at openjdk.org  Mon Nov 25 19:40:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Nov 2024 19:40:26 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v7]
In-Reply-To: <jARwQdGF0AwLvyOr3vTGCyx3ADrvH05BtmJWMzSJWC4=.086fb71c-e489-42b9-b30b-7af976288456@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <pfyUmffRAmQm2IUdA4jJYfv5tXgYcuvyIbqiVFx_XFc=.f4c0aa5f-2e6c-405d-b73f-e1414c41738b@github.com>
 <JoN2wTjeWNg8rMKxkRUgWho9bqRxw46npCjGXj1_Fr4=.6cdb8814-7173-4dd1-95e4-76ab93352438@github.com>
 <i-MvUaoe8n9apI8y-4hQ9ohIGHwSPaufWIkdMLyBdLM=.3bcf0c60-9b68-4552-b425-45a8ff946664@github.com>
 <idpksYdcncdiU3JIH_SR7gPS0tyz-dNp_4uRYHP33tE=.328cd779-dda1-4e4d-a60f-63e6500d94d8@github.com>
 <APW2rDug_Eyva5EePCc_Z5H2APJbXTuIqHObJ1ZEafU=.be7edeef-e18c-4ee7-8cd5-8beff3c77bb1@github.com>
 <jARwQdGF0AwLvyOr3vTGCyx3ADrvH05BtmJWMzSJWC4=.086fb71c-e489-42b9-b30b-7af976288456@github.com>
Message-ID: <uCZnSq4iDPSfcNkTMz3WsF48erxpFl2a1IL8QCXDvis=.8cb11ea3-2c6e-4928-b778-9c683d8f3cf9@github.com>

On Mon, 25 Nov 2024 18:22:09 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> I'll make some adjustments here. Note that `OLD_MARKING` also implies (and sets) `MARKING`, and - unlike `YOUNG_MARKING` - is not exclusive with `EVACUTION` and `UPDATEREFS`. Perhaps an explanatory comment is the best option here?
>
> Thing is, the description for `MARKING` says "marking is in progress" (for any mode), while `YOUNG_MARKING` says that it is only true for yougn marking. So for a casual reader, checking for `YOUNG_MARKING` in non-generational code looks weird. I don't know the best way out of this, really. I think this is the only place when we check `YOUNG_MARKING` for non-gen mode? If so, we should probably fix it right here?

Something like this maybe: https://bugs.openjdk.org/browse/JDK-8344985?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1857241797

From xpeng at openjdk.org  Mon Nov 25 22:25:39 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Mon, 25 Nov 2024 22:25:39 GMT
Subject: RFR: 8338737: Shenandoah: Reset marking bitmaps after the cycle
 [v16]
In-Reply-To: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
References: <4Vf5hDoYaZyyFJ3IS3Fxr9kNMhSoiJY5K4BqAaKXI-I=.60fe828e-6998-4438-a75a-0859604bae44@github.com>
Message-ID: <vIbu5XmN02KaE98zc0TwAS2ot-zFsrV3K-3ZgomzT2g=.6bcfeead-7389-47a9-a795-a6dc3a4f77c1@github.com>

> All the shenandoah passed, still waiting for our test farm to verify performance.
> 
> Reset marking bitmaps after collection cycle, for GenShen only do this for young generation, also choose not do this for Degen and full GC since both are running at safepoint, we should leave safepoint as ASAP.
> 
> I have run same workload for 30s with shenandoah in generational mode and classic mode, average average time of concurrent reset dropped significantly(283us to 109us for GenShen, 276us to 167us for classic shenandoah. 
> 
> Classic shenandoah baseline:
> 
> [30.973s][info][gc,stats    ] Concurrent Reset               =    0.148 s (a =      276 us) (n =   536) (lvls, us =      141,      205,      240,      291,      694)
> [30.973s][info][gc,stats    ] Pause Init Mark (G)            =    0.136 s (a =      253 us) (n =   536) (lvls, us =      119,      242,      254,      264,      926)
> [30.973s][info][gc,stats    ] Pause Init Mark (N)            =    0.047 s (a =       87 us) (n =   536) (lvls, us =       62,       84,       88,       91,      110)
> ...
> 
> 
> Classic shenandoah reset bitmaps after cycle:
> 
> [30.967s][info][gc,stats    ] Concurrent Reset               =    0.109 s (a =      167 us) (n =   652) (lvls, us =       48,      113,      139,      160,      888)
> [30.967s][info][gc,stats    ] Concurrent Reset After Collect =    0.085 s (a =      132 us) (n =   648) (lvls, us =      107,      121,      125,      131,      532)
> [30.967s][info][gc,stats    ] Pause Init Mark (G)            =    0.189 s (a =      289 us) (n =   652) (lvls, us =      117,      260,      277,      291,     2297)
> [30.967s][info][gc,stats    ] Pause Init Mark (N)            =    0.058 s (a =       89 us) (n =   652) (lvls, us =       62,       85,       89,       94,      143)
> ...
> 
> 
> GenShen baseline
> 
> [31.008s][info][gc,stats    ] Concurrent Reset               =    0.107 s (a =      283 us) (n =   379) (lvls, us =      143,      225,      283,      330,      753)
> [31.008s][info][gc,stats    ] Pause Init Mark (G)            =    0.098 s (a =      259 us) (n =   379) (lvls, us =      104,      227,      262,      305,     1033)
> [31.008s][info][gc,stats    ] Pause Init Mark (N)            =    0.034 s (a =       90 us) (n =   379) (lvls, us =       67,       81,       89,       99,      130)
> ...
> 
> 
> GenShen reset bitmaps after cycle
> 
> 
> [30.977s][info][gc,stats    ] Concurrent Reset               =    0.050 s (a =      109 us) (n =   462) (lvls, us =       54,       77,      100,      125,      496)
> [30.977s][...

Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits:

 - Merge branch 'shenandoah' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Move set_mark_incomplete into reset_mark_bitmap
 - fix test failures
 - set_mark_incomplete should be always set after resetting bitmap
 - Merge branch 'openjdk:master' into genshen-JDK-8338737
 - Not reset bitmap after ShenandoahOldGC
 - Capture Top At Mark Start for *this* generation
 - ... and 30 more: https://git.openjdk.org/shenandoah/compare/e22e8d56...375f2bd3

-------------

Changes: https://git.openjdk.org/shenandoah/pull/516/files
  Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=516&range=15
  Stats: 111 lines in 6 files changed: 96 ins; 2 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah/pull/516.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/516/head:pull/516

PR: https://git.openjdk.org/shenandoah/pull/516

From wkemper at openjdk.org  Mon Nov 25 23:27:09 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 25 Nov 2024 23:27:09 GMT
Subject: Integrated: 8344985: GenShen: Refactor arraycopy barrier for
 generational mode
In-Reply-To: <9mhJhHK2sJXcIcxudHmf5rCE36eeo2cIMJD5jw17sUc=.3724a0ed-4863-4258-882c-48c8250104ca@github.com>
References: <9mhJhHK2sJXcIcxudHmf5rCE36eeo2cIMJD5jw17sUc=.3724a0ed-4863-4258-882c-48c8250104ca@github.com>
Message-ID: <h2Lk_jICImKUrBcUHOhoMwGGtQjD2ZEQTC8cdRv2dgI=.a6db6828-1332-4972-b20e-d55e88ea18a2@github.com>

On Mon, 25 Nov 2024 19:12:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Make the difference in behavior for generational and other modes more clear.

This pull request has now been integrated.

Changeset: 34d334d8
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/34d334d84aca0fe1724d8f38080d8017dbf8bb88
Stats:     14 lines in 1 file changed: 6 ins; 7 del; 1 mod

8344985: GenShen: Refactor arraycopy barrier for generational mode

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/shenandoah/pull/553

From wkemper at openjdk.org  Tue Nov 26 00:37:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 26 Nov 2024 00:37:32 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v11]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - 8344985: GenShen: Refactor arraycopy barrier for generational mode
   
   Reviewed-by: shade

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21273/files
  - new: https://git.openjdk.org/jdk/pull/21273/files/381af0cd..08b59bc3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=10
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=09-10

  Stats: 14 lines in 1 file changed: 6 ins; 7 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From wkemper at openjdk.org  Tue Nov 26 01:07:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 26 Nov 2024 01:07:39 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v8]
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <HripUUPUYfiN85INiVDn05zpI3FJiNjX6kEFXjp3Ucs=.c71a1a08-9b42-41a1-bb6b-09151eac3b62@github.com>

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 18 additional commits since the last revision:

 - Log uncommitted delta and capacity
 - Merge remote-tracking branch 'jdk/master' into shen-uncommit-thread
 - Restore logging format, show change in committed heap, rather than usage
 - Allow commits initially
 - Use idiomatic name for CADR class
 - Improve comments
 - Do not notify uncommit thread when uncommit is forbidden
 - Prevent uncommit thread from running during GC
 - Style and formatting fixes
 - Alphabetize includes in shenandoahHeap.cpp
 - ... and 8 more: https://git.openjdk.org/jdk/compare/f2fab487...847a2593

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22019/files
  - new: https://git.openjdk.org/jdk/pull/22019/files/2cb71140..847a2593

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=06-07

  Stats: 225409 lines in 4393 files changed: 87200 ins; 121188 del; 17021 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From shade at openjdk.org  Tue Nov 26 09:44:02 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 26 Nov 2024 09:44:02 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v11]
In-Reply-To: <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
Message-ID: <BV9-mRewY4Obqkb9_HYDaWfL85k1THGMwF5pWk1HweA=.134b2a95-5158-4a0c-b72e-39c481a16143@github.com>

On Tue, 26 Nov 2024 00:37:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344985: GenShen: Refactor arraycopy barrier for generational mode
>    
>    Reviewed-by: shade

I checked all review threads, and I think my review feedback was appropriately addressed. I see no (new) test failures in my local testing as well.

Thank you all, this is a huge piece of engineering work, and it looks good for experimental feature. We would probably make a few touchups and simplifications after the integration. I hope this would stabilize in the future releases to become a new default mode for Shenandoah!

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2460929110

From shade at openjdk.org  Tue Nov 26 10:21:47 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 26 Nov 2024 10:21:47 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v8]
In-Reply-To: <HripUUPUYfiN85INiVDn05zpI3FJiNjX6kEFXjp3Ucs=.c71a1a08-9b42-41a1-bb6b-09151eac3b62@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <HripUUPUYfiN85INiVDn05zpI3FJiNjX6kEFXjp3Ucs=.c71a1a08-9b42-41a1-bb6b-09151eac3b62@github.com>
Message-ID: <WD1RwtNlh4ftiRfkFnjanV227bvT1xZviGCZ1mw0rw0=.37015485-10e7-4202-8a41-1d3de4ec30bb@github.com>

On Tue, 26 Nov 2024 01:07:39 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 18 additional commits since the last revision:
> 
>  - Log uncommitted delta and capacity
>  - Merge remote-tracking branch 'jdk/master' into shen-uncommit-thread
>  - Restore logging format, show change in committed heap, rather than usage
>  - Allow commits initially
>  - Use idiomatic name for CADR class
>  - Improve comments
>  - Do not notify uncommit thread when uncommit is forbidden
>  - Prevent uncommit thread from running during GC
>  - Style and formatting fixes
>  - Alphabetize includes in shenandoahHeap.cpp
>  - ... and 8 more: https://git.openjdk.org/jdk/compare/a518ae81...847a2593

Changes requested by shade (Reviewer).

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 75:

> 73:       MonitorLocker locker(&_stop_lock, Mutex::_no_safepoint_check_flag);
> 74:       if (!_stop_requested.is_set()) {
> 75:         locker.wait((int64_t)shrink_period);

I tried to test this on some of my toy examples, and realized this particular line may end up as `locker.wait(0)`, which means "wait indefinitely, until notified". This breaks periodic commits. The old code rode on control thread doing `MAX2(1, ...)`, so we never feed `0` into `wait`. I am also confused about units. The comment above says `shrink_period` is in seconds, but `locker.wait` accepts milliseconds?

src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 181:

> 179:   log_info(gc)("%s " PROPERFMT "(" PROPERFMT ") %.3fms",
> 180:                msg, PROPERFMTARGS(committed_start - committed_end), PROPERFMTARGS(_heap->capacity()),
> 181:                elapsed * MILLIUNITS);

I think we want an additional space. I see the current output is:


[11.366s][info][gc] Concurrent uncommit 32768K(192M) 2.506ms


Should probably be:


[11.366s][info][gc] Concurrent uncommit 32768K (192M) 2.506ms

-------------

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2461003962
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1858191192
PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1858203087

From shade at openjdk.org  Tue Nov 26 10:21:48 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 26 Nov 2024 10:21:48 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v8]
In-Reply-To: <WD1RwtNlh4ftiRfkFnjanV227bvT1xZviGCZ1mw0rw0=.37015485-10e7-4202-8a41-1d3de4ec30bb@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <HripUUPUYfiN85INiVDn05zpI3FJiNjX6kEFXjp3Ucs=.c71a1a08-9b42-41a1-bb6b-09151eac3b62@github.com>
 <WD1RwtNlh4ftiRfkFnjanV227bvT1xZviGCZ1mw0rw0=.37015485-10e7-4202-8a41-1d3de4ec30bb@github.com>
Message-ID: <yIvCmwS_txWMPRAVB65hXCB10-Okzv169gL-3hb_jZ4=.ef28efb3-c29e-481b-8404-26629dd53de5@github.com>

On Tue, 26 Nov 2024 10:10:42 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 18 additional commits since the last revision:
>> 
>>  - Log uncommitted delta and capacity
>>  - Merge remote-tracking branch 'jdk/master' into shen-uncommit-thread
>>  - Restore logging format, show change in committed heap, rather than usage
>>  - Allow commits initially
>>  - Use idiomatic name for CADR class
>>  - Improve comments
>>  - Do not notify uncommit thread when uncommit is forbidden
>>  - Prevent uncommit thread from running during GC
>>  - Style and formatting fixes
>>  - Alphabetize includes in shenandoahHeap.cpp
>>  - ... and 8 more: https://git.openjdk.org/jdk/compare/a518ae81...847a2593
>
> src/hotspot/share/gc/shenandoah/shenandoahUncommitThread.cpp line 75:
> 
>> 73:       MonitorLocker locker(&_stop_lock, Mutex::_no_safepoint_check_flag);
>> 74:       if (!_stop_requested.is_set()) {
>> 75:         locker.wait((int64_t)shrink_period);
> 
> I tried to test this on some of my toy examples, and realized this particular line may end up as `locker.wait(0)`, which means "wait indefinitely, until notified". This breaks periodic commits. The old code rode on control thread doing `MAX2(1, ...)`, so we never feed `0` into `wait`. I am also confused about units. The comment above says `shrink_period` is in seconds, but `locker.wait` accepts milliseconds?

It sounds like this line should be:


        locker.wait(MAX2<int64_t>(1, shrink_period * 1000));

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22019#discussion_r1858200969

From rkennke at openjdk.org  Tue Nov 26 11:16:07 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 26 Nov 2024 11:16:07 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v11]
In-Reply-To: <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
Message-ID: <UNg5h8LyziWxMnjRQxXXeER54Fn-gEIUqPAWGjEZxDk=.c38f013e-6748-4ea1-983e-cd21cc14592b@github.com>

On Tue, 26 Nov 2024 00:37:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344985: GenShen: Refactor arraycopy barrier for generational mode
>    
>    Reviewed-by: shade

I also looked again at any new changes since I reviewed last time. Still looks good. Ship it! This is great work!

Roman

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2461152113

From gziemski at openjdk.org  Tue Nov 26 15:24:51 2024
From: gziemski at openjdk.org (Gerard Ziemski)
Date: Tue, 26 Nov 2024 15:24:51 GMT
Subject: RFR: 8328944: NMT reports "unknown" memory [v4]
In-Reply-To: <OWT0yvNSRVILU5oqnU28W9iTXGWvOjZtxVzDGwXmxE0=.ee837d76-1901-47c7-b849-9526d373e5fc@github.com>
References: <q410XZ4jLQtnc1oedcMow7N8Dhhb9nw1dJYggeY2f6U=.00b23ac6-e99c-4784-ab1e-150f04eb7cc8@github.com>
 <WE3g3ujzzV9COo98K9kSBuhjku1qQBsh7Lp3ajsZsDg=.53795dab-550d-47b2-96b1-55cdfe82d111@github.com>
 <OWT0yvNSRVILU5oqnU28W9iTXGWvOjZtxVzDGwXmxE0=.ee837d76-1901-47c7-b849-9526d373e5fc@github.com>
Message-ID: <irPzFtbV9xDNAssiSg7Uf2DVilT95kSQkMP-OTevGXs=.6efef11b-ae33-4eb0-853d-2f2803dba477@github.com>

On Mon, 25 Nov 2024 19:05:58 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> This LGTM and the chosen categories seem to make sense.

Thank you for the review!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21843#issuecomment-2501130432

From mdoerr at openjdk.org  Tue Nov 26 17:43:06 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Tue, 26 Nov 2024 17:43:06 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v11]
In-Reply-To: <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
Message-ID: <DQFCYDvT7j17XZjc1v-KX1GuBISl7i-gAiKGR0oJ4bE=.020de971-e9a3-4975-b126-a9d800414bff@github.com>

On Tue, 26 Nov 2024 00:37:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344985: GenShen: Refactor arraycopy barrier for generational mode
>    
>    Reviewed-by: shade

Can you also add the platform code contributors to the Contributors, please?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2501563056

From wkemper at openjdk.org  Tue Nov 26 19:08:58 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 26 Nov 2024 19:08:58 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v9]
In-Reply-To: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
Message-ID: <z3-Eyj1kI9WS3QyAg8WEDOaDBkTA956IwB2tUKm1aOQ=.1a70b2d4-44ee-4064-aa0f-725e8ff88d50@github.com>

> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Use count of regions uncommitted to compute uncommit delta
 - Decouple polling interval from uncommit time out

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/22019/files
  - new: https://git.openjdk.org/jdk/pull/22019/files/847a2593..8d3c3926

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=08
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22019&range=07-08

  Stats: 15 lines in 1 file changed: 2 ins; 4 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/22019.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/22019/head:pull/22019

PR: https://git.openjdk.org/jdk/pull/22019

From shade at openjdk.org  Tue Nov 26 19:28:42 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 26 Nov 2024 19:28:42 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v9]
In-Reply-To: <z3-Eyj1kI9WS3QyAg8WEDOaDBkTA956IwB2tUKm1aOQ=.1a70b2d4-44ee-4064-aa0f-725e8ff88d50@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <z3-Eyj1kI9WS3QyAg8WEDOaDBkTA956IwB2tUKm1aOQ=.1a70b2d4-44ee-4064-aa0f-725e8ff88d50@github.com>
Message-ID: <Hf75U2n32CRCaBLegNTb6AalTfITP46qxNvPf5lurko=.bea6702e-1864-4a81-a58a-943563c3ea43@github.com>

On Tue, 26 Nov 2024 19:08:58 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Currently, Shenandoah uncommits regions from its control thread. The control thread is responsible for starting GC cycles in a timely fashion. Uncommitting memory from this thread may introduce unwanted delays in the control thread's response to GC pressure.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Use count of regions uncommitted to compute uncommit delta
>  - Decouple polling interval from uncommit time out

Looks good!

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22019#pullrequestreview-2462504290

From wkemper at openjdk.org  Tue Nov 26 19:52:03 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 26 Nov 2024 19:52:03 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v11]
In-Reply-To: <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
Message-ID: <pDCEXlT-8LW1qux9JFLPQFux9KT9XNhmX90NIHmiftU=.735ac320-1474-4df6-b3b8-e43ed3c23ab5@github.com>

On Tue, 26 Nov 2024 00:37:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344985: GenShen: Refactor arraycopy barrier for generational mode
>    
>    Reviewed-by: shade

Thank you for getting us started:

Thank you for the PPC64 port:

Thank you for the RISCV port:

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2501791242

From wkemper at openjdk.org  Tue Nov 26 20:13:42 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 26 Nov 2024 20:13:42 GMT
Subject: RFR: 8342444: Shenandoah: Uncommit regions from a separate,
 STS aware thread [v3]
In-Reply-To: <2aAK-4vQxs3-P-cscvtJqOwn40B2aduezOZkQIKK-BY=.f1b9a066-63ef-4fe3-bb3d-2efcfac2b6af@github.com>
References: <tmBUHpyr0Pozu_mzKD1HROlb1dV2vgPKW9Kb4tJEZ7s=.5a8bd9b3-76ba-4781-b9f7-31243ab24e81@github.com>
 <9SXRh1N-RjMfm_G4LvQFsuF5_DvRxMCrcCXMOtsAwpM=.2b37fe99-1107-440d-a4cb-468fe415b3be@github.com>
 <2aAK-4vQxs3-P-cscvtJqOwn40B2aduezOZkQIKK-BY=.f1b9a066-63ef-4fe3-bb3d-2efcfac2b6af@github.com>
Message-ID: <BPF7V_OX7MH5Du0e0N8zHHJ9UhFbxfeBz-XPsRpvFRw=.e395128c-ff54-467b-91ac-99d37e27a06b@github.com>

On Tue, 19 Nov 2024 02:03:18 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Prevent uncommit thread from running during GC
>
> Looks good to me. A few documentation comment requests.
> 
> Also please share performance data in this PR or in the ticket, especially from the perf/benchmark that may have precipitated this change.

@ysramakrishna - I ran several iterations of specjbb2015 with different variations of polling interval. Results show that 1/10th of `ShenandoahUncommitDelay` is reasonable, and avoids unintentional commit delays when the polling interval is equal or greater than `ShenandoahUncommitDelay`.


                      Category |  Count |         Total |      GeoMean |      Average |     Trim 0.1 |       StdDev |      Minimum |      Maximum
openjdk:master   critical_jops |      5 |     50862.000 |    10162.095 |    10172.400 |    10172.400 |      513.605 |     9630.000 |    10882.000                                
30ms polling     critical_jops |      5 |     48035.000 |     9582.113 |     9607.000 |     9607.000 |      778.036 |     8808.000 |    10692.000
30s polling      critical_jops |      5 |     56398.000 |    11272.026 |    11279.600 |    11279.600 |      460.355 |    10627.000 |    11842.000
no polling       critical_jops |      5 |     55917.000 |    11176.046 |    11183.400 |    11183.400 |      460.960 |    10899.000 |    11995.000

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22019#issuecomment-2501826875

From ysr at openjdk.org  Wed Nov 27 02:30:01 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 27 Nov 2024 02:30:01 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v6]
In-Reply-To: <0FH7N5Xa3DLDGZcWdxwWQHivNSTxGj9w3p-hWFSgMIs=.654c7bd9-90bb-436d-aedd-08d9fa77efc8@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <-XAjLoN795CLkoG36JK_Xsb64u55hizy5ym3e6qsfVE=.fbbdc54c-e117-4b84-bad1-e9593f2c048c@github.com>
 <PvuT82ARwnCUpMLGeQY-pEqTnlDjF9xG7gfyVFdUTKw=.ede17597-adce-4c49-bcaa-f4d8d3c708b7@github.com>
 <0FH7N5Xa3DLDGZcWdxwWQHivNSTxGj9w3p-hWFSgMIs=.654c7bd9-90bb-436d-aedd-08d9fa77efc8@github.com>
Message-ID: <oDWiBSoQ5bD_GHq4nlUCPAB2hS277o3LijIEeRc8EaM=.ad1fedda-026f-476a-8127-5b57c91eb0b4@github.com>

On Wed, 20 Nov 2024 00:01:49 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 474:
>> 
>>> 472:     // elision safe.
>>> 473:     return;
>>> 474:   }
>> 
>> Why is this safe for Shenandoah? I suspect it needs `CardTableBarrierSet::on_slowpath_allocation_exit` to work. `G1BarrierSetC2` gets it by subclassing `CardTableBarrierSetC2`. But `ShenandoahBarrierSetC2` does not. Should it?
>
> Good question. I'm tracking at https://bugs.openjdk.org/browse/JDK-8344593 , and will follow up there.

A code review indicates that the code is fine, but the comment could be improved. A quick test of performance didn't indicate (with specjbb) any difference in performance (done only for GenShen) but a more complete suite of performance tests is being run to assess the efficacy of the optimization. I'm inclined to leave this in, since eliding card-marks doesn't hurt, even if it doesn't seem to help much either. The comment will be adjusted as part of the above ticket.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21273#discussion_r1859698145

From mdoerr at openjdk.org  Wed Nov 27 09:44:04 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Wed, 27 Nov 2024 09:44:04 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v11]
In-Reply-To: <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
Message-ID: <KmklsHNhqNJjcYWgpt7jZlGLzK7pZpUlKpnqBTGUtdc=.f5647e77-f8e7-40a8-b6d3-05a6aaca07d2@github.com>

On Tue, 26 Nov 2024 00:37:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344985: GenShen: Refactor arraycopy barrier for generational mode
>    
>    Reviewed-by: shade

Thanks! I've run another round of tests on all our platforms and the only failure I have seen is [JDK-8344312](https://bugs.openjdk.org/browse/JDK-8344312). I can do more stress testing by making GenShen the default GC. But that can still be done after integration.
Could you please add the failing test to the problem list to avoid failures in our CI?
gc/shenandoah/oom/TestAllocOutOfMemory.java 8344312 linux-ppc64le

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21273#issuecomment-2503393423

From phh at openjdk.org  Wed Nov 27 23:09:05 2024
From: phh at openjdk.org (Paul Hohensee)
Date: Wed, 27 Nov 2024 23:09:05 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v11]
In-Reply-To: <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <tJ8mTTopxljOg_VuIcZuImWEzMztO2tBCmikbfwKTRo=.7b40cc85-42d1-4242-800c-a0a8ecd9aa4d@github.com>
Message-ID: <tV19LjULHhodaIEbJUyEev4ZzI5wAyyU-f_kVrrglBc=.88750bf3-d2a2-4841-a231-68018d21a4d6@github.com>

On Tue, 26 Nov 2024 00:37:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344985: GenShen: Refactor arraycopy barrier for generational mode
>    
>    Reviewed-by: shade

Marked as reviewed by phh (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2466068316

From wkemper at openjdk.org  Thu Nov 28 14:25:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 28 Nov 2024 14:25:13 GMT
Subject: RFR: Merge openjdk/jdk21u:master
Message-ID: <6Lk7gCLo6ZKKSqz2IUNGHp7hsXvgj9IhofQU3JZr1Mo=.80af9c39-5031-4c7c-a373-0f496766a2c8@github.com>

Merges tag jdk-21.0.6+5

-------------

Commit messages:
 - 8343877: Test AsyncClose.java intermittent fails - Socket.getInputStream().read() wasn't preempted
 - 8338344: Test TestPrivilegedMode.java intermittent fails java.lang.NoClassDefFoundError: jdk/test/lib/Platform
 - 8316893: Compile without -fno-delete-null-pointer-checks
 - 8340552: Harden TzdbZoneRulesCompiler against missing zone names
 - 8316907: Fix nonnull-compare warnings
 - 8343884: [s390x]  Disallow OptoScheduling
 - 8333248: VectorGatherMaskFoldingTest.java failed when maximum vector bits is 64
 - 8319678: Several tests from corelibs areas ignore VM flags
 - 8339637: (tz) Update Timezone Data to 2024b
 - 8328619: sun/management/jmxremote/bootstrap/SSLConfigFilePermissionTest.java failed with BindException: Address already in use
 - ... and 7 more: https://git.openjdk.org/shenandoah-jdk21u/compare/4b1365ea...50aa07ef

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/139/files
  Stats: 1877 lines in 45 files changed: 678 ins; 170 del; 1029 mod
  Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/139.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/139/head:pull/139

PR: https://git.openjdk.org/shenandoah-jdk21u/pull/139

From wkemper at openjdk.org  Fri Nov 29 14:18:20 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 29 Nov 2024 14:18:20 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <LvcuPfQE_I5zLPyT4VqQnEoERmDcgWfb_z4EXbMRZgQ=.22e893b1-b8fb-43ee-b68f-0e9dfd7a6b19@github.com>

Merges tag jdk-24+26

-------------

Commit messages:
 - 8344822: CDS BulkLoaderTest.java#dynamic fails with COH
 - 8344913: Improve -Xlog:cds+map+oop logging for Java mirrors
 - 8342280: Deprecate for removal java.awt.AWTPermission
 - 8345001: java/awt/doc-files/FocusSpec.html has SecurityManager references
 - 8344824: CDS dump crashes when member_method of a lambda proxy is null
 - 8345126: [BACKOUT] JDK-8318127: align_up has potential overflow
 - 8344220: Remove calls to SecurityManager and doPrivileged in java.net.InetAddress and sun.net.util.IPAddressUtil after JEP 486 integration
 - 8345073: Remove SecurityManager checks from sun.awt.image classes
 - 8035271: Incorrect indentation of LineNumberTable/LocalVariableTable/Exception table/LocalVariableTypeTable/StackMapTable/RuntimeVisibleTypeAnnotations in verbose mode
 - 8344882: (bf) Temporary direct buffers should not count against the upper limit on direct buffer memory
 - ... and 140 more: https://git.openjdk.org/shenandoah/compare/4fbf2720...8485cb1c

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/554/files
  Stats: 43775 lines in 857 files changed: 22578 ins; 17018 del; 4179 mod
  Patch: https://git.openjdk.org/shenandoah/pull/554.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/554/head:pull/554

PR: https://git.openjdk.org/shenandoah/pull/554

From wkemper at openjdk.org  Fri Nov 29 16:00:52 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 29 Nov 2024 16:00:52 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v12]
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <jDpTG8MOf0LaMQuuGE2wXRA-713aq37593vrhaXWjss=.8ce7dfa0-aca9-4051-ba3a-0bcd56d0b7dc@github.com>

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 533 commits:

 - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
 - Problem list Shenandoah TestAllocOutOfMemory on ppc64
 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - 8344985: GenShen: Refactor arraycopy barrier for generational mode
   
   Reviewed-by: shade
 - Remove incomplete test case
 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - Merge
 - 8344779: GenShen: Consolidate shared heap region closures into one header
   
   Reviewed-by: kdnilsen, ysr
 - 8344797: GenShen: Update and rename confusing method
   
   Reviewed-by: ysr, kdnilsen
 - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
 - ... and 523 more: https://git.openjdk.org/jdk/compare/a80ccf2c...f29d741c

-------------

Changes: https://git.openjdk.org/jdk/pull/21273/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21273&range=11
  Stats: 22878 lines in 229 files changed: 21122 ins; 913 del; 843 mod
  Patch: https://git.openjdk.org/jdk/pull/21273.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21273/head:pull/21273

PR: https://git.openjdk.org/jdk/pull/21273

From shade at openjdk.org  Fri Nov 29 16:57:03 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 29 Nov 2024 16:57:03 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v12]
In-Reply-To: <jDpTG8MOf0LaMQuuGE2wXRA-713aq37593vrhaXWjss=.8ce7dfa0-aca9-4051-ba3a-0bcd56d0b7dc@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <jDpTG8MOf0LaMQuuGE2wXRA-713aq37593vrhaXWjss=.8ce7dfa0-aca9-4051-ba3a-0bcd56d0b7dc@github.com>
Message-ID: <oNcCeTZ93a32237iHX_qTPgELSRl9MR2YkMW2nHbAOM=.ef440c30-bf05-4f5a-9db6-198d51efbce3@github.com>

On Fri, 29 Nov 2024 16:00:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 533 commits:
> 
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Problem list Shenandoah TestAllocOutOfMemory on ppc64
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344985: GenShen: Refactor arraycopy barrier for generational mode
>    
>    Reviewed-by: shade
>  - Remove incomplete test case
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8344779: GenShen: Consolidate shared heap region closures into one header
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8344797: GenShen: Update and rename confusing method
>    
>    Reviewed-by: ysr, kdnilsen
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - ... and 523 more: https://git.openjdk.org/jdk/compare/a80ccf2c...f29d741c

Marked as reviewed by shade (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2470311607

From rkennke at openjdk.org  Fri Nov 29 17:58:01 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 29 Nov 2024 17:58:01 GMT
Subject: RFR: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental) [v12]
In-Reply-To: <jDpTG8MOf0LaMQuuGE2wXRA-713aq37593vrhaXWjss=.8ce7dfa0-aca9-4051-ba3a-0bcd56d0b7dc@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
 <jDpTG8MOf0LaMQuuGE2wXRA-713aq37593vrhaXWjss=.8ce7dfa0-aca9-4051-ba3a-0bcd56d0b7dc@github.com>
Message-ID: <eQLnJZtaX23fHX24rHl0yPwGVd08MjcIEcKGxMknR7w=.d93fc3fc-8ad0-4ae1-824e-07ef65e70f02@github.com>

On Fri, 29 Nov 2024 16:00:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 533 commits:
> 
>  - Merge remote-tracking branch 'jdk/master' into great-genshen-pr-redux
>  - Problem list Shenandoah TestAllocOutOfMemory on ppc64
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - 8344985: GenShen: Refactor arraycopy barrier for generational mode
>    
>    Reviewed-by: shade
>  - Remove incomplete test case
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - Merge
>  - 8344779: GenShen: Consolidate shared heap region closures into one header
>    
>    Reviewed-by: kdnilsen, ysr
>  - 8344797: GenShen: Update and rename confusing method
>    
>    Reviewed-by: ysr, kdnilsen
>  - Merge remote-tracking branch 'shenandoah/master' into great-genshen-pr-redux
>  - ... and 523 more: https://git.openjdk.org/jdk/compare/a80ccf2c...f29d741c

Marked as reviewed by rkennke (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/21273#pullrequestreview-2470373501

From wkemper at openjdk.org  Fri Nov 29 20:56:04 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 29 Nov 2024 20:56:04 GMT
Subject: Integrated: 8337511: Implement JEP 404: Generational Shenandoah
 (Experimental)
In-Reply-To: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
References: <8N7AiGx8AZc-d6MgBEKVw5R-qk8J_1FBZH-SbzmydGg=.d7ac9a04-5fa3-47e3-8d24-c8efd28a51f7@github.com>
Message-ID: <f-Y7SYTuFD0MWY0skVO3yLgvrnocM67z1qmPWeKQUns=.9744df95-e5a8-4895-aa5f-13b4d3f43ca8@github.com>

On Mon, 30 Sep 2024 22:02:45 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This PR merges JEP 404, a generational mode for the Shenandoah garbage collector. The JEP can be viewed here: https://openjdk.org/jeps/404. We would like to target JDK24 with this PR.

This pull request has now been integrated.

Changeset: 28ae281b
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/28ae281b42cd00f471e275db544a5d23a42df59c
Stats:     22878 lines in 229 files changed: 21122 ins; 913 del; 843 mod

8337511: Implement JEP 404: Generational Shenandoah (Experimental)

Co-authored-by: Kelvin Nilsen <kdnilsen at openjdk.org>
Co-authored-by: Y. Srinivas Ramakrishna <ysr at openjdk.org>
Co-authored-by: Bernd Mathiske <bmathiske at openjdk.org>
Co-authored-by: Martin Doerr <mdoerr at openjdk.org>
Co-authored-by: Fei Yang <fyang at openjdk.org>
Reviewed-by: rkennke, shade, phh

-------------

PR: https://git.openjdk.org/jdk/pull/21273