From ysr at openjdk.org Sat Jun 1 00:00:06 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 1 Jun 2024 00:00:06 GMT
Subject: RFR: 8333005: Deadlock when setting or updating the inline cache
In-Reply-To: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
References: <jhP_p6-WKFgPhOuhFGZUXoAjRQWyNPUK9h_kp0zfxhY=.92572c3b-7885-41ea-95a2-a0ebe8763fd0@github.com>
Message-ID: <oh3eewfHl1PfaJV_9P03oxnG_sNIgQ-YbE4tj30X_I0=.95e46e84-c553-456f-a48c-9008097bb441@github.com>
On Wed, 29 May 2024 07:32:55 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
> In our concurrently class unloading collectors (ZGC, Generational ZGC, Shenandoah), there is a per-nmethod lock. This lock is used to protect the nmethod oops, and is used by nmethod entry barriers and for lazily computing the value of is_unloading, for example. It is also used to protect some other random stuff, and it is also used to protect the state machine of inline caches, which is otherwise completely orthogonal to any of the GC stuff.
>
> Because the lock is used to protect the inline caches (is taken by the CompiledICLocker), you are not allowed to call is_unloading() on *other* nmethods while holding it. Because when we need access to the oops to compute is_unloading, we need to take the nmethod lock. So if two nmethods have inline caches pointing at each other, and calls are resolved at the same time, while concurrent class unloading is going on, we can sometimes get a deadlock.
>
> I accidentally introduced such a bug when I removed the ICStubs (https://bugs.openjdk.org/browse/JDK-8322630), where this has indeed been observed.
>
> The intention with this patch is to make the system less fragile. While it's possible to move around the call to is_unloading() to get rid of the deadlocks, I think I will sleep better at night knowing that you can call is_unloading() anywhere, at least in the shared runtime code, without knowing the GC implementation details. So I'm adding a per-nmethod inline cache lock that protects the completely orthogonal inline cache state for the CompiledICLocker. This way these deadlocks can't happen.
>
> Tested ZGC tests tier1-7, and it looks green. The reproducer that caught the problem, also has stopped reproducing.
Thanks for the fix! Would it be possible to add to the associated ticket a stack retrace of the deadlocked process with the reproducer before the fix, and some information about the reproducer used and any stress flags to induce the deadlock?
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19446#issuecomment-2143131720
From ysr at openjdk.org Mon Jun 3 21:29:08 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 3 Jun 2024 21:29:08 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v13]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <d2B8EkbETYUSbPjAHNMCcubOXYC8EnGkT0gB_x31_J4=.b39ccb48-4073-4d8a-852c-48d0e8463ca4@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
Code called from LRB cannot rely on SGH::_gc_generation and must instead
use the _active_generation field.
Still need to aduit remaining _gc_generation uses for safety.
Removed too strong assert introduced for debugging in previous commit.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/407/files
- new: https://git.openjdk.org/shenandoah/pull/407/files/62934983..b2ff5e43
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=12
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=11-12
Stats: 5 lines in 3 files changed: 1 ins; 2 del; 2 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From wkemper at openjdk.org Mon Jun 3 21:30:18 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 3 Jun 2024 21:30:18 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational code
Message-ID: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
The remembered set is moved into the old generation and the age census is moved into the generational mode heap.
-------------
Commit messages:
- Cannot initialize age census before heap
- Clean up headers
- Merge remote-tracking branch 'shenandoah/master' into syncup-worker-initialization
- Fix zero build
- Move age census out of shHeap.
- Remove some forwarding methods now that includes have been fixed
- Fix initialization order errors
- Move remembered set instance into old generation
- Merge remote-tracking branch 'earthling/fix-isolate-gc-helpers' into syncup-worker-initialization
- Restore cancellation check
- ... and 3 more: https://git.openjdk.org/shenandoah/compare/840c1825...5b47ddb5
Changes: https://git.openjdk.org/shenandoah/pull/443/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=443&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333457
Stats: 284 lines in 28 files changed: 118 ins; 77 del; 89 mod
Patch: https://git.openjdk.org/shenandoah/pull/443.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/443/head:pull/443
PR: https://git.openjdk.org/shenandoah/pull/443
From kdnilsen at openjdk.org Mon Jun 3 23:07:15 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 3 Jun 2024 23:07:15 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code
In-Reply-To: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
Message-ID: <-Mz8rCascg9y7lrKOdG6uY0Gv835Mk9XVf2R94wGX_8=.45ab6443-fed3-4bd0-924e-0c136397a21d@github.com>
On Mon, 3 Jun 2024 21:24:02 GMT, William Kemper <wkemper at openjdk.org> wrote:
> The remembered set is moved into the old generation and the age census is moved into the generational mode heap.
Thanks for this refactoring.
src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 210:
> 208: shenandoah_assert_safepoint();
> 209:
> 210: // TODO: Eventually, we want replace this with a constant-time exchange of pointers.
we want *to* replace. (preexisting typo in comment, might as well fix...)
-------------
Marked as reviewed by kdnilsen (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/443#pullrequestreview-2094990659
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625134464
From wkemper at openjdk.org Mon Jun 3 23:59:49 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 3 Jun 2024 23:59:49 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v2]
In-Reply-To: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
Message-ID: <uktFbkjCr7jo1T-U0PIg3worZN3OgfJyeSRf9oErrwA=.dde90582-00d9-4785-94e2-b8c2ac855312@github.com>
> The remembered set is moved into the old generation and the age census is moved into the generational mode heap.
William Kemper has updated the pull request incrementally with one additional commit since the last revision:
Fix typo in comment
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/443/files
- new: https://git.openjdk.org/shenandoah/pull/443/files/5b47ddb5..954be8eb
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=443&range=01
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=443&range=00-01
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/shenandoah/pull/443.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/443/head:pull/443
PR: https://git.openjdk.org/shenandoah/pull/443
From wkemper at openjdk.org Mon Jun 3 23:59:49 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 3 Jun 2024 23:59:49 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v2]
In-Reply-To: <-Mz8rCascg9y7lrKOdG6uY0Gv835Mk9XVf2R94wGX_8=.45ab6443-fed3-4bd0-924e-0c136397a21d@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
<-Mz8rCascg9y7lrKOdG6uY0Gv835Mk9XVf2R94wGX_8=.45ab6443-fed3-4bd0-924e-0c136397a21d@github.com>
Message-ID: <wP4_PD9ytYjY1dQWua24_juwf7Z6b133yULUJAHG4hc=.dcff665b-2d15-4d88-86fc-9918c2af3353@github.com>
On Mon, 3 Jun 2024 22:45:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Fix typo in comment
>
> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 210:
>
>> 208: shenandoah_assert_safepoint();
>> 209:
>> 210: // TODO: Eventually, we want replace this with a constant-time exchange of pointers.
>
> we want *to* replace. (preexisting typo in comment, might as well fix...)
Done
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625170981
From ysr at openjdk.org Tue Jun 4 01:27:29 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 4 Jun 2024 01:27:29 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v2]
In-Reply-To: <uktFbkjCr7jo1T-U0PIg3worZN3OgfJyeSRf9oErrwA=.dde90582-00d9-4785-94e2-b8c2ac855312@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
<uktFbkjCr7jo1T-U0PIg3worZN3OgfJyeSRf9oErrwA=.dde90582-00d9-4785-94e2-b8c2ac855312@github.com>
Message-ID: <K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
On Mon, 3 Jun 2024 23:59:49 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> The remembered set is moved into the old generation and the age census is moved into the generational mode heap.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>
> Fix typo in comment
I didn't follow all of the logic in ShenandoahMark::mark_through_ref, but otherwise looks good.
Left a few remarks, none of them super-important.
Thanks for making these changes, further separating out the generational code from the non-generational original.
src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 95:
> 93: RememberedScanner* _scanner;
> 94: public:
> 95: ShenandoahMergeWriteTable() : _heap(ShenandoahHeap::heap()), _scanner(_heap->card_scan()) {}
Wow :-)
src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 106:
> 104: };
> 105:
> 106: class ShenandoahCopyWriteCardTableToRead: public ShenandoahHeapRegionClosure {
Provides appropriate gravitas to the operation that was lacking in the old name, which conjured up images of [Scrat](https://en.wikipedia.org/wiki/Scrat) :-)
src/hotspot/share/gc/shenandoah/shenandoahGenerationalEvacuationTask.cpp line 171:
> 169: assert(fill_size >= ShenandoahHeap::min_fill_size(), "previously allocated objects known to be larger than min_size");
> 170: ShenandoahHeap::fill_with_object(obj_addr, fill_size);
> 171: old_gen->card_scan()->register_object_without_lock(obj_addr);
May be just stash the RememberedScanner pointer in a const local, and use it everywhere here.
src/hotspot/share/gc/shenandoah/shenandoahGenerationalEvacuationTask.cpp line 236:
> 234:
> 235: ShenandoahOldGeneration* const old_generation = _heap->old_generation();
> 236: ShenandoahGeneration* const young_generation = _heap->young_generation();
Could rename `old_gen, youn_gen` for brevity like at lines 143,144 above. (But not crucial; ignore if it's too much change for no gain.)
src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.hpp line 50:
> 48: // True when regions and objects should be aged during the current cycle
> 49: ShenandoahSharedFlag _is_aging_cycle;
> 50: // Age census used for adapting tenuring threshold in generational mode
Here and at line 62 below, the "in generational mode" in the documentation comment can probably now be dropped, since we are now part of the generational subtype of Shenandoah heap?
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1:
> 1: /*
Nice cleanups & refactoring!
src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 1:
> 1: /*
Nice cleanup, thank you!!
src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 740:
> 738: void clear_cards_for(ShenandoahHeapRegion* region);
> 739: void mark_card_as_dirty(void* location);
> 740:
Hurray!!
src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.hpp line 32:
> 30: #include "gc/shared/spaceDecorator.hpp"
> 31: #include "gc/shenandoah/shenandoahAffiliation.hpp"
> 32: #include "gc/shenandoah/shenandoahAgeCensus.hpp"
This seemed very curious. Is it just for `CENSUS_NOISE()`? If so, I wonder if we should just pull it up into `shenandoah_globals.hpp` and avoid this seemingly curious include; or may be that's worse than the curious include? :-)
Doesn't need to be done here necessarily, but the question might come up in a review when pushing this upstream in the future.
src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp line 293:
> 291:
> 292: assert((GENERATION == GLOBAL || GENERATION == NON_GEN), "Unexpected generation type");
> 293: return true;
For safety should this return `heap->is_in(obj)`, or if always returning `true` here, should it `assert(heap->is_in(obj), ...);`?
We should document the semantics of both `ShenandoahMark::in_generation()` and `ShenandoahGeneration::contains()` more precsiely than currently done, and audit uses to ensure the callers are cognizant of those sematics. For instance, it sometimes makes sense to have `_or_null` variants of these when a nullptr may be passed to these methods by callers, etc.
src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp line 330:
> 328: // Old mark, found a young pointer.
> 329: // TODO: Rethink this: may be redundant with dirtying of cards identified during young-gen remembered set scanning
> 330: // and by mutator write barriers. Assert
I assume "Assert" here meant: "Assert card is dirty, instead of marking it."
Would be worth doing that. It makes sense that this card dirtying is redundant, since now old mark has been cleared before that needs to be remarked, nor a new cross-gen pointer being created here.
May be we need a ticket to get rid of this separately.
src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 193:
> 191:
> 192: if (ShenandoahCardBarrier) {
> 193: // TODO: Old and young generations should only be instantiated for generational mode
I assume that'll be in a follow-up as discussed at meeting this morning?
src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 786:
> 784: }
> 785:
> 786: void ShenandoahOldGeneration::mark_card_as_dirty(void* location) {
Do we need a paranoid assert here?
assert(is_in(location), ...);
-------------
Marked as reviewed by ysr (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/443#pullrequestreview-2094971710
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625123649
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625124542
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625179047
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625181249
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625168605
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625166926
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625165578
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625165267
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625163101
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625155504
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625213267
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625144445
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1625139784
From varadam at openjdk.org Tue Jun 4 09:19:20 2024
From: varadam at openjdk.org (Varada M)
Date: Tue, 4 Jun 2024 09:19:20 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at most
places
Message-ID: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
-------------
Commit messages:
- [PPC64] saving and restoring CR is not needed at most places
- [PPC64] saving and restoring CR is not needed at most places
Changes: https://git.openjdk.org/jdk/pull/19494/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8331733
Stats: 75 lines in 12 files changed: 13 ins; 3 del; 59 mod
Patch: https://git.openjdk.org/jdk/pull/19494.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19494/head:pull/19494
PR: https://git.openjdk.org/jdk/pull/19494
From mdoerr at openjdk.org Tue Jun 4 09:19:20 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Tue, 4 Jun 2024 09:19:20 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places
In-Reply-To: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
Message-ID: <39aW2Z66k-tf-QGM-sDRe3SmfQOHC-_xTarZ2xdW8W4=.d8b21d43-41ff-4e52-b1df-c69bd892c37f@github.com>
On Fri, 31 May 2024 08:56:36 GMT, Varada M <varadam at openjdk.org> wrote:
> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
src/hotspot/cpu/ppc/gc/shared/barrierSetAssembler_ppc.cpp line 345:
> 343: }
> 344: } else if (vm_reg->is_ConditionRegister()) {
> 345: // NOP. Conditions registers are covered by save_LR
This comment is no longer correct. I don't think that we ever save or restore condition registers at this point. So, I think we can replace this comment by `ShouldNotReachHere(); // live condition registers are unexpected at this point`.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19494#discussion_r1622505227
From varadam at openjdk.org Tue Jun 4 09:19:20 2024
From: varadam at openjdk.org (Varada M)
Date: Tue, 4 Jun 2024 09:19:20 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places
In-Reply-To: <39aW2Z66k-tf-QGM-sDRe3SmfQOHC-_xTarZ2xdW8W4=.d8b21d43-41ff-4e52-b1df-c69bd892c37f@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<39aW2Z66k-tf-QGM-sDRe3SmfQOHC-_xTarZ2xdW8W4=.d8b21d43-41ff-4e52-b1df-c69bd892c37f@github.com>
Message-ID: <iZOuEa4yMCNr-j8rL6s2MwjRqNa_MTQZWDmAPzoFyig=.d9e04634-0719-44bf-89a0-dc6206ee7bcc@github.com>
On Fri, 31 May 2024 14:24:46 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>
> src/hotspot/cpu/ppc/gc/shared/barrierSetAssembler_ppc.cpp line 345:
>
>> 343: }
>> 344: } else if (vm_reg->is_ConditionRegister()) {
>> 345: // NOP. Conditions registers are covered by save_LR
>
> This comment is no longer correct. I don't think that we ever save or restore condition registers at this point. So, I think we can replace this comment by `ShouldNotReachHere(); // live condition registers are unexpected at this point`.
Thanks @TheRealMDoerr . I have changed the comment
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19494#discussion_r1625649264
From mdoerr at openjdk.org Tue Jun 4 16:11:10 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Tue, 4 Jun 2024 16:11:10 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places
In-Reply-To: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
Message-ID: <_oOJA6AJ729zZn5EZpXF-W5h9CfU_IoX5NtY119trrg=.737da5ff-5a8f-42a5-b823-02fb3e000bf4@github.com>
On Fri, 31 May 2024 08:56:36 GMT, Varada M <varadam at openjdk.org> wrote:
> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
> Fastdebug: build and tier1 testing successful. [unrelated failures]
Please make xBarrierSetAssembler_ppc.cpp consistent with the normal BarrierSetAssembler. "// NOP. Conditions registers are covered by save_LR_CR" should get replaced, too.
Saving and restoring CR in `RegisterSaver::push_frame_reg_args_and_save_live_registers` and `RegisterSaver::restore_live_registers_and_pop_frame` are not needed, either.
Otherwise, this PR looks good to me. I'll run tests. (Will retest when you make updates.)
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19494#issuecomment-2147765836
From wkemper at openjdk.org Tue Jun 4 17:04:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 4 Jun 2024 17:04:32 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v2]
In-Reply-To: <K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
<uktFbkjCr7jo1T-U0PIg3worZN3OgfJyeSRf9oErrwA=.dde90582-00d9-4785-94e2-b8c2ac855312@github.com>
<K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
Message-ID: <CKvfdHH9L1GLwZtNSlsZruZuYdJ3fsGpEj6B73Ne9fY=.c9550f24-44cb-42c2-9248-b426a7d452f8@github.com>
On Mon, 3 Jun 2024 23:23:43 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Fix typo in comment
>
> src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp line 293:
>
>> 291:
>> 292: assert((GENERATION == GLOBAL || GENERATION == NON_GEN), "Unexpected generation type");
>> 293: return true;
>
> For safety should this return `heap->is_in(obj)`, or if always returning `true` here, should it `assert(heap->is_in(obj), ...);`?
>
> We should document the semantics of both `ShenandoahMark::in_generation()` and `ShenandoahGeneration::contains()` more precsiely than currently done, and audit uses to ensure the callers are cognizant of those sematics. For instance, it sometimes makes sense to have `_or_null` variants of these when a nullptr may be passed to these methods by callers, etc.
It makes sense to `assert(heap->is_in(obj))` here.
> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 193:
>
>> 191:
>> 192: if (ShenandoahCardBarrier) {
>> 193: // TODO: Old and young generations should only be instantiated for generational mode
>
> I assume that'll be in a follow-up as discussed at meeting this morning?
Yes, the old gen constructor should be able to assert that `ShenandoahCardBarrier` is on. The check is necessary because non-generational modes still instantiate the old generation.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1626342862
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1626341800
From wkemper at openjdk.org Tue Jun 4 17:29:15 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 4 Jun 2024 17:29:15 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v2]
In-Reply-To: <K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
<uktFbkjCr7jo1T-U0PIg3worZN3OgfJyeSRf9oErrwA=.dde90582-00d9-4785-94e2-b8c2ac855312@github.com>
<K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
Message-ID: <QCtT_MO0KMvjBO5IIExB03IkgpZo0QzmvURCPAnXj4I=.89ec1127-bf11-40c2-b9bd-45c5b7ce66dd@github.com>
On Mon, 3 Jun 2024 22:54:30 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Fix typo in comment
>
> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 786:
>
>> 784: }
>> 785:
>> 786: void ShenandoahOldGeneration::mark_card_as_dirty(void* location) {
>
> Do we need a paranoid assert here?
>
>
> assert(is_in(location), ...);
This ultimately bottoms out in `CardTable::index_for`, which does assert that the location is in the heap: https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shared/cardTable.hpp#L160
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1626374415
From wkemper at openjdk.org Tue Jun 4 17:43:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 4 Jun 2024 17:43:27 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v2]
In-Reply-To: <K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
<uktFbkjCr7jo1T-U0PIg3worZN3OgfJyeSRf9oErrwA=.dde90582-00d9-4785-94e2-b8c2ac855312@github.com>
<K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
Message-ID: <O8bQWQ4ZdNvzCPoRUh1TjUWIMGrykoUCGQNDQAy2Lus=.02f7e4eb-a95b-4363-8954-49d3d94c8080@github.com>
On Mon, 3 Jun 2024 23:42:24 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Fix typo in comment
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 740:
>
>> 738: void clear_cards_for(ShenandoahHeapRegion* region);
>> 739: void mark_card_as_dirty(void* location);
>> 740:
>
> Hurray!!
;)
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.hpp line 32:
>
>> 30: #include "gc/shared/spaceDecorator.hpp"
>> 31: #include "gc/shenandoah/shenandoahAffiliation.hpp"
>> 32: #include "gc/shenandoah/shenandoahAgeCensus.hpp"
>
> This seemed very curious. Is it just for `CENSUS_NOISE()`? If so, I wonder if we should just pull it up into `shenandoah_globals.hpp` and avoid this seemingly curious include; or may be that's worse than the curious include? :-)
>
> Doesn't need to be done here necessarily, but the question might come up in a review when pushing this upstream in the future.
Yes, it's necessary for the definition of `CENSUS_NOISE`. I think it's fine to keep it in `shAgeCensus.hpp`. In my mind, `sh_globals.hpp` is really just for command line options.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1626393964
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1626393696
From wkemper at openjdk.org Tue Jun 4 19:03:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 4 Jun 2024 19:03:30 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v3]
In-Reply-To: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
Message-ID: <II-un12xuOzTD1_4tzVygx6H3TPddQNguUr44I6x51Y=.f0742a9d-44d7-4ca8-b6bb-427c328cd7f3@github.com>
> The remembered set is moved into the old generation and the age census is moved into the generational mode heap.
William Kemper has updated the pull request incrementally with one additional commit since the last revision:
Improve variable names, comments and add assertions
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/443/files
- new: https://git.openjdk.org/shenandoah/pull/443/files/954be8eb..2b1d49ca
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=443&range=02
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=443&range=01-02
Stats: 25 lines in 3 files changed: 5 ins; 0 del; 20 mod
Patch: https://git.openjdk.org/shenandoah/pull/443.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/443/head:pull/443
PR: https://git.openjdk.org/shenandoah/pull/443
From wkemper at openjdk.org Tue Jun 4 19:03:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 4 Jun 2024 19:03:31 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v2]
In-Reply-To: <K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
<uktFbkjCr7jo1T-U0PIg3worZN3OgfJyeSRf9oErrwA=.dde90582-00d9-4785-94e2-b8c2ac855312@github.com>
<K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
Message-ID: <78Vnh7iNhuK-jt_WAewHVE3HoV6n0-PHTWCrAnNlzoY=.876ae83e-5215-4693-8287-1c2e84db5d1f@github.com>
On Tue, 4 Jun 2024 01:19:08 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Fix typo in comment
>
> src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp line 330:
>
>> 328: // Old mark, found a young pointer.
>> 329: // TODO: Rethink this: may be redundant with dirtying of cards identified during young-gen remembered set scanning
>> 330: // and by mutator write barriers. Assert
>
> I assume "Assert" here meant: "Assert card is dirty, instead of marking it."
> Would be worth doing that. It makes sense that this card dirtying is redundant, since no old card mark has been cleared that needs to be remarked, nor is a new cross-gen pointer being created here.
>
> May be we need a ticket to get rid of this separately.
This assert passes tier1. I'll run it through the full pipeline and see if it pops. If this code turns out to be redundant, I'd remove it on a different PR.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/443#discussion_r1626479801
From wkemper at openjdk.org Tue Jun 4 19:06:22 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 4 Jun 2024 19:06:22 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v2]
In-Reply-To: <K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
<uktFbkjCr7jo1T-U0PIg3worZN3OgfJyeSRf9oErrwA=.dde90582-00d9-4785-94e2-b8c2ac855312@github.com>
<K4eiQ27Ou5FWZM7Dj2B94fl2qQBQyH9FGOLSRCAeudc=.1c3991c3-fb14-48e1-9cbe-5d945e2dddcf@github.com>
Message-ID: <phbpXfpDHClUAAnZ8ibhwxfwIbH8uNXxf5y36H3x490=.a6905d20-4efd-4b4b-bd70-282ffb230362@github.com>
On Tue, 4 Jun 2024 01:24:17 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Fix typo in comment
>
> I didn't follow all of the logic in ShenandoahMark::mark_through_ref, but otherwise looks good.
>
> Left a few remarks, none of them super-important.
>
> Thanks for making these changes, further separating out the generational code from the non-generational original.
@ysramakrishna , the change for `mark_through_ref` creates a specialization for the NON_GEN case (i.e., single gen modes). It's likely the compiler could generate an optimized function for the NON_GEN template parameter, but I wanted to be able to use the `ShenandoahGenerationalHeap` type here without a runtime check on the mode.
-------------
PR Comment: https://git.openjdk.org/shenandoah/pull/443#issuecomment-2148211732
From wkemper at openjdk.org Tue Jun 4 23:41:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 4 Jun 2024 23:41:40 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code [v4]
In-Reply-To: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
Message-ID: <ZBGqU0OzMIGf2WjVWwwhZv-l3UUh3HPcoJlNenXivps=.d9622737-2f6c-4d28-a9c5-e75efd5ea5f5@github.com>
> The remembered set is moved into the old generation and the age census is moved into the generational mode heap.
William Kemper has updated the pull request incrementally with one additional commit since the last revision:
Disable assertion that old marking only finds young pointers with marked cards
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/443/files
- new: https://git.openjdk.org/shenandoah/pull/443/files/2b1d49ca..08996baa
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=443&range=03
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=443&range=02-03
Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/shenandoah/pull/443.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/443/head:pull/443
PR: https://git.openjdk.org/shenandoah/pull/443
From varadam at openjdk.org Wed Jun 5 07:00:26 2024
From: varadam at openjdk.org (Varada M)
Date: Wed, 5 Jun 2024 07:00:26 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v2]
In-Reply-To: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
Message-ID: <4QAGcf4iBInupqh6dYvZ02y6LzpCMrOKXnca74Uny9A=.c85d59eb-4e2c-4f3d-be00-71b2f4cb3269@github.com>
> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
> Fastdebug: build and tier1 testing successful. [unrelated failures]
Varada M has updated the pull request incrementally with one additional commit since the last revision:
[PPC64] saving and restoring CR is not needed at most places
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19494/files
- new: https://git.openjdk.org/jdk/pull/19494/files/75b9afa9..6dee9281
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=01
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=00-01
Stats: 7 lines in 2 files changed: 0 ins; 5 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19494.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19494/head:pull/19494
PR: https://git.openjdk.org/jdk/pull/19494
From varadam at openjdk.org Wed Jun 5 07:03:57 2024
From: varadam at openjdk.org (Varada M)
Date: Wed, 5 Jun 2024 07:03:57 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places
In-Reply-To: <_oOJA6AJ729zZn5EZpXF-W5h9CfU_IoX5NtY119trrg=.737da5ff-5a8f-42a5-b823-02fb3e000bf4@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<_oOJA6AJ729zZn5EZpXF-W5h9CfU_IoX5NtY119trrg=.737da5ff-5a8f-42a5-b823-02fb3e000bf4@github.com>
Message-ID: <Xu-Jtf_j9v9zvexb-PedokUrM6Tn0vypdSL-DcCXhBI=.5cf13a4e-79ba-4d96-ba4e-6c40bbed850a@github.com>
On Tue, 4 Jun 2024 15:02:43 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Please make xBarrierSetAssembler_ppc.cpp consistent with the normal BarrierSetAssembler. "// NOP. Conditions registers are covered by save_LR_CR" should get replaced, too.
> Saving and restoring CR in `RegisterSaver::push_frame_reg_args_and_save_live_registers` and `RegisterSaver::restore_live_registers_and_pop_frame` are not needed, either.
> Otherwise, this PR looks good to me. I'll run tests. (Will retest when you make updates.)
Thank you @TheRealMDoerr . I have made the changes.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19494#issuecomment-2149030309
From mdoerr at openjdk.org Wed Jun 5 08:24:57 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Wed, 5 Jun 2024 08:24:57 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v2]
In-Reply-To: <4QAGcf4iBInupqh6dYvZ02y6LzpCMrOKXnca74Uny9A=.c85d59eb-4e2c-4f3d-be00-71b2f4cb3269@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<4QAGcf4iBInupqh6dYvZ02y6LzpCMrOKXnca74Uny9A=.c85d59eb-4e2c-4f3d-be00-71b2f4cb3269@github.com>
Message-ID: <WMOMEhg0AxreU9gYRSyvfpgAJ1WzMItXyo2yZhUCjxI=.aefa6ec3-870f-4ce4-b0b4-c610cbe2f2ae@github.com>
On Wed, 5 Jun 2024 07:00:26 GMT, Varada M <varadam at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Varada M has updated the pull request incrementally with one additional commit since the last revision:
>
> [PPC64] saving and restoring CR is not needed at most places
Thanks for the updates! Sorry, my assumption is wrong. I have found a test which runs into the `ShouldNotReachHere()`: jdk/incubator/vector/VectorMaxConversionTests.java#ZGenerational
I think we should revert all changes in `SaveLiveRegisters` (barrierSetAssembler_ppc.cpp) and `XSaveLiveRegisters` (xBarrierSetAssembler_ppc.cpp). We can keep them saving and restoring CR. Sorry for not finding this earlier.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19494#issuecomment-2149186342
From varadam at openjdk.org Wed Jun 5 10:24:23 2024
From: varadam at openjdk.org (Varada M)
Date: Wed, 5 Jun 2024 10:24:23 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v3]
In-Reply-To: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
Message-ID: <yQ0wTnHjUpTOgW5djd6TO-AKQqCxxvS9RgWHzf4fkYI=.8ffbd9cb-6b2d-4855-88b7-9cd02fbef820@github.com>
> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
> Fastdebug: build and tier1 testing successful. [unrelated failures]
Varada M has updated the pull request incrementally with one additional commit since the last revision:
[PPC64] saving and restoring CR is not needed at most places
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19494/files
- new: https://git.openjdk.org/jdk/pull/19494/files/6dee9281..f01de445
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=02
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=01-02
Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod
Patch: https://git.openjdk.org/jdk/pull/19494.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19494/head:pull/19494
PR: https://git.openjdk.org/jdk/pull/19494
From varadam at openjdk.org Wed Jun 5 10:24:23 2024
From: varadam at openjdk.org (Varada M)
Date: Wed, 5 Jun 2024 10:24:23 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v2]
In-Reply-To: <WMOMEhg0AxreU9gYRSyvfpgAJ1WzMItXyo2yZhUCjxI=.aefa6ec3-870f-4ce4-b0b4-c610cbe2f2ae@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<4QAGcf4iBInupqh6dYvZ02y6LzpCMrOKXnca74Uny9A=.c85d59eb-4e2c-4f3d-be00-71b2f4cb3269@github.com>
<WMOMEhg0AxreU9gYRSyvfpgAJ1WzMItXyo2yZhUCjxI=.aefa6ec3-870f-4ce4-b0b4-c610cbe2f2ae@github.com>
Message-ID: <x-Wbg95AatH4RouKlbeavWNdhdaO1hm06H04PH4W3qQ=.dee81872-43fc-4ef4-8362-ee9d011701b5@github.com>
On Wed, 5 Jun 2024 08:22:24 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:
> Thanks for the updates! Sorry, my assumption is wrong. I have found a test which runs into the `ShouldNotReachHere()`: jdk/incubator/vector/VectorMaxConversionTests.java#ZGenerational I think we should revert all changes in `SaveLiveRegisters` (barrierSetAssembler_ppc.cpp) and `XSaveLiveRegisters` (xBarrierSetAssembler_ppc.cpp). We can keep them saving and restoring CR. Sorry for not finding this earlier.
No problem @TheRealMDoerr . Thanks for fix. I have reverted the changes and now the test jdk/incubator/vector/VectorMaxConversionTests.java is passing. I will run tier1 one more time
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19494#issuecomment-2149435569
From mdoerr at openjdk.org Wed Jun 5 10:29:57 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Wed, 5 Jun 2024 10:29:57 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v3]
In-Reply-To: <yQ0wTnHjUpTOgW5djd6TO-AKQqCxxvS9RgWHzf4fkYI=.8ffbd9cb-6b2d-4855-88b7-9cd02fbef820@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<yQ0wTnHjUpTOgW5djd6TO-AKQqCxxvS9RgWHzf4fkYI=.8ffbd9cb-6b2d-4855-88b7-9cd02fbef820@github.com>
Message-ID: <FJRiCoNzu0RxKQGvDVA1zt5U3X-zLHE_Ko18N5WUthg=.a010e029-0314-4bc1-97a5-ed1a39fd91af@github.com>
On Wed, 5 Jun 2024 10:24:23 GMT, Varada M <varadam at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Varada M has updated the pull request incrementally with one additional commit since the last revision:
>
> [PPC64] saving and restoring CR is not needed at most places
Please also revert the two ShouldNotReachHere changes. The old comments are correct again, now. Note that VectorMaxConversionTests.java is in tier3.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19494#issuecomment-2149446914
From varadam at openjdk.org Wed Jun 5 10:35:26 2024
From: varadam at openjdk.org (Varada M)
Date: Wed, 5 Jun 2024 10:35:26 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v4]
In-Reply-To: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
Message-ID: <t2qyl4ZnC0htDXultA4ctwrSFqhGRmroSfVo_uW6PGY=.12801e1b-5f04-40c8-ae56-91a1f0e3e0b0@github.com>
> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
> Fastdebug: build and tier1 testing successful. [unrelated failures]
Varada M has updated the pull request incrementally with one additional commit since the last revision:
[PPC64] saving and restoring CR is not needed at most places
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19494/files
- new: https://git.openjdk.org/jdk/pull/19494/files/f01de445..3445e9ac
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=03
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=02-03
Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19494.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19494/head:pull/19494
PR: https://git.openjdk.org/jdk/pull/19494
From mdoerr at openjdk.org Wed Jun 5 10:41:05 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Wed, 5 Jun 2024 10:41:05 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v4]
In-Reply-To: <t2qyl4ZnC0htDXultA4ctwrSFqhGRmroSfVo_uW6PGY=.12801e1b-5f04-40c8-ae56-91a1f0e3e0b0@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<t2qyl4ZnC0htDXultA4ctwrSFqhGRmroSfVo_uW6PGY=.12801e1b-5f04-40c8-ae56-91a1f0e3e0b0@github.com>
Message-ID: <CZjDNa8QFCzMJoZXexurSVxWaL0cZj8ZO_h1KGAJI98=.d1c55db7-ea7a-4b09-b640-2aa40387b48e@github.com>
On Wed, 5 Jun 2024 10:35:26 GMT, Varada M <varadam at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Varada M has updated the pull request incrementally with one additional commit since the last revision:
>
> [PPC64] saving and restoring CR is not needed at most places
Looks good, now. Thanks. I'll rerun the tests.
-------------
PR Review: https://git.openjdk.org/jdk/pull/19494#pullrequestreview-2098741697
From amitkumar at openjdk.org Wed Jun 5 13:41:03 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Wed, 5 Jun 2024 13:41:03 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v4]
In-Reply-To: <t2qyl4ZnC0htDXultA4ctwrSFqhGRmroSfVo_uW6PGY=.12801e1b-5f04-40c8-ae56-91a1f0e3e0b0@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<t2qyl4ZnC0htDXultA4ctwrSFqhGRmroSfVo_uW6PGY=.12801e1b-5f04-40c8-ae56-91a1f0e3e0b0@github.com>
Message-ID: <Cum98KWK6jTHAygRJ8gEz8NiiEEYBc8Wu5fCIG0_Rgs=.058240e5-f8b7-4f12-80a5-075ba727b766@github.com>
On Wed, 5 Jun 2024 10:35:26 GMT, Varada M <varadam at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Varada M has updated the pull request incrementally with one additional commit since the last revision:
>
> [PPC64] saving and restoring CR is not needed at most places
Please update the copyright headers as well.
-------------
Changes requested by amitkumar (Committer).
PR Review: https://git.openjdk.org/jdk/pull/19494#pullrequestreview-2099238884
From wkemper at openjdk.org Wed Jun 5 15:48:22 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 5 Jun 2024 15:48:22 GMT
Subject: Integrated: 8333457: GenShen: Move remembered set into new
generational code
In-Reply-To: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
References: <049l2LJB2s-M9ByCgH_9OWGQabx98acUNtTN4ilIg5k=.f66e6c0a-357d-4ddd-9df4-b48510e8f809@github.com>
Message-ID: <dCcuyOmmUMCH8rXVfpmzClp4HOWD_ogMjLnpooxbV1Y=.5188cdeb-27e0-4f85-b133-0471c30eb642@github.com>
On Mon, 3 Jun 2024 21:24:02 GMT, William Kemper <wkemper at openjdk.org> wrote:
> The remembered set is moved into the old generation and the age census is moved into the generational mode heap.
This pull request has now been integrated.
Changeset: a32f36bb
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/a32f36bbbc4fc2a36872f3a0aaec68950bfc6cf6
Stats: 300 lines in 28 files changed: 124 ins; 77 del; 99 mod
8333457: GenShen: Move remembered set into new generational code
Reviewed-by: kdnilsen, ysr
-------------
PR: https://git.openjdk.org/shenandoah/pull/443
From ysr at openjdk.org Wed Jun 5 18:21:20 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 5 Jun 2024 18:21:20 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v14]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <ssdFEEMOAeEwQtzvJntRxEQm8BezYU80Sgtb9fMpVb8=.8b0f38c4-7456-4677-b859-f410f14cc142@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits:
- Merge branch 'master' into active_generation
- Code called from LRB cannot rely on SGH::_gc_generation and must instead
use the _active_generation field.
Still need to aduit remaining _gc_generation uses for safety.
Removed too strong assert introduced for debugging in previous commit.
- Merge branch 'master' into active_generation
- Some asserts to catch a tricky race. These assertions may be too strong
in general but would help with debugging a rare crash.
- Merge branch 'master' into active_generation
- Remove vestigial thread() method in ShenandoahController.
- cosmetic
- jcheck white-space
- Remove static _thread field introduced in ShenandoahController, as its
role is already served by the _control_thread field in ShenandoahHeap.
- Macro-ize previously inlined shenandoah assert.
- ... and 34 more: https://git.openjdk.org/shenandoah/compare/a32f36bb...4e6a61ee
-------------
Changes: https://git.openjdk.org/shenandoah/pull/407/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=13
Stats: 168 lines in 17 files changed: 134 ins; 1 del; 33 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From duke at openjdk.org Wed Jun 5 20:45:22 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Wed, 5 Jun 2024 20:45:22 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v4]
In-Reply-To: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
Message-ID: <aumjLSAUm1CNEhg3JWjsPPQt4TlWISRK2cer4oyYspU=.2305c90a-57f6-421c-8463-b0fc818c6eca@github.com>
> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
>
> **Verification**
>
> Shenendoah
>
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>
> ZGC
>
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>
> **Issue**
> https://bugs.openjdk.org/browse/JDK-8331911
Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
8331911: Remove is_armed check from callers
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19285/files
- new: https://git.openjdk.org/jdk/pull/19285/files/c59034da..d474c1b7
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19285&range=03
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19285&range=02-03
Stats: 13 lines in 2 files changed: 9 ins; 2 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19285.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19285/head:pull/19285
PR: https://git.openjdk.org/jdk/pull/19285
From duke at openjdk.org Wed Jun 5 21:04:06 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Wed, 5 Jun 2024 21:04:06 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v3]
In-Reply-To: <SM6Tj6DXxNSXZbkM-daZA4gB8baJiPtwcua4JKOcaUg=.2bd3dffb-143c-4dc5-b0ef-166536c0d848@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
<uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>
<-GBU6zDyLGsnEyT3yEMfW_TnA-stpR1tLeqfvfeVcQ0=.c4b13824-35a4-4e22-8132-721a668a0425@github.com>
<SM6Tj6DXxNSXZbkM-daZA4gB8baJiPtwcua4JKOcaUg=.2bd3dffb-143c-4dc5-b0ef-166536c0d848@github.com>
Message-ID: <Q9HzKTdYmzEx4oaMVlpIsPRZZegwIwmr4f461IepiaA=.a39f08a9-dc7c-4927-92ea-6bbb95bcc964@github.com>
On Thu, 30 May 2024 16:04:04 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
>>> It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.
>>
>> Thanks for the feedback! Looking around the code, I think there are a few places where we can do more changes.
>>
>> First, remove check here:
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/code/nmethod.cpp#L852-L855
>>
>> This would force us to add the check in super-class implementation here:
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L83
>>
>> Second, we can remove the check here:
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L175-L181
>>
>> But it does not seem straightforward, because we currently skip cross-modification fence based on?is_armed(...)?check. Unfortunately, we cannot easily know if?nmethod_entry_barrier?acted or not, we only know if method is safe or not.? Can we / should we do these refactoring separately?
>
>> > It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.
>>
>>
>>
>> Thanks for the feedback! Looking around the code, I think there are a few places where we can do more changes.
>>
>>
>>
>> First, remove check here:
>>
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/code/nmethod.cpp#L852-L855
>>
>>
>>
>> This would force us to add the check in super-class implementation here:
>>
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L83
>>
>>
>>
>> Second, we can remove the check here:
>>
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L175-L181
>>
>>
>>
>> But it does not seem straightforward, because we currently skip cross-modification fence based on?is_armed(...)?check. Unfortunately, we cannot easily know if?nmethod_entry_barrier?acted or not, we only know if method is safe or not.? Can we / should we do these refactoring separately?
>
> I see your point. However, this PR is refactoring the code to iron out who is responsible for checking is_armed, so I would prefer if we got that right in this PR. We say it should be the backend code doing that, so the callers shouldn't. I agree with all the changes you just listed and if you make them I would be happy.
>
> Regarding the cross modifying fence, I strongly prefer to not try and be clever. Just run the cross modifying fence unconditionally after calling the backend code. We get there because the barrier was armed anyway.
@fisk
I've addressed the feedback. Can you take a look?
I did not remove the check [here](https://github.com/openjdk/jdk/blob/5dcb7a627e1cfb360719a25722588180e5de9d09/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L175-L177). Removing this check resulted in time out when `-XX:+DeoptimizeNMethodBarriersALot` flag set as it [executes deoptimization code path](https://github.com/openjdk/jdk/blob/5dcb7a627e1cfb360719a25722588180e5de9d09/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L200-L203)
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2150955996
From eosterlund at openjdk.org Thu Jun 6 10:52:46 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Thu, 6 Jun 2024 10:52:46 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v4]
In-Reply-To: <aumjLSAUm1CNEhg3JWjsPPQt4TlWISRK2cer4oyYspU=.2305c90a-57f6-421c-8463-b0fc818c6eca@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<aumjLSAUm1CNEhg3JWjsPPQt4TlWISRK2cer4oyYspU=.2305c90a-57f6-421c-8463-b0fc818c6eca@github.com>
Message-ID: <O00aYc40vpwu8dO-_v4rroH68mQ94h3iiboY743nMTE=.aed856f7-7a6a-424f-a82d-f0612f2346ee@github.com>
On Wed, 5 Jun 2024 20:45:22 GMT, Neethu Prasad <duke at openjdk.org> wrote:
>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>>
>> **Verification**
>>
>> Shenendoah
>>
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>>
>> ZGC
>>
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>>
>> **Issue**
>> https://bugs.openjdk.org/browse/JDK-8331911
>
> Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
>
> 8331911: Remove is_armed check from callers
Changes requested by eosterlund (Reviewer).
src/hotspot/share/gc/shared/barrierSetNMethod.cpp line 184:
> 182: // too often. nmethod_entry_barrier checks for disarmed status itself,
> 183: // but we have no visibility into whether the barrier acted or not.
> 184: if (!bs_nm->is_armed(nm)) {
I would still like this check gone, together with the comment above.
-------------
PR Review: https://git.openjdk.org/jdk/pull/19285#pullrequestreview-2101609057
PR Review Comment: https://git.openjdk.org/jdk/pull/19285#discussion_r1629278896
From mdoerr at openjdk.org Thu Jun 6 10:59:46 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Thu, 6 Jun 2024 10:59:46 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v4]
In-Reply-To: <t2qyl4ZnC0htDXultA4ctwrSFqhGRmroSfVo_uW6PGY=.12801e1b-5f04-40c8-ae56-91a1f0e3e0b0@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<t2qyl4ZnC0htDXultA4ctwrSFqhGRmroSfVo_uW6PGY=.12801e1b-5f04-40c8-ae56-91a1f0e3e0b0@github.com>
Message-ID: <JOz39r3igKRRspDcs6ygwaYYGefWIDs7KjTMbhGLeY8=.c2c6147a-2992-4c99-b30e-a337fa421ff0@github.com>
On Wed, 5 Jun 2024 10:35:26 GMT, Varada M <varadam at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Varada M has updated the pull request incrementally with one additional commit since the last revision:
>
> [PPC64] saving and restoring CR is not needed at most places
The tests have passed on our side. I can approve it once the Copyright years are updated.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19494#issuecomment-2152013268
From shade at openjdk.org Thu Jun 6 11:06:46 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 6 Jun 2024 11:06:46 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v4]
In-Reply-To: <O00aYc40vpwu8dO-_v4rroH68mQ94h3iiboY743nMTE=.aed856f7-7a6a-424f-a82d-f0612f2346ee@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<aumjLSAUm1CNEhg3JWjsPPQt4TlWISRK2cer4oyYspU=.2305c90a-57f6-421c-8463-b0fc818c6eca@github.com>
<O00aYc40vpwu8dO-_v4rroH68mQ94h3iiboY743nMTE=.aed856f7-7a6a-424f-a82d-f0612f2346ee@github.com>
Message-ID: <BmxAP8Q5rxl7zlcgc_OQXqS-WbZG9I9fCUjkWpdFqYg=.63f8b03a-74d1-4ac8-bdec-306c206a63b9@github.com>
On Thu, 6 Jun 2024 10:49:32 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
>> Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
>>
>> 8331911: Remove is_armed check from callers
>
> src/hotspot/share/gc/shared/barrierSetNMethod.cpp line 184:
>
>> 182: // too often. nmethod_entry_barrier checks for disarmed status itself,
>> 183: // but we have no visibility into whether the barrier acted or not.
>> 184: if (!bs_nm->is_armed(nm)) {
>
> I would still like this check gone, together with the comment above.
I agree with Neethu, though: I think we should proceed to the `DeoptimizeNMethodBarrierALot` block and cross-modify fences only when barrier acted. We know from testing that it hurts otherwise. I would prefer this change not to introduce new performance potholes, even for fastdebug modes.
If the argument is cleanliness on who is checking "armed", and that we decide it should be solely in backend, then the middle ground might be adding the out-parameter, like `nmethod_entry_barrier(nmethod* nm, bool has_acted)`, and checking that before proceeding here? That feels uglier than just leaving the check here.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19285#discussion_r1629299546
From varadam at openjdk.org Thu Jun 6 11:24:02 2024
From: varadam at openjdk.org (Varada M)
Date: Thu, 6 Jun 2024 11:24:02 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v5]
In-Reply-To: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
Message-ID: <QVQ-I14eNw64efIFubZPee_82T07QppA-fsQEvzbEYE=.63f3b5d9-1924-46c8-817e-106d05297f67@github.com>
> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
> Fastdebug: build and tier1 testing successful. [unrelated failures]
Varada M has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
- Merge branch 'master' into LR
- [PPC64] saving and restoring CR is not needed at most places
- [PPC64] saving and restoring CR is not needed at most places
- [PPC64] saving and restoring CR is not needed at most places
- [PPC64] saving and restoring CR is not needed at most places
- [PPC64] saving and restoring CR is not needed at most places
- [PPC64] saving and restoring CR is not needed at most places
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19494/files
- new: https://git.openjdk.org/jdk/pull/19494/files/3445e9ac..5938878b
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=04
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19494&range=03-04
Stats: 90827 lines in 1720 files changed: 65308 ins; 17165 del; 8354 mod
Patch: https://git.openjdk.org/jdk/pull/19494.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19494/head:pull/19494
PR: https://git.openjdk.org/jdk/pull/19494
From varadam at openjdk.org Thu Jun 6 11:24:02 2024
From: varadam at openjdk.org (Varada M)
Date: Thu, 6 Jun 2024 11:24:02 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v4]
In-Reply-To: <JOz39r3igKRRspDcs6ygwaYYGefWIDs7KjTMbhGLeY8=.c2c6147a-2992-4c99-b30e-a337fa421ff0@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<t2qyl4ZnC0htDXultA4ctwrSFqhGRmroSfVo_uW6PGY=.12801e1b-5f04-40c8-ae56-91a1f0e3e0b0@github.com>
<JOz39r3igKRRspDcs6ygwaYYGefWIDs7KjTMbhGLeY8=.c2c6147a-2992-4c99-b30e-a337fa421ff0@github.com>
Message-ID: <Bx9oJ0SEgGHr9g9YsZVw6fdWf7SPPxnLggclxhYHEqg=.0b3f0ea5-87fd-4408-a1a4-687ae384b00a@github.com>
On Thu, 6 Jun 2024 10:56:52 GMT, Martin Doerr <mdoerr at openjdk.org> wrote:
>> Varada M has updated the pull request incrementally with one additional commit since the last revision:
>>
>> [PPC64] saving and restoring CR is not needed at most places
>
> The tests have passed on our side. I can approve it once the Copyright years are updated.
Thanks @TheRealMDoerr for running the tests. Only tier1 is completed from my side which looks fine. @offamitkumar I have updated the copyright headers. Thank you
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19494#issuecomment-2152096579
From eosterlund at openjdk.org Thu Jun 6 11:46:47 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Thu, 6 Jun 2024 11:46:47 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v3]
In-Reply-To: <SM6Tj6DXxNSXZbkM-daZA4gB8baJiPtwcua4JKOcaUg=.2bd3dffb-143c-4dc5-b0ef-166536c0d848@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
<uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>
<-GBU6zDyLGsnEyT3yEMfW_TnA-stpR1tLeqfvfeVcQ0=.c4b13824-35a4-4e22-8132-721a668a0425@github.com>
<SM6Tj6DXxNSXZbkM-daZA4gB8baJiPtwcua4JKOcaUg=.2bd3dffb-143c-4dc5-b0ef-166536c0d848@github.com>
Message-ID: <Q0Pyyx9YaLS55a7KmCZX6tEA5arxJ0qxVKXW1VAPWKE=.d09724f3-796b-4d20-bfc4-f39a1deb35ea@github.com>
On Thu, 30 May 2024 16:04:04 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
>>> It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.
>>
>> Thanks for the feedback! Looking around the code, I think there are a few places where we can do more changes.
>>
>> First, remove check here:
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/code/nmethod.cpp#L852-L855
>>
>> This would force us to add the check in super-class implementation here:
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L83
>>
>> Second, we can remove the check here:
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L175-L181
>>
>> But it does not seem straightforward, because we currently skip cross-modification fence based on?is_armed(...)?check. Unfortunately, we cannot easily know if?nmethod_entry_barrier?acted or not, we only know if method is safe or not.? Can we / should we do these refactoring separately?
>
>> > It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.
>>
>>
>>
>> Thanks for the feedback! Looking around the code, I think there are a few places where we can do more changes.
>>
>>
>>
>> First, remove check here:
>>
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/code/nmethod.cpp#L852-L855
>>
>>
>>
>> This would force us to add the check in super-class implementation here:
>>
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L83
>>
>>
>>
>> Second, we can remove the check here:
>>
>> https://github.com/openjdk/jdk/blob/bc7d9e3d0bc663bbbeb068889082da4a9f0fa8de/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L175-L181
>>
>>
>>
>> But it does not seem straightforward, because we currently skip cross-modification fence based on?is_armed(...)?check. Unfortunately, we cannot easily know if?nmethod_entry_barrier?acted or not, we only know if method is safe or not.? Can we / should we do these refactoring separately?
>
> I see your point. However, this PR is refactoring the code to iron out who is responsible for checking is_armed, so I would prefer if we got that right in this PR. We say it should be the backend code doing that, so the callers shouldn't. I agree with all the changes you just listed and if you make them I would be happy.
>
> Regarding the cross modifying fence, I strongly prefer to not try and be clever. Just run the cross modifying fence unconditionally after calling the backend code. We get there because the barrier was armed anyway.
> @fisk
>
> I've addressed the feedback. Can you take a look?
>
> I did not remove the check [here](https://github.com/openjdk/jdk/blob/5dcb7a627e1cfb360719a25722588180e5de9d09/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L175-L177). Removing this check resulted in time out when `-XX:+DeoptimizeNMethodBarriersALot` flag set as it [executes deoptimization code path](https://github.com/openjdk/jdk/blob/5dcb7a627e1cfb360719a25722588180e5de9d09/src/hotspot/share/gc/shared/barrierSetNMethod.cpp#L200-L203)
I am quite nervous about having that silly optimization there. So I'm going to have to insist on removing it. Perhaps though, it should be fixed as a separate issue. Allow me to explain myself why it makes me nervous.
The nmethod entry barriers guard modifications to instructions and data through a mixed bag of synchronous and asynchronous cross modifying code.
Synchronous cross modifying code is the sane thing to do; we modify instructions, guarded from concurrent execution by a data flag. After modifying the instructions, the data flag is flipped, and observers are allowed to execute the instructions *after executing an instruction cross modification fence*.
On for example AArch64 we only perform synchronous cross modifying code, and the stub has the fencing machinery, so it should be okay.
However, on x86_64, we perform a mix of asynchronous and synchronous cross modifying code. The guard word is the immediate part of a compare instruction. If the new disarmed immediate is observed by concurrent execution, instruction cache coherency guarantees that we will correctly observe the cross modified instructions when they are subsequently executed.
However, when we go into the stub slow path, and check is_armed etc, these are *data reads*. That makes the dance entirely different as it suddenly performs synchronous cross modifying code. If a data read observes that the instructions have been modified, we don't have the same level of guarantees any longer, unless we perform an instruction cross modification fence.
So my concern here, is that the silly optimization to fix some verification code timeout or whatever, is in fact causing a real correctness problem for real release builds, on x86_64. By skipping the cross modification fence we perform an incomplete synchronous instruction cross modification dance that isn't sound.
Having said that, perhaps we should file a separate issue to remove that check, since it seems to fix an actual bug, while I guess this was meant more as an optimization. What do you think?
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2152174420
From amitkumar at openjdk.org Thu Jun 6 12:10:46 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Thu, 6 Jun 2024 12:10:46 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v5]
In-Reply-To: <QVQ-I14eNw64efIFubZPee_82T07QppA-fsQEvzbEYE=.63f3b5d9-1924-46c8-817e-106d05297f67@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<QVQ-I14eNw64efIFubZPee_82T07QppA-fsQEvzbEYE=.63f3b5d9-1924-46c8-817e-106d05297f67@github.com>
Message-ID: <Oafpx2cVufuiFwOeACT-nbnZ_GKfjW9g2cp4fALbdHo=.836ac555-707d-4e2f-9970-efa9b44b997d@github.com>
On Thu, 6 Jun 2024 11:24:02 GMT, Varada M <varadam at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Varada M has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
>
> - Merge branch 'master' into LR
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
Looks good to me.
I ran tier1 & vector test on `ppc-le` and these are the results:
failures in tier1 (looks unrelated to this PR):
gtest/GTestWrapper.java
java/util/ResourceBundle/Control/MissingResourceCauseTestRun.java
jdk/javadoc/doclet/testIOException/TestIOException.java
Vector tests:
Test summary
==============================
TEST TOTAL PASS FAIL ERROR
jtreg:./test/jdk/jdk/incubator/vector 78 78 0 0
==============================
TEST SUCCESS
``
-------------
Marked as reviewed by amitkumar (Committer).
PR Review: https://git.openjdk.org/jdk/pull/19494#pullrequestreview-2101765929
From mdoerr at openjdk.org Thu Jun 6 12:24:47 2024
From: mdoerr at openjdk.org (Martin Doerr)
Date: Thu, 6 Jun 2024 12:24:47 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v5]
In-Reply-To: <QVQ-I14eNw64efIFubZPee_82T07QppA-fsQEvzbEYE=.63f3b5d9-1924-46c8-817e-106d05297f67@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<QVQ-I14eNw64efIFubZPee_82T07QppA-fsQEvzbEYE=.63f3b5d9-1924-46c8-817e-106d05297f67@github.com>
Message-ID: <TJyPAb1jkG_J-xwTiix_IKYztUE2nT1KFpfqRKLdUgM=.928b5565-7ebf-45c4-b343-0a8b3b8122ea@github.com>
On Thu, 6 Jun 2024 11:24:02 GMT, Varada M <varadam at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Varada M has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
>
> - Merge branch 'master' into LR
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
Marked as reviewed by mdoerr (Reviewer).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19494#pullrequestreview-2101793809
From shade at openjdk.org Thu Jun 6 13:57:46 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 6 Jun 2024 13:57:46 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v3]
In-Reply-To: <Q0Pyyx9YaLS55a7KmCZX6tEA5arxJ0qxVKXW1VAPWKE=.d09724f3-796b-4d20-bfc4-f39a1deb35ea@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
<uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>
<-GBU6zDyLGsnEyT3yEMfW_TnA-stpR1tLeqfvfeVcQ0=.c4b13824-35a4-4e22-8132-721a668a0425@github.com>
<SM6Tj6DXxNSXZbkM-daZA4gB8baJiPtwcua4JKOcaUg=.2bd3dffb-143c-4dc5-b0ef-166536c0d848@github.com>
<Q0Pyyx9YaLS55a7KmCZX6tEA5arxJ0qxVKXW1VAPWKE=.d09724f3-796b-4d20-bfc4-f39a1deb35ea@github.com>
Message-ID: <YsEp4fgCBxneEuJnfidh98QfR0LAUV_H6GVN6PzPqys=.bf6c999d-4cab-4176-be79-10f23d385a7d@github.com>
On Thu, 6 Jun 2024 11:44:08 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
> Having said that, perhaps we should file a separate issue to remove that check, since it seems to fix an actual bug, while I guess this was meant more as an optimization. What do you think?
FTR, I don't mind executing cross-modify-fence unconditionally. I do mind going into deopts too often. I do also think that we want to stay on performance-positive side for at least an easy variant of fix, and do potentially regressing things separately. The initial motivation for this work was to resolve an issue in a service workload that runs many threads with similar stacks, and get something that we are sure about for a prompt backport.
To that end, we can continue working out the final shape of the patch here, while we mitigate our current service problems with picking up a limited version of this patch with [JDK-8333716](https://bugs.openjdk.org/browse/JDK-8333716) -- it resolves only Shenandoah parts of it, though. Or, we can integrate this patch in its current form, resolving the issue on both Shenandoah and ZGC paths, and work out the check removal as the follow up of [JDK-8310239](https://bugs.openjdk.org/browse/JDK-8310239).
I think the latter alternative is more pragmatic.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2152603188
From wkemper at openjdk.org Thu Jun 6 18:55:17 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 6 Jun 2024 18:55:17 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode
Message-ID: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
Other modes should not instantiate or touch young/old generations.
-------------
Commit messages:
- Add missing precompiled header
- Fix merge error
- Better assert messages for inappropriate generation access
- Merge branch 'shenandoah/master' into move-young-old-generations
- Back out generation getter/helpers (members will stay in sh heap)
- Factor region age management into gen heap, remove duplication
- Move generation size and generation initialization out of shHeap
- Fix copy/pasta in ShenandoahGenerationalMemoryPool
- Simplify generational memory pools
- Merge branch 'syncup-worker-initialization' into move-young-old-generations
- ... and 20 more: https://git.openjdk.org/shenandoah/compare/a32f36bb...5e744806
Changes: https://git.openjdk.org/shenandoah/pull/444/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=444&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333750
Stats: 881 lines in 20 files changed: 469 ins; 318 del; 94 mod
Patch: https://git.openjdk.org/shenandoah/pull/444.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/444/head:pull/444
PR: https://git.openjdk.org/shenandoah/pull/444
From cslucas at openjdk.org Thu Jun 6 19:15:55 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 6 Jun 2024 19:15:55 GMT
Subject: RFR: 8333566: Remove unused methods
Message-ID: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
-------------
Commit messages:
- Temove trailing whitespace.
- Removing unused methods in aarch64
- Merge remote-tracking branch 'origin/main' into unused-methods
- Merge remote-tracking branch 'origin/main' into unused-methods
- Remove defined but unused methods.
Changes: https://git.openjdk.org/jdk/pull/19550/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19550&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333566
Stats: 1398 lines in 139 files changed: 1 ins; 1290 del; 107 mod
Patch: https://git.openjdk.org/jdk/pull/19550.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19550/head:pull/19550
PR: https://git.openjdk.org/jdk/pull/19550
From amitkumar at openjdk.org Thu Jun 6 19:15:56 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Thu, 6 Jun 2024 19:15:56 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <jqtn_6blyyHKNx57BY-D9lHTtIFAyGhkWVcKWdAc_WI=.05e794c0-3bc7-4a09-8393-63298ed8b99b@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
src/hotspot/cpu/s390/vm_version_s390.hpp line 516:
> 514: static void set_has_CompareTrap() { _features[0] |= GnrlInstrExtFacilityMask; }
> 515: static void set_has_RelativeLoadStore() { _features[0] |= GnrlInstrExtFacilityMask; }
> 516: static void set_has_GnrlInstrExtensions() { _features[0] |= GnrlInstrExtFacilityMask; }
I know this PR is still in draft state. Just a thought: I would like to keep the methods in `vm_version_s390.hpp` file for now. I'm planning to remove the checks applicable to older hardware. So it would be better, If I clean these methods as a part of that PR :-)
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19550#discussion_r1628627936
From cslucas at openjdk.org Thu Jun 6 19:15:56 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 6 Jun 2024 19:15:56 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <jqtn_6blyyHKNx57BY-D9lHTtIFAyGhkWVcKWdAc_WI=.05e794c0-3bc7-4a09-8393-63298ed8b99b@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
<jqtn_6blyyHKNx57BY-D9lHTtIFAyGhkWVcKWdAc_WI=.05e794c0-3bc7-4a09-8393-63298ed8b99b@github.com>
Message-ID: <_Nv0J1Vf3F9z9S9N-eKLMy0JOOt9KHBa7l6D7OepBjc=.9a24926d-33ec-469d-bdee-5c0efed17fb5@github.com>
On Thu, 6 Jun 2024 01:28:00 GMT, Amit Kumar <amitkumar at openjdk.org> wrote:
>> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>>
>> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>>
>> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
>
> src/hotspot/cpu/s390/vm_version_s390.hpp line 516:
>
>> 514: static void set_has_CompareTrap() { _features[0] |= GnrlInstrExtFacilityMask; }
>> 515: static void set_has_RelativeLoadStore() { _features[0] |= GnrlInstrExtFacilityMask; }
>> 516: static void set_has_GnrlInstrExtensions() { _features[0] |= GnrlInstrExtFacilityMask; }
>
> I know this PR is still in draft state. Just a thought: I would like to keep the methods in `vm_version_s390.hpp` file for now. I'm planning to remove the checks applicable to older hardware. So it would be better, If I clean these methods as a part of that PR :-)
Sounds good to me!
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19550#discussion_r1629762442
From amitkumar at openjdk.org Thu Jun 6 19:23:46 2024
From: amitkumar at openjdk.org (Amit Kumar)
Date: Thu, 6 Jun 2024 19:23:46 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <Pg1shNNmDwZmFWEYewuUkQnOz5T1FrtQC-J955qdaDw=.672adbdb-d67c-4d08-a8e8-4ef783f76c3e@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
src/hotspot/cpu/s390/vm_version_s390.hpp line 516:
> 514: static void set_has_CompareTrap() { _features[0] |= GnrlInstrExtFacilityMask; }
> 515: static void set_has_RelativeLoadStore() { _features[0] |= GnrlInstrExtFacilityMask; }
> 516: static void set_has_ProcessorAssist() { _features[0] |= ProcessorAssistMask; }
This looks incorrect; there exist a second definition below;
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19550#discussion_r1630102121
From kvn at openjdk.org Thu Jun 6 19:31:48 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 6 Jun 2024 19:31:48 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <wGHuC6LZrSmqMhq5Vc9lU-k_92ZHQ-ysME0Y9yqYpCM=.8b5d8e50-a8b4-42fd-8d23-9ab81a541aeb@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
src/hotspot/cpu/x86/macroAssembler_x86.hpp line 992:
> 990: // * No condition for this * void ALWAYSINLINE jecxz(Label& L, bool maybe_short = true) { jcc(Assembler::cxz, L, maybe_short); }
> 991:
> 992: // Short versions of the above
These all branch instructions were added recently [#18893](https://github.com/openjdk/jdk/pull/18893) for JDK-8320448 which is not pushed yet. So I will suggest to not remove them.
src/hotspot/cpu/x86/vm_version_x86.hpp line 666:
> 664: // Feature identification which can be affected by VM settings
> 665: //
> 666: static bool supports_cpuid() { return _features != 0; }
I suggest to not touch this file. Some CPU features could used in a future.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19550#discussion_r1630110159
PR Review Comment: https://git.openjdk.org/jdk/pull/19550#discussion_r1630112304
From wkemper at openjdk.org Thu Jun 6 21:12:50 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 6 Jun 2024 21:12:50 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v2]
In-Reply-To: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
Message-ID: <d3TgxylixrAMhUOFOrSWqWma--uhZL1Vfyy_Jl-GOPw=.c894beb0-683b-4fee-94bd-7121cdc59421@github.com>
> Other modes should not instantiate or touch young/old generations.
William Kemper has updated the pull request incrementally with one additional commit since the last revision:
Fix zero build
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/444/files
- new: https://git.openjdk.org/shenandoah/pull/444/files/5e744806..b28513f7
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=444&range=01
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=444&range=00-01
Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/444.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/444/head:pull/444
PR: https://git.openjdk.org/shenandoah/pull/444
From duke at openjdk.org Fri Jun 7 03:03:15 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Fri, 7 Jun 2024 03:03:15 GMT
Subject: RFR: 8333716: Shenandoah: Check for disarmed method before taking the
nmethod lock
Message-ID: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
**Notes**
We are spending significant time on acquiring the per-nmethod as all the
threads are in same nmethod.
Adding double-check lock by calling is_armed before lock acquisition.
**Verification**
1. hotspot_gc tests
2.
dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
[0.617s][info][gc] GC(0) Concurrent marking roots 5.553ms
[0.643s][info][gc] GC(1) Concurrent marking roots 3.148ms
[0.666s][info][gc] GC(2) Concurrent marking roots 3.219ms
[0.687s][info][gc] GC(3) Concurrent marking roots 2.796ms
[0.708s][info][gc] GC(4) Concurrent marking roots 2.892ms
[0.946s][info][gc] GC(5) Concurrent marking roots 3.393ms
[1.023s][info][gc] GC(6) Concurrent marking roots 2.710ms
[1.095s][info][gc] GC(7) Concurrent marking roots 2.466ms
[1.173s][info][gc] GC(8) Concurrent marking roots 2.734ms
[1.247s][info][gc] GC(9) Concurrent marking roots 2.706ms
[1.325s][info][gc] GC(10) Concurrent marking roots 2.451ms
[1.397s][info][gc] GC(11) Concurrent marking roots 2.381ms
[1.477s][info][gc] GC(12) Concurrent marking roots 2.411ms
[1.555s][info][gc] GC(13) Concurrent marking roots 2.222ms
[1.628s][info][gc] GC(14) Concurrent marking roots 2.599ms
-------------
Commit messages:
- 8333716: Shenandoah - Check for disarmed method before taking the nmethod lock
Changes: https://git.openjdk.org/jdk/pull/19587/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19587&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333716
Stats: 8 lines in 1 file changed: 6 ins; 0 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19587.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19587/head:pull/19587
PR: https://git.openjdk.org/jdk/pull/19587
From varadam at openjdk.org Fri Jun 7 06:56:19 2024
From: varadam at openjdk.org (Varada M)
Date: Fri, 7 Jun 2024 06:56:19 GMT
Subject: RFR: 8331733: [PPC64] saving and restoring CR is not needed at
most places [v5]
In-Reply-To: <QVQ-I14eNw64efIFubZPee_82T07QppA-fsQEvzbEYE=.63f3b5d9-1924-46c8-817e-106d05297f67@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
<QVQ-I14eNw64efIFubZPee_82T07QppA-fsQEvzbEYE=.63f3b5d9-1924-46c8-817e-106d05297f67@github.com>
Message-ID: <wpEklji_d7-uvFV1QOOB9vz0Pwbl3Nw4xGbntOcbZUY=.45a4201b-d3e2-423e-8ac4-fe5757c39f80@github.com>
On Thu, 6 Jun 2024 11:24:02 GMT, Varada M <varadam at openjdk.org> wrote:
>> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
>> Fastdebug: build and tier1 testing successful. [unrelated failures]
>
> Varada M has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
>
> - Merge branch 'master' into LR
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
> - [PPC64] saving and restoring CR is not needed at most places
Thank you,
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19494#issuecomment-2154207330
From shade at openjdk.org Fri Jun 7 08:19:13 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 7 Jun 2024 08:19:13 GMT
Subject: RFR: 8333716: Shenandoah: Check for disarmed method before taking
the nmethod lock
In-Reply-To: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
References: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
Message-ID: <jhiBcxc1SnSPnTZ7L5FWmxo5200wbdg-QGzVx8xRDA0=.ec21ca2a-0820-41bd-9163-8fd10bc46cb2@github.com>
On Thu, 6 Jun 2024 21:19:16 GMT, Neethu Prasad <duke at openjdk.org> wrote:
> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
>
> **Verification**
> 1. hotspot_gc tests
>
> 2.
>
> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
> [0.617s][info][gc] GC(0) Concurrent marking roots 5.553ms
> [0.643s][info][gc] GC(1) Concurrent marking roots 3.148ms
> [0.666s][info][gc] GC(2) Concurrent marking roots 3.219ms
> [0.687s][info][gc] GC(3) Concurrent marking roots 2.796ms
> [0.708s][info][gc] GC(4) Concurrent marking roots 2.892ms
> [0.946s][info][gc] GC(5) Concurrent marking roots 3.393ms
> [1.023s][info][gc] GC(6) Concurrent marking roots 2.710ms
> [1.095s][info][gc] GC(7) Concurrent marking roots 2.466ms
> [1.173s][info][gc] GC(8) Concurrent marking roots 2.734ms
> [1.247s][info][gc] GC(9) Concurrent marking roots 2.706ms
> [1.325s][info][gc] GC(10) Concurrent marking roots 2.451ms
> [1.397s][info][gc] GC(11) Concurrent marking roots 2.381ms
> [1.477s][info][gc] GC(12) Concurrent marking roots 2.411ms
> [1.555s][info][gc] GC(13) Concurrent marking roots 2.222ms
> [1.628s][info][gc] GC(14) Concurrent marking roots 2.599ms
Looks good, thanks! Note @kdnilsen.
For Shenandoah patches, please make sure to run `hotspot_gc_shenandoah`. I am pretty sure `hotspot_gc` includes the majority of it, but some compiler tests might be missing.
-------------
Marked as reviewed by shade (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19587#pullrequestreview-2103923873
PR Comment: https://git.openjdk.org/jdk/pull/19587#issuecomment-2154330475
From varadam at openjdk.org Fri Jun 7 08:53:16 2024
From: varadam at openjdk.org (Varada M)
Date: Fri, 7 Jun 2024 08:53:16 GMT
Subject: Integrated: 8331733: [PPC64] saving and restoring CR is not needed at
most places
In-Reply-To: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
References: <P_njVxwa9D-KHAmzVxJMP4OXtl6VgvbbD4pl1ZHpHwA=.efc95f0b-a68f-4c5a-b900-2f7272254013@github.com>
Message-ID: <-AsHbsg6KtJSUFLfY5mklB12miz3_NnAl_gOBrMx5zs=.db86b746-9603-4f74-81d1-b8c86b642da2@github.com>
On Fri, 31 May 2024 08:56:36 GMT, Varada M <varadam at openjdk.org> wrote:
> PPC64 uses save/restore CR less often. Only LR is critical, CR is mainly needed for native-to-Java calls.
> Fastdebug: build and tier1 testing successful. [unrelated failures]
This pull request has now been integrated.
Changeset: 40b2fbd8
Author: Varada M <varadam at openjdk.org>
Committer: Martin Doerr <mdoerr at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/40b2fbd8207404961d3d23375b288cceafc3f902
Stats: 93 lines in 11 files changed: 13 ins; 8 del; 72 mod
8331733: [PPC64] saving and restoring CR is not needed at most places
Reviewed-by: mdoerr, amitkumar
-------------
PR: https://git.openjdk.org/jdk/pull/19494
From lucy at openjdk.org Fri Jun 7 11:14:23 2024
From: lucy at openjdk.org (Lutz Schmidt)
Date: Fri, 7 Jun 2024 11:14:23 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <IY1GbRJzPTS55MrrDshP9oKMhGUmm-P_mORqRYRTapk=.f7a7f02c-da19-4817-a6e2-32146fb36f65@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
I feel very uncomfortable with this PR, at least as far as s390 is concerned. Many of the methods now set to be removed have been implemented with "implementation completeness" in mind. Not being used currently does not allow the implication of being obsolete. The approach on s390 has always been to provide support for potentially useful new instructions once they become available. Later exploitation can then focus on the use, without bloating the PR with low-level assembler* declarations.
There are a few exceptions, though. One is the code around `_atomic_memory_operation_lock`. That seems to be a leftover which simply was forgotten. Same seems true for `pd_relocate_CodeBlob`
-------------
Changes requested by lucy (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19550#pullrequestreview-2104286082
From jsjolen at openjdk.org Fri Jun 7 11:52:12 2024
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Fri, 7 Jun 2024 11:52:12 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <cIKcOMrD0I329G6aR8Z4fHxvXLykvpbs2hD1k6WUuMk=.0acec0e4-6849-4c03-843d-ad752f8784b0@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
Hi,
Removing dead code is great, but it has to be done with reasoning behind it and preferably in smaller batches, so as to be reviewable. I also don't understand why you comment out tests.
If you can make these into smaller and more localized PRs, then I'll be happy to take a look if I think I'm the right person to review it.
All the best,
Johan
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19550#issuecomment-2154676801
From eosterlund at openjdk.org Fri Jun 7 12:15:18 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Fri, 7 Jun 2024 12:15:18 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <b0PHGns6xAmpvPxzxFkXupNOBjFw2pPhMjoDS-Og1-8=.fd58d0ef-e219-4866-80d9-4c500b664d43@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
Some parts of the code, such as the assemblers, can be seen as tools that we have in our shed so that we can write other powerful code. If you have a shed full of tools, then naturally you can go through the shed and get rid of the tools we don't seem to currently use. Who needs a spade anyway? Nobody has used that spade for a year!
Except that eventually, the day always comes when you need a spade. Since you have now thrown away the only spade in the shed, you will find yourself with the option to either 1) try to make do with a trowel, which is horrible but might work as a hack. Or 2) you have to make a new spade yet again. And no, we can't buy a ready made spade.
It can be very annoying when you have what would seemingly be a trivial patch, but then you find out you won the lottery and you are apparently the first person in a while that needed a testl with a memory operand comparing against a 32 bit immediate, and have to go and read ISA manuals to figure out how to encode this thing correctly. It adds a large amount of extra work to add support for something that we should be able to take for granted.
I'm not a big fan of throwing away all the tools we have in the shed just because they haven't been used in a while. I don't want to dig my next hole with a trowel, nor do I want to build a new spade that we already have.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19550#issuecomment-2154709857
From jwilhelm at openjdk.org Fri Jun 7 13:20:13 2024
From: jwilhelm at openjdk.org (Jesper Wilhelmsson)
Date: Fri, 7 Jun 2024 13:20:13 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <TLpqb-7PGnBWLutlEGBh5jajEqSPlfEiqdIckfNUjas=.377d5468-8f85-45c0-9a87-b68511c8eba7@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
This seems to me like a classic case of not reading the instructions. @JohnTortugo, please read the [OpenJDK Developers' Guide](https://openjdk.org/guide/) before posting a PR with a huge change. There are sections in there that covers this exact case. The section [Contributing to an OpenJDK Project](https://openjdk.org/guide/#contributing-to-an-openjdk-project) really is mandatory reading before doing anything in OpenJDK.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19550#issuecomment-2154820179
From wkemper at openjdk.org Fri Jun 7 14:17:04 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Jun 2024 14:17:04 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <KSwcYbCYc5BSws2jdlBnIfvewzSFUapBMtfWYLBn3h8=.48e05c25-98fc-48db-9bd7-958a56d9aa98@github.com>
Merges tag jdk-23+26
-------------
Commit messages:
- 8333743: Change .jcheck/conf branches property to match valid branches
- 8332550: [macos] Voice Over: java.awt.IllegalComponentStateException: component must be showing on the screen to determine its location
- 8333674: Disable CollectorPolicy.young_min_ergo_vm for PPC64
- 8333647: C2 SuperWord: some additional PopulateIndex tests
- 8333270: HandlersOnComplexResetUpdate and HandlersOnComplexUpdate tests fail with "Unexpected reference" if timeoutFactor is less than 1/3
- 8333560: -Xlint:restricted does not work with --release
- 8332670: C1 clone intrinsic needs memory barriers
- 8333622: ubsan: relocInfo_x86.cpp:101:56: runtime error: pointer index expression with base (-1) overflowed
- 8332865: ubsan: os::attempt_reserve_memory_between reports overflow
- 6942632: Hotspot should be able to use more than 64 logical processors on Windows
- ... and 324 more: https://git.openjdk.org/shenandoah/compare/2f10a316...31696a44
The webrevs contain the conflicts with master and the adjustments done while merging with regards to each parent branch:
- merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=445&range=00.conflicts
- master: https://webrevs.openjdk.org/?repo=shenandoah&pr=445&range=00.0
- openjdk/jdk:master: https://webrevs.openjdk.org/?repo=shenandoah&pr=445&range=00.1
Changes: https://git.openjdk.org/shenandoah/pull/445/files
Stats: 91727 lines in 1735 files changed: 66138 ins; 17207 del; 8382 mod
Patch: https://git.openjdk.org/shenandoah/pull/445.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/445/head:pull/445
PR: https://git.openjdk.org/shenandoah/pull/445
From duke at openjdk.org Fri Jun 7 14:33:23 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Fri, 7 Jun 2024 14:33:23 GMT
Subject: RFR: 8333716: Shenandoah: Check for disarmed method before taking
the nmethod lock [v2]
In-Reply-To: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
References: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
Message-ID: <j_lU0IBLiblj8Dd59DZtUs54GDL7X-Ve9cfV9AoXWx8=.4e3b620b-3104-4600-9ff1-ece18cd2ead3@github.com>
> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
>
> **Verification**
> 1. hotspot_gc tests
>
> 2.
>
> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
> [0.617s][info][gc] GC(0) Concurrent marking roots 5.553ms
> [0.643s][info][gc] GC(1) Concurrent marking roots 3.148ms
> [0.666s][info][gc] GC(2) Concurrent marking roots 3.219ms
> [0.687s][info][gc] GC(3) Concurrent marking roots 2.796ms
> [0.708s][info][gc] GC(4) Concurrent marking roots 2.892ms
> [0.946s][info][gc] GC(5) Concurrent marking roots 3.393ms
> [1.023s][info][gc] GC(6) Concurrent marking roots 2.710ms
> [1.095s][info][gc] GC(7) Concurrent marking roots 2.466ms
> [1.173s][info][gc] GC(8) Concurrent marking roots 2.734ms
> [1.247s][info][gc] GC(9) Concurrent marking roots 2.706ms
> [1.325s][info][gc] GC(10) Concurrent marking roots 2.451ms
> [1.397s][info][gc] GC(11) Concurrent marking roots 2.381ms
> [1.477s][info][gc] GC(12) Concurrent marking roots 2.411ms
> [1.555s][info][gc] GC(13) Concurrent marking roots 2.222ms
> [1.628s][info][gc] GC(14) Concurrent marking roots 2.599ms
Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
Update full name
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19587/files
- new: https://git.openjdk.org/jdk/pull/19587/files/65123cd0..9649f1e3
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19587&range=01
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19587&range=00-01
Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/jdk/pull/19587.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19587/head:pull/19587
PR: https://git.openjdk.org/jdk/pull/19587
From duke at openjdk.org Fri Jun 7 14:33:24 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Fri, 7 Jun 2024 14:33:24 GMT
Subject: RFR: 8333716: Shenandoah: Check for disarmed method before taking
the nmethod lock
In-Reply-To: <jhiBcxc1SnSPnTZ7L5FWmxo5200wbdg-QGzVx8xRDA0=.ec21ca2a-0820-41bd-9163-8fd10bc46cb2@github.com>
References: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
<jhiBcxc1SnSPnTZ7L5FWmxo5200wbdg-QGzVx8xRDA0=.ec21ca2a-0820-41bd-9163-8fd10bc46cb2@github.com>
Message-ID: <GZ3-_SWV6pHvTqe80OrO2z0n2kZmZ9tFkBQ-CjT2ZnY=.5e3a4add-5779-43ef-8af9-170b92b171de@github.com>
On Fri, 7 Jun 2024 08:16:22 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> For Shenandoah patches, please make sure to run `hotspot_gc_shenandoah`. I am pretty sure `hotspot_gc` includes the majority of it, but some compiler tests might be missing.
Ran hotspot_gc_shenandoah.
Verified that all the tests succeeded.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19587#issuecomment-2154969377
From ysr at openjdk.org Fri Jun 7 15:46:13 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 7 Jun 2024 15:46:13 GMT
Subject: RFR: 8333716: Shenandoah: Check for disarmed method before taking
the nmethod lock [v2]
In-Reply-To: <j_lU0IBLiblj8Dd59DZtUs54GDL7X-Ve9cfV9AoXWx8=.4e3b620b-3104-4600-9ff1-ece18cd2ead3@github.com>
References: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
<j_lU0IBLiblj8Dd59DZtUs54GDL7X-Ve9cfV9AoXWx8=.4e3b620b-3104-4600-9ff1-ece18cd2ead3@github.com>
Message-ID: <OZqDsr5vvkBWtMZJTwVe9rFdILEJWyq2BmerVKkXZvA=.7d12f4d6-db2a-4c97-81cb-27ed061acc65@github.com>
On Fri, 7 Jun 2024 14:33:23 GMT, Neethu Prasad <duke at openjdk.org> wrote:
>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>>
>> **Verification**
>> 1. hotspot_gc tests
>>
>> 2.
>>
>> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.617s][info][gc] GC(0) Concurrent marking roots 5.553ms
>> [0.643s][info][gc] GC(1) Concurrent marking roots 3.148ms
>> [0.666s][info][gc] GC(2) Concurrent marking roots 3.219ms
>> [0.687s][info][gc] GC(3) Concurrent marking roots 2.796ms
>> [0.708s][info][gc] GC(4) Concurrent marking roots 2.892ms
>> [0.946s][info][gc] GC(5) Concurrent marking roots 3.393ms
>> [1.023s][info][gc] GC(6) Concurrent marking roots 2.710ms
>> [1.095s][info][gc] GC(7) Concurrent marking roots 2.466ms
>> [1.173s][info][gc] GC(8) Concurrent marking roots 2.734ms
>> [1.247s][info][gc] GC(9) Concurrent marking roots 2.706ms
>> [1.325s][info][gc] GC(10) Concurrent marking roots 2.451ms
>> [1.397s][info][gc] GC(11) Concurrent marking roots 2.381ms
>> [1.477s][info][gc] GC(12) Concurrent marking roots 2.411ms
>> [1.555s][info][gc] GC(13) Concurrent marking roots 2.222ms
>> [1.628s][info][gc] GC(14) Concurrent marking roots 2.599ms
>
> Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
>
> Update full name
LGTM. ?
In addition to the "after" figures, may be also include the "before" figures for the microbenchmark you shared? If you can include, but without any details, the impact of this change on the customer's GC metrics, that would be even better, but isn't essential.
-------------
Marked as reviewed by ysr (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19587#pullrequestreview-2104889107
From wkemper at openjdk.org Fri Jun 7 18:00:18 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Jun 2024 18:00:18 GMT
Subject: RFR: 8333716: Shenandoah: Check for disarmed method before taking
the nmethod lock [v2]
In-Reply-To: <j_lU0IBLiblj8Dd59DZtUs54GDL7X-Ve9cfV9AoXWx8=.4e3b620b-3104-4600-9ff1-ece18cd2ead3@github.com>
References: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
<j_lU0IBLiblj8Dd59DZtUs54GDL7X-Ve9cfV9AoXWx8=.4e3b620b-3104-4600-9ff1-ece18cd2ead3@github.com>
Message-ID: <rBpje-HY83hhdfWUsgJhWZXWwfJAL6pWxQueLJK0yik=.24f3e664-8d43-4165-b2c8-8dd7b03960c0@github.com>
On Fri, 7 Jun 2024 14:33:23 GMT, Neethu Prasad <duke at openjdk.org> wrote:
>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>>
>> **Verification**
>> 1. hotspot_gc tests
>>
>> 2. Benchmarking on [c6a.12xlarge](https://aws.amazon.com/ec2/instance-types/c6a/)
>> **Prior to this PR:**
>>
>>
>> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.697s][info][gc] GC(0) Concurrent marking roots 82.772ms
>> [0.800s][info][gc] GC(1) Concurrent marking roots 82.675ms
>> [0.883s][info][gc] GC(2) Concurrent marking roots 67.893ms
>> [0.976s][info][gc] GC(3) Concurrent marking roots 76.811ms
>> [1.064s][info][gc] GC(4) Concurrent marking roots 73.170ms
>> [1.276s][info][gc] GC(5) Concurrent marking roots 68.098ms
>> [1.386s][info][gc] GC(6) Concurrent marking roots 78.548ms
>> [1.496s][info][gc] GC(7) Concurrent marking roots 77.666ms
>> [1.601s][info][gc] GC(8) Concurrent marking roots 69.263ms
>> [1.714s][info][gc] GC(9) Concurrent marking roots 70.956ms
>> [1.838s][info][gc] GC(10) Concurrent marking roots 75.879ms
>> [1.956s][info][gc] GC(11) Concurrent marking roots 71.753ms
>> [2.085s][info][gc] GC(12) Concurrent marking roots 78.325ms
>> [2.214s][info][gc] GC(13) Concurrent marking roots 72.238ms
>> [2.337s][info][gc] GC(14) Concurrent marking roots 75.127ms
>>
>>
>> **After:**
>>
>>
>> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.617s][info][gc] GC(0) Concurrent marking roots 5.553ms
>> [0.643s][info][gc] GC(1) Concurrent marking roots 3.148ms
>> [0.666s][info][gc] GC(2) Concurrent marking roots 3.219ms
>> [0.687s][info][gc] GC(3) Concurrent marking roots 2.796ms
>> [0.708s][info][gc] GC(4) Concurrent marking roots 2.892ms
>> [0.946s][info][gc] GC(5) Concurrent marking roots 3.393ms
>> [1.023s][info][gc] GC(6) Concurrent marking roots 2.710ms
>> [1.095s][info][gc] GC(7) Concurrent marking roots 2.466ms
>> [1.173s][info][gc] GC(8) Concurrent marking roots 2.734ms
>> [1.247s][info][gc] GC(9) Concurrent marking roots 2.706ms
>> [1.325s][info][gc] GC(10) Concurrent marking roots 2.451ms
>> [1.397s][info][gc] GC(11) Concurrent marking roots 2.381ms
>> [1.477s][info][gc] GC(12) Concurrent marking roots 2.411ms
>> [1.555s][info][gc] GC(13) Concurrent mark...
>
> Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
>
> Update full name
Marked as reviewed by wkemper (Committer).
That's a nice improvement!
-------------
PR Review: https://git.openjdk.org/jdk/pull/19587#pullrequestreview-2105101357
PR Comment: https://git.openjdk.org/jdk/pull/19587#issuecomment-2155281303
From duke at openjdk.org Fri Jun 7 18:27:12 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Fri, 7 Jun 2024 18:27:12 GMT
Subject: RFR: 8333716: Shenandoah: Check for disarmed method before taking
the nmethod lock [v2]
In-Reply-To: <j_lU0IBLiblj8Dd59DZtUs54GDL7X-Ve9cfV9AoXWx8=.4e3b620b-3104-4600-9ff1-ece18cd2ead3@github.com>
References: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
<j_lU0IBLiblj8Dd59DZtUs54GDL7X-Ve9cfV9AoXWx8=.4e3b620b-3104-4600-9ff1-ece18cd2ead3@github.com>
Message-ID: <weHWKa-jLfn2mluuVnmaHEpIfE0Yxhawi6fB6_Itung=.af40d26f-0d61-46e2-b76a-a77f7d377c84@github.com>
On Fri, 7 Jun 2024 14:33:23 GMT, Neethu Prasad <duke at openjdk.org> wrote:
>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>>
>> **Verification**
>> 1. hotspot_gc tests
>>
>> 2. Benchmarking on [c6a.12xlarge](https://aws.amazon.com/ec2/instance-types/c6a/)
>> **Prior to this PR:**
>>
>>
>> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.697s][info][gc] GC(0) Concurrent marking roots 82.772ms
>> [0.800s][info][gc] GC(1) Concurrent marking roots 82.675ms
>> [0.883s][info][gc] GC(2) Concurrent marking roots 67.893ms
>> [0.976s][info][gc] GC(3) Concurrent marking roots 76.811ms
>> [1.064s][info][gc] GC(4) Concurrent marking roots 73.170ms
>> [1.276s][info][gc] GC(5) Concurrent marking roots 68.098ms
>> [1.386s][info][gc] GC(6) Concurrent marking roots 78.548ms
>> [1.496s][info][gc] GC(7) Concurrent marking roots 77.666ms
>> [1.601s][info][gc] GC(8) Concurrent marking roots 69.263ms
>> [1.714s][info][gc] GC(9) Concurrent marking roots 70.956ms
>> [1.838s][info][gc] GC(10) Concurrent marking roots 75.879ms
>> [1.956s][info][gc] GC(11) Concurrent marking roots 71.753ms
>> [2.085s][info][gc] GC(12) Concurrent marking roots 78.325ms
>> [2.214s][info][gc] GC(13) Concurrent marking roots 72.238ms
>> [2.337s][info][gc] GC(14) Concurrent marking roots 75.127ms
>>
>>
>> **After:**
>>
>>
>> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.617s][info][gc] GC(0) Concurrent marking roots 5.553ms
>> [0.643s][info][gc] GC(1) Concurrent marking roots 3.148ms
>> [0.666s][info][gc] GC(2) Concurrent marking roots 3.219ms
>> [0.687s][info][gc] GC(3) Concurrent marking roots 2.796ms
>> [0.708s][info][gc] GC(4) Concurrent marking roots 2.892ms
>> [0.946s][info][gc] GC(5) Concurrent marking roots 3.393ms
>> [1.023s][info][gc] GC(6) Concurrent marking roots 2.710ms
>> [1.095s][info][gc] GC(7) Concurrent marking roots 2.466ms
>> [1.173s][info][gc] GC(8) Concurrent marking roots 2.734ms
>> [1.247s][info][gc] GC(9) Concurrent marking roots 2.706ms
>> [1.325s][info][gc] GC(10) Concurrent marking roots 2.451ms
>> [1.397s][info][gc] GC(11) Concurrent marking roots 2.381ms
>> [1.477s][info][gc] GC(12) Concurrent marking roots 2.411ms
>> [1.555s][info][gc] GC(13) Concurrent mark...
>
> Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
>
> Update full name
Thanks all for the approval.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19587#issuecomment-2155316954
From ysr at openjdk.org Fri Jun 7 18:29:51 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 7 Jun 2024 18:29:51 GMT
Subject: RFR: 8333825: GenShen: Revert/Remove ShenandoahMaxEvacLABRatio
Message-ID: <ShKnWYQVibD4QuWTe5ujonhvf4Lp3gHi3iQBmiVhj8k=.44f8cd9f-a2eb-40eb-aa35-5f3d3b912239@github.com>
ShenandoahMaxEvacLABRatio was intended as an experimental/research option and guardrail to manually bound the ceiling of PLABs used in Shenandoah and GenShen at the time that GenShen's LAB auto-sizing and allocation heuristics were somewhat nascent and immature. There is however a natural upper-bound of the heap region size for these local allocation buffers which prevent runaway growth, and the manual throttle has never had to be used, and the option thus appears somewhat vestigial. Removing it also reduces differences with upstream Shenandoah which should simplify GenShen's upstreaming reviews.
- Removed the option and its uses in Shen and GenShen code
- Adjusted logging to be consistent, and adjusted some comments in GenShen
- Tested with codepipeline, GC tests, and exercised the new log lines
-------------
Commit messages:
- Cosmetic change to debug level log.
- Try and minimize changes wrt jdk tip.
- Remove vestigial option ShenandoahMaxEvacLABRatio; move and adjust debug
Changes: https://git.openjdk.org/shenandoah/pull/446/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=446&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333825
Stats: 53 lines in 3 files changed: 7 ins; 36 del; 10 mod
Patch: https://git.openjdk.org/shenandoah/pull/446.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/446/head:pull/446
PR: https://git.openjdk.org/shenandoah/pull/446
From wkemper at openjdk.org Fri Jun 7 18:41:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Jun 2024 18:41:33 GMT
Subject: RFR: 8333825: GenShen: Revert/Remove ShenandoahMaxEvacLABRatio
In-Reply-To: <ShKnWYQVibD4QuWTe5ujonhvf4Lp3gHi3iQBmiVhj8k=.44f8cd9f-a2eb-40eb-aa35-5f3d3b912239@github.com>
References: <ShKnWYQVibD4QuWTe5ujonhvf4Lp3gHi3iQBmiVhj8k=.44f8cd9f-a2eb-40eb-aa35-5f3d3b912239@github.com>
Message-ID: <chgP4cCxxv9o5BotEy0-BfnFacZnTcEqjemieo5VyVE=.1367d8d8-e061-450d-acd8-5b3b9bf6d663@github.com>
On Fri, 7 Jun 2024 18:24:26 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
> ShenandoahMaxEvacLABRatio was intended as an experimental/research option and guardrail to manually bound the ceiling of PLABs used in Shenandoah and GenShen at the time that GenShen's LAB auto-sizing and allocation heuristics were somewhat nascent and immature. There is however a natural upper-bound of the heap region size for these local allocation buffers which prevent runaway growth, and the manual throttle has never had to be used, and the option thus appears somewhat vestigial. Removing it also reduces differences with upstream Shenandoah which should simplify GenShen's upstreaming reviews.
>
> - Removed the option and its uses in Shen and GenShen code
> - Adjusted logging to be consistent, and adjusted some comments in GenShen
> - Tested with codepipeline, GC tests, and exercised the new log lines
LGTM
-------------
Marked as reviewed by wkemper (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/446#pullrequestreview-2105170521
From cslucas at openjdk.org Fri Jun 7 19:19:19 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Fri, 7 Jun 2024 19:19:19 GMT
Subject: RFR: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <KQJ8XGFr19_pV1BcVU9FrzkjMFNCCy3SU53ACQuHaqI=.abace4fb-044d-402c-a9e9-534c4ca73cd8@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
Closing this as not relevant. Thank you for the reviews.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19550#issuecomment-2155383826
From cslucas at openjdk.org Fri Jun 7 19:19:19 2024
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Fri, 7 Jun 2024 19:19:19 GMT
Subject: Withdrawn: 8333566: Remove unused methods
In-Reply-To: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
References: <3vCVNjjgFzEw_YgoboTLXemBuzwht9uEeZxC8yg5Zog=.a7600184-c4ca-4a93-8b76-74df7bdff405@github.com>
Message-ID: <GY73UyeqW_Y3z1JpU3Dl83UCtXLyDu9xV0XgIIUGKcY=.27ac46d5-4791-47d8-9342-7fef1a99bcd2@github.com>
On Tue, 4 Jun 2024 20:51:52 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:
> Please, consider this patch to remove unused methods from the code base. To the best of my knowledge, these methods are only defined but never used.
>
> Here is a list with names of delete methods: https://gist.github.com/JohnTortugo/fccc29781a1b584c03162aa4e160e874
>
> Tested with Linux x86_64 tier1-4, GHA, and only cross building to other platforms.
This pull request has been closed without being integrated.
-------------
PR: https://git.openjdk.org/jdk/pull/19550
From duke at openjdk.org Fri Jun 7 20:06:16 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Fri, 7 Jun 2024 20:06:16 GMT
Subject: Integrated: 8333716: Shenandoah: Check for disarmed method before
taking the nmethod lock
In-Reply-To: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
References: <YO3LHp0PTPxm3a67WZNkwRwi77aie6YwbBbDnM1xYkY=.65f1c3f2-83d1-4d6c-9ae6-aa807a2f1e7a@github.com>
Message-ID: <j3HDFuvi6aGh8Lq7t9EErjySjwfUuX1Md91pZ7MmFrY=.14d19a43-8a60-4d90-b35a-e8568082c0d4@github.com>
On Thu, 6 Jun 2024 21:19:16 GMT, Neethu Prasad <duke at openjdk.org> wrote:
> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
>
> **Verification**
> 1. hotspot_gc tests
>
> 2. Benchmarking on [c6a.12xlarge](https://aws.amazon.com/ec2/instance-types/c6a/)
> **Prior to this PR:**
>
>
> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
> [0.697s][info][gc] GC(0) Concurrent marking roots 82.772ms
> [0.800s][info][gc] GC(1) Concurrent marking roots 82.675ms
> [0.883s][info][gc] GC(2) Concurrent marking roots 67.893ms
> [0.976s][info][gc] GC(3) Concurrent marking roots 76.811ms
> [1.064s][info][gc] GC(4) Concurrent marking roots 73.170ms
> [1.276s][info][gc] GC(5) Concurrent marking roots 68.098ms
> [1.386s][info][gc] GC(6) Concurrent marking roots 78.548ms
> [1.496s][info][gc] GC(7) Concurrent marking roots 77.666ms
> [1.601s][info][gc] GC(8) Concurrent marking roots 69.263ms
> [1.714s][info][gc] GC(9) Concurrent marking roots 70.956ms
> [1.838s][info][gc] GC(10) Concurrent marking roots 75.879ms
> [1.956s][info][gc] GC(11) Concurrent marking roots 71.753ms
> [2.085s][info][gc] GC(12) Concurrent marking roots 78.325ms
> [2.214s][info][gc] GC(13) Concurrent marking roots 72.238ms
> [2.337s][info][gc] GC(14) Concurrent marking roots 75.127ms
>
>
> **After:**
>
>
> dev-dsk-neethp-jdk-2c-ad54955c % jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
> [0.617s][info][gc] GC(0) Concurrent marking roots 5.553ms
> [0.643s][info][gc] GC(1) Concurrent marking roots 3.148ms
> [0.666s][info][gc] GC(2) Concurrent marking roots 3.219ms
> [0.687s][info][gc] GC(3) Concurrent marking roots 2.796ms
> [0.708s][info][gc] GC(4) Concurrent marking roots 2.892ms
> [0.946s][info][gc] GC(5) Concurrent marking roots 3.393ms
> [1.023s][info][gc] GC(6) Concurrent marking roots 2.710ms
> [1.095s][info][gc] GC(7) Concurrent marking roots 2.466ms
> [1.173s][info][gc] GC(8) Concurrent marking roots 2.734ms
> [1.247s][info][gc] GC(9) Concurrent marking roots 2.706ms
> [1.325s][info][gc] GC(10) Concurrent marking roots 2.451ms
> [1.397s][info][gc] GC(11) Concurrent marking roots 2.381ms
> [1.477s][info][gc] GC(12) Concurrent marking roots 2.411ms
> [1.555s][info][gc] GC(13) Concurrent marking roots 2.222ms
> [1.628s][info][gc] GC(14) Concurrent marking roots 2.599ms
This pull request has now been integrated.
Changeset: 18e7d7b5
Author: Neethu Prasad <neethp at amazon.com>
Committer: Paul Hohensee <phh at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/18e7d7b5e710b24e49b995777906a197e35795e6
Stats: 8 lines in 1 file changed: 6 ins; 0 del; 2 mod
8333716: Shenandoah: Check for disarmed method before taking the nmethod lock
Reviewed-by: shade, ysr, wkemper
-------------
PR: https://git.openjdk.org/jdk/pull/19587
From eosterlund at openjdk.org Fri Jun 7 20:17:13 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Fri, 7 Jun 2024 20:17:13 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v4]
In-Reply-To: <aumjLSAUm1CNEhg3JWjsPPQt4TlWISRK2cer4oyYspU=.2305c90a-57f6-421c-8463-b0fc818c6eca@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<aumjLSAUm1CNEhg3JWjsPPQt4TlWISRK2cer4oyYspU=.2305c90a-57f6-421c-8463-b0fc818c6eca@github.com>
Message-ID: <yy5tetunUEl724IuecatIJdl8SHniS9cG9rwpv4kv_0=.6b60b877-5c95-43a8-84a5-03445e265abe@github.com>
On Wed, 5 Jun 2024 20:45:22 GMT, Neethu Prasad <duke at openjdk.org> wrote:
>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>>
>> **Verification**
>>
>> Shenendoah
>>
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>>
>> ZGC
>>
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>>
>> **Issue**
>> https://bugs.openjdk.org/browse/JDK-8331911
>
> Neethu Prasad has updated the pull request incrementally with one additional commit since the last revision:
>
> 8331911: Remove is_armed check from callers
Marked as reviewed by eosterlund (Reviewer).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19285#pullrequestreview-2105296053
From eosterlund at openjdk.org Fri Jun 7 20:17:14 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Fri, 7 Jun 2024 20:17:14 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v3]
In-Reply-To: <YsEp4fgCBxneEuJnfidh98QfR0LAUV_H6GVN6PzPqys=.bf6c999d-4cab-4176-be79-10f23d385a7d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
<uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>
<-GBU6zDyLGsnEyT3yEMfW_TnA-stpR1tLeqfvfeVcQ0=.c4b13824-35a4-4e22-8132-721a668a0425@github.com>
<SM6Tj6DXxNSXZbkM-daZA4gB8baJiPtwcua4JKOcaUg=.2bd3dffb-143c-4dc5-b0ef-166536c0d848@github.com>
<Q0Pyyx9YaLS55a7KmCZX6tEA5arxJ0qxVKXW1VAPWKE=.d09724f3-796b-4d20-bfc4-f39a1deb35ea@github.com>
<YsEp4fgCBxneEuJnfidh98QfR0LAUV_H6GVN6PzPqys=.bf6c999d-4cab-4176-be79-10f23d385a7d@github.com>
Message-ID: <KSVOZh0Op3yfFGg0C0Vg_eMtkEulKnAkDpUjDCY6If4=.6dbf481a-2f27-47dd-8dab-ba348bc447e2@github.com>
On Thu, 6 Jun 2024 13:54:26 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> > Having said that, perhaps we should file a separate issue to remove that check, since it seems to fix an actual bug, while I guess this was meant more as an optimization. What do you think?
>
> FTR, I don't mind executing cross-modify-fence unconditionally. I do mind going into deopts too often. I do also think that we want to stay on performance-positive side for at least an easy variant of fix, and do potentially regressing things separately. The initial motivation for this work was to resolve an issue in a service workload that runs many threads with similar stacks, and get something that we are sure about for a prompt backport.
Fair enough. For what it's worth, aside for the deopt stressing option with arbitrary frequency we can update, we will not deopt more. We just perform an extra cross modifying fence when racingly entering an nmethod concurrently being disarmed. Not performing it might be slightly faster, but is a bug. But I see your point.
> To that end, we can continue working out the final shape of the patch here, while we mitigate our current service problems with picking up a limited version of this patch with [JDK-8333716](https://bugs.openjdk.org/browse/JDK-8333716) -- it resolves only Shenandoah parts of it, though. Or, we can integrate this patch in its current form, resolving the issue on both Shenandoah and ZGC paths, and work out the check removal as the follow up of [JDK-8310239](https://bugs.openjdk.org/browse/JDK-8310239).
>
> I think the latter alternative is more pragmatic.
I'm okay with approving this patch, and we fix the actual bug separately. Sounds good? Then this is a refactoring and optimization, without the bug fix.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2155485496
From ysr at openjdk.org Fri Jun 7 21:34:27 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 7 Jun 2024 21:34:27 GMT
Subject: Integrated: 8333825: GenShen: Revert/Remove ShenandoahMaxEvacLABRatio
In-Reply-To: <ShKnWYQVibD4QuWTe5ujonhvf4Lp3gHi3iQBmiVhj8k=.44f8cd9f-a2eb-40eb-aa35-5f3d3b912239@github.com>
References: <ShKnWYQVibD4QuWTe5ujonhvf4Lp3gHi3iQBmiVhj8k=.44f8cd9f-a2eb-40eb-aa35-5f3d3b912239@github.com>
Message-ID: <I4VpGGBmUkfEPQaePGrkCl9VW8Zzkbk6V4scpWPXOjo=.239659da-3b08-4d9c-ad78-38cdd63715b0@github.com>
On Fri, 7 Jun 2024 18:24:26 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
> ShenandoahMaxEvacLABRatio was intended as an experimental/research option and guardrail to manually bound the ceiling of PLABs used in Shenandoah and GenShen at the time that GenShen's LAB auto-sizing and allocation heuristics were somewhat nascent and immature. There is however a natural upper-bound of the heap region size for these local allocation buffers which prevent runaway growth, and the manual throttle has never had to be used, and the option thus appears somewhat vestigial. Removing it also reduces differences with upstream Shenandoah which should simplify GenShen's upstreaming reviews.
>
> - Removed the option and its uses in Shen and GenShen code
> - Adjusted logging to be consistent, and adjusted some comments in GenShen
> - Tested with codepipeline, GC tests, and exercised the new log lines
This pull request has now been integrated.
Changeset: 6071b4a9
Author: Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/6071b4a9a170b2d91522971a3c9b32bbf44eee3b
Stats: 53 lines in 3 files changed: 7 ins; 36 del; 10 mod
8333825: GenShen: Revert/Remove ShenandoahMaxEvacLABRatio
Reviewed-by: wkemper
-------------
PR: https://git.openjdk.org/shenandoah/pull/446
From wkemper at openjdk.org Fri Jun 7 23:39:57 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 7 Jun 2024 23:39:57 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
Message-ID: <DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
> Other modes should not instantiate or touch young/old generations.
William Kemper has updated the pull request incrementally with three additional commits since the last revision:
- Make age and youth const
- Do not age regions after completing old mark
- Detect cancellation of old GC after final mark.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/444/files
- new: https://git.openjdk.org/shenandoah/pull/444/files/b28513f7..d8215be6
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=444&range=02
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=444&range=01-02
Stats: 14 lines in 5 files changed: 8 ins; 1 del; 5 mod
Patch: https://git.openjdk.org/shenandoah/pull/444.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/444/head:pull/444
PR: https://git.openjdk.org/shenandoah/pull/444
From kdnilsen at openjdk.org Mon Jun 10 17:12:44 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Jun 2024 17:12:44 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
Message-ID: <oOIVEXXSFecZ-QrX7HQ3wqcBSL6LqWMiOEnAEH0ECYE=.8b274914-b5ef-4096-a3e9-bd89650a0089@github.com>
On Fri, 7 Jun 2024 23:39:57 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Other modes should not instantiate or touch young/old generations.
>
> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
>
> - Make age and youth const
> - Do not age regions after completing old mark
> - Detect cancellation of old GC after final mark.
Thanks for this refactoring. All much deserved and important. I wonder if the title of this PR should be a bit more general, since the refactoring appears to be more far reaching than just not instantiating young/old generations in non-generational mode.
src/hotspot/share/gc/shenandoah/shenandoahOldGC.cpp line 119:
> 117: return false;
> 118: }
> 119:
This looks like it might be a bug fix, vs the rest of this PR, which is mostly isolation refactoring. Should the bug fix be separated out into a different PR?
-------------
Marked as reviewed by kdnilsen (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/444#pullrequestreview-2108335069
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1633573810
From wkemper at openjdk.org Mon Jun 10 18:31:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 18:31:39 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <oOIVEXXSFecZ-QrX7HQ3wqcBSL6LqWMiOEnAEH0ECYE=.8b274914-b5ef-4096-a3e9-bd89650a0089@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
<oOIVEXXSFecZ-QrX7HQ3wqcBSL6LqWMiOEnAEH0ECYE=.8b274914-b5ef-4096-a3e9-bd89650a0089@github.com>
Message-ID: <PTd4Hx2PZOQmTgDCbFjOoK7zqn5_sGAzmj0SeuLt5ZA=.050afed0-2e6b-4962-badd-e0d7f86af9e3@github.com>
On Mon, 10 Jun 2024 17:07:49 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
>>
>> - Make age and youth const
>> - Do not age regions after completing old mark
>> - Detect cancellation of old GC after final mark.
>
> src/hotspot/share/gc/shenandoah/shenandoahOldGC.cpp line 119:
>
>> 117: return false;
>> 118: }
>> 119:
>
> This looks like it might be a bug fix, vs the rest of this PR, which is mostly isolation refactoring. Should the bug fix be separated out into a different PR?
Yes, it is a bug fix. I'll pull it out into a separate PR.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1633672241
From ysr at openjdk.org Mon Jun 10 18:55:45 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 10 Jun 2024 18:55:45 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v15]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <zoS-70khEbK7GAVp79OhpV5J-rebNIq25peZ64Q-BJk=.6be21c9b-faf7-42de-8e0a-ddc4bdcbf0a6@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 45 commits:
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Code called from LRB cannot rely on SGH::_gc_generation and must instead
use the _active_generation field.
Still need to aduit remaining _gc_generation uses for safety.
Removed too strong assert introduced for debugging in previous commit.
- Merge branch 'master' into active_generation
- Some asserts to catch a tricky race. These assertions may be too strong
in general but would help with debugging a rare crash.
- Merge branch 'master' into active_generation
- Remove vestigial thread() method in ShenandoahController.
- cosmetic
- jcheck white-space
- Remove static _thread field introduced in ShenandoahController, as its
role is already served by the _control_thread field in ShenandoahHeap.
- ... and 35 more: https://git.openjdk.org/shenandoah/compare/6071b4a9...561c2f1a
-------------
Changes: https://git.openjdk.org/shenandoah/pull/407/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=14
Stats: 168 lines in 17 files changed: 134 ins; 1 del; 33 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From kdnilsen at openjdk.org Mon Jun 10 19:37:00 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 10 Jun 2024 19:37:00 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation of
FreeSet into GenShen
Message-ID: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
into Generational Shenandoah.
-------------
Commit messages:
- Remove TestHumongousThreshold test from problem list
- Fix white space
- Merge branch 'master' of https://git.openjdk.org/shenandoah into harmonize-free-set
- Remove debugging instrumentation
- Merge remote-tracking branch 'origin/master' into harmonize-free-set
- Merge branch 'openjdk:master' into master
- Enhance diagnostic message for verification error
- Some fixes related to TestThreadFailure.java
- Disable debug instrumentation
- Fix two errors and cleanup some debug instrumentation
- ... and 46 more: https://git.openjdk.org/shenandoah/compare/6071b4a9...51f4136a
Changes: https://git.openjdk.org/shenandoah/pull/440/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8327000
Stats: 6600 lines in 125 files changed: 4580 ins; 962 del; 1058 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From wkemper at openjdk.org Mon Jun 10 21:44:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 21:44:06 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <m5W5QAhpH_NmUujV1pHqNA-e5jPTwYLfvm6-MENGlpM=.633152f0-cec2-4dd4-b43e-bd803ca97c91@github.com>
Merge tag jdk-24+1, without free set changes.
-------------
Commit messages:
- Merge tag 'jdk-24+1' into merge-jdk-24+1
- 8026127: Deflater/Inflater documentation incomplete/misleading
- 8333456: CompactNumberFormat integer parsing fails when string has no suffix
- 8333743: Change .jcheck/conf branches property to match valid branches
- 8330182: Start of release updates for JDK 24
- 8332550: [macos] Voice Over: java.awt.IllegalComponentStateException: component must be showing on the screen to determine its location
- 8333674: Disable CollectorPolicy.young_min_ergo_vm for PPC64
- 8333647: C2 SuperWord: some additional PopulateIndex tests
- 8333270: HandlersOnComplexResetUpdate and HandlersOnComplexUpdate tests fail with "Unexpected reference" if timeoutFactor is less than 1/3
- 8333560: -Xlint:restricted does not work with --release
- ... and 328 more: https://git.openjdk.org/shenandoah/compare/6071b4a9...3b1646dd
The webrevs contain the adjustments done while merging with regards to each parent branch:
- master: https://webrevs.openjdk.org/?repo=shenandoah&pr=447&range=00.0
- openjdk/jdk:master: https://webrevs.openjdk.org/?repo=shenandoah&pr=447&range=00.1
Changes: https://git.openjdk.org/shenandoah/pull/447/files
Stats: 91891 lines in 1767 files changed: 66786 ins; 16868 del; 8237 mod
Patch: https://git.openjdk.org/shenandoah/pull/447.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/447/head:pull/447
PR: https://git.openjdk.org/shenandoah/pull/447
From wkemper at openjdk.org Mon Jun 10 21:44:06 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 21:44:06 GMT
Subject: RFR: Merge openjdk/jdk:master
In-Reply-To: <m5W5QAhpH_NmUujV1pHqNA-e5jPTwYLfvm6-MENGlpM=.633152f0-cec2-4dd4-b43e-bd803ca97c91@github.com>
References: <m5W5QAhpH_NmUujV1pHqNA-e5jPTwYLfvm6-MENGlpM=.633152f0-cec2-4dd4-b43e-bd803ca97c91@github.com>
Message-ID: <I0yDaSNxtpRMxw53_D3T2GgTh-k5w2L7SlLt7C6x2Sw=.22a28455-ed36-4ad4-998c-7de38eb7b332@github.com>
On Mon, 10 Jun 2024 21:39:25 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Merge tag jdk-24+1, without free set changes.
==============================
Test summary
==============================
TEST TOTAL PASS FAIL ERROR
jtreg:test/hotspot/jtreg:tier1_gc_shenandoah 91 91 0 0
==============================
TEST SUCCESS
-------------
PR Comment: https://git.openjdk.org/shenandoah/pull/447#issuecomment-2159331508
From wkemper at openjdk.org Mon Jun 10 22:19:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 22:19:45 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <m5W5QAhpH_NmUujV1pHqNA-e5jPTwYLfvm6-MENGlpM=.633152f0-cec2-4dd4-b43e-bd803ca97c91@github.com>
References: <m5W5QAhpH_NmUujV1pHqNA-e5jPTwYLfvm6-MENGlpM=.633152f0-cec2-4dd4-b43e-bd803ca97c91@github.com>
Message-ID: <kij2iB6uhjo007OAopQoE2TAl2pkjw3Rdkd0S1jrBn0=.a076151b-da14-4aa1-a87b-28ef4abfd01d@github.com>
On Mon, 10 Jun 2024 21:39:25 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Merge tag jdk-24+1, without free set changes.
This pull request has now been integrated.
Changeset: 56611de7
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/56611de7744a9e9d872ccc85b745734abb6cb354
Stats: 91891 lines in 1767 files changed: 66786 ins; 16868 del; 8237 mod
Merge
-------------
PR: https://git.openjdk.org/shenandoah/pull/447
From wkemper at openjdk.org Mon Jun 10 22:20:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 22:20:01 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <hKShEAF4YkXn29eJigBB3RVLjRQ4UKdgJq-mIhMRpB4=.22cc2608-ea19-45a2-a966-8401b9bf63ad@github.com>
References: <hKShEAF4YkXn29eJigBB3RVLjRQ4UKdgJq-mIhMRpB4=.22cc2608-ea19-45a2-a966-8401b9bf63ad@github.com>
Message-ID: <XDiGq6ESce-RF3Aj3lqBDfHgI7YlGn2vuYtJSVbW7H4=.ddb0a52b-62ce-4b7b-bcaa-d70907a129bf@github.com>
> Merges tag jdk-23+24
William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/438/files
- new: https://git.openjdk.org/shenandoah/pull/438/files/9d332e65..9d332e65
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=438&range=01
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=438&range=00-01
Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/438.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/438/head:pull/438
PR: https://git.openjdk.org/shenandoah/pull/438
From duke at openjdk.org Mon Jun 10 22:20:01 2024
From: duke at openjdk.org (duke)
Date: Mon, 10 Jun 2024 22:20:01 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <hKShEAF4YkXn29eJigBB3RVLjRQ4UKdgJq-mIhMRpB4=.22cc2608-ea19-45a2-a966-8401b9bf63ad@github.com>
References: <hKShEAF4YkXn29eJigBB3RVLjRQ4UKdgJq-mIhMRpB4=.22cc2608-ea19-45a2-a966-8401b9bf63ad@github.com>
Message-ID: <1b4RaR30QqxG-Ogs8lz9edZzATgBpeS5RJuV2lneGqk=.fc48ed01-d118-48e3-81be-e9d9c460eb0d@github.com>
On Fri, 24 May 2024 14:11:32 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Merges tag jdk-23+24
This pull request has been closed without being integrated.
-------------
PR: https://git.openjdk.org/shenandoah/pull/438
From wkemper at openjdk.org Mon Jun 10 22:20:10 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 22:20:10 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <cZF_aBjMPKsInzDiq8G9ga7m7onD0OXZL0ZCy_I4P0k=.7d96d8d9-6765-41c9-88bd-20fbefb78b88@github.com>
References: <cZF_aBjMPKsInzDiq8G9ga7m7onD0OXZL0ZCy_I4P0k=.7d96d8d9-6765-41c9-88bd-20fbefb78b88@github.com>
Message-ID: <yjXUJ-WXwOZrxiIdzgY2kseNWNSnPQ15FDXJOjHwpL0=.51766f10-abce-43a5-a58a-24f502966690@github.com>
> Merges tag jdk-23+25
William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/442/files
- new: https://git.openjdk.org/shenandoah/pull/442/files/789ac8b2..789ac8b2
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=442&range=01
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=442&range=00-01
Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/442.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/442/head:pull/442
PR: https://git.openjdk.org/shenandoah/pull/442
From duke at openjdk.org Mon Jun 10 22:20:10 2024
From: duke at openjdk.org (duke)
Date: Mon, 10 Jun 2024 22:20:10 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <cZF_aBjMPKsInzDiq8G9ga7m7onD0OXZL0ZCy_I4P0k=.7d96d8d9-6765-41c9-88bd-20fbefb78b88@github.com>
References: <cZF_aBjMPKsInzDiq8G9ga7m7onD0OXZL0ZCy_I4P0k=.7d96d8d9-6765-41c9-88bd-20fbefb78b88@github.com>
Message-ID: <gU7dBGF_izUZ-CeyZqb52ztm6sgHldMogvq66mBVxT4=.bac43f67-23b0-4ac2-a15f-ebe1102bc859@github.com>
On Fri, 31 May 2024 14:10:35 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Merges tag jdk-23+25
This pull request has been closed without being integrated.
-------------
PR: https://git.openjdk.org/shenandoah/pull/442
From wkemper at openjdk.org Mon Jun 10 22:27:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 22:27:00 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <KSwcYbCYc5BSws2jdlBnIfvewzSFUapBMtfWYLBn3h8=.48e05c25-98fc-48db-9bd7-958a56d9aa98@github.com>
References: <KSwcYbCYc5BSws2jdlBnIfvewzSFUapBMtfWYLBn3h8=.48e05c25-98fc-48db-9bd7-958a56d9aa98@github.com>
Message-ID: <fsBhnqgX6SL3uLWLcMJErvPHL1So643a9tX0_dgLZ-s=.41ce3b14-94b1-4526-b0ed-ca71e3d109e2@github.com>
> Merges tag jdk-23+26
William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/445/files
- new: https://git.openjdk.org/shenandoah/pull/445/files/31696a44..31696a44
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=445&range=01
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=445&range=00-01
Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/445.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/445/head:pull/445
PR: https://git.openjdk.org/shenandoah/pull/445
From wkemper at openjdk.org Mon Jun 10 22:27:00 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 22:27:00 GMT
Subject: RFR: Merge openjdk/jdk:master
In-Reply-To: <KSwcYbCYc5BSws2jdlBnIfvewzSFUapBMtfWYLBn3h8=.48e05c25-98fc-48db-9bd7-958a56d9aa98@github.com>
References: <KSwcYbCYc5BSws2jdlBnIfvewzSFUapBMtfWYLBn3h8=.48e05c25-98fc-48db-9bd7-958a56d9aa98@github.com>
Message-ID: <94sFnZM1CVxTPqUPhBD2pUQnByCturToGPyMw5HhvGs=.5f599407-9c47-4beb-aa06-deae16842572@github.com>
On Fri, 7 Jun 2024 14:12:05 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Merges tag jdk-23+26
Superseded by https://github.com/openjdk/shenandoah/pull/447
-------------
PR Comment: https://git.openjdk.org/shenandoah/pull/445#issuecomment-2159409300
From wkemper at openjdk.org Mon Jun 10 22:27:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 22:27:01 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <KSwcYbCYc5BSws2jdlBnIfvewzSFUapBMtfWYLBn3h8=.48e05c25-98fc-48db-9bd7-958a56d9aa98@github.com>
References: <KSwcYbCYc5BSws2jdlBnIfvewzSFUapBMtfWYLBn3h8=.48e05c25-98fc-48db-9bd7-958a56d9aa98@github.com>
Message-ID: <6ifhjN_cl2aqC5Mko4skIcvn6M00AjklHUjKN0TKSd0=.a3d07c43-c167-4dd9-98de-aaa45ddeada4@github.com>
On Fri, 7 Jun 2024 14:12:05 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Merges tag jdk-23+26
This pull request has been closed without being integrated.
-------------
PR: https://git.openjdk.org/shenandoah/pull/445
From wkemper at openjdk.org Mon Jun 10 22:38:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 22:38:35 GMT
Subject: RFR: 8333925: Shenandoah: Heuristics should have an option to ignore
abbreviated cycles
Message-ID: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
After concurrent marking is complete, Shenandoah will skip the evacuation phase if a sufficient amount of garbage is found in regions that contain _no live objects_. These abbreviated cycles are much shorter than a cycle that performs evacuation and update references and tend to lower the average cycle time used by the heuristic to predict cycle times. This may cause the heuristic to wait too long to initiate a cycle and may lead to degenerated cycles. This change has the heuristic ignore abbreviated cycle times by default, with an option to have the heuristic count them as it does now.
-------------
Commit messages:
- Adaptive heuristic should ignore abbreviated cycles
Changes: https://git.openjdk.org/jdk/pull/19640/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19640&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333925
Stats: 48 lines in 7 files changed: 26 ins; 2 del; 20 mod
Patch: https://git.openjdk.org/jdk/pull/19640.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19640/head:pull/19640
PR: https://git.openjdk.org/jdk/pull/19640
From wkemper at openjdk.org Mon Jun 10 22:57:23 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 10 Jun 2024 22:57:23 GMT
Subject: RFR: 8333926: Shenandoah: Lower default immediate garbage threshold
Message-ID: <VOQFUcrOsWrnFrE4RURyTfe6_VX6pAZZRWu8urrtJLE=.10d5367a-4eb6-46e2-8e8e-174de302dfe0@github.com>
After marking, when Shenandoah finds a certain percentage threshold of garbage within regions that contain no live objects it will skip the evacuation phase. This threshold is currently 90%, but we (Amazon) have found that many internal workloads benefit significantly by reducing this threshold to 70%. We have also seen improvements on dacapo/sunflow and specjbb2015. We have also not seen any performance degradation caused by lowering this threshold.
-------------
Commit messages:
- Default immediate threshold to 70
Changes: https://git.openjdk.org/jdk/pull/19641/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19641&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333926
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/jdk/pull/19641.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19641/head:pull/19641
PR: https://git.openjdk.org/jdk/pull/19641
From wkemper at openjdk.org Tue Jun 11 00:11:45 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 00:11:45 GMT
Subject: RFR: 8333930: GenShen: Check for cancellation of old mark after final
mark
Message-ID: <BOLgAnyGi_gSPujLcTwQiZmA-oy9_9kmFpQCPIPAT9o=.9ae7a67b-7439-4378-aa10-81dee5a3fbdf@github.com>
This is the old generation mark version of https://github.com/openjdk/jdk/pull/19434
-------------
Commit messages:
- Check for cancellation after final mark
Changes: https://git.openjdk.org/shenandoah/pull/448/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=448&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333930
Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/448.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/448/head:pull/448
PR: https://git.openjdk.org/shenandoah/pull/448
From wkemper at openjdk.org Tue Jun 11 00:25:29 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 00:25:29 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <PTd4Hx2PZOQmTgDCbFjOoK7zqn5_sGAzmj0SeuLt5ZA=.050afed0-2e6b-4962-badd-e0d7f86af9e3@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
<oOIVEXXSFecZ-QrX7HQ3wqcBSL6LqWMiOEnAEH0ECYE=.8b274914-b5ef-4096-a3e9-bd89650a0089@github.com>
<PTd4Hx2PZOQmTgDCbFjOoK7zqn5_sGAzmj0SeuLt5ZA=.050afed0-2e6b-4962-badd-e0d7f86af9e3@github.com>
Message-ID: <fanqkXcWg1Xj2sXuwU8N4yC3scOC96mYVAn_BqMediw=.386aa0ae-dd3e-4ffa-93c4-38d1683adb17@github.com>
On Mon, 10 Jun 2024 18:28:56 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahOldGC.cpp line 119:
>>
>>> 117: return false;
>>> 118: }
>>> 119:
>>
>> This looks like it might be a bug fix, vs the rest of this PR, which is mostly isolation refactoring. Should the bug fix be separated out into a different PR?
>
> Yes, it is a bug fix. I'll pull it out into a separate PR.
Separate PR for this here: https://github.com/openjdk/shenandoah/pull/448
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634002836
From ysr at openjdk.org Tue Jun 11 03:27:32 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 11 Jun 2024 03:27:32 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
Message-ID: <OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
On Fri, 7 Jun 2024 23:39:57 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> * Other modes should not instantiate or touch young/old generations.
>> * The generation sizer has been put in its own file and moved out of shHeap
>> * The logic for aging regions has been decoupled from non-generational final update refs
>
> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
>
> - Make age and youth const
> - Do not age regions after completing old mark
> - Detect cancellation of old GC after final mark.
LGTM; left a few minor/optional comments, none of which require a re-review.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 734:
> 732: } else {
> 733: _heap->generation_for(r->affiliation())->increment_affiliated_region_count();
> 734: }
Can the affiliation update be done unconditionally outside of the if-else, as when the affiliation is set at line 721, or if it needs to be done after the old region coalesce/fill/card-clearing, then after that at line 733, but unconditionally, skipping the update at line 731 for the old gen?
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2044:
> 2042: }
> 2043:
> 2044: global_generation()->cancel_marking();
Not that this matters, but there are a few methods `foo()` on the heap that do an operation on global and, in generational mode, on old and young. If possible, I'd use the same idiom in them uniformly(unless there is a good reason not to), i.e.
`ShenandoahHeap::foo() {
global_gen->foo();
if (generational) {
young->foo();
old->foo();
}
`
In fact, you could just do this by means of a macro like `ALL_GENS(x)` which does this for you?
e.g. `set_mark_complete`, `set_mark_incomplete`, etc. (Not sure if there are several others in the same fashion, and therefore if it's worthwhile to macro-ize or not.
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2369:
> 2367:
> 2368: void ShenandoahSynchronizePinnedRegionStates::heap_region_do(ShenandoahHeapRegion* r) {
> 2369: // Drop unnecessary "pinned" state from regions that does not have CP marks
May be "critical pins" instead of the more abstruse "CP marks"?
also "does" -> "do".
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2408:
> 2406: }
> 2407:
> 2408: void ShenandoahHeap::final_update_refs_do_regions() {
The name of this method is too generic, and doesn't tell you if and what you are doing? Is there a better possible name?
I see that it's a virtual method and in one case does pinning recalibration and in the other additionally region age updates. Is there a good name for these two?
May be `final_update_refs_update_region_states()` analogous to the name of the timing phase, like for the `trash_cset` case below?
src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 460:
> 458: // Final update region states
> 459: void update_heap_region_states(bool concurrent);
> 460: virtual void final_update_refs_do_regions();
Or, vide my suggestion above.
You could rename the current `update_heap_region_states()` to `update_state_and_trash()`, and then rename the virtual method to `update_heap_region_states()`. I realize this will cause changes to upstream legacy Shenandoah code, but makes the naming clearer.
This is a "weak" suggestion; you should do what feels better here (in terms of yr earlier point re "diminishing returns").
src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 505:
> 503: return _old_generation;
> 504: }
> 505:
I presume that the presence of these methods here indicates that the "isolation factoring" still isn't complete, but I realize that's a tough bar to get to (and as you stated may be we are at a point of "diminishing benefits").
src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 140:
> 138: if (heap->mode()->is_generational()) {
> 139: heap->young_generation()->increment_affiliated_region_count();
> 140: }
I assume the strict order here is on account of assertion checks in some of these methods? Otherwise, one might simplify it by moving the affiliation setting out from the sandwich of the count adjustments which are conditional on generational mode.
Ideally the count for young gen in non-generational mode would work for the generational case, and we could just adjust the old count like now and drop the adjustment of young count (lines 138-140) entirely, because they are covered by line 137.
-------------
Marked as reviewed by ysr (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/444#pullrequestreview-2105412096
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634024066
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634058105
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634061284
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634093821
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634110341
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634111111
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634113529
From ysr at openjdk.org Tue Jun 11 03:27:33 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 11 Jun 2024 03:27:33 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v2]
In-Reply-To: <d3TgxylixrAMhUOFOrSWqWma--uhZL1Vfyy_Jl-GOPw=.c894beb0-683b-4fee-94bd-7121cdc59421@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<d3TgxylixrAMhUOFOrSWqWma--uhZL1Vfyy_Jl-GOPw=.c894beb0-683b-4fee-94bd-7121cdc59421@github.com>
Message-ID: <lHUqIwkk5M86DKC-EiEL2qdFaSHu7L95qWXb0uorIJU=.8b5356e7-76b0-4bed-b327-04c1b05e1d20@github.com>
On Thu, 6 Jun 2024 21:12:50 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> * Other modes should not instantiate or touch young/old generations.
>> * The generation sizer has been put in its own file and moved out of shHeap
>> * The logic for aging regions has been decoupled from non-generational final update refs
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>
> Fix zero build
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.cpp line 138:
> 136: // This is safe iff it is assured that each PLAB is a whole-number multiple of card-mark memory size and each
> 137: // PLAB is aligned with the start of each card's memory range.
> 138: // TODO: Assert this in retire_plab?
I recall seeing this comment in other code I recently touched. (I made the comment a bit more succinct there.) Looking at this here and at that place, I now feel this portion of the comment actually belongs in the implementation of `retire_plab()` where the filling & registration are done, rather than being repeated at each place where we call `retire_plab()` as is the case currently.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1631717951
From ysr at openjdk.org Tue Jun 11 03:27:33 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 11 Jun 2024 03:27:33 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <fanqkXcWg1Xj2sXuwU8N4yC3scOC96mYVAn_BqMediw=.386aa0ae-dd3e-4ffa-93c4-38d1683adb17@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
<oOIVEXXSFecZ-QrX7HQ3wqcBSL6LqWMiOEnAEH0ECYE=.8b274914-b5ef-4096-a3e9-bd89650a0089@github.com>
<PTd4Hx2PZOQmTgDCbFjOoK7zqn5_sGAzmj0SeuLt5ZA=.050afed0-2e6b-4962-badd-e0d7f86af9e3@github.com>
<fanqkXcWg1Xj2sXuwU8N4yC3scOC96mYVAn_BqMediw=.386aa0ae-dd3e-4ffa-93c4-38d1683adb17@github.com>
Message-ID: <mpcLKFy_Xw2msAl8umOY3ovFcqiSTx523xpyd3HzsY0=.52311a60-e7ef-4332-a95d-55b6c9cdec61@github.com>
On Tue, 11 Jun 2024 00:23:06 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Yes, it is a bug fix. I'll pull it out into a separate PR.
>
> Separate PR for this here: https://github.com/openjdk/shenandoah/pull/448
Thanks for pulling this out separately. I do think this is subtle and we should (in the fullness of time) make this idiom a bit cleaner and less fragile and uniform in terms of the representation of the state machine that can be used by both the concurrent and the degenerate resumptions.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1634123989
From ysr at openjdk.org Tue Jun 11 03:30:36 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 11 Jun 2024 03:30:36 GMT
Subject: RFR: 8333930: GenShen: Check for cancellation of old mark after
final mark
In-Reply-To: <BOLgAnyGi_gSPujLcTwQiZmA-oy9_9kmFpQCPIPAT9o=.9ae7a67b-7439-4378-aa10-81dee5a3fbdf@github.com>
References: <BOLgAnyGi_gSPujLcTwQiZmA-oy9_9kmFpQCPIPAT9o=.9ae7a67b-7439-4378-aa10-81dee5a3fbdf@github.com>
Message-ID: <bVsuTCjmzCJ115t4bJfoAQ13Cu9AP_w83ix8FABP6Co=.5a554546-41da-4078-ad18-2523fb462607@github.com>
On Tue, 11 Jun 2024 00:08:18 GMT, William Kemper <wkemper at openjdk.org> wrote:
> This is the old generation mark version of https://github.com/openjdk/jdk/pull/19434
Marked as reviewed by ysr (Committer).
-------------
PR Review: https://git.openjdk.org/shenandoah/pull/448#pullrequestreview-2109223141
From dholmes at openjdk.org Tue Jun 11 05:41:12 2024
From: dholmes at openjdk.org (David Holmes)
Date: Tue, 11 Jun 2024 05:41:12 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort
In-Reply-To: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
Message-ID: <nYycWzpNJUUKPhesxmPR2PUcsUXCwkn0aoVcgcosz70=.0031103d-b722-4daf-9cf1-469a37c129cb@github.com>
On Wed, 29 May 2024 18:52:03 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
> The "idempotent" argument is removed from that function, with associated
> simplifications to the implementation. Callers are updated to remove that
> argument. Callers that were providing a false value are unaffected in their
> behavior. The 3 callers that were providing a true value to request the
> associated feature are also unaffected (other than by being made faster),
> because the arrays involved don't contain any equivalent pairs.
>
> There are also some miscellaneous cleanups, including using the swap utility
> and fixing some comments.
>
> Testing: mach5 tier1-3
So .... IIUC the only code that would be affected by this change would be code that passes true, which could also have equivalent elements to sort, and which requires the sort order to always be the same regardless of the order the elements are found. I think only the archive related code cares about deterministic order, and package and module names should be unique, so this seems fine.
One pre-existing nit in a comment but otherwise looks good.
Thanks
src/hotspot/share/utilities/quickSort.hpp line 43:
> 41: // We swap these three values into the right place in the array. This
> 42: // means that this method not only returns the index of the pivot
> 43: // element. It also alters the array so that:
Pre-existing nit: this should be one sentence: "... element, it also ..."
-------------
Marked as reviewed by dholmes (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19464#pullrequestreview-2109357549
PR Review Comment: https://git.openjdk.org/jdk/pull/19464#discussion_r1634213493
From kdnilsen at openjdk.org Tue Jun 11 16:04:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Jun 2024 16:04:12 GMT
Subject: RFR: 8333926: Shenandoah: Lower default immediate garbage
threshold
In-Reply-To: <VOQFUcrOsWrnFrE4RURyTfe6_VX6pAZZRWu8urrtJLE=.10d5367a-4eb6-46e2-8e8e-174de302dfe0@github.com>
References: <VOQFUcrOsWrnFrE4RURyTfe6_VX6pAZZRWu8urrtJLE=.10d5367a-4eb6-46e2-8e8e-174de302dfe0@github.com>
Message-ID: <-TznARKzcTOQ4EXhPiI3w_z52RV-euUO2t8UZyhkkd0=.a139ae69-d948-4f68-bb5e-fb12fab178a6@github.com>
On Mon, 10 Jun 2024 22:53:07 GMT, William Kemper <wkemper at openjdk.org> wrote:
> After marking, when Shenandoah finds a certain percentage threshold of garbage within regions that contain no live objects it will skip the evacuation phase. This threshold is currently 90%, but we (Amazon) have found that many internal workloads benefit significantly by reducing this threshold to 70%. We have also seen improvements on dacapo/sunflow and specjbb2015. We have also not seen any performance degradation caused by lowering this threshold.
Marked as reviewed by kdnilsen (no project role).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19641#pullrequestreview-2110886323
From kdnilsen at openjdk.org Tue Jun 11 16:05:15 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Jun 2024 16:05:15 GMT
Subject: RFR: 8333925: Shenandoah: Heuristics should have an option to
ignore abbreviated cycles
In-Reply-To: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
References: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
Message-ID: <OWsDe0PxX8TF4qBCSkAK6qhquyZWTjFyq0Q_7I1hBpc=.c1e91ab1-a1f6-4360-8081-bd5ea2e49ab4@github.com>
On Mon, 10 Jun 2024 22:34:00 GMT, William Kemper <wkemper at openjdk.org> wrote:
> After concurrent marking is complete, Shenandoah will skip the evacuation phase if a sufficient amount of garbage is found in regions that contain _no live objects_. These abbreviated cycles are much shorter than a cycle that performs evacuation and update references and tend to lower the average cycle time used by the heuristic to predict cycle times. This may cause the heuristic to wait too long to initiate a cycle and may lead to degenerated cycles. This change has the heuristic ignore abbreviated cycle times by default, with an option to have the heuristic count them as it does now.
Marked as reviewed by kdnilsen (no project role).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19640#pullrequestreview-2110889128
From wkemper at openjdk.org Tue Jun 11 16:14:34 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 16:14:34 GMT
Subject: Integrated: 8333930: GenShen: Check for cancellation of old mark after
final mark
In-Reply-To: <BOLgAnyGi_gSPujLcTwQiZmA-oy9_9kmFpQCPIPAT9o=.9ae7a67b-7439-4378-aa10-81dee5a3fbdf@github.com>
References: <BOLgAnyGi_gSPujLcTwQiZmA-oy9_9kmFpQCPIPAT9o=.9ae7a67b-7439-4378-aa10-81dee5a3fbdf@github.com>
Message-ID: <9kCKtyGIKgk8qZcjtMf26F-PMNbLZxMv-O92FGb4Xyk=.6741b3b5-6606-470e-87f5-ebea6a70c026@github.com>
On Tue, 11 Jun 2024 00:08:18 GMT, William Kemper <wkemper at openjdk.org> wrote:
> This is the old generation mark version of https://github.com/openjdk/jdk/pull/19434
This pull request has now been integrated.
Changeset: e3c3098a
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/e3c3098a682be9041045140cb89bdda43f4bff30
Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod
8333930: GenShen: Check for cancellation of old mark after final mark
Reviewed-by: ysr
-------------
PR: https://git.openjdk.org/shenandoah/pull/448
From wkemper at openjdk.org Tue Jun 11 18:06:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 18:06:38 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
<OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
Message-ID: <9bdi7ySmV1wcOJpBOopujlDSFZIYO0je3u7T4CtYlts=.012d0c23-e954-4bf5-bb60-50a165e28b4c@github.com>
On Tue, 11 Jun 2024 01:06:42 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
>>
>> - Make age and youth const
>> - Do not age regions after completing old mark
>> - Detect cancellation of old GC after final mark.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 734:
>
>> 732: } else {
>> 733: _heap->generation_for(r->affiliation())->increment_affiliated_region_count();
>> 734: }
>
> Can the affiliation update be done unconditionally outside of the if-else, as when the affiliation is set at line 721, or if it needs to be done after the old region coalesce/fill/card-clearing, then after that at line 733, but unconditionally, skipping the update at line 731 for the old gen?
Yes, that should be fine.
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2044:
>
>> 2042: }
>> 2043:
>> 2044: global_generation()->cancel_marking();
>
> Not that this matters, but there are a few methods `foo()` on the heap that do an operation on global and, in generational mode, on old and young. If possible, I'd use the same idiom in them uniformly(unless there is a good reason not to), i.e.
> `ShenandoahHeap::foo() {
> global_gen->foo();
> if (generational) {
> young->foo();
> old->foo();
> }
> `
>
> In fact, you could just do this by means of a macro like `ALL_GENS(x)` which does this for you?
>
> e.g. `set_mark_complete`, `set_mark_incomplete`, etc. (Not sure if there are several others in the same fashion, and therefore if it's worthwhile to macro-ize or not.
I'll consider this for another PR. I think there are only a couple of places that would fit such a macro precisely and I'm not sure the obfuscating effects would be worth it.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1635284848
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1635291010
From kdnilsen at openjdk.org Tue Jun 11 18:23:01 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Jun 2024 18:23:01 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v2]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <KGTGhmXTBMaRr7TDrBSercHd0aRHU_whZfxGevawaOA=.c4df1517-7a4c-444d-8bba-25e6803e9550@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 28 commits:
- Merge remote-tracking branch 'origin/master' into harmonize-free-set
- Merge branch 'openjdk:master' into master
- Remove TestHumongousThreshold test from problem list
- Fix white space
- Merge branch 'master' of https://git.openjdk.org/shenandoah into harmonize-free-set
- Remove debugging instrumentation
- Merge remote-tracking branch 'origin/master' into harmonize-free-set
- Merge branch 'openjdk:master' into master
- Enhance diagnostic message for verification error
- Some fixes related to TestThreadFailure.java
- ... and 18 more: https://git.openjdk.org/shenandoah/compare/e3c3098a...bc3cba66
-------------
Changes: https://git.openjdk.org/shenandoah/pull/440/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=01
Stats: 1659 lines in 14 files changed: 865 ins; 219 del; 575 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From wkemper at openjdk.org Tue Jun 11 18:35:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 18:35:39 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
<OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
Message-ID: <_0ufb1wv8PrdKo1vK2kpJ-_ggivjFzxNOS3shCeVb2Y=.9b800ddd-39cf-4c3e-ad6c-037b795f1f7c@github.com>
On Tue, 11 Jun 2024 02:45:21 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
>>
>> - Make age and youth const
>> - Do not age regions after completing old mark
>> - Detect cancellation of old GC after final mark.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2408:
>
>> 2406: }
>> 2407:
>> 2408: void ShenandoahHeap::final_update_refs_do_regions() {
>
> The name of this method is too generic, and doesn't tell you if and what you are doing? Is there a better possible name?
>
> I see that it's a virtual method and in one case does pinning recalibration and in the other additionally region age updates. Is there a good name for these two?
>
> May be `final_update_refs_update_region_states()` analogous to the name of the timing phase, like for the `trash_cset` case below?
Yes, I thought the original name was too generic. `update_heap_region_states` doesn't indicate which phase of the collection it pertains to. I'll go with `final_update_refs_update_region_states` to match the timing phase.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1635322211
From wkemper at openjdk.org Tue Jun 11 18:48:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 18:48:28 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
<OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
Message-ID: <6Zi4yK0TjiP3fNwEsDxvtWDiOGeA4GI26dQn-l6KD38=.96ea968b-983f-4024-b991-5b1cf4b00f7d@github.com>
On Tue, 11 Jun 2024 03:03:51 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
>>
>> - Make age and youth const
>> - Do not age regions after completing old mark
>> - Detect cancellation of old GC after final mark.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 140:
>
>> 138: if (heap->mode()->is_generational()) {
>> 139: heap->young_generation()->increment_affiliated_region_count();
>> 140: }
>
> I assume the strict order here is on account of assertion checks in some of these methods? Otherwise, one might simplify it by moving the affiliation setting out from the sandwich of the count adjustments which are conditional on generational mode.
>
> Ideally the count for young gen in non-generational mode would work for the generational case, and we could just adjust the old count like now and drop the adjustment of young count (lines 138-140) entirely, because they are covered by line 137.
We should be able to remove one of the mode checks. We don't maintain affiliated region counts in non-generational modes. There is a `TODO` for this: https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp#L872
I think this PR makes it easier to tackle that `TODO`, but it will be a separate PR.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1635336191
From wkemper at openjdk.org Tue Jun 11 18:58:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 18:58:28 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <9bdi7ySmV1wcOJpBOopujlDSFZIYO0je3u7T4CtYlts=.012d0c23-e954-4bf5-bb60-50a165e28b4c@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
<OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
<9bdi7ySmV1wcOJpBOopujlDSFZIYO0je3u7T4CtYlts=.012d0c23-e954-4bf5-bb60-50a165e28b4c@github.com>
Message-ID: <8pvktaF6ADDs5PYG0SRI-PIbq3YJDO4Iyf6VDt4Ocak=.65f36a5c-8ad0-4d3a-8653-d4adfc317910@github.com>
On Tue, 11 Jun 2024 17:58:51 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 734:
>>
>>> 732: } else {
>>> 733: _heap->generation_for(r->affiliation())->increment_affiliated_region_count();
>>> 734: }
>>
>> Can the affiliation update be done unconditionally outside of the if-else, as when the affiliation is set at line 721, or if it needs to be done after the old region coalesce/fill/card-clearing, then after that at line 733, but unconditionally, skipping the update at line 731 for the old gen?
>
> Yes, that should be fine.
Yes, this should be fine.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1635346127
From wkemper at openjdk.org Tue Jun 11 18:58:28 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 18:58:28 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v3]
In-Reply-To: <OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
<DR1GcMWG0tlo5JW0qAl6YdBs1iE9DJmIUxJXV7gIyiY=.b80c39fa-cbda-46df-9271-4942f4745de3@github.com>
<OFHmiaGi-AB6EinxfB2_yiJ6LoxyiHQ7qpTQYBRYV60=.5c14c12f-b435-46d9-83b7-36a0c163b5c4@github.com>
Message-ID: <sbass6jT2Eb4olmyJ7y0d9uHd-dN3wTQvI8Xx4cc6u0=.36625217-ea3f-4c38-ab51-4aecefc1f55b@github.com>
On Tue, 11 Jun 2024 02:07:00 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with three additional commits since the last revision:
>>
>> - Make age and youth const
>> - Do not age regions after completing old mark
>> - Detect cancellation of old GC after final mark.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2369:
>
>> 2367:
>> 2368: void ShenandoahSynchronizePinnedRegionStates::heap_region_do(ShenandoahHeapRegion* r) {
>> 2369: // Drop unnecessary "pinned" state from regions that does not have CP marks
>
> May be "critical pins" instead of the more abstruse "CP marks"?
>
> also "does" -> "do".
I've simplified this comment.
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 505:
>
>> 503: return _old_generation;
>> 504: }
>> 505:
>
> I presume that the presence of these methods here indicates that the "isolation factoring" still isn't complete, but I realize that's a tough bar to get to (and as you stated may be we are at a point of "diminishing benefits").
Yes, there are some usages internal to `ShenandoahHeap` that would be difficult to factor out without making certain "hot" methods `virtual` (on the allocation path, rebuilding free set, e.g.). I'm reluctant to do this for performance reasons, but perhaps it's worth the experiment?
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1635343709
PR Review Comment: https://git.openjdk.org/shenandoah/pull/444#discussion_r1635343244
From kdnilsen at openjdk.org Tue Jun 11 19:41:55 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Jun 2024 19:41:55 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v3]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <NcxBajUceJ7Y0MZYHPQc7wRwL79tshwccvhSt-CzJ9E=.cf0ab476-369c-4b31-8d44-d3bbcb721fe3@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Resolve conflicts introduced by manual resolution of merge
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/bc3cba66..ac9ecfe9
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=02
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=01-02
Stats: 50 lines in 4 files changed: 23 ins; 15 del; 12 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Tue Jun 11 19:48:01 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 11 Jun 2024 19:48:01 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v4]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <O6C8ppLpTjaDq804QzcZtC41qvOK5jSJW-zK8Cs2Dt0=.7ddc3445-d03f-4b98-81a8-da316e85bdc4@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Fix whitespace
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/ac9ecfe9..67c0d742
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=03
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=02-03
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From ysr at openjdk.org Tue Jun 11 20:03:16 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 11 Jun 2024 20:03:16 GMT
Subject: RFR: 8333925: Shenandoah: Heuristics should have an option to
ignore abbreviated cycles
In-Reply-To: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
References: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
Message-ID: <GtmRdDUl_KkPILQte9rzAJu9fMJiIhY1lkNQn9rzSPM=.6e32e8a9-fe7e-4183-a221-bcbd44fd0629@github.com>
On Mon, 10 Jun 2024 22:34:00 GMT, William Kemper <wkemper at openjdk.org> wrote:
> After concurrent marking is complete, Shenandoah will skip the evacuation phase if a sufficient amount of garbage is found in regions that contain _no live objects_. These abbreviated cycles are much shorter than a cycle that performs evacuation and update references and tend to lower the average cycle time used by the heuristic to predict cycle times. This may cause the heuristic to wait too long to initiate a cycle and may lead to degenerated cycles. This change has the heuristic ignore abbreviated cycle times by default, with an option to have the heuristic count them as it does now.
LGTM.
Any performance numbers or improvements to share?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.cpp line 3:
> 1: /*
> 2: * Copyright (c) 2018, 2020, Red Hat, Inc. All rights reserved.
> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
Remove?
src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 152:
> 150: range(0,1.0) \
> 151: \
> 152: product(bool, ShenandoahAdaptiveIgnoreShortCycles, true, EXPERIMENTAL, \
Abbreviated instead of Short,for consistency with use of terms in code?
-------------
Marked as reviewed by ysr (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19640#pullrequestreview-2111327384
PR Review Comment: https://git.openjdk.org/jdk/pull/19640#discussion_r1635405682
PR Review Comment: https://git.openjdk.org/jdk/pull/19640#discussion_r1635411480
From ysr at openjdk.org Tue Jun 11 20:07:14 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 11 Jun 2024 20:07:14 GMT
Subject: RFR: 8333926: Shenandoah: Lower default immediate garbage
threshold
In-Reply-To: <VOQFUcrOsWrnFrE4RURyTfe6_VX6pAZZRWu8urrtJLE=.10d5367a-4eb6-46e2-8e8e-174de302dfe0@github.com>
References: <VOQFUcrOsWrnFrE4RURyTfe6_VX6pAZZRWu8urrtJLE=.10d5367a-4eb6-46e2-8e8e-174de302dfe0@github.com>
Message-ID: <2HNTTChjhpd9v_YY_UCLUeyylkj9tdcN-Ejm8H-96xc=.48ec13ff-aeb5-4d16-9be1-533f44636408@github.com>
On Mon, 10 Jun 2024 22:53:07 GMT, William Kemper <wkemper at openjdk.org> wrote:
> After marking, when Shenandoah finds a certain percentage threshold of garbage within regions that contain no live objects it will skip the evacuation phase. This threshold is currently 90%, but we (Amazon) have found that many internal workloads benefit significantly by reducing this threshold to 70%. We have also seen improvements on dacapo/sunflow and specjbb2015. We have also not seen any performance degradation caused by lowering this threshold.
Marked as reviewed by ysr (Reviewer).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19641#pullrequestreview-2111356199
From wkemper at openjdk.org Tue Jun 11 20:20:40 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 20:20:40 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode [v4]
In-Reply-To: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
Message-ID: <MNcSUmQ1_IhZbF2zVBa3J--oDAYmCqZWjUSyfYMZ8MY=.0b718da8-38c7-4f05-a3e5-a118fb1ee9b1@github.com>
> * Other modes should not instantiate or touch young/old generations.
> * The generation sizer has been put in its own file and moved out of shHeap
> * The logic for aging regions has been decoupled from non-generational final update refs
William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 38 commits:
- Simplify affiliation accounting on allocation path
- Reduce mode checks when making a region young
- Improve method name and comments
- Merge remote-tracking branch 'shenandoah/master' into move-young-old-generations
- Make age and youth const
- Do not age regions after completing old mark
- Detect cancellation of old GC after final mark.
- Fix zero build
- Add missing precompiled header
- Fix merge error
- ... and 28 more: https://git.openjdk.org/shenandoah/compare/e3c3098a...694a4d29
-------------
Changes: https://git.openjdk.org/shenandoah/pull/444/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=444&range=03
Stats: 890 lines in 22 files changed: 471 ins; 323 del; 96 mod
Patch: https://git.openjdk.org/shenandoah/pull/444.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/444/head:pull/444
PR: https://git.openjdk.org/shenandoah/pull/444
From wkemper at openjdk.org Tue Jun 11 22:12:15 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 22:12:15 GMT
Subject: Integrated: 8333926: Shenandoah: Lower default immediate garbage
threshold
In-Reply-To: <VOQFUcrOsWrnFrE4RURyTfe6_VX6pAZZRWu8urrtJLE=.10d5367a-4eb6-46e2-8e8e-174de302dfe0@github.com>
References: <VOQFUcrOsWrnFrE4RURyTfe6_VX6pAZZRWu8urrtJLE=.10d5367a-4eb6-46e2-8e8e-174de302dfe0@github.com>
Message-ID: <dO4ewSfQ__vlU0yUGDH5_38B0ZlTMi7bKwN_GHprMk0=.8c45eb2a-5091-48ab-84dd-6e1ffe70d14a@github.com>
On Mon, 10 Jun 2024 22:53:07 GMT, William Kemper <wkemper at openjdk.org> wrote:
> After marking, when Shenandoah finds a certain percentage threshold of garbage within regions that contain no live objects it will skip the evacuation phase. This threshold is currently 90%, but we (Amazon) have found that many internal workloads benefit significantly by reducing this threshold to 70%. We have also seen improvements on dacapo/sunflow and specjbb2015. We have also not seen any performance degradation caused by lowering this threshold.
This pull request has now been integrated.
Changeset: a7205cc6
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/a7205cc6512796466fefe17d171082995e0966de
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
8333926: Shenandoah: Lower default immediate garbage threshold
Reviewed-by: kdnilsen, ysr
-------------
PR: https://git.openjdk.org/jdk/pull/19641
From wkemper at openjdk.org Tue Jun 11 22:36:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 22:36:26 GMT
Subject: RFR: 8333925: Shenandoah: Heuristics should have an option to
ignore abbreviated cycles [v2]
In-Reply-To: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
References: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
Message-ID: <RJwopj5bf7i9sI0-cKmY7A0VwQbBsIrLJmul7OhLZLo=.3276b9a4-3bdf-46ec-aa36-f8cfd2c53721@github.com>
> After concurrent marking is complete, Shenandoah will skip the evacuation phase if a sufficient amount of garbage is found in regions that contain _no live objects_. These abbreviated cycles are much shorter than a cycle that performs evacuation and update references and tend to lower the average cycle time used by the heuristic to predict cycle times. This may cause the heuristic to wait too long to initiate a cycle and may lead to degenerated cycles. This change has the heuristic ignore abbreviated cycle times by default, with an option to have the heuristic count them as it does now.
William Kemper has updated the pull request incrementally with one additional commit since the last revision:
Rename option to ShenandoahAdaptiveIgnoreAbbreviated, remove unnecessary Amazon copyright from header
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19640/files
- new: https://git.openjdk.org/jdk/pull/19640/files/d8036f31..0028d3fa
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19640&range=01
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19640&range=00-01
Stats: 3 lines in 2 files changed: 0 ins; 1 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19640.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19640/head:pull/19640
PR: https://git.openjdk.org/jdk/pull/19640
From wkemper at openjdk.org Tue Jun 11 22:44:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 11 Jun 2024 22:44:14 GMT
Subject: RFR: 8333925: Shenandoah: Heuristics should have an option to
ignore abbreviated cycles [v2]
In-Reply-To: <GtmRdDUl_KkPILQte9rzAJu9fMJiIhY1lkNQn9rzSPM=.6e32e8a9-fe7e-4183-a221-bcbd44fd0629@github.com>
References: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
<GtmRdDUl_KkPILQte9rzAJu9fMJiIhY1lkNQn9rzSPM=.6e32e8a9-fe7e-4183-a221-bcbd44fd0629@github.com>
Message-ID: <byTd3CWTuScx2NBrubF7VIJ6HuRx7vHAPuEcPutOsB0=.9cb4e0a6-f21b-4bf4-a588-137ba7ab7d33@github.com>
On Tue, 11 Jun 2024 19:52:23 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Rename option to ShenandoahAdaptiveIgnoreAbbreviated, remove unnecessary Amazon copyright from header
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.cpp line 3:
>
>> 1: /*
>> 2: * Copyright (c) 2018, 2020, Red Hat, Inc. All rights reserved.
>> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
>
> Remove?
Okay.
> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 152:
>
>> 150: range(0,1.0) \
>> 151: \
>> 152: product(bool, ShenandoahAdaptiveIgnoreShortCycles, true, EXPERIMENTAL, \
>
> Abbreviated instead of Short,for consistency with use of terms in code?
Good idea.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19640#discussion_r1635555954
PR Review Comment: https://git.openjdk.org/jdk/pull/19640#discussion_r1635556040
From wkemper at openjdk.org Wed Jun 12 17:03:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 12 Jun 2024 17:03:43 GMT
Subject: Integrated: 8333750: GenShen: Only instantiate young/old generations
in generational mode
In-Reply-To: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
References: <JpZPTYezDOhlJi_YRn4atR1LxgYwGNvCXv9bB_FtQIU=.8ba84dbb-52eb-4b97-9f51-1327b5fa80a4@github.com>
Message-ID: <slBEh5Y_OE4IhvLnzGTLrKjjB1N5rASNARDljujCCyI=.861a1826-6e58-469e-a7f5-5e0199d1744f@github.com>
On Thu, 6 Jun 2024 18:51:27 GMT, William Kemper <wkemper at openjdk.org> wrote:
> * Other modes should not instantiate or touch young/old generations.
> * The generation sizer has been put in its own file and moved out of shHeap
> * The logic for aging regions has been decoupled from non-generational final update refs
This pull request has now been integrated.
Changeset: 2abf4cb4
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/2abf4cb43e84dda883a38abc6bcbac5aba3d7ce8
Stats: 890 lines in 22 files changed: 471 ins; 323 del; 96 mod
8333750: GenShen: Only instantiate young/old generations in generational mode
Reviewed-by: kdnilsen, ysr
-------------
PR: https://git.openjdk.org/shenandoah/pull/444
From ysr at openjdk.org Wed Jun 12 21:58:03 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 12 Jun 2024 21:58:03 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v16]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <7T_n4NkvVNHUDrI_jc_TzkIzd9rOvuKlhfdYxIQ7Ks4=.bc308f63-77e4-40ab-b893-063c99a3e56c@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits:
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Code called from LRB cannot rely on SGH::_gc_generation and must instead
use the _active_generation field.
Still need to aduit remaining _gc_generation uses for safety.
Removed too strong assert introduced for debugging in previous commit.
- Merge branch 'master' into active_generation
- Some asserts to catch a tricky race. These assertions may be too strong
in general but would help with debugging a rare crash.
- Merge branch 'master' into active_generation
- Remove vestigial thread() method in ShenandoahController.
- cosmetic
- jcheck white-space
- ... and 36 more: https://git.openjdk.org/shenandoah/compare/2abf4cb4...b513abae
-------------
Changes: https://git.openjdk.org/shenandoah/pull/407/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=15
Stats: 167 lines in 17 files changed: 134 ins; 1 del; 32 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From kdnilsen at openjdk.org Thu Jun 13 14:10:02 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 13 Jun 2024 14:10:02 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v5]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <6cFufjAesAi6XkBPiZ2iMpE6wh7c3cYUVjkYXgpAMvM=.bf0df2dc-5d7f-4fd6-b6e4-51952e1a2695@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Fix GHA errors
One of these errors was introduced by auto-merge. The other addresses
more stringent restrictions enforced by the Microsoft compiler.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/67c0d742..5f22805a
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=04
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=03-04
Stats: 5 lines in 1 file changed: 1 ins; 1 del; 3 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Thu Jun 13 15:41:55 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 13 Jun 2024 15:41:55 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v6]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <r_zZ2-ZAXztY0hk7cf-T4d-L3jH1D6R2ktLcc8z1NxE=.3c7aabb0-d265-4723-930d-9779ce44cbec@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Fix anticipated merge conflicts
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/5f22805a..f4485195
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=05
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=04-05
Stats: 28 lines in 2 files changed: 11 ins; 7 del; 10 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Thu Jun 13 16:36:54 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 13 Jun 2024 16:36:54 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v7]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <7Ve0aQZTBmSBgR9QLjWf9eqsBrARxWWx85ERojepv70=.aeece178-e2db-44fd-b3c4-cf51c60fae46@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 34 commits:
- Merge remote-tracking branch 'origin/master' into harmonize-free-set
- Merge branch 'openjdk:master' into master
- Fix anticipated merge conflicts
- Fix GHA errors
One of these errors was introduced by auto-merge. The other addresses
more stringent restrictions enforced by the Microsoft compiler.
- Fix whitespace
- Resolve conflicts introduced by manual resolution of merge
- Merge remote-tracking branch 'origin/master' into harmonize-free-set
- Merge branch 'openjdk:master' into master
- Remove TestHumongousThreshold test from problem list
- Fix white space
- ... and 24 more: https://git.openjdk.org/shenandoah/compare/2abf4cb4...f50528b3
-------------
Changes: https://git.openjdk.org/shenandoah/pull/440/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=06
Stats: 1673 lines in 14 files changed: 872 ins; 217 del; 584 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From duke at openjdk.org Thu Jun 13 18:40:44 2024
From: duke at openjdk.org (Xiaolong Peng)
Date: Thu, 13 Jun 2024 18:40:44 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock
Message-ID: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
### Notes
While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x0, 0x1, 0x3, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
Also noticed there was regression in Dacapo h2 benchmark, after deep dive and debug we decided to let non-java threads to only spin, which favors GC threads a little over Java threads at contented lock. Some follow-up tasks will be taken to reduce lock contention from Shenandoah GC, e.g. https://bugs.openjdk.org/browse/JDK-8334147
#### Test code
public class Alloc {
static final int THREADS = 1280; //32 threads per CPU core, 40 cores
static final Object[] sinks = new Object[64*THREADS];
static volatile boolean start;
static volatile boolean stop;
public static void main(String... args) throws Throwable {
for (int t = 0; t < THREADS; t++) {
int ft = t;
new Thread(() -> work(ft * 64)).start();
}
Thread.sleep(1000);
start = true;
Thread.sleep(30_000);
stop = true;
}
public static void work(int idx) {
while (!start) { Thread.onSpinWait(); }
while (!stop) {
sinks[idx] = new byte[128];
}
}
}
Run it like this and observe TTSP times:
java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
#### Metrics from tests(TTSP, allocation rate)
##### Heavy contention(1280 threads, 32 per CPU core)
| Test | Count | AVG | TRIMMEAN 2% | MAX | MIN | AVG allocation rate(M/s) |
| ---------- | ----- | ------ | ----------- | -------- | ----- | ------------------------ |
| Baseline | 19 | 940270 | 940270 | 5956928 | 75562 | 23.34 |
| No spin | 164 | 222958 | 204822 | 3330053 | 53819 | 238.9 |
| 0x01 | 172 | 189173 | 186601 | 750715 | 64864 | 244.1 |
| 0x03 | 174 | 286892 | 217739 | 12412225 | 55891 | 239.5 |
| 0x07 | 187 | 194440 | 183894 | 2284615 | 55256 | 235.9 |
| 0x0F | 182 | 657904 | 354898 | 55801594 | 55239 | 232.6 |
| 0x1F | 159 | 230107 | 213839 | 2973310 | 41083 | 212.3 |
| 0x3F | 139 | 284702 | 272033 | 2258179 | 46941 | 197 |
| 0x7F | 171 | 511573 | 280797 | 39980104 | 44162 | 197.53 |
| 0xFF | 158 | 608347 | 348863 | 41647457 | 48753 | 175 |
##### Light contention(40 threads, 1 per CPU core)
| Test | Count | AVG | TRIMMEAN 2% | MAX | MIN | allocation rate(M/s) |
| ---------- | ----- | ------ | ----------- | ------- | ---- | -------------------- |
| Baseline | 86 | 188536 | 188536 | 3159699 | 5072 | 164.7249794 |
| No spin | 128 | 23431 | 66482 | 128061 | 4332 | 258.7864942 |
| 0x01 | 132 | 28958 | 24035 | 689788 | 3164 | 262.067242 |
| 0x03 | 128 | 47344 | 22843 | 3175792 | 5932 | 259.4006259 |
| 0x07 | 125 | 21319 | 20566 | 128729 | 6494 | 252.633891 |
| 0x0F | 125 | 36923 | 20528 | 2084257 | 6245 | 255.295953 |
| 0x1F | 119 | 25430 | 23788 | 236583 | 6383 | 241.1196582 |
| 0x3F | 116 | 34549 | 29085 | 685559 | 6532 | 230.5788333 |
| 0x7F | 104 | 50870 | 49425 | 243020 | 6119 | 203.4185628 |
| 0xFF | 95 | 101771 | 101771 | 2081392 | 2551 | 188.4154766 |
Overall the results shows less spin delvers better performance in terms of TTSP and allocation rate, but it seems too aggressive to spin only once or no spin at all, still want to be conservative for the spin count, we decided to use 0x1F.
-------------
Commit messages:
- Merge branch 'openjdk:master' into JDK-8331411
- minor adjustments
- Minor fix
- Merge branch 'openjdk:master' into JDK-8331411
- Typo
- Bug fix
- Revert the fix specifically h2 and polish the code
- Yield up to 5 times
- Apply TTAS for fast lock
- Remove trailing whitespace
- ... and 5 more: https://git.openjdk.org/jdk/compare/cff048c7...d5d8b65f
Changes: https://git.openjdk.org/jdk/pull/19570/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8331411
Stats: 49 lines in 2 files changed: 20 ins; 17 del; 12 mod
Patch: https://git.openjdk.org/jdk/pull/19570.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19570/head:pull/19570
PR: https://git.openjdk.org/jdk/pull/19570
From kdnilsen at openjdk.org Thu Jun 13 22:22:01 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 13 Jun 2024 22:22:01 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Minor refinements to test programs
TestAllocIntArrays: comments to explain behavior.
TestOldGrowthTriggers: reduce the number of loop iterations so this test
will not time out on less powerful test platforms.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/f50528b3..0350ce08
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=07
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=06-07
Stats: 8 lines in 2 files changed: 5 ins; 0 del; 3 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kbarrett at openjdk.org Sat Jun 15 07:38:30 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Sat, 15 Jun 2024 07:38:30 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort
In-Reply-To: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
Message-ID: <X23eTytvPT0yKjFjbPYduZ4epuSWQ3aj0D1gdIqsjF4=.613e2c3b-de99-46de-a19d-b267619204b0@github.com>
On Wed, 29 May 2024 18:52:03 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
> The "idempotent" argument is removed from that function, with associated
> simplifications to the implementation. Callers are updated to remove that
> argument. Callers that were providing a false value are unaffected in their
> behavior. The 3 callers that were providing a true value to request the
> associated feature are also unaffected (other than by being made faster),
> because the arrays involved don't contain any equivalent pairs.
>
> There are also some miscellaneous cleanups, including using the swap utility
> and fixing some comments.
>
> Testing: mach5 tier1-3
Thanks for the review.
> So .... IIUC
Not exactly.
> the only code that would be affected by this change would be code that
> passes true,
Correct.
> which could also have equivalent elements to sort,
Correct.
> and which requires the sort order to always be the same regardless of the
> order the elements are found.
It does not provide any such thing. All the flag does is prevent swapping of
equivalent elements, which doesn't give us any interesting additional ordering
property.
We can only detect the effect of the flag if there are elements that are
equivalent according to the sort function but are distinguishable by some
other means. Depending on what you mean, either
(1) All permutations of the sequence will sort to the same order. This isn't
implementable.
(2) The sort doesn't change the relative order of equivalent elements, e.g.
the sort is stable. Simple quicksort is not stable. It's possible to make it
stable via O(N) extra memory, or (I've read) using a complex variant that is
significantly slower. We're not doing either of those.
> I think only the archive related code cares about deterministic order, and
> package and module names should be unique, so this seems fine.
Correct.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19464#issuecomment-2169182340
From dholmes at openjdk.org Mon Jun 17 04:51:11 2024
From: dholmes at openjdk.org (David Holmes)
Date: Mon, 17 Jun 2024 04:51:11 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort
In-Reply-To: <X23eTytvPT0yKjFjbPYduZ4epuSWQ3aj0D1gdIqsjF4=.613e2c3b-de99-46de-a19d-b267619204b0@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
<X23eTytvPT0yKjFjbPYduZ4epuSWQ3aj0D1gdIqsjF4=.613e2c3b-de99-46de-a19d-b267619204b0@github.com>
Message-ID: <nl2ACgjveuSPgnZ_zFS_POtPWwnpzMBDkxqZkT5cgoo=.b7f84bd9-bf42-4efd-8b24-d7ba725a174a@github.com>
On Sat, 15 Jun 2024 07:35:41 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
> It does not provide any such thing. All the flag does is prevent swapping of
equivalent elements, which doesn't give us any interesting additional ordering
property.
I only meant the sort order of the equivalent elements would be maintained.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19464#issuecomment-2172231564
From fweimer at openjdk.org Mon Jun 17 12:56:18 2024
From: fweimer at openjdk.org (Florian Weimer)
Date: Mon, 17 Jun 2024 12:56:18 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort
In-Reply-To: <nl2ACgjveuSPgnZ_zFS_POtPWwnpzMBDkxqZkT5cgoo=.b7f84bd9-bf42-4efd-8b24-d7ba725a174a@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
<X23eTytvPT0yKjFjbPYduZ4epuSWQ3aj0D1gdIqsjF4=.613e2c3b-de99-46de-a19d-b267619204b0@github.com>
<nl2ACgjveuSPgnZ_zFS_POtPWwnpzMBDkxqZkT5cgoo=.b7f84bd9-bf42-4efd-8b24-d7ba725a174a@github.com>
Message-ID: <sAlRFyLsVXfzvSC5kK8CbH46w9fth7KwOx9LyDZsP8g=.c85611be-6375-4ef1-a2eb-97083a64c94f@github.com>
On Mon, 17 Jun 2024 04:48:46 GMT, David Holmes <dholmes at openjdk.org> wrote:
> > It does not provide any such thing. All the flag does is prevent swapping of
> > equivalent elements, which doesn't give us any interesting additional ordering
> > property.
>
> I only meant the sort order of the equivalent elements would be maintained.
I think the partitioning phase swaps inequal elements based on comparison with the pivot, and this can move elements equivalent to the pivot past the pivot, with or without that additional equality check.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19464#issuecomment-2173322389
From duke at openjdk.org Mon Jun 17 16:31:52 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Mon, 17 Jun 2024 16:31:52 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v5]
In-Reply-To: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
Message-ID: <MFHmR3Lcd4AcIe1HIW41xRDsC17zhantIIdw-P921pk=.de168ec2-1f4f-4d54-8d60-ed8502f80e9d@github.com>
> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
>
> **Verification**
>
> Shenendoah
>
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>
> ZGC
>
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>
> **Issue**
> https://bugs.openjdk.org/browse/JDK-8331911
Neethu Prasad has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits:
- Merge branch 'openjdk:master' into JDK-8331911
- 8331911: Remove is_armed check from callers
- Update full name
- Address feedback on whitespace and comments
- 8331911: Fix whitespace
- 8331911: Reconsider locking for recently disarmed nmethods
-------------
Changes: https://git.openjdk.org/jdk/pull/19285/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19285&range=04
Stats: 30 lines in 4 files changed: 22 ins; 2 del; 6 mod
Patch: https://git.openjdk.org/jdk/pull/19285.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19285/head:pull/19285
PR: https://git.openjdk.org/jdk/pull/19285
From wkemper at openjdk.org Mon Jun 17 20:38:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 20:38:26 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <XTUdcHSBHS7PyxCwOKzgs8exyw9p2m7bKnYAi2aY7CE=.50adc38e-e1b3-4a13-b1a3-6331fe0b9a3d@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
test/hotspot/jtreg/gc/shenandoah/TestAllocIntArrays.java line 211:
> 209: final int max = 384 * 1024;
> 210: // Each allocated int array is assumed to consume 16 bytes for alignment and header, plus
> 211: // an everage of 4 * the average number of elements in the array.
s/everage/average
test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 49:
> 47: List<Object> root = new ArrayList<Object>();
> 48: while (true) {
> 49: root.add(new Object[SIZE]);
Accidental? Rest of file uses 4 spaces for indent.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643402343
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643403554
From wkemper at openjdk.org Mon Jun 17 20:43:36 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 20:43:36 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <SSODN7yOFMTXNfEYcTQSGKexUkK2QhRjx589cStFOmU=.61d76999-a4c4-43d3-828f-c5e4af88efec@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
Minor suggestions. I think it will be easier to compare the freeset changes here with upstream (i.e., openjdk:master).
src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 138:
> 136: if (is_old()) {
> 137: heap->old_generation()->decrement_affiliated_region_count();
> 138: heap->old_generation()->decrease_capacity(region_size_bytes());
Should capacity be updated every time affiliated region count changes? If yes, then we could have increase/decrease affiliated region count increase/decrease capacity directly. If no, is this something peculiar to full GC?
-------------
Changes requested by wkemper (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/440#pullrequestreview-2123863941
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643406703
From wkemper at openjdk.org Mon Jun 17 21:33:25 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 21:33:25 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
It looks like the comments for `ShenandoahFreeSet::internal_fragmentation` and `ShenandoahFreeSet::external_fragmentation` have been moved from the header into the implementation file when comparing against upstream (`jdk:master`). Can we keep them in-sync with upstream for the sake of minimizing differences here?
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 184:
> 182: _rightmosts_empty[int(ShenandoahFreeSetPartitionId::Mutator)] = mutator_rightmost_empty;
> 183:
> 184: _region_counts[int(ShenandoahFreeSetPartitionId::Mutator)] = mutator_region_count;
This statement duplicates statement on line 178.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 208:
> 206: _rightmosts_empty[int(ShenandoahFreeSetPartitionId::OldCollector)] = old_collector_rightmost_empty;
> 207:
> 208: _region_counts[int(ShenandoahFreeSetPartitionId::OldCollector)] = old_collector_region_count;
Same duplication from `establish_mutator_intervals`.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 344:
> 342: // At start of update refs: Collector => Mutator
> 343: // OldCollector Empty => Mutator
> 344: assert (((available <= _region_size_bytes) &&
This is quite hard for my feeble brain to parse. Could we break this into multiple asserts? or break some of the expressions out into functions?
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 975:
> 973: assert(!r->is_affiliated(), "New region " SIZE_FORMAT " should be unaffiliated", r->index());
> 974: r->set_affiliation(req.affiliation());
> 975: ShenandoahMarkingContext* const ctx = _heap->complete_marking_context();
Can we move this `ctx` closer to the `asserts` that use it, or perhaps inline the variable in the asserts?
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1178:
> 1176:
> 1177: size_t remainder = words_size & ShenandoahHeapRegion::region_size_words_mask();
> 1178: ShenandoahMarkingContext* const ctx = _heap->complete_marking_context();
`ctx` looks unused.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1226:
> 1224: }
> 1225:
> 1226: void ShenandoahFreeSet::try_recycle_trashed(ShenandoahHeapRegion *r, bool is_generational) {
`is_generational` looks unused.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 2066:
> 2064: "Boundaries or find_first_set_bit failed: " SSIZE_FORMAT, index);
> 2065: ShenandoahHeapRegion* r = _heap->get_region(index);
> 2066: if (r->is_empty()) {
This algorithm has changed from upstream. Is that intentional?
-------------
Changes requested by wkemper (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/440#pullrequestreview-2123889333
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643422326
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643423178
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643434756
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643436394
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643438862
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643441101
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643450046
From wkemper at openjdk.org Mon Jun 17 21:59:50 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 21:59:50 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master [v2]
In-Reply-To: <9cJnMe4iAmTOAvT8MT1p57xQq3o4iPNnJm_yYrPKNGY=.76d16fd2-9e6e-40c8-8b2f-af68e25ad741@github.com>
References: <9cJnMe4iAmTOAvT8MT1p57xQq3o4iPNnJm_yYrPKNGY=.76d16fd2-9e6e-40c8-8b2f-af68e25ad741@github.com>
Message-ID: <BjUI6YB2UAtVJli66Aa44vt55UvKOdjls3Ov2LW8kuo=.6b0a1c9c-f2fa-4fbb-9598-2bf14e7abca5@github.com>
> Merges tag jdk-21.0.4+5
William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 18 commits:
- Merge branch 'shenandoah-jdk21u/master' into merge-jdk-21.0.4+5
- 8320692: Null icon returned for .exe without custom icon
Backport-of: 6212264cc6fe428e8d15b7f33e2979c081e432d7
- 8330275: Crash in XMark::follow_array
Reviewed-by: stuefe
Backport-of: 42b1d858d15fd06de9ce41b08b430b12724652e9
- 8329862: libjli GetApplicationHome cleanups and enhance jli tracing
Backport-of: 377f2e538ae0fc94fc5483700a3ae70175017741
- 8323635: Test gc/g1/TestHumongousAllocConcurrentStart.java fails with -XX:TieredStopAtLevel=3
Backport-of: 5045839cb2095105a5c6c9eebc633a78b1e3213e
- 8295111: dpkg appears to have problems resolving symbolically linked native libraries
Backport-of: 32946e1882e9b22c983cbba3c6bda3cc7295946a
- 8331031: unify os::dont_yield and os::naked_yield across Posix platforms
Backport-of: c9442014e561f8e1cb43a0e9f18a9a5ae2a7a2da
- 8329223: Parallel: Parallel GC resizes heap even if -Xms = -Xmx
Backport-of: aca1e8365bf0f64bf18caf798bbca1d25b3c4117
- 8330464: hserr generic events - add entry for the before_exit calls
Backport-of: 45ed97f15b96071e5d6b6d21e8f0129eda5aa447
- 8329605: hs errfile generic events - move memory protections and nmethod flushes to separate sections
Backport-of: 397d94831033e91c7a849774bf4e80d8f1c8ec66
- ... and 8 more: https://git.openjdk.org/shenandoah-jdk21u/compare/0dda663b...7cb6b6bf
-------------
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/52/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=52&range=01
Stats: 1234 lines in 152 files changed: 442 ins; 332 del; 460 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/52.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/52/head:pull/52
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/52
From wkemper at openjdk.org Mon Jun 17 22:13:39 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:13:39 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v16]
In-Reply-To: <7T_n4NkvVNHUDrI_jc_TzkIzd9rOvuKlhfdYxIQ7Ks4=.bc308f63-77e4-40ab-b893-063c99a3e56c@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
<7T_n4NkvVNHUDrI_jc_TzkIzd9rOvuKlhfdYxIQ7Ks4=.bc308f63-77e4-40ab-b893-063c99a3e56c@github.com>
Message-ID: <W0hbPypmFWrIJy5janVs8N4TW9Bg4PDaJImstnIc0AI=.b28ef645-d7c4-4807-ac6c-febe1ea066a4@github.com>
On Wed, 12 Jun 2024 21:58:03 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>>
>> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
>> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
>> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>>
>> Asserts check the protocol for setting and clearing the variables.
>>
>> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>>
>> *Testing*:
>> - [x] code pipeline
>> - [x] specjbb testing
>> - [ ] specjbb performance
>> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
>> - [x] GHA
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits:
>
> - Merge branch 'master' into active_generation
> - Merge branch 'master' into active_generation
> - Merge branch 'master' into active_generation
> - Code called from LRB cannot rely on SGH::_gc_generation and must instead
> use the _active_generation field.
>
> Still need to aduit remaining _gc_generation uses for safety.
>
> Removed too strong assert introduced for debugging in previous commit.
> - Merge branch 'master' into active_generation
> - Some asserts to catch a tricky race. These assertions may be too strong
> in general but would help with debugging a rare crash.
> - Merge branch 'master' into active_generation
> - Remove vestigial thread() method in ShenandoahController.
> - cosmetic
> - jcheck white-space
> - ... and 36 more: https://git.openjdk.org/shenandoah/compare/2abf4cb4...b513abae
src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.cpp line 796:
> 794: ShenandoahHeapRegion* r = _regions->next();
> 795: // We update references for global, old, and young collections.
> 796: ShenandoahGeneration* const gc_generation = _heap->gc_generation();
It would certainly involve more changes, but we could pass the correct generation from `shConcurrentGC` in this case. I think we could do this in most cases (save for the barriers, but you've established a safe pattern for them to access the active generation already).
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1643483138
From wkemper at openjdk.org Mon Jun 17 22:23:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:23:43 GMT
Subject: Integrated: 8332913: GenShen: Restore shared update refs iterator
Message-ID: <lmDfZuBsljB0sMTepgWMEeT_MGkv0BLEl9Y0XWHp2UI=.92abfe44-e67b-497b-95e3-c014dbc6c718@github.com>
Clean backport.
-------------
Commit messages:
- Backport 31bca3076634a3119a049283587dedd1c3de9f16
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/53/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=53&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8332913
Stats: 21 lines in 4 files changed: 11 ins; 3 del; 7 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/53.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/53/head:pull/53
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/53
From wkemper at openjdk.org Mon Jun 17 22:23:43 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:23:43 GMT
Subject: Integrated: 8332913: GenShen: Restore shared update refs iterator
In-Reply-To: <lmDfZuBsljB0sMTepgWMEeT_MGkv0BLEl9Y0XWHp2UI=.92abfe44-e67b-497b-95e3-c014dbc6c718@github.com>
References: <lmDfZuBsljB0sMTepgWMEeT_MGkv0BLEl9Y0XWHp2UI=.92abfe44-e67b-497b-95e3-c014dbc6c718@github.com>
Message-ID: <GfROeyxKSs5D2b6g7iAvUkrBA8y4tNjNpYf1g7YmV88=.dc7807fe-61a4-42f5-a381-d4aa3ab2b53a@github.com>
On Mon, 17 Jun 2024 22:17:52 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Clean backport.
This pull request has now been integrated.
Changeset: 51783094
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/51783094c6c2b809dd8a33f66c13fc7bd25418b7
Stats: 21 lines in 4 files changed: 11 ins; 3 del; 7 mod
8332913: GenShen: Restore shared update refs iterator
Backport-of: 31bca3076634a3119a049283587dedd1c3de9f16
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/53
From wkemper at openjdk.org Mon Jun 17 22:36:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:36:01 GMT
Subject: RFR: 8333109: GenShen: Factor generational mode out of gc helpers
(redo)
Message-ID: <Qx7Tg8ONv1-s34RyebGKXPG3pLv0W3YtvZmNdHwQazk=.406cd8fa-7a57-47c1-86c0-c93aaba3f775@github.com>
Clean backport
-------------
Commit messages:
- Backport 840c1825f38a11485243a648b171b1ba5ec78de5
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/54/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=54&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333109
Stats: 345 lines in 6 files changed: 132 ins; 140 del; 73 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/54.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/54/head:pull/54
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/54
From wkemper at openjdk.org Mon Jun 17 22:36:55 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:36:55 GMT
Subject: RFR: 8333825: GenShen: Revert/Remove ShenandoahMaxEvacLABRatio
Message-ID: <zbhUn0j6R1_TzXxSU1LTg8pY-g1rUGOARgdZC95ySL4=.ad4e50af-7a18-4f30-8565-faa7082e160a@github.com>
Clean backport
-------------
Commit messages:
- Backport 6071b4a9a170b2d91522971a3c9b32bbf44eee3b
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/55/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=55&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333825
Stats: 53 lines in 3 files changed: 7 ins; 36 del; 10 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/55.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/55/head:pull/55
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/55
From wkemper at openjdk.org Mon Jun 17 22:38:04 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:38:04 GMT
Subject: RFR: 8333930: GenShen: Check for cancellation of old mark after final
mark
Message-ID: <CCU1m7lfr4rA9sWORQmDmgRPy1wtHm2kciSH8a516A4=.168fe565-850d-473e-9d3d-ae1af0cf81ed@github.com>
Clean backport
-------------
Commit messages:
- Backport e3c3098a682be9041045140cb89bdda43f4bff30
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/56/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=56&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333930
Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/56.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/56/head:pull/56
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/56
From wkemper at openjdk.org Mon Jun 17 22:57:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:57:26 GMT
Subject: Integrated: 8333109: GenShen: Factor generational mode out of gc
helpers (redo)
In-Reply-To: <Qx7Tg8ONv1-s34RyebGKXPG3pLv0W3YtvZmNdHwQazk=.406cd8fa-7a57-47c1-86c0-c93aaba3f775@github.com>
References: <Qx7Tg8ONv1-s34RyebGKXPG3pLv0W3YtvZmNdHwQazk=.406cd8fa-7a57-47c1-86c0-c93aaba3f775@github.com>
Message-ID: <xmt5m7sPAZWRhBeh26IW23LCuxdh9pAv2pZX4T_6Is8=.ae74392a-9f77-4f8a-8480-6f54698bc4b1@github.com>
On Mon, 17 Jun 2024 22:30:21 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Clean backport
This pull request has now been integrated.
Changeset: 0fde7590
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/0fde7590a378648181b9626ba60672ae633f9730
Stats: 345 lines in 6 files changed: 132 ins; 140 del; 73 mod
8333109: GenShen: Factor generational mode out of gc helpers (redo)
Backport-of: 840c1825f38a11485243a648b171b1ba5ec78de5
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/54
From wkemper at openjdk.org Mon Jun 17 22:57:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:57:26 GMT
Subject: Integrated: 8333930: GenShen: Check for cancellation of old mark after
final mark
In-Reply-To: <CCU1m7lfr4rA9sWORQmDmgRPy1wtHm2kciSH8a516A4=.168fe565-850d-473e-9d3d-ae1af0cf81ed@github.com>
References: <CCU1m7lfr4rA9sWORQmDmgRPy1wtHm2kciSH8a516A4=.168fe565-850d-473e-9d3d-ae1af0cf81ed@github.com>
Message-ID: <aFnwVNIxvsQYnhmfNwoXwrPhQ2fb3_1FAqbOgCbxHP0=.5be8b1c5-28dc-4a0b-bba1-d861ca1de8d0@github.com>
On Mon, 17 Jun 2024 22:31:05 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Clean backport
This pull request has now been integrated.
Changeset: 06973243
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/06973243c43af27700c49329714deee221295dff
Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod
8333930: GenShen: Check for cancellation of old mark after final mark
Backport-of: e3c3098a682be9041045140cb89bdda43f4bff30
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/56
From wkemper at openjdk.org Mon Jun 17 22:57:35 2024
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 17 Jun 2024 22:57:35 GMT
Subject: Integrated: 8333825: GenShen: Revert/Remove ShenandoahMaxEvacLABRatio
In-Reply-To: <zbhUn0j6R1_TzXxSU1LTg8pY-g1rUGOARgdZC95ySL4=.ad4e50af-7a18-4f30-8565-faa7082e160a@github.com>
References: <zbhUn0j6R1_TzXxSU1LTg8pY-g1rUGOARgdZC95ySL4=.ad4e50af-7a18-4f30-8565-faa7082e160a@github.com>
Message-ID: <nLeVNfpDi67drrHxPAjPemjEzTn7nd0JtRWrs6FbXdo=.47d1ad19-6ccf-408f-a55a-76e9885b4109@github.com>
On Mon, 17 Jun 2024 22:30:51 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Clean backport
This pull request has now been integrated.
Changeset: ba6aa296
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/ba6aa296a4e7f412c59457ce1e7a2882e63538b5
Stats: 53 lines in 3 files changed: 7 ins; 36 del; 10 mod
8333825: GenShen: Revert/Remove ShenandoahMaxEvacLABRatio
Backport-of: 6071b4a9a170b2d91522971a3c9b32bbf44eee3b
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/55
From wkemper at openjdk.org Tue Jun 18 13:42:38 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 13:42:38 GMT
Subject: Integrated: Merge openjdk/jdk21u-dev:master
In-Reply-To: <9cJnMe4iAmTOAvT8MT1p57xQq3o4iPNnJm_yYrPKNGY=.76d16fd2-9e6e-40c8-8b2f-af68e25ad741@github.com>
References: <9cJnMe4iAmTOAvT8MT1p57xQq3o4iPNnJm_yYrPKNGY=.76d16fd2-9e6e-40c8-8b2f-af68e25ad741@github.com>
Message-ID: <UFC_X88qfBEeefJUJy3-iP3-5de4AJSW1CcLqJ5WT8I=.c1845638-a2aa-4f8a-b12e-849b46117ba1@github.com>
On Thu, 30 May 2024 14:17:52 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Merges tag jdk-21.0.4+5
This pull request has now been integrated.
Changeset: 4c603253
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/4c6032539714e27da67cac5049bf165353fd0865
Stats: 1234 lines in 152 files changed: 442 ins; 332 del; 460 mod
Merge
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/52
From wkemper at openjdk.org Tue Jun 18 14:27:56 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 14:27:56 GMT
Subject: RFR: 8327388: GenShen: census during marking is partial
Message-ID: <qTy5OBMowEzhermvuntfK4vzBUvuo1SISWnLP8QiBXQ=.6c792136-9fe0-4017-8d79-0c00b9fd95d3@github.com>
Clean backport
-------------
Commit messages:
- Backport 10febd95d093165ca2e91c12e385e3754dc47ea0
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/57/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=57&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8327388
Stats: 166 lines in 11 files changed: 121 ins; 27 del; 18 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/57.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/57/head:pull/57
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/57
From wkemper at openjdk.org Tue Jun 18 14:31:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 14:31:26 GMT
Subject: Integrated: 8327388: GenShen: census during marking is partial
In-Reply-To: <qTy5OBMowEzhermvuntfK4vzBUvuo1SISWnLP8QiBXQ=.6c792136-9fe0-4017-8d79-0c00b9fd95d3@github.com>
References: <qTy5OBMowEzhermvuntfK4vzBUvuo1SISWnLP8QiBXQ=.6c792136-9fe0-4017-8d79-0c00b9fd95d3@github.com>
Message-ID: <lEENnBWBWAekAqfewh_sh80o7mKOHyPGat-bzTu787Y=.f6407365-3e05-41e3-b890-06635a711541@github.com>
On Tue, 18 Jun 2024 14:23:10 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Clean backport
This pull request has now been integrated.
Changeset: 31996856
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/3199685631f9f4f938c5913a3f844085b80fea09
Stats: 166 lines in 11 files changed: 121 ins; 27 del; 18 mod
8327388: GenShen: census during marking is partial
Backport-of: 10febd95d093165ca2e91c12e385e3754dc47ea0
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/57
From kdnilsen at openjdk.org Tue Jun 18 15:08:13 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 15:08:13 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <XTUdcHSBHS7PyxCwOKzgs8exyw9p2m7bKnYAi2aY7CE=.50adc38e-e1b3-4a13-b1a3-6331fe0b9a3d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<XTUdcHSBHS7PyxCwOKzgs8exyw9p2m7bKnYAi2aY7CE=.50adc38e-e1b3-4a13-b1a3-6331fe0b9a3d@github.com>
Message-ID: <YSjW1w0Oau9xIRvwQ4dBr0LY7DnLD9N5Sz6Cr6pbVX8=.9d1bcd7a-b9fd-450a-a207-9206460c1f6e@github.com>
On Mon, 17 Jun 2024 20:35:59 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 49:
>
>> 47: List<Object> root = new ArrayList<Object>();
>> 48: while (true) {
>> 49: root.add(new Object[SIZE]);
>
> Accidental? Rest of file uses 4 spaces for indent.
Thanks. fixing.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644622472
From kdnilsen at openjdk.org Tue Jun 18 15:08:12 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 15:08:12 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v9]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <cqYEs6i_ZzEfDLjykcOO2S4deiP9Efp2E4hYfTRurBY=.636d05dc-1755-4ea7-ae49-cfde9a3bc333@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Fix comment and formatting errors in test programs
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/0350ce08..ebc56a87
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=08
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=07-08
Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Tue Jun 18 15:11:46 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 15:11:46 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <SSODN7yOFMTXNfEYcTQSGKexUkK2QhRjx589cStFOmU=.61d76999-a4c4-43d3-828f-c5e4af88efec@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<SSODN7yOFMTXNfEYcTQSGKexUkK2QhRjx589cStFOmU=.61d76999-a4c4-43d3-828f-c5e4af88efec@github.com>
Message-ID: <sClur0I75youKUvspJi9NUyOu6jIn_FwfpQijgh_vgg=.41fd6365-67ab-4032-8273-0967aa46b1f3@github.com>
On Mon, 17 Jun 2024 20:39:33 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 138:
>
>> 136: if (is_old()) {
>> 137: heap->old_generation()->decrement_affiliated_region_count();
>> 138: heap->old_generation()->decrease_capacity(region_size_bytes());
>
> Should capacity be updated every time affiliated region count changes? If yes, then we could have increase/decrease affiliated region count increase/decrease capacity directly. If no, is this something peculiar to full GC?
This is a good observation and suggestion. This doesn't seem to be a consistent behavior. promote_in_place() adjusts affiliated regions without adjusting capacity, but maybe it should. I'll explore further.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644632868
From kdnilsen at openjdk.org Tue Jun 18 15:40:43 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 15:40:43 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <sClur0I75youKUvspJi9NUyOu6jIn_FwfpQijgh_vgg=.41fd6365-67ab-4032-8273-0967aa46b1f3@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<SSODN7yOFMTXNfEYcTQSGKexUkK2QhRjx589cStFOmU=.61d76999-a4c4-43d3-828f-c5e4af88efec@github.com>
<sClur0I75youKUvspJi9NUyOu6jIn_FwfpQijgh_vgg=.41fd6365-67ab-4032-8273-0967aa46b1f3@github.com>
Message-ID: <Ip5z3ZxH61d89NZhW7xwv-SKn6sLwwtOuPwnXAgR8ms=.2ae316f6-fdc9-4cca-965f-83dc52a7e7d5@github.com>
On Tue, 18 Jun 2024 15:09:24 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 138:
>>
>>> 136: if (is_old()) {
>>> 137: heap->old_generation()->decrement_affiliated_region_count();
>>> 138: heap->old_generation()->decrease_capacity(region_size_bytes());
>>
>> Should capacity be updated every time affiliated region count changes? If yes, then we could have increase/decrease affiliated region count increase/decrease capacity directly. If no, is this something peculiar to full GC?
>
> This is a good observation and suggestion. This doesn't seem to be a consistent behavior. promote_in_place() adjusts affiliated regions without adjusting capacity, but maybe it should. I'll explore further.
Looks like this is something special about full gc. The call to make_young_maybe() only occurs when !_is_generational(). kind of a peculiar name for this function, given that we're non-generational here. Still exploring.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644675322
From kdnilsen at openjdk.org Tue Jun 18 15:44:29 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 15:44:29 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <Ip5z3ZxH61d89NZhW7xwv-SKn6sLwwtOuPwnXAgR8ms=.2ae316f6-fdc9-4cca-965f-83dc52a7e7d5@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<SSODN7yOFMTXNfEYcTQSGKexUkK2QhRjx589cStFOmU=.61d76999-a4c4-43d3-828f-c5e4af88efec@github.com>
<sClur0I75youKUvspJi9NUyOu6jIn_FwfpQijgh_vgg=.41fd6365-67ab-4032-8273-0967aa46b1f3@github.com>
<Ip5z3ZxH61d89NZhW7xwv-SKn6sLwwtOuPwnXAgR8ms=.2ae316f6-fdc9-4cca-965f-83dc52a7e7d5@github.com>
Message-ID: <v2e4dN4fjMDAzdTSWieEs8WNIXJ6Q6Ldk2YwGJ1hrR4=.0a286207-ba62-4cf4-b864-45c172be012d@github.com>
On Tue, 18 Jun 2024 15:38:16 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> This is a good observation and suggestion. This doesn't seem to be a consistent behavior. promote_in_place() adjusts affiliated regions without adjusting capacity, but maybe it should. I'll explore further.
>
> Looks like this is something special about full gc. The call to make_young_maybe() only occurs when !_is_generational(). kind of a peculiar name for this function, given that we're non-generational here. Still exploring.
That code should not even be reached. I'm testing a change here.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644680248
From wkemper at openjdk.org Tue Jun 18 16:47:53 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 16:47:53 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational code
Message-ID: <cwcDLsyuHcPYV6CfJvMSRKyEjfVGgc-sC_KBHfgS3dI=.ed9ee337-b930-4d76-9725-5268a843c6e7@github.com>
Resolved minor conflict where tenuring threshold was passed to age table when printing it to console.
-------------
Commit messages:
- 8333457: GenShen: Move remembered set into new generational code
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/58/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=58&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333457
Stats: 304 lines in 28 files changed: 126 ins; 78 del; 100 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/58.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/58/head:pull/58
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/58
From kdnilsen at openjdk.org Tue Jun 18 17:07:34 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 17:07:34 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code
In-Reply-To: <cwcDLsyuHcPYV6CfJvMSRKyEjfVGgc-sC_KBHfgS3dI=.ed9ee337-b930-4d76-9725-5268a843c6e7@github.com>
References: <cwcDLsyuHcPYV6CfJvMSRKyEjfVGgc-sC_KBHfgS3dI=.ed9ee337-b930-4d76-9725-5268a843c6e7@github.com>
Message-ID: <xdD1k9xZvJVggMfH0IcQ_6GqWbK27QUm4hxt2xtQGd0=.f04fb115-c28d-48e4-9253-76cb24ea4311@github.com>
On Tue, 18 Jun 2024 16:41:46 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Resolved minor conflict where tenuring threshold was passed to age table when printing it to console.
Thanks for pulling this together.
-------------
Marked as reviewed by kdnilsen (Committer).
PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/58#pullrequestreview-2126067754
From kdnilsen at openjdk.org Tue Jun 18 17:29:04 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 17:29:04 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v10]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <6qHAR4xFVyvddqzHPVfz0IpvqhJPGoLp9Fe-jDaVLFc=.e9620560-08a1-40aa-9493-d4d3b2bfeaa1@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Remove dead code in make_young_maybe
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/ebc56a87..8d3f676c
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=09
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=08-09
Stats: 13 lines in 1 file changed: 1 ins; 12 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From ysr at openjdk.org Tue Jun 18 17:29:44 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 18 Jun 2024 17:29:44 GMT
Subject: RFR: 8333457: GenShen: Move remembered set into new generational
code
In-Reply-To: <cwcDLsyuHcPYV6CfJvMSRKyEjfVGgc-sC_KBHfgS3dI=.ed9ee337-b930-4d76-9725-5268a843c6e7@github.com>
References: <cwcDLsyuHcPYV6CfJvMSRKyEjfVGgc-sC_KBHfgS3dI=.ed9ee337-b930-4d76-9725-5268a843c6e7@github.com>
Message-ID: <CbktAp6z8dI9y3Vd-jgn5Av7tCqBVwPL3o2OZjqpsgw=.ed8d5a1f-5d9b-41ef-8533-e38ad57fd080@github.com>
On Tue, 18 Jun 2024 16:41:46 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Resolved minor conflict where tenuring threshold was passed to age table when printing it to console.
? ... Thank you!
-------------
Marked as reviewed by ysr (Committer).
PR Review: https://git.openjdk.org/shenandoah-jdk21u/pull/58#pullrequestreview-2126108448
From kdnilsen at openjdk.org Tue Jun 18 17:32:30 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 17:32:30 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <v2e4dN4fjMDAzdTSWieEs8WNIXJ6Q6Ldk2YwGJ1hrR4=.0a286207-ba62-4cf4-b864-45c172be012d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<SSODN7yOFMTXNfEYcTQSGKexUkK2QhRjx589cStFOmU=.61d76999-a4c4-43d3-828f-c5e4af88efec@github.com>
<sClur0I75youKUvspJi9NUyOu6jIn_FwfpQijgh_vgg=.41fd6365-67ab-4032-8273-0967aa46b1f3@github.com>
<Ip5z3ZxH61d89NZhW7xwv-SKn6sLwwtOuPwnXAgR8ms=.2ae316f6-fdc9-4cca-965f-83dc52a7e7d5@github.com>
<v2e4dN4fjMDAzdTSWieEs8WNIXJ6Q6Ldk2YwGJ1hrR4=.0a286207-ba62-4cf4-b864-45c172be012d@github.com>
Message-ID: <cpACVA7Tx7rEKyNo-ggT2223xF5ONCPqetZ9EQjgQF0=.93af63b7-e344-4122-800f-8c595c283944@github.com>
On Tue, 18 Jun 2024 15:41:41 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> Looks like this is something special about full gc. The call to make_young_maybe() only occurs when !_is_generational(). kind of a peculiar name for this function, given that we're non-generational here. Still exploring.
>
> That code should not even be reached. I'm testing a change here.
Have replace this code with an assert. Thanks for catching the inconsistency.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644825633
From wkemper at openjdk.org Tue Jun 18 17:56:31 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 17:56:31 GMT
Subject: Integrated: 8333457: GenShen: Move remembered set into new
generational code
In-Reply-To: <cwcDLsyuHcPYV6CfJvMSRKyEjfVGgc-sC_KBHfgS3dI=.ed9ee337-b930-4d76-9725-5268a843c6e7@github.com>
References: <cwcDLsyuHcPYV6CfJvMSRKyEjfVGgc-sC_KBHfgS3dI=.ed9ee337-b930-4d76-9725-5268a843c6e7@github.com>
Message-ID: <pbRX39VXmfMIS51HXpMVRI61xe-8mMW_1T7L-yaEkAU=.bea1580b-84e9-4c03-9988-01e459d77259@github.com>
On Tue, 18 Jun 2024 16:41:46 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Resolved minor conflict where tenuring threshold was passed to age table when printing it to console.
This pull request has now been integrated.
Changeset: 3fa0b69c
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/3fa0b69c2c82f01854919a87ed8cbe8ea58a51d5
Stats: 304 lines in 28 files changed: 126 ins; 78 del; 100 mod
8333457: GenShen: Move remembered set into new generational code
Reviewed-by: kdnilsen, ysr
Backport-of: a32f36bbbc4fc2a36872f3a0aaec68950bfc6cf6
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/58
From kdnilsen at openjdk.org Tue Jun 18 17:56:52 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 17:56:52 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v11]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <S5D7qAn7dzikrPl5t5NsIIQovdl5UX_pwKSkpxFWlRE=.efec16fd-f817-430a-b204-92c8cd6329b1@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Move comments describing fragmentation functions to header file
for consistency with upstream code
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/8d3f676c..d675bed4
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=10
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=09-10
Stats: 69 lines in 2 files changed: 35 ins; 34 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Tue Jun 18 17:56:52 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 17:56:52 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
Message-ID: <BBWI5xe6SO-ZyAagPWqUWvyXCBjKmr2JMt66ALon2Io=.805c6250-b3b0-487c-b326-1e2d34a9d0ce@github.com>
On Mon, 17 Jun 2024 21:31:00 GMT, William Kemper <wkemper at openjdk.org> wrote:
> It looks like the comments for `ShenandoahFreeSet::internal_fragmentation` and `ShenandoahFreeSet::external_fragmentation` have been moved from the header into the implementation file when comparing against upstream (`jdk:master`). Can we keep them in-sync with upstream for the sake of minimizing differences here?
Agree. I'm making this change.
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 184:
>
>> 182: _rightmosts_empty[int(ShenandoahFreeSetPartitionId::Mutator)] = mutator_rightmost_empty;
>> 183:
>> 184: _region_counts[int(ShenandoahFreeSetPartitionId::Mutator)] = mutator_region_count;
>
> This statement duplicates statement on line 178.
Thanks. Removing.
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 208:
>
>> 206: _rightmosts_empty[int(ShenandoahFreeSetPartitionId::OldCollector)] = old_collector_rightmost_empty;
>> 207:
>> 208: _region_counts[int(ShenandoahFreeSetPartitionId::OldCollector)] = old_collector_region_count;
>
> Same duplication from `establish_mutator_intervals`.
Thanks here also. Removing.
-------------
PR Comment: https://git.openjdk.org/shenandoah/pull/440#issuecomment-2176651406
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644850917
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644851614
From wkemper at openjdk.org Tue Jun 18 18:12:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 18:12:37 GMT
Subject: RFR: 8334491: GenShen: Revert changes to Shenandoah defaults
Message-ID: <tnIkkjs-ko-HygX3eItnvGvjgx4uQpQunL4T2ezjzd4=.0b08474d-c6e6-478a-8b3d-81c7aba9cc39@github.com>
On the genshen branch, we have changed the adaptive heuristics decay factor to 0.1 and increased the learning cycles to 10. Testing with the original defaults of 0.5 and 5 (respectively) shows modest performance improvements in specjbb and extremem.
-------------
Commit messages:
- Sync decay factor, learning steps with upstream
Changes: https://git.openjdk.org/shenandoah/pull/450/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=450&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8334491
Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.org/shenandoah/pull/450.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/450/head:pull/450
PR: https://git.openjdk.org/shenandoah/pull/450
From kdnilsen at openjdk.org Tue Jun 18 18:29:27 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 18:29:27 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
Message-ID: <rSGlhL1iVgupQEl9pO0_QshziGu_nqFy8LLjakd2LDQ=.36980070-c516-483a-b31e-261ed14b576b@github.com>
On Mon, 17 Jun 2024 21:10:44 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 344:
>
>> 342: // At start of update refs: Collector => Mutator
>> 343: // OldCollector Empty => Mutator
>> 344: assert (((available <= _region_size_bytes) &&
>
> This is quite hard for my feeble brain to parse. Could we break this into multiple asserts? or break some of the expressions out into functions?
I agree that code is very difficult to understand. I'm replacing with some function calls to make it more readable.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644885021
From kdnilsen at openjdk.org Tue Jun 18 19:48:48 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 19:48:48 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v12]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <6LfgxS25UJce6ha_2Pm5HDWf8X_LfNtq5d6Qlmxts-I=.2c543251-119b-4e80-a2d9-04964962c392@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Make complex assertion a bit easier to read
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/d675bed4..1bc1c4fb
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=11
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=10-11
Stats: 36 lines in 2 files changed: 22 ins; 8 del; 6 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Tue Jun 18 20:08:29 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 20:08:29 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
Message-ID: <PUDOuU3_X6jV2MHLsyaQRxBBYtj6bah3W6Phwd9xkyg=.d3377a43-4a5c-4b60-b1d8-ac34b560e6c6@github.com>
On Mon, 17 Jun 2024 21:12:39 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 975:
>
>> 973: assert(!r->is_affiliated(), "New region " SIZE_FORMAT " should be unaffiliated", r->index());
>> 974: r->set_affiliation(req.affiliation());
>> 975: ShenandoahMarkingContext* const ctx = _heap->complete_marking_context();
>
> Can we move this `ctx` closer to the `asserts` that use it, or perhaps inline the variable in the asserts?
Thanks. Done.
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1178:
>
>> 1176:
>> 1177: size_t remainder = words_size & ShenandoahHeapRegion::region_size_words_mask();
>> 1178: ShenandoahMarkingContext* const ctx = _heap->complete_marking_context();
>
> `ctx` looks unused.
Removed.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644995791
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1644995913
From kdnilsen at openjdk.org Tue Jun 18 20:15:25 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 20:15:25 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
Message-ID: <_jt0VeeDMN07ooQClm-i8oanMwPAi6BZtuUPbbPLHqk=.946e24bb-74f8-4a7b-bb8f-fa3c77ee0a9f@github.com>
On Mon, 17 Jun 2024 21:18:03 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1226:
>
>> 1224: }
>> 1225:
>> 1226: void ShenandoahFreeSet::try_recycle_trashed(ShenandoahHeapRegion *r, bool is_generational) {
>
> `is_generational` looks unused.
Thanks. Removing this unneeded argument.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645002286
From kdnilsen at openjdk.org Tue Jun 18 20:18:37 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 20:18:37 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v13]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <htM5SKqWNKW8kO3YrioBs7cFBdmL3_PD-qHYUrJie3g=.3de53a2a-13ee-4d1a-bb6b-f5c05ac9f16b@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Remove is_generational argument from try_recycle_trashed()
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/1bc1c4fb..d113ee73
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=12
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=11-12
Stats: 15 lines in 2 files changed: 2 ins; 5 del; 8 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Tue Jun 18 20:23:30 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 20:23:30 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
Message-ID: <N9NxlUFABJt_QVEgLxwrTVaTqoPzFDZX_r6Wq9rZ9Zs=.21fab225-8246-425b-88e0-08aac704594c@github.com>
On Mon, 17 Jun 2024 21:28:10 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 2066:
>
>> 2064: "Boundaries or find_first_set_bit failed: " SSIZE_FORMAT, index);
>> 2065: ShenandoahHeapRegion* r = _heap->get_region(index);
>> 2066: if (r->is_empty()) {
>
> This algorithm has changed from upstream. Is that intentional?
That change came in when I merged from master. I had assumed the change had also been applied upstream. Is your diff compared to a "recent" upstream?
dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2037) ShenandoahHeapRegion* r = _heap->get_region(index);
dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2038) if (r->is_empty()) {
dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2039) free += ShenandoahHeapRegion::region_size_bytes();
dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2040) if (last_idx + 1 == index) {
dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2041) empty_contig++;
f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2042) } else {
f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2043) empty_contig = 0;
f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2044) }
f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2045)
f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2046) max_contig = MAX2(max_contig, empty_contig);
f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2047) last_idx = index;
f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2048) }
5f22805a50d4 (Kelvin Nilsen 2024-06-13 14:04:30 +0000 2049) index = _partitions.find_index_of_next_available_region(Shenando\
ahFreeSetPartitionId::Mutator, index + 1);
f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2050) }
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645010444
From kdnilsen at openjdk.org Tue Jun 18 20:34:27 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 20:34:27 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <N9NxlUFABJt_QVEgLxwrTVaTqoPzFDZX_r6Wq9rZ9Zs=.21fab225-8246-425b-88e0-08aac704594c@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
<N9NxlUFABJt_QVEgLxwrTVaTqoPzFDZX_r6Wq9rZ9Zs=.21fab225-8246-425b-88e0-08aac704594c@github.com>
Message-ID: <PzggWK2fF_7xHa_jr_-es_XAZUKBpEc6LOauyv4XtX0=.45abd722-6109-4af6-b1a6-2511a74dbe23@github.com>
On Tue, 18 Jun 2024 20:21:17 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 2066:
>>
>>> 2064: "Boundaries or find_first_set_bit failed: " SSIZE_FORMAT, index);
>>> 2065: ShenandoahHeapRegion* r = _heap->get_region(index);
>>> 2066: if (r->is_empty()) {
>>
>> This algorithm has changed from upstream. Is that intentional?
>
> That change came in when I merged from master. I had assumed the change had also been applied upstream. Is your diff compared to a "recent" upstream?
>
> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2037) ShenandoahHeapRegion* r = _heap->get_region(index);
> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2038) if (r->is_empty()) {
> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2039) free += ShenandoahHeapRegion::region_size_bytes();
> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2040) if (last_idx + 1 == index) {
> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2041) empty_contig++;
> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2042) } else {
> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2043) empty_contig = 0;
> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2044) }
> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2045)
> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2046) max_contig = MAX2(max_contig, empty_contig);
> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2047) last_idx = index;
> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2048) }
> 5f22805a50d4 (Kelvin Nilsen 2024-06-13 14:04:30 +0000 2049) index = _partitions.find_index_of_next_available_region(Shenando\
> ahFreeSetPartitionId::Mutator, index + 1);
> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2050) }
I think we may have drifted from openjdk/jdk after we forked our generational shenandoah work?
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645021724
From kdnilsen at openjdk.org Tue Jun 18 20:41:48 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 20:41:48 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v14]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <6Ef4fpuSWW5gE1AbiKxLBq2_VGexufn_lRN6P2TXg1U=.94a83181-275e-41e5-afa8-3bc84981a31b@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Resolve delta from openjdk/jdk in external_fragmentation() function
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/d113ee73..f7cb541b
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=13
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=12-13
Stats: 8 lines in 1 file changed: 4 ins; 1 del; 3 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Tue Jun 18 20:41:48 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 20:41:48 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <PzggWK2fF_7xHa_jr_-es_XAZUKBpEc6LOauyv4XtX0=.45abd722-6109-4af6-b1a6-2511a74dbe23@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<bHR03B8N6-GaBbYbhU75lE89OzESwGOiL6hI0YYhXic=.e4f1a94d-b20d-4fc3-8e84-0ba48300c505@github.com>
<N9NxlUFABJt_QVEgLxwrTVaTqoPzFDZX_r6Wq9rZ9Zs=.21fab225-8246-425b-88e0-08aac704594c@github.com>
<PzggWK2fF_7xHa_jr_-es_XAZUKBpEc6LOauyv4XtX0=.45abd722-6109-4af6-b1a6-2511a74dbe23@github.com>
Message-ID: <ZPneHXy8FTCNDxL9FPMD3wF8w49PzB0OTc8ka0lGyQg=.ae1073a3-40e8-4b96-b394-5043aacb6694@github.com>
On Tue, 18 Jun 2024 20:31:57 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> That change came in when I merged from master. I had assumed the change had also been applied upstream. Is your diff compared to a "recent" upstream?
>>
>> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2037) ShenandoahHeapRegion* r = _heap->get_region(index);
>> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2038) if (r->is_empty()) {
>> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2039) free += ShenandoahHeapRegion::region_size_bytes();
>> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2040) if (last_idx + 1 == index) {
>> dc184f1099e0 (Kelvin Nilsen 2024-05-16 16:47:09 +0000 2041) empty_contig++;
>> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2042) } else {
>> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2043) empty_contig = 0;
>> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2044) }
>> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2045)
>> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2046) max_contig = MAX2(max_contig, empty_contig);
>> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2047) last_idx = index;
>> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2048) }
>> 5f22805a50d4 (Kelvin Nilsen 2024-06-13 14:04:30 +0000 2049) index = _partitions.find_index_of_next_available_region(Shenando\
>> ahFreeSetPartitionId::Mutator, index + 1);
>> f37149b1c510 (Aleksey Shipilev 2020-03-23 19:14:01 +0100 2050) }
>
> I think we may have drifted from openjdk/jdk after we forked our generational shenandoah work?
I must have messed something up during manual merge conflict resolution.
I'm fixing this code to match upstream.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645026202
From wkemper at openjdk.org Tue Jun 18 20:57:54 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 20:57:54 GMT
Subject: RFR: 8333750: GenShen: Only instantiate young/old generations in
generational mode
Message-ID: <aNwl91iS6b1vG8acEoR9UzMZXBDgs8xYFxYMiOEuvUc=.30738a4f-64b9-47a9-bffc-d948b59f07b0@github.com>
Clean backport
-------------
Commit messages:
- Backport 2abf4cb43e84dda883a38abc6bcbac5aba3d7ce8
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/59/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=59&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333750
Stats: 890 lines in 22 files changed: 471 ins; 323 del; 96 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/59.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/59/head:pull/59
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/59
From wkemper at openjdk.org Tue Jun 18 21:16:32 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 21:16:32 GMT
Subject: Integrated: 8333750: GenShen: Only instantiate young/old generations
in generational mode
In-Reply-To: <aNwl91iS6b1vG8acEoR9UzMZXBDgs8xYFxYMiOEuvUc=.30738a4f-64b9-47a9-bffc-d948b59f07b0@github.com>
References: <aNwl91iS6b1vG8acEoR9UzMZXBDgs8xYFxYMiOEuvUc=.30738a4f-64b9-47a9-bffc-d948b59f07b0@github.com>
Message-ID: <Urr6PferXDL51coRfX0tUd71RFvd60RwrodK9c6s54M=.fe7c8ed0-84c4-46cb-86c6-64bd038e5b87@github.com>
On Tue, 18 Jun 2024 20:53:33 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Clean backport
This pull request has now been integrated.
Changeset: 25b860e7
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/25b860e79a02b4ea09d7620d4fa70eaaa3b7e1f0
Stats: 890 lines in 22 files changed: 471 ins; 323 del; 96 mod
8333750: GenShen: Only instantiate young/old generations in generational mode
Backport-of: 2abf4cb43e84dda883a38abc6bcbac5aba3d7ce8
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/59
From kdnilsen at openjdk.org Tue Jun 18 21:19:24 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 18 Jun 2024 21:19:24 GMT
Subject: RFR: 8334491: GenShen: Revert changes to Shenandoah defaults
In-Reply-To: <tnIkkjs-ko-HygX3eItnvGvjgx4uQpQunL4T2ezjzd4=.0b08474d-c6e6-478a-8b3d-81c7aba9cc39@github.com>
References: <tnIkkjs-ko-HygX3eItnvGvjgx4uQpQunL4T2ezjzd4=.0b08474d-c6e6-478a-8b3d-81c7aba9cc39@github.com>
Message-ID: <3rtc19jfP6oTsqNwm5NGkBBZrhhJlZ5xqVjcRGowxr8=.741605c2-9fc6-44be-92e4-03a62ded1f35@github.com>
On Tue, 18 Jun 2024 18:08:02 GMT, William Kemper <wkemper at openjdk.org> wrote:
> On the genshen branch, we have changed the adaptive heuristics decay factor to 0.1 and increased the learning cycles to 10. Testing with the original defaults of 0.5 and 5 (respectively) shows modest performance improvements in specjbb and extremem.
Marked as reviewed by kdnilsen (Committer).
-------------
PR Review: https://git.openjdk.org/shenandoah/pull/450#pullrequestreview-2126511373
From wkemper at openjdk.org Tue Jun 18 23:09:27 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 18 Jun 2024 23:09:27 GMT
Subject: Integrated: 8334491: GenShen: Revert changes to Shenandoah defaults
In-Reply-To: <tnIkkjs-ko-HygX3eItnvGvjgx4uQpQunL4T2ezjzd4=.0b08474d-c6e6-478a-8b3d-81c7aba9cc39@github.com>
References: <tnIkkjs-ko-HygX3eItnvGvjgx4uQpQunL4T2ezjzd4=.0b08474d-c6e6-478a-8b3d-81c7aba9cc39@github.com>
Message-ID: <E1FP0Ns042Ft4hMNYgg4gvqK2X9DFZE5gSRbRsoCN68=.f805ab12-98a6-41bb-b1d4-8e07bc12b20f@github.com>
On Tue, 18 Jun 2024 18:08:02 GMT, William Kemper <wkemper at openjdk.org> wrote:
> On the genshen branch, we have changed the adaptive heuristics decay factor to 0.1 and increased the learning cycles to 10. Testing with the original defaults of 0.5 and 5 (respectively) shows modest performance improvements in specjbb and extremem.
This pull request has now been integrated.
Changeset: 744fc265
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/744fc265aafe2fd59c2ca53c798ee930119f5d8d
Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
8334491: GenShen: Revert changes to Shenandoah defaults
Reviewed-by: kdnilsen
-------------
PR: https://git.openjdk.org/shenandoah/pull/450
From ysr at openjdk.org Wed Jun 19 01:01:29 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:01:29 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v14]
In-Reply-To: <6Ef4fpuSWW5gE1AbiKxLBq2_VGexufn_lRN6P2TXg1U=.94a83181-275e-41e5-afa8-3bc84981a31b@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<6Ef4fpuSWW5gE1AbiKxLBq2_VGexufn_lRN6P2TXg1U=.94a83181-275e-41e5-afa8-3bc84981a31b@github.com>
Message-ID: <J5E-WUuwsGPHyZgJUxQCaOTZWm_AEljvEpX35BHrYAg=.0c14b31b-60b6-4148-ba9f-a5791c9920fe@github.com>
On Tue, 18 Jun 2024 20:41:48 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Resolve delta from openjdk/jdk in external_fragmentation() function
Here's an initial tranche of comments. More as I complete the two remaining files or so. Apologies for the delay.
test/hotspot/jtreg/ProblemList.txt line 104:
> 102: gc/shenandoah/TestHumongousThreshold.java#16b 8327000 generic-all
> 103: gc/shenandoah/TestHumongousThreshold.java#generational 8327000 generic-all
> 104: gc/shenandoah/TestHumongousThreshold.java#generational-16b 8327000 generic-all
Very nice to see the problem list shrink!
-------------
PR Review: https://git.openjdk.org/shenandoah/pull/440#pullrequestreview-2124124227
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643582516
From ysr at openjdk.org Wed Jun 19 01:01:30 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:01:30 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 757:
> 755: heap->free_set()->prepare_to_rebuild(young_cset_regions, old_cset_regions, first_old, last_old, num_old);
> 756: // Free set construction uses reserve quantities, because they are known to be valid here
> 757: heap->free_set()->finish_rebuild(young_cset_regions, old_cset_regions, num_old, true);
is `num_old` different than the size of the `old_cset_regions` set?
src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp line 150:
> 148: void increase_allocated(size_t bytes);
> 149:
> 150: // These methods change the capacity of the region by adding or subtracting the given number of bytes from the current
I would say `generation`, instead of `region` here to avoid a little bit of unnecessary terminology confusion with the `heap region` object.
Also may be add something like: `Return the new capacity of the generation following the change.`
src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp line 155:
> 153: size_t decrease_capacity(size_t decrement);
> 154:
> 155: size_t establish_capacity(size_t byte_size);
Why not simply `set_capacity` ? Also a 1-line comment.
// set the capacity of the generation, returning the value set.
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 992:
> 990: // gc_no_progress_count is incremented following each degen or full GC that fails to achieve is_good_progress().
> 991: // Note that Generational Shenandoah may increment no_progress_count faster than traditional Shenandoah because young
> 992: // GCs, which may degenerate, typically occur more frequently than single-generation Global GCs.
Should this documentation comment about the expected difference in rates of incrementation occur in the vicinity of any differences (or otherwise) in the setting of the default value of `ShenandoahNoProgressThreshold` in each case -- or if the default values are the same, then where the option is documented?
I also wonder if `is_good_progress()` and `gc_no_progress_count` should apply uniformly in both cases to whole heap collections rather than to young collections (in the generational case). Otherwise, don't we run the risk of declaring no progress prematurely without a global collection in the case of generational, thus breaking in spirit the requirement of "best efforts attempt to reclaim unused memory"?
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1029:
> 1027: // the better remediation than repeated attempts to allocate following repeated GC cycles.
> 1028:
> 1029: while ((result == nullptr) && (original_count == shenandoah_policy()->full_gc_count())) {
I'd go a bot farther in the direction of being conservative and declare best effort only after a full/global GC (whether concurrent or stop-world, although stop-world would be better). I realize this may be much worse latency-wise than customers would care for, but I think the notion of latency being poor for a concurrent collector should probably be separated from the notion of cpu overhead for GC and space remaining after a best effort to reclaim unused memory. I realize that in the concurrent collector case where low latency is a design requirement, this would make the notion of OOM for exceeding gc overhead unlikely to ever engage, with latency already being very poor.
If we want to control that separately, I'd have some notion of frequency (or time density if you will) of latency-violating events that would trigger an OOM, although that would be new behaviour. Most JVM users typically use JMX/introspection to self-destruct or to raise an alarm about malfunction. It can of course be argued that that would cover the existing OOM upon gc overhead too, because ultimately a very slow JVM because of GC overhead would translate to worsening latency for user code.
Still, I would keep the notion of gc overhead and space available separate from the _latency_ of a stop-world or concurrent event and always ensure that we look at available space only following a global/full gc to decide if an OOM due to GC overhead should be thrown. Can discuss this more offline in person.
src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 121:
> 119:
> 120: // Change affiliation to YOUNG_GENERATION if _state is not _pinned_cset, _regular, or _pinned. This implements
> 121: // behavior previously performed as a side effect of make_regular_bypass(). This is used by Full GC
For this and other `make_*` methods of `ShenandoahHeapRegion`, I wonder if it makes sense to provide documentation in the header file. (Replicating it in the implementation is fine too, but not necessary.) What for example is the semantics of the `_bypass` suffix in the names of some of the methods? I was not able to easily tell from reading the code in the methods, so thought a documentation comment might be useful for at least the non-obvious cases.
src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 132:
> 130:
> 131:
> 132:
delete. The 3 blank lines seem to be a mismerge?
src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.inline.hpp line 78:
> 76: assert(new_top <= end(), "PLAB cannot span end of heap region");
> 77: set_top(new_top);
> 78: // We do not req.set_actual_size() here. The caller sets it.
Not needed now, but I would be much happier with some regular structure in the allocation paths of when the actual size of an allocation request is populated. I am guessing it is in the method in the call chain where the allocation first succeeds, or some other natural structure to the allocation paths. I'll try and see about getting a clearer mental map of those paths in the future.
src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 429:
> 427: size_t generation_capacity = generation->max_capacity();
> 428: guarantee(stats.span() <= generation_capacity,
> 429: "%s: generation (%s) size spanned by regions (" SIZE_FORMAT ") * region size: " PROPERFMT
For consistency, place the region size value also in circular braces `( ... )` like for the other values?
test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 93:
> 91: ProcessBuilder pb = ProcessTools.createLimitedTestJavaProcessBuilder(
> 92: "-Xmx32m",
> 93: "-XX:+UnlockExperimentalVMOptions", "-XX:ShenandoahNoProgressThreshold=24",
See my previous comment about when `ShenandoahNoProgressThreshold` should increment earlier in code.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643630442
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643625733
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643624102
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643584901
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643592499
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643614533
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643604825
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643616071
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643617762
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643619067
From ysr at openjdk.org Wed Jun 19 01:01:31 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:01:31 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
Message-ID: <AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
On Tue, 18 Jun 2024 00:41:09 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 992:
>
>> 990: // gc_no_progress_count is incremented following each degen or full GC that fails to achieve is_good_progress().
>> 991: // Note that Generational Shenandoah may increment no_progress_count faster than traditional Shenandoah because young
>> 992: // GCs, which may degenerate, typically occur more frequently than single-generation Global GCs.
>
> Should this documentation comment about the expected difference in rates of incrementation occur in the vicinity of any differences (or otherwise) in the setting of the default value of `ShenandoahNoProgressThreshold` in each case -- or if the default values are the same, then where the option is documented?
>
> I also wonder if `is_good_progress()` and `gc_no_progress_count` should apply uniformly in both cases to whole heap collections rather than to young collections (in the generational case). Otherwise, don't we run the risk of declaring no progress prematurely without a global collection in the case of generational, thus breaking in spirit the requirement of "best efforts attempt to reclaim unused memory"?
In particular, how is the notion of progress treated in other generational collectors?
> test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 93:
>
>> 91: ProcessBuilder pb = ProcessTools.createLimitedTestJavaProcessBuilder(
>> 92: "-Xmx32m",
>> 93: "-XX:+UnlockExperimentalVMOptions", "-XX:ShenandoahNoProgressThreshold=24",
>
> See my previous comment about when `ShenandoahNoProgressThreshold` should increment earlier in code.
Hmm, interesting, it looks like this does indeed increment only at a full gc, or at least that's the intent. Should check that this is also respected in the generational case. Vide:
product(uintx, ShenandoahNoProgressThreshold, 5, EXPERIMENTAL, \
"After this number of consecutive Full GCs fail to make " \
"progress, Shenandoah will raise out of memory errors. Note " \
"that progress is determined by ShenandoahCriticalFreeThreshold") \
\
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643587192
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643622489
From ysr at openjdk.org Wed Jun 19 01:01:31 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:01:31 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <SSODN7yOFMTXNfEYcTQSGKexUkK2QhRjx589cStFOmU=.61d76999-a4c4-43d3-828f-c5e4af88efec@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<SSODN7yOFMTXNfEYcTQSGKexUkK2QhRjx589cStFOmU=.61d76999-a4c4-43d3-828f-c5e4af88efec@github.com>
Message-ID: <3ZpXVQYYstqMUZ0AAZ3jV0pM9EHupRPslcIh_KOtB6k=.4e816c99-96dc-4523-b6e1-d56ac84e6d40@github.com>
On Mon, 17 Jun 2024 20:39:33 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 138:
>
>> 136: if (is_old()) {
>> 137: heap->old_generation()->decrement_affiliated_region_count();
>> 138: heap->old_generation()->decrease_capacity(region_size_bytes());
>
> Should capacity be updated every time affiliated region count changes? If yes, then we could have increase/decrease affiliated region count increase/decrease capacity directly. If no, is this something peculiar to full GC?
What William asks above.
Additionally, it is also curious to me that the only caller of `make_young_maybe()` from full gc here: https://github.com/openjdk/shenandoah/blob/0350ce08f7362bc6091569f051760ef7c870e804/src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp#L993 seems to call it in only the non-generational case, while here we change affiliation from old (and adjust capacity & count) quite naturally only in the generational case (which is where old gen affiliation is possible).
I am clearly missing something here.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1643612458
From ysr at openjdk.org Wed Jun 19 01:15:30 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:15:30 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <UNATztrz0rkT2oRtOUQcLItnKONDmo4typw4omUsYpI=.463a6723-28f1-4bb9-878e-3ce1daefc6ef@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 367:
> 365: // evacuation reserves are unknown, is based in part on anticipated promotion as determined by analysis of live data
> 366: // found during the previous GC pass which is one less than the current tenure age.
> 367: void finish_rebuild(size_t young_cset_regions, size_t old_cset_regions, size_t num_old_regions,
For the documentation comments, for the individual public methods themselves, viz. `prepare_to_rebuild`, `finish_rebuild`, `find_regions_with_alloc_capacity` etc., it's easier to have a terser comment that describes the parameters to each call and their side-effect/result.
Then, in the callers of these methods (the implementation of the rebuild) you can place the commentary about how the rebuild is happening by stitching these methods together. Otherwise the header file can be confusing in the form of a narrative.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645267905
From ysr at openjdk.org Wed Jun 19 01:18:36 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:18:36 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <wc-NsFOp_rs76M3jgu6QMMFlUw0A5T7W_1kdR-Ve_Jw=.7589f23f-58d3-4d87-b295-e7d9a31f919c@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 382:
> 380: // of the regions recycled from the collection set will be available. In the very unlikely event that there
> 381: // are fewer regions in the collection set than remain in the collector's free set, we limit the transfer in order
> 382: // to assure that the replenished Collector reserves can be sufficiently large.
Apropos my previous comment, this part would seem to belong in the caller(s?) of the method.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645269790
From ysr at openjdk.org Wed Jun 19 01:33:26 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:33:26 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <1vKm5lgmDiVn1kywz1UM9hnlN7HzdNfF6DtME5o_kow=.8d13ce98-0bd8-4f1f-9728-1e56c5d5986a@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1419:
> 1417: }
> 1418:
> 1419: void ShenandoahFreeSet::move_regions_from_collector_to_mutator(size_t max_xfer_regions) {
This method has 3 isomorphic loops. I wonder if it might read better to extract a separate work method for the loop and invoke it thrice from this method with appropriate parms. Also I'd advise using a local variable to countdown the regions to be transferred rather than modifying the method's input parm.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645277383
From ysr at openjdk.org Wed Jun 19 01:37:27 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:37:27 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <GyisTO4qqSYHpt5VYD8yPGbdUfRsSvurqXnhk4CO8Ws=.17e7ae3d-8fe4-4b43-91a4-0c5bccd54e84@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 408:
> 406: // capacity as NotFree. Subsequently, we will move some of the mutator regions into the collector and old collector
> 407: // partitions with the intent of packing old collector memory into the highest (far rightmost) addresses of the heap,
> 408: // young collector memory into higher address, and mutator memory consuming the lowest addresses of the heap.
Is the part of the comment that starts at "Subsequently, we will ..." done as part of this method? If not, then it should go into the caller of this method (after this method is called), rather than stating what the caller will do. I'd keep documentation of these methods very simple: what its parms are and what it does.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645279602
From ysr at openjdk.org Wed Jun 19 01:50:25 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:50:25 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <Dfq-3agINZmuDBzMHZNu4_fO6WEP0fXxzSgU2OGJqmU=.d38c7120-a610-4110-9f21-e9d5405c9936@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 418:
> 416: // old collector partition to assure that old collector partition has at least old_reserve. Update old_region_count
> 417: // to represent the total number of regions in the old generation by adding the number of regions moved from the
> 418: // mutator partition to the old collector partition.
How about:
Ensure that Collector has at least `to_reserve` regions, and OldCollector has at
least `old_reserve` regions.
Upon return `old_region_count` holds the number of regions in OldCollector.
This makes me wonder about the asymmetry of the return value. Also could it be a plain return value from the method rather than var parameter to the method?
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 423:
> 421:
> 422:
> 423: void reserve_regions(size_t young_reserve, size_t old_reserve);
Is the old variant without the var parameter still used somewhere?
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645283876
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645285901
From ysr at openjdk.org Wed Jun 19 01:55:25 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:55:25 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <KvSJGDQocvUwldX2wRC0guST3fK50lYDZ1oUxkDyIew=.51a89494-d439-4180-8a75-7e735f8dfb86@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 264:
> 262: // collection set (cset).
> 263: //
> 264: // The ShenandoahFreeSet endeavors to congregrate survivor objects (objects that have been evacuated at least once) at the
May be "tries to colocate" is simpler than "endeavors to congregate"?
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645288342
From ysr at openjdk.org Wed Jun 19 01:58:23 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 01:58:23 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <hkjZzY9tRnog1qWiaQYKDkX6d2eXbFCH8SzyxKhJYCA=.0485cb67-164f-4057-b7bb-a280553d6639@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 280:
> 278: // sure there is enough memory reserved at the high end of memory to hold the objects that might need to be evacuated
> 279: // during the next GC pass.
> 280:
This is a very useful block comment, thanks!
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645289853
From ysr at openjdk.org Wed Jun 19 02:03:33 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 19 Jun 2024 02:03:33 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
Message-ID: <RP8bvZ4fMB1d6B8HObsciIk1cshdj5cOEitpvvzxQLQ=.39feb61b-6205-4bea-9393-e358c006bb7e@github.com>
On Thu, 13 Jun 2024 22:22:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Minor refinements to test programs
>
> TestAllocIntArrays: comments to explain behavior.
> TestOldGrowthTriggers: reduce the number of loop iterations so this test
> will not time out on less powerful test platforms.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 290:
> 288:
> 289: // Satisfy collector allocation request req by finding memory that matches affiliation from within the
> 290: // free partition associated with which_partition.
Returns the address allocated and `in_new_region` indicates if the allocation was
satisfied from a new, previously empty region.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1645292351
From kbarrett at openjdk.org Wed Jun 19 08:41:24 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Wed, 19 Jun 2024 08:41:24 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort [v2]
In-Reply-To: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
Message-ID: <vEw_CJ5GwZgEBnk3P27bxURjZlWH0oWUCyK8KPtRg5Y=.97e5b0e0-ae39-4976-a536-a6348093c068@github.com>
> The "idempotent" argument is removed from that function, with associated
> simplifications to the implementation. Callers are updated to remove that
> argument. Callers that were providing a false value are unaffected in their
> behavior. The 3 callers that were providing a true value to request the
> associated feature are also unaffected (other than by being made faster),
> because the arrays involved don't contain any equivalent pairs.
>
> There are also some miscellaneous cleanups, including using the swap utility
> and fixing some comments.
>
> Testing: mach5 tier1-3
Kim Barrett has updated the pull request incrementally with one additional commit since the last revision:
improve find_pivot description
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19464/files
- new: https://git.openjdk.org/jdk/pull/19464/files/154a5ed0..5cee3b81
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19464&range=01
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19464&range=00-01
Stats: 6 lines in 1 file changed: 2 ins; 2 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19464.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19464/head:pull/19464
PR: https://git.openjdk.org/jdk/pull/19464
From xpeng at openjdk.org Wed Jun 19 16:40:22 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 19 Jun 2024 16:40:22 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v2]
In-Reply-To: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
Message-ID: <S5v4mM4qM_tYkHDzsyDq8hG80HaEqvzyHxVxJeSHOCg=.f6d472f0-63ec-4f5d-aa18-817ce164e49a@github.com>
> ### Notes
> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x0, 0x1, 0x3, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>
> Also noticed there was regression in Dacapo h2 benchmark, after deep dive and debug we decided to let non-java threads to only spin, which favors GC threads a little over Java threads at contented lock. Some follow-up tasks will be taken to reduce lock contention from Shenandoah GC, e.g. https://bugs.openjdk.org/browse/JDK-8334147
>
> #### Test code
>
> public class Alloc {
> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
> static final Object[] sinks = new Object[64*THREADS];
> static volatile boolean start;
> static volatile boolean stop;
>
> public static void main(String... args) throws Throwable {
> for (int t = 0; t < THREADS; t++) {
> int ft = t;
> new Thread(() -> work(ft * 64)).start();
> }
>
> Thread.sleep(1000);
> start = true;
> Thread.sleep(30_000);
> stop = true;
> }
>
> public static void work(int idx) {
> while (!start) { Thread.onSpinWait(); }
> while (!stop) {
> sinks[idx] = new byte[128];
> }
> }
> }
>
>
> Run it like this and observe TTSP times:
>
>
> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>
>
> #### Metrics from tests(TTSP, allocation rate)
> ##### Heavy contention(1280 threads, 32 per CPU core)
> | Test | Count | AVG | TRIMMEAN 2% | MAX | MIN | AVG allocation rate(M/s) |
> | ---------- | ----- | ------ | ----------- | -------- | ----- | ------------------------ |
> | Baseline | 19 | 940270 | 940270 | 5956928 | 75562 | 23.34 |
> | No spin | 164 | 222958 | 204822 | 3330053 | 53819 | 238.9 |
> | 0x01 | 172 | 189173 | 186601 | 750715 | 64864 | 244.1 |
> | 0x03 | 174 | 286892 | 217739 | 12412225 | 55891 | 239.5 |
> | 0x07 | 187 | 194440 | 183894 | 2284615 | 55256 | 235.9 |
> | 0x0F | 1...
Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
- Wait on STS_lock to suspend Java thread acquiring heap lock when SP is synchronizing
- Test code
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19570/files
- new: https://git.openjdk.org/jdk/pull/19570/files/d5d8b65f..f388b344
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=01
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=00-01
Stats: 35 lines in 2 files changed: 20 ins; 6 del; 9 mod
Patch: https://git.openjdk.org/jdk/pull/19570.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19570/head:pull/19570
PR: https://git.openjdk.org/jdk/pull/19570
From kdnilsen at openjdk.org Wed Jun 19 17:56:37 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 17:56:37 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <XTUdcHSBHS7PyxCwOKzgs8exyw9p2m7bKnYAi2aY7CE=.50adc38e-e1b3-4a13-b1a3-6331fe0b9a3d@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<XTUdcHSBHS7PyxCwOKzgs8exyw9p2m7bKnYAi2aY7CE=.50adc38e-e1b3-4a13-b1a3-6331fe0b9a3d@github.com>
Message-ID: <LCsQfXZMEE4bQZy3IxdIOMay4SQze2OB2_afIAmE8vU=.c83b062a-2159-4f12-a35f-cb51cc27aa1f@github.com>
On Mon, 17 Jun 2024 20:34:59 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> test/hotspot/jtreg/gc/shenandoah/TestAllocIntArrays.java line 211:
>
>> 209: final int max = 384 * 1024;
>> 210: // Each allocated int array is assumed to consume 16 bytes for alignment and header, plus
>> 211: // an everage of 4 * the average number of elements in the array.
>
> s/everage/average
Thanks. Corrected.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646558688
From kdnilsen at openjdk.org Wed Jun 19 18:05:38 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 18:05:38 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
<AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
Message-ID: <nymh3yPGccULWMvwJcFZtP0C8ULrxxZjAiSEUCoB5Gw=.4b3fcac4-d134-4fba-aaef-7480a5c51443@github.com>
On Tue, 18 Jun 2024 00:45:59 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 992:
>>
>>> 990: // gc_no_progress_count is incremented following each degen or full GC that fails to achieve is_good_progress().
>>> 991: // Note that Generational Shenandoah may increment no_progress_count faster than traditional Shenandoah because young
>>> 992: // GCs, which may degenerate, typically occur more frequently than single-generation Global GCs.
>>
>> Should this documentation comment about the expected difference in rates of incrementation occur in the vicinity of any differences (or otherwise) in the setting of the default value of `ShenandoahNoProgressThreshold` in each case -- or if the default values are the same, then where the option is documented?
>>
>> I also wonder if `is_good_progress()` and `gc_no_progress_count` should apply uniformly in both cases to whole heap collections rather than to young collections (in the generational case). Otherwise, don't we run the risk of declaring no progress prematurely without a global collection in the case of generational, thus breaking in spirit the requirement of "best efforts attempt to reclaim unused memory"?
>
> In particular, how is the notion of progress treated in other generational collectors?
Thanks for asking these questions and digging into the details. I see in your other comment that it is the intention that we only increment gc_no_progress_count for consecutive full GCs. However, our implementation seems to have drifted from that intention. We also increment following a degen that does not make progress. And then we increment again in the Full GC after we upgrade to full. Maybe there are better ways to fix this. I'm investigating.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646564625
From kdnilsen at openjdk.org Wed Jun 19 18:13:35 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 18:13:35 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <nymh3yPGccULWMvwJcFZtP0C8ULrxxZjAiSEUCoB5Gw=.4b3fcac4-d134-4fba-aaef-7480a5c51443@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
<AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
<nymh3yPGccULWMvwJcFZtP0C8ULrxxZjAiSEUCoB5Gw=.4b3fcac4-d134-4fba-aaef-7480a5c51443@github.com>
Message-ID: <gOiIvHxl_rA6L1-eRJqTV5Sgu-VOQ6bXHKnt9kJ5GlE=.e1846fca-8b29-4e53-a424-dab0ce08724f@github.com>
On Wed, 19 Jun 2024 18:02:28 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> In particular, how is the notion of progress treated in other generational collectors?
>
> Thanks for asking these questions and digging into the details. I see in your other comment that it is the intention that we only increment gc_no_progress_count for consecutive full GCs. However, our implementation seems to have drifted from that intention. We also increment following a degen that does not make progress. And then we increment again in the Full GC after we upgrade to full. Maybe there are better ways to fix this. I'm investigating.
I'm of the "opinion" that consecutive Full GC failures means no intervening successful concurrent or degenerated collections.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646573071
From kdnilsen at openjdk.org Wed Jun 19 18:24:33 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 18:24:33 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <gOiIvHxl_rA6L1-eRJqTV5Sgu-VOQ6bXHKnt9kJ5GlE=.e1846fca-8b29-4e53-a424-dab0ce08724f@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
<AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
<nymh3yPGccULWMvwJcFZtP0C8ULrxxZjAiSEUCoB5Gw=.4b3fcac4-d134-4fba-aaef-7480a5c51443@github.com>
<gOiIvHxl_rA6L1-eRJqTV5Sgu-VOQ6bXHKnt9kJ5GlE=.e1846fca-8b29-4e53-a424-dab0ce08724f@github.com>
Message-ID: <aZid_J1sW5hF-AYWs6VuPYmVBXwl5DoFklDjQK0Nf9c=.457cd8f2-1bda-4b77-8f4e-604433bade3c@github.com>
On Wed, 19 Jun 2024 18:11:08 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> Thanks for asking these questions and digging into the details. I see in your other comment that it is the intention that we only increment gc_no_progress_count for consecutive full GCs. However, our implementation seems to have drifted from that intention. We also increment following a degen that does not make progress. And then we increment again in the Full GC after we upgrade to full. Maybe there are better ways to fix this. I'm investigating.
>
> I'm of the "opinion" that consecutive Full GC failures means no intervening successful concurrent or degenerated collections.
I'm proposing two changes:
1. Do not increment gc_no_progress in failed degens
2. Reset gc_no_progress to zero after successful concurrent gc
I will remove this comment, and I will test this change. I believe this may allow me to backoff the extreme testing overrides of ShenandoahNoProgressThreshold.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646580580
From kdnilsen at openjdk.org Wed Jun 19 18:57:46 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 18:57:46 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v15]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <7dc6WsL2GSMcRv4Tml1dNkYB9yvdNVHV9tu7n6gB-68=.e35d1d36-d69c-45f1-8f18-4bfc05112930@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Only increment gc_no_progess count following back-to-back failed fullgc
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/f7cb541b..ad42243a
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=14
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=13-14
Stats: 8 lines in 4 files changed: 6 ins; 2 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Wed Jun 19 20:54:30 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 20:54:30 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
Message-ID: <GnERPP6FLntpoaDzEpDxikZbKiECKm47Q7-dWg7CA2w=.74c333f7-04d1-466b-b535-db67895385b0@github.com>
On Tue, 18 Jun 2024 00:56:30 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1029:
>
>> 1027: // the better remediation than repeated attempts to allocate following repeated GC cycles.
>> 1028:
>> 1029: while ((result == nullptr) && (original_count == shenandoah_policy()->full_gc_count())) {
>
> I'd go a bot farther in the direction of being conservative and declare best effort only after a full/global GC (whether concurrent or stop-world, although stop-world would be better). I realize this may be much worse latency-wise than customers would care for, but I think the notion of latency being poor for a concurrent collector should probably be separated from the notion of cpu overhead for GC and space remaining after a best effort to reclaim unused memory. I realize that in the concurrent collector case where low latency is a design requirement, this would make the notion of OOM for exceeding gc overhead unlikely to ever engage, with latency already being very poor.
>
> If we want to control that separately, I'd have some notion of frequency (or time density if you will) of latency-violating events that would trigger an OOM, although that would be new behaviour. Most JVM users typically use JMX/introspection to self-destruct or to raise an alarm about malfunction. It can of course be argued that that would cover the existing OOM upon gc overhead too, because ultimately a very slow JVM because of GC overhead would translate to worsening latency for user code.
>
> Still, I would keep the notion of gc overhead and space available separate from the _latency_ of a stop-world or concurrent event and always ensure that we look at available space only following a global/full gc to decide if an OOM due to GC overhead should be thrown. Can discuss this more offline in person.
These are subtle issues. I'm not sure how best to address. Will go with consensus. One other observation:
Exception in thread thread_name: java.lang.OutOfMemoryError: GC Overhead limit exceeded
Cause: The detail message "GC overhead limit exceeded" indicates that the garbage collector is running all the time and Java program is making very slow progress. After a garbage collection, if the Java process is spending more than approximately 98% of its time doing garbage collection and if it is recovering less than 2% of the heap and has been doing so far the last 5 (compile time constant) consecutive garbage collections, then a java.lang.OutOfMemoryError is thrown. This exception is typically thrown because the amount of live data barely fits into the Java heap having little free space for new allocations.
from https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html
This description aligns with our default behavior of ShenandoahNoProgressThreshold default of 5, which would trigger in this PR's current implementation if we do 5 back-to-back unproductive full GCs (regardless of how much CPU time is consumed by GC vs mutator threads). Unproductive is defined as NOT is_good_progress(), where is_good_progress() (see shenandoahMetrics.cpp) is:
1. free memory is at least 1% of the heap capacity, and
a. The amount of used memory is decreased by at least the size of one region, or
b. We have decreased internal fragmentation by at least 1%, or
c. We have decreased external fragmentation by at least 1%
One problem that I have observed with certain test programs is that this definition of "is_good_progress" is pretty generous. We can have many back-to-back full gcs that "make progress" according to this generous definition, while a mutator thread repeatedly retries and fails to satisfy its allocation request.
My judgement is that the mutator is much better served by getting a quick OOM exception rather than chugging away in this retry loop.
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 132:
>
>> 130:
>> 131:
>> 132:
>
> delete. The 3 blank lines seem to be a mismerge?
Thanks. Removed.
> src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 429:
>
>> 427: size_t generation_capacity = generation->max_capacity();
>> 428: guarantee(stats.span() <= generation_capacity,
>> 429: "%s: generation (%s) size spanned by regions (" SIZE_FORMAT ") * region size: " PROPERFMT
>
> For consistency, place the region size value also in circular braces `( ... )` like for the other values?
Thanks. Fixed.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646673274
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646673607
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646673995
From kdnilsen at openjdk.org Wed Jun 19 21:00:26 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 21:00:26 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
Message-ID: <TTscx1mbOdFSU1gEMN0fZv3mYqQubZECHImAaGM24Po=.f15c3ff9-530f-4e00-b29d-3918a235fac3@github.com>
On Tue, 18 Jun 2024 01:54:13 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp line 155:
>
>> 153: size_t decrease_capacity(size_t decrement);
>> 154:
>> 155: size_t establish_capacity(size_t byte_size);
>
> Why not simply `set_capacity` ? Also a 1-line comment.
>
> // set the capacity of the generation, returning the value set.
Thanks. Fixed.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646681507
From kdnilsen at openjdk.org Wed Jun 19 21:00:27 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 21:00:27 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
<AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
Message-ID: <2wDffC_YF6eu9FcvWaM8Y4KoSNkzwCslrzT9RbfcKhY=.b53f155e-1575-4281-881c-e88c99ce979b@github.com>
On Tue, 18 Jun 2024 01:51:02 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 93:
>>
>>> 91: ProcessBuilder pb = ProcessTools.createLimitedTestJavaProcessBuilder(
>>> 92: "-Xmx32m",
>>> 93: "-XX:+UnlockExperimentalVMOptions", "-XX:ShenandoahNoProgressThreshold=24",
>>
>> See my previous comment about when `ShenandoahNoProgressThreshold` should increment earlier in code.
>
> Hmm, interesting, it looks like this does indeed increment only at a full gc, or at least that's the intent. Should check that this is also respected in the generational case. Vide:
>
>
> product(uintx, ShenandoahNoProgressThreshold, 5, EXPERIMENTAL, \
> "After this number of consecutive Full GCs fail to make " \
> "progress, Shenandoah will raise out of memory errors. Note " \
> "that progress is determined by ShenandoahCriticalFreeThreshold") \
> \
I'm testing now a configuration that honors the intent of this comment, to only increment gc_no_progress count if "consecutive Full GCs" fail to make progress. This is not how it was implemented before, because our implementation has been also incrementing gc_no_progress count if degenerated fails to make progress, and we have not been resetting the count when we experience a productive concurrent GC. My interpretation of the intent is that a concurrent GC happening between two unproductive Full GCs does not count as two "consecutive" unproductive full GCs.
If this change behaves well, we may be able to remove the ShenandoahNoProgressThreshold override on TestThreadFailure#generational.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646679059
From kdnilsen at openjdk.org Wed Jun 19 21:06:29 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 21:06:29 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
Message-ID: <yNeSuAPfAgsNwegMyF9DxM68Jx08cV_MzjKD_c35iOs=.cba0860e-a8a1-457d-893d-b8bd75f98bc8@github.com>
On Tue, 18 Jun 2024 01:56:15 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp line 150:
>
>> 148: void increase_allocated(size_t bytes);
>> 149:
>> 150: // These methods change the capacity of the region by adding or subtracting the given number of bytes from the current
>
> I would say `generation`, instead of `region` here to avoid a little bit of unnecessary terminology confusion with the `heap region` object.
>
> Also may be add something like: `Return the new capacity of the generation following the change.`
Agree. Fixed.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646686817
From kdnilsen at openjdk.org Wed Jun 19 22:00:29 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 22:00:29 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
Message-ID: <Qy05gl3ZsYMRBB4GOVj5Vu6iJBxBuIKnRISiXEO_9qs=.54ae9b0d-85d5-4025-b7f3-9f59a54aab94@github.com>
On Tue, 18 Jun 2024 02:01:01 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 757:
>
>> 755: heap->free_set()->prepare_to_rebuild(young_cset_regions, old_cset_regions, first_old, last_old, num_old);
>> 756: // Free set construction uses reserve quantities, because they are known to be valid here
>> 757: heap->free_set()->finish_rebuild(young_cset_regions, old_cset_regions, num_old, true);
>
> is `num_old` different than the size of the `old_cset_regions` set?
I'll add a comment. num_old is the number of regions that currently can be allocated from within the Collector set (excluding the old-gen cset).
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646715262
From kdnilsen at openjdk.org Wed Jun 19 22:18:23 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 22:18:23 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <aZid_J1sW5hF-AYWs6VuPYmVBXwl5DoFklDjQK0Nf9c=.457cd8f2-1bda-4b77-8f4e-604433bade3c@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
<AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
<nymh3yPGccULWMvwJcFZtP0C8ULrxxZjAiSEUCoB5Gw=.4b3fcac4-d134-4fba-aaef-7480a5c51443@github.com>
<gOiIvHxl_rA6L1-eRJqTV5Sgu-VOQ6bXHKnt9kJ5GlE=.e1846fca-8b29-4e53-a424-dab0ce08724f@github.com>
<aZid_J1sW5hF-AYWs6VuPYmVBXwl5DoFklDjQK0Nf9c=.457cd8f2-1bda-4b77-8f4e-604433bade3c@github.com>
Message-ID: <W0JFMI22nV9rzax8m47a27h6snaQgFdZ17CEHz_6mAs=.dd42ac2a-158e-49f1-9a89-5ec73929b6f9@github.com>
On Wed, 19 Jun 2024 18:22:07 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> I'm of the "opinion" that consecutive Full GC failures means no intervening successful concurrent or degenerated collections.
>
> I'm proposing two changes:
> 1. Do not increment gc_no_progress in failed degens
> 2. Reset gc_no_progress to zero after successful concurrent gc
>
> I will remove this comment, and I will test this change. I believe this may allow me to backoff the extreme testing overrides of ShenandoahNoProgressThreshold.
If we don't reset gc_no_progress to zero after successful concurrent gc, it seems the use of ShenandoahNoProgressThreshold might be kind of "arbitrary". For example, TestThreadFailure repeatedly drives the heap to exhaustion, and then it cleans up and does it again. Each time it reaches exhaustion, we escalate to FullGC and throw OOM, which causes the mutator to release its memory. Having released its memory, we'll run multiple successful concurrent GCs until we once again have exhausted memory, at which time we'll once again escalate to FullGC, which will be unproductive and we'll thrown OOM.
The sixth time though this loop (the sixth consecutive Full GC with many successful intervening concurrent GCs), we'll see that _gc_no_progress_count exceeds ShenandoahNoProgressThreshold and will throw OOM without even trying to do a Full GC. I think this is not what we want.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646721865
From kdnilsen at openjdk.org Wed Jun 19 22:18:24 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 22:18:24 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <GnERPP6FLntpoaDzEpDxikZbKiECKm47Q7-dWg7CA2w=.74c333f7-04d1-466b-b535-db67895385b0@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
<GnERPP6FLntpoaDzEpDxikZbKiECKm47Q7-dWg7CA2w=.74c333f7-04d1-466b-b535-db67895385b0@github.com>
Message-ID: <YM5lSVBpKNCSTkBYRi7t6R2EHzP6darMNgdqqHxsf2s=.3a6304e6-b582-46c0-ac34-58dab8fdcfcf@github.com>
On Wed, 19 Jun 2024 20:51:06 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 132:
>>
>>> 130:
>>> 131:
>>> 132:
>>
>> delete. The 3 blank lines seem to be a mismerge?
>
> Thanks. Removed.
Fixed. Thanks.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646722585
From kdnilsen at openjdk.org Wed Jun 19 22:30:31 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 22:30:31 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <UNATztrz0rkT2oRtOUQcLItnKONDmo4typw4omUsYpI=.463a6723-28f1-4bb9-878e-3ce1daefc6ef@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<UNATztrz0rkT2oRtOUQcLItnKONDmo4typw4omUsYpI=.463a6723-28f1-4bb9-878e-3ce1daefc6ef@github.com>
Message-ID: <taiH-0YR0clLNhgnKenITziNmT7QBAX2C5sBIxh98Mo=.e6a4ee86-f7a8-44e6-acbf-8ed5caa697c8@github.com>
On Wed, 19 Jun 2024 01:12:40 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 367:
>
>> 365: // evacuation reserves are unknown, is based in part on anticipated promotion as determined by analysis of live data
>> 366: // found during the previous GC pass which is one less than the current tenure age.
>> 367: void finish_rebuild(size_t young_cset_regions, size_t old_cset_regions, size_t num_old_regions,
>
> For the documentation comments, for the individual public methods themselves, viz. `prepare_to_rebuild`, `finish_rebuild`, `find_regions_with_alloc_capacity` etc., it's easier to have a terser comment that describes the parameters to each call and their side-effect/result.
>
> Then, in the callers of these methods (the implementation of the rebuild) you can place the commentary about how the rebuild is happening by stitching these methods together. Otherwise the header file can be confusing in the form of a narrative.
I've taken a pass at improving descriptions of all arguments to these functions.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646727798
From kdnilsen at openjdk.org Wed Jun 19 22:53:27 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 19 Jun 2024 22:53:27 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <wc-NsFOp_rs76M3jgu6QMMFlUw0A5T7W_1kdR-Ve_Jw=.7589f23f-58d3-4d87-b295-e7d9a31f919c@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<wc-NsFOp_rs76M3jgu6QMMFlUw0A5T7W_1kdR-Ve_Jw=.7589f23f-58d3-4d87-b295-e7d9a31f919c@github.com>
Message-ID: <8ORGOZXP7vYTOtvqjBpUqtWG6_JAGqEsw0bOGdkirz0=.01bb3273-a017-4dd7-a5dd-9fddb6ab5de7@github.com>
On Wed, 19 Jun 2024 01:16:02 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 382:
>
>> 380: // of the regions recycled from the collection set will be available. In the very unlikely event that there
>> 381: // are fewer regions in the collection set than remain in the collector's free set, we limit the transfer in order
>> 382: // to assure that the replenished Collector reserves can be sufficiently large.
>
> Apropos my previous comment, this part would seem to belong in the caller(s?) of the method.
I'm moving a variation of this comment to the callers. Thanks.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1646737669
From ysr at openjdk.org Thu Jun 20 01:08:25 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 20 Jun 2024 01:08:25 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v15]
In-Reply-To: <7dc6WsL2GSMcRv4Tml1dNkYB9yvdNVHV9tu7n6gB-68=.e35d1d36-d69c-45f1-8f18-4bfc05112930@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<7dc6WsL2GSMcRv4Tml1dNkYB9yvdNVHV9tu7n6gB-68=.e35d1d36-d69c-45f1-8f18-4bfc05112930@github.com>
Message-ID: <NVrZH_u2JFLU8jQsAonpV21pJ7RQGKNpTP722wfOrXM=.6709226e-57a1-405c-a5e4-e345a340c0ab@github.com>
On Wed, 19 Jun 2024 18:57:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Only increment gc_no_progess count following back-to-back failed fullgc
I think that's most of my comments. Looks mostly good otherwise, so I'm approving, but let me know if you should need a quick re-review for any substantive changes stemming from reviews (with the caveat that I might not get to rereview before Tuesday June 25 on account of PTO).
-------------
Marked as reviewed by ysr (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/440#pullrequestreview-2129204039
From ysr at openjdk.org Thu Jun 20 02:07:02 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 20 Jun 2024 02:07:02 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v17]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <5y0Pa7akLojTMgbp4nPmpcQZmAwya3wNsXnoZWaA1lE=.72b63ae7-f922-4b75-aba6-d36be254428b@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
Revert debugging detritus.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/407/files
- new: https://git.openjdk.org/shenandoah/pull/407/files/b513abae..556e5cd8
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=16
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=15-16
Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From ysr at openjdk.org Thu Jun 20 02:13:00 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 20 Jun 2024 02:13:00 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v3]
In-Reply-To: <L-pDWdxZYQL15bdgT2UPPCIDU30Xt2lzyjNAsUakCd0=.42803be7-3c39-46d7-b971-c87e918513e7@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
<wDQSfRdHe1FwUgjxrRuWvJ5_6wojOjDr2IwiZx9h0ZE=.f140d429-946d-45c0-b64c-b128c79b488e@github.com>
<L-pDWdxZYQL15bdgT2UPPCIDU30Xt2lzyjNAsUakCd0=.42803be7-3c39-46d7-b971-c87e918513e7@github.com>
Message-ID: <EU9P1x25NvgX8vK3NXuUooNVC3uQFtB5BLckfiqg6m4=.190a4439-1c0c-4cf5-a0fd-2283a3e81f5a@github.com>
On Fri, 17 May 2024 17:23:34 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits:
>>
>> - jcheck whitespace
>> - Fixed mismerge because of code moving between files in merge from
>> master; encapsulate assert & provide more useful debugging info. We
>> aren't done yet... It might be the case that we do away with the
>> sync/async split of the variable and instead move the relevant state
>> into the closure if possible. That might have to wait for another day
>> though based on whether it'll work well everywhere or not.
>> - Merge branch 'master' into active_generation
>> - Greater separation of gc_generation and active_generation fields of
>> ShHeap. WIP.
>> - Clean ups.
>> - jcheck whitespace
>> - Banish forcing entirely. Fix one case where full gc sets
>> active_generation directly. Might be other cases as well. Will need a
>> cleaner and more natural abstraction different from the unseemly
>> under-the-covers groveling that we do currently.
>> - Relax an assert that is too strong; still iterating execution paths,
>> more cleanups planned.
>> - Disallow forcing.
>> Test.
>> - Remove an incorrect gc status setting.
>> - ... and 22 more: https://git.openjdk.org/shenandoah/compare/dfa78866...61848c82
>
> test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java line 81:
>
>> 79: ProcessBuilder pb = ProcessTools.createLimitedTestJavaProcessBuilder(
>> 80: "-Xmx32m",
>> 81: // "-XX:+UnlockExperimentalVMOptions", "-XX:ShenandoahNoProgressThreshold=12", "-XX:+ShowMessageBoxOnError",
>
> Ignore this. Will be removed; debugging detritus left in place until we believe we are ready.
Removed in latest. Resolving.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1646834814
From ysr at openjdk.org Thu Jun 20 02:12:58 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 20 Jun 2024 02:12:58 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v18]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <O9oZXLGGwlR5rv2uKfDhns4KEeZSoKmDskn_OxHk5XU=.24d2b800-0101-4441-8cc7-c1183b260786@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 48 commits:
- Merge branch 'master' into active_generation
- Revert debugging detritus.
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Code called from LRB cannot rely on SGH::_gc_generation and must instead
use the _active_generation field.
Still need to aduit remaining _gc_generation uses for safety.
Removed too strong assert introduced for debugging in previous commit.
- Merge branch 'master' into active_generation
- Some asserts to catch a tricky race. These assertions may be too strong
in general but would help with debugging a rare crash.
- Merge branch 'master' into active_generation
- Remove vestigial thread() method in ShenandoahController.
- ... and 38 more: https://git.openjdk.org/shenandoah/compare/744fc265...73ea4694
-------------
Changes: https://git.openjdk.org/shenandoah/pull/407/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=17
Stats: 166 lines in 16 files changed: 133 ins; 1 del; 32 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From kdnilsen at openjdk.org Thu Jun 20 04:40:47 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 04:40:47 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v16]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <0SszF4w8LNQKmmYPv8GcIaeDtq2y9DznjxUKpSyhGhk=.ea5af431-4cb5-4457-ad8c-b5743f47d0a3@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Respond to reviewer feedback
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/ad42243a..ca743b76
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=15
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=14-15
Stats: 157 lines in 7 files changed: 78 ins; 43 del; 36 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From shade at openjdk.org Thu Jun 20 09:16:22 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 20 Jun 2024 09:16:22 GMT
Subject: RFR: 8334594: Generational ZGC: Deadlock after OopMap rewrites in
8331572
Message-ID: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
As shown in the bug, there are cases when acquiring the `ServiceLock` for opportunistic notification leads to deadlock. We can untie the deadlock by checking if `ServiceLock` can be acquired on triggering path, and never blocking otherwise.
Additional testing:
- [ ] Linux x86_64 service fastdebug, `all`
- [ ] Linux AArch64 service fastdebug, `all`
-------------
Commit messages:
- Fix
Changes: https://git.openjdk.org/jdk/pull/19800/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19800&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8334594
Stats: 12 lines in 6 files changed: 3 ins; 0 del; 9 mod
Patch: https://git.openjdk.org/jdk/pull/19800.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19800/head:pull/19800
PR: https://git.openjdk.org/jdk/pull/19800
From stefank at openjdk.org Thu Jun 20 09:46:18 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 20 Jun 2024 09:46:18 GMT
Subject: RFR: 8334594: Generational ZGC: Deadlock after OopMap rewrites in
8331572
In-Reply-To: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
References: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
Message-ID: <2FpJxRnMwrCrs1VPpDIrvA2KwtC0aJSacgceNUXW_K4=.27f6882b-23ee-4cb9-a4dc-8ce3b568a9d4@github.com>
On Thu, 20 Jun 2024 09:11:04 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> As shown in the bug, there are cases when acquiring the `ServiceLock` for opportunistic notification leads to deadlock. We can untie the deadlock by checking if `ServiceLock` can be acquired on triggering path, and never blocking otherwise.
>
> Additional testing:
> - [ ] Linux x86_64 service fastdebug, `all`
> - [ ] Linux AArch64 service fastdebug, `all`
Looks good to me. Thanks for fixing this.
For those not reading the JBS entry: We might want to do a follow-up to limit the usage of the Service_lock so that we don't hold it while processing the JVMTI oops.
-------------
Marked as reviewed by stefank (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19800#pullrequestreview-2129957643
From eosterlund at openjdk.org Thu Jun 20 10:00:11 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Thu, 20 Jun 2024 10:00:11 GMT
Subject: RFR: 8334594: Generational ZGC: Deadlock after OopMap rewrites in
8331572
In-Reply-To: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
References: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
Message-ID: <W1Y19wN-7een1XgYWH1gqQUefwyNHxDCPlHUPISvRfw=.c8680686-4574-4d1c-a2b4-206d834d2bc3@github.com>
On Thu, 20 Jun 2024 09:11:04 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> As shown in the bug, there are cases when acquiring the `ServiceLock` for opportunistic notification leads to deadlock. We can untie the deadlock by checking if `ServiceLock` can be acquired on triggering path, and never blocking otherwise.
>
> Additional testing:
> - [ ] Linux x86_64 service fastdebug, `all`
> - [ ] Linux AArch64 service fastdebug, `all`
This looks good as a direct fix to the bug. I agree though with the assessment that we should use a different lock for the queue going forward.
It's also interesting to read the placement of these hooks. For the GC VM operation there is a comment saying that we probably just used the oop map cache so now is a good time to trigger cleanup.
The same comment is present for ZGC and Shenandoah where we perform safepoints. But there the situation is typically the exact opposite to what the comment suggests then. Since we perform concurrent root scanning, we are *just about to* use the oop map cache, and the placement is probably the most unfortunate instead. It seems like we clean the caches *just before* using them instead just after. But again, that seems like a follow-up thing.
-------------
Marked as reviewed by eosterlund (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19800#pullrequestreview-2129994072
From shade at openjdk.org Thu Jun 20 10:22:11 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 20 Jun 2024 10:22:11 GMT
Subject: RFR: 8334594: Generational ZGC: Deadlock after OopMap rewrites in
8331572
In-Reply-To: <W1Y19wN-7een1XgYWH1gqQUefwyNHxDCPlHUPISvRfw=.c8680686-4574-4d1c-a2b4-206d834d2bc3@github.com>
References: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
<W1Y19wN-7een1XgYWH1gqQUefwyNHxDCPlHUPISvRfw=.c8680686-4574-4d1c-a2b4-206d834d2bc3@github.com>
Message-ID: <Ab0Z0qsmrR_o9g7iCEoxP3dy9l-nB7VUDJiYCk_c79I=.ab342cc2-82ed-480f-bf87-83bc9db4e5a3@github.com>
On Thu, 20 Jun 2024 09:57:18 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
> The same comment is present for ZGC and Shenandoah where we perform safepoints. But there the situation is typically the exact opposite to what the comment suggests then. Since we perform concurrent root scanning, we are just about to use the oop map cache, and the placement is probably the most unfortunate instead. It seems like we clean the caches just before using them instead just after. But again, that seems like a follow-up thing.
Yeah, we did it out of symmetry with other (STW) GC ops. We can actually move these around to the exact places in Shenandoah and ZGC right after concurrent root handling happens. Relaxing the requirement for actually locking ServiceLock with this particular fix would help safety when we move the trigger around.
(Re phasing: at least for Shenandoah, the start-update-refs op would start soon after root processing and evac is over, so we get the _after_ effect as well. There might be additional contention on reclamation queue and GlobalCounter that we can avoid, though).
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19800#issuecomment-2180325904
From coleenp at openjdk.org Thu Jun 20 12:41:12 2024
From: coleenp at openjdk.org (Coleen Phillimore)
Date: Thu, 20 Jun 2024 12:41:12 GMT
Subject: RFR: 8334594: Generational ZGC: Deadlock after OopMap rewrites in
8331572
In-Reply-To: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
References: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
Message-ID: <hoZWKtVmTwVA_faz5bf79VE0_MprqFnDAiiIPBvG6Gc=.b16fdb57-99ef-4956-ad59-c7dd00ad7f7d@github.com>
On Thu, 20 Jun 2024 09:11:04 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> As shown in the bug, there are cases when acquiring the `ServiceLock` for opportunistic notification leads to deadlock. We can untie the deadlock by checking if `ServiceLock` can be acquired on triggering path, and never blocking otherwise.
>
> Additional testing:
> - [ ] Linux x86_64 service fastdebug, `all`
> - [ ] Linux AArch64 service fastdebug, `all`
Seems like a good workaround.
-------------
Marked as reviewed by coleenp (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19800#pullrequestreview-2130311043
From zgu at openjdk.org Thu Jun 20 13:17:11 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Thu, 20 Jun 2024 13:17:11 GMT
Subject: RFR: 8334594: Generational ZGC: Deadlock after OopMap rewrites in
8331572
In-Reply-To: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
References: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
Message-ID: <Fl7sq6eDrFgD-qVnuAJ-aHuDNfVtvqeTalQ1PR-WxFg=.17e8d44e-8832-4c39-ba45-986801e7d938@github.com>
On Thu, 20 Jun 2024 09:11:04 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> As shown in the bug, there are cases when acquiring the `ServiceLock` for opportunistic notification leads to deadlock. We can untie the deadlock by checking if `ServiceLock` can be acquired on triggering path, and never blocking otherwise.
>
> Additional testing:
> - [ ] Linux x86_64 service fastdebug, `all`
> - [ ] Linux AArch64 service fastdebug, `all`
LGTM
> This looks good as a direct fix to the bug. I agree though with the assessment that we should use a different lock for the queue going forward.
>
> It's also interesting to read the placement of these hooks. For the GC VM operation there is a comment saying that we probably just used the oop map cache so now is a good time to trigger cleanup.
>
> The same comment is present for ZGC and Shenandoah where we perform safepoints. But there the situation is typically the exact opposite to what the comment suggests then. Since we perform concurrent root scanning, we are _just about to_ use the oop map cache, and the placement is probably the most unfortunate instead. It seems like we clean the caches _just before_ using them instead just after. But again, that seems like a follow-up thing.
The cleanup is on already stalled/evicted oop map entries, which are no longer accessible, so I don't the placement issue.
-------------
Marked as reviewed by zgu (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19800#pullrequestreview-2130401926
PR Comment: https://git.openjdk.org/jdk/pull/19800#issuecomment-2180657257
From kdnilsen at openjdk.org Thu Jun 20 13:26:48 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 13:26:48 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v17]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <ye4tUGFfTPvE0R2uCpqjGfRT0oQVNbAY_ZPTWSkfB50=.572aaf57-bb56-49ee-9e6e-509299a8faa1@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Fix whitespace
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/ca743b76..b3427186
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=16
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=15-16
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From shade at openjdk.org Thu Jun 20 14:10:10 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 20 Jun 2024 14:10:10 GMT
Subject: RFR: 8334594: Generational ZGC: Deadlock after OopMap rewrites in
8331572
In-Reply-To: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
References: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
Message-ID: <V186OW60CnDsen9UBRgglA7gJNSQCVY_1hoNU6fW4zs=.ea6a19cd-116d-4865-a585-9dc912fcaf88@github.com>
On Thu, 20 Jun 2024 09:11:04 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> As shown in the bug, there are cases when acquiring the `ServiceLock` for opportunistic notification leads to deadlock. We can untie the deadlock by checking if `ServiceLock` can be acquired on triggering path, and never blocking otherwise.
>
> Additional testing:
> - [x] Linux x86_64 service fastdebug, `all`
> - [x] Linux AArch64 service fastdebug, `all`
All tests pass. I'll wait until 24 hours expire and then integrate.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19800#issuecomment-2180802390
From kdnilsen at openjdk.org Thu Jun 20 16:30:56 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 16:30:56 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v18]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <nIeMK1gWULvbZRUVMfzq5DwgiGv-X5oyLUo2lJkhlF4=.91b32e17-ec9f-4b18-871f-4e0dc143b74d@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Add missing lock request
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/b3427186..d2486552
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=17
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=16-17
Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Thu Jun 20 16:40:31 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 16:40:31 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <1vKm5lgmDiVn1kywz1UM9hnlN7HzdNfF6DtME5o_kow=.8d13ce98-0bd8-4f1f-9728-1e56c5d5986a@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<1vKm5lgmDiVn1kywz1UM9hnlN7HzdNfF6DtME5o_kow=.8d13ce98-0bd8-4f1f-9728-1e56c5d5986a@github.com>
Message-ID: <k1YXfP82DMzYJb4HeAGkh2TpZc1DItFOAQhX1cQPAos=.b8855594-38ba-4790-829a-d1a6313dab57@github.com>
On Wed, 19 Jun 2024 01:30:13 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1419:
>
>> 1417: }
>> 1418:
>> 1419: void ShenandoahFreeSet::move_regions_from_collector_to_mutator(size_t max_xfer_regions) {
>
> This method has 3 isomorphic loops. I wonder if it might read better to extract a separate work method for the loop and invoke it thrice from this method with appropriate parms. Also I'd advise using a local variable to countdown the regions to be transferred rather than modifying the method's input parm.
I've factored out the loop bodies into two function calls. Good suggestions. Thanks.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1647855443
From kdnilsen at openjdk.org Thu Jun 20 17:41:39 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 17:41:39 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <GyisTO4qqSYHpt5VYD8yPGbdUfRsSvurqXnhk4CO8Ws=.17e7ae3d-8fe4-4b43-91a4-0c5bccd54e84@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<GyisTO4qqSYHpt5VYD8yPGbdUfRsSvurqXnhk4CO8Ws=.17e7ae3d-8fe4-4b43-91a4-0c5bccd54e84@github.com>
Message-ID: <UiUzS3GCxkEOPaXFL769WVxanlpQZ15aQV2Ngpfoi_M=.1df849ba-f12f-409a-b41b-57aeabd6ca06@github.com>
On Wed, 19 Jun 2024 01:34:38 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 408:
>
>> 406: // capacity as NotFree. Subsequently, we will move some of the mutator regions into the collector and old collector
>> 407: // partitions with the intent of packing old collector memory into the highest (far rightmost) addresses of the heap,
>> 408: // young collector memory into higher address, and mutator memory consuming the lowest addresses of the heap.
>
> Is the part of the comment that starts at "Subsequently, we will ..." done as part of this method? If not, then it should go into the caller of this method (after this method is called), rather than stating what the caller will do. I'd keep documentation of these methods very simple: what its parms are and what it does.
Thanks. I've adjusted the comments as guided.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1647927053
From kdnilsen at openjdk.org Thu Jun 20 17:53:31 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 17:53:31 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <Dfq-3agINZmuDBzMHZNu4_fO6WEP0fXxzSgU2OGJqmU=.d38c7120-a610-4110-9f21-e9d5405c9936@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<Dfq-3agINZmuDBzMHZNu4_fO6WEP0fXxzSgU2OGJqmU=.d38c7120-a610-4110-9f21-e9d5405c9936@github.com>
Message-ID: <EEM2jIok2pvdWINo4GBDH-XnHnZQzdWQ4gc-qURTo94=.9426139b-7b75-4438-86f4-d87671662999@github.com>
On Wed, 19 Jun 2024 01:43:36 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 418:
>
>> 416: // old collector partition to assure that old collector partition has at least old_reserve. Update old_region_count
>> 417: // to represent the total number of regions in the old generation by adding the number of regions moved from the
>> 418: // mutator partition to the old collector partition.
>
> How about:
>
>
> Ensure that Collector has at least `to_reserve` regions, and OldCollector has at
> least `old_reserve` regions.
> Upon return `old_region_count` holds the number of regions in OldCollector.
>
>
> This makes me wonder about the asymmetry of the return value. Also could it be a plain return value from the method rather than var parameter to the method?
I've adjusted this comment as suggested. old_region_count is both input and output, so I'm leaving as is. This also more closely mirrors the openjdk/jdk implementation.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1647940732
From kdnilsen at openjdk.org Thu Jun 20 18:07:38 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 18:07:38 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <Dfq-3agINZmuDBzMHZNu4_fO6WEP0fXxzSgU2OGJqmU=.d38c7120-a610-4110-9f21-e9d5405c9936@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<Dfq-3agINZmuDBzMHZNu4_fO6WEP0fXxzSgU2OGJqmU=.d38c7120-a610-4110-9f21-e9d5405c9936@github.com>
Message-ID: <KDtDW4NrkzDttSJKM36UkzccpFfnJzwGQLhMM8DUXyU=.d52d7d34-20aa-4b69-a2b2-fdb72e97207b@github.com>
On Wed, 19 Jun 2024 01:47:49 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 423:
>
>> 421:
>> 422:
>> 423: void reserve_regions(size_t young_reserve, size_t old_reserve);
>
> Is the old variant without the var parameter still used somewhere?
Good catch. That is no longer relevant. Removing.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1647956646
From kdnilsen at openjdk.org Thu Jun 20 18:12:32 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 18:12:32 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <KvSJGDQocvUwldX2wRC0guST3fK50lYDZ1oUxkDyIew=.51a89494-d439-4180-8a75-7e735f8dfb86@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<KvSJGDQocvUwldX2wRC0guST3fK50lYDZ1oUxkDyIew=.51a89494-d439-4180-8a75-7e735f8dfb86@github.com>
Message-ID: <0ty1AxoaHT_--szM-COP324-2pQRUPJggRHqE6_SkDU=.589f2a02-6727-4926-9de2-59e2fd47187b@github.com>
On Wed, 19 Jun 2024 01:52:54 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 264:
>
>> 262: // collection set (cset).
>> 263: //
>> 264: // The ShenandoahFreeSet endeavors to congregrate survivor objects (objects that have been evacuated at least once) at the
>
> May be "tries to colocate" is simpler than "endeavors to congregate"?
Revised this comment. Thanks.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1647961264
From kdnilsen at openjdk.org Thu Jun 20 18:17:36 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 20 Jun 2024 18:17:36 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <RP8bvZ4fMB1d6B8HObsciIk1cshdj5cOEitpvvzxQLQ=.39feb61b-6205-4bea-9393-e358c006bb7e@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<RP8bvZ4fMB1d6B8HObsciIk1cshdj5cOEitpvvzxQLQ=.39feb61b-6205-4bea-9393-e358c006bb7e@github.com>
Message-ID: <RLFRqYFo4i9wf_yVaSK9juY296JGi-zl7sMnGdfCJT4=.a39bf5ea-2e08-4313-b212-d2d08a4ccb9d@github.com>
On Wed, 19 Jun 2024 02:00:27 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Minor refinements to test programs
>>
>> TestAllocIntArrays: comments to explain behavior.
>> TestOldGrowthTriggers: reduce the number of loop iterations so this test
>> will not time out on less powerful test platforms.
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 290:
>
>> 288:
>> 289: // Satisfy collector allocation request req by finding memory that matches affiliation from within the
>> 290: // free partition associated with which_partition.
>
> Returns the address allocated and `in_new_region` indicates if the allocation was
> satisfied from a new, previously empty region.
Thanks. Have updated this comment.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1647965581
From kdnilsen at openjdk.org Fri Jun 21 04:15:44 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 21 Jun 2024 04:15:44 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v19]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <apYSnkSeZUjz5AdBLglz8F8KeNM2SUbt7lMPdnBmbhc=.61e2a616-53ae-4dca-8394-c130e3835b97@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with five additional commits since the last revision:
- Shrink iterations again so TestOldGrowth won't timeout on macos-aarch64
- Improve comments
- Remove extrameous reserve_regions() declaration
- Update comments as guided by reviewer requests
- Adjust locking to reduce frequency of lock acquisition
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/d2486552..7a0c4817
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=18
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=17-18
Stats: 42 lines in 3 files changed: 9 ins; 15 del; 18 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From wkemper at openjdk.org Fri Jun 21 16:12:01 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 21 Jun 2024 16:12:01 GMT
Subject: RFR: 8334681: GenShen: Do not use gtest skip test feature
Message-ID: <hL7hsNmngJRojXN93nuhU4l8L9oEmy-88E6r7nf1jLA=.3c9b251b-21dc-4567-9ad6-bf878e853531@github.com>
Recent versions of gtest provide a means to skip tests [1]. Unfortunately, the test result parsing in the build counts them as errors and fails the build.
[1] https://google.github.io/googletest/advanced.html#skipping-test-execution
-------------
Commit messages:
- Test result parser fails skipped tests
Changes: https://git.openjdk.org/shenandoah/pull/452/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=452&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8334681
Stats: 23 lines in 1 file changed: 18 ins; 3 del; 2 mod
Patch: https://git.openjdk.org/shenandoah/pull/452.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/452/head:pull/452
PR: https://git.openjdk.org/shenandoah/pull/452
From kdnilsen at openjdk.org Fri Jun 21 16:39:28 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 21 Jun 2024 16:39:28 GMT
Subject: RFR: 8334681: GenShen: Do not use gtest skip test feature
In-Reply-To: <hL7hsNmngJRojXN93nuhU4l8L9oEmy-88E6r7nf1jLA=.3c9b251b-21dc-4567-9ad6-bf878e853531@github.com>
References: <hL7hsNmngJRojXN93nuhU4l8L9oEmy-88E6r7nf1jLA=.3c9b251b-21dc-4567-9ad6-bf878e853531@github.com>
Message-ID: <7Wsytw8GCD_jl9ihAyen4RaufFRjru3Ih8EWizx2q00=.49b85597-a0eb-4364-887b-e2df289cb099@github.com>
On Fri, 21 Jun 2024 16:07:28 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Recent versions of gtest provide a means to skip tests [1]. Unfortunately, the test result parsing in the build counts them as errors and fails the build.
>
> [1] https://google.github.io/googletest/advanced.html#skipping-test-execution
Thanks.
-------------
Marked as reviewed by kdnilsen (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/452#pullrequestreview-2133034277
From wkemper at openjdk.org Fri Jun 21 16:50:30 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 21 Jun 2024 16:50:30 GMT
Subject: Integrated: 8334681: GenShen: Do not use gtest skip test feature
In-Reply-To: <hL7hsNmngJRojXN93nuhU4l8L9oEmy-88E6r7nf1jLA=.3c9b251b-21dc-4567-9ad6-bf878e853531@github.com>
References: <hL7hsNmngJRojXN93nuhU4l8L9oEmy-88E6r7nf1jLA=.3c9b251b-21dc-4567-9ad6-bf878e853531@github.com>
Message-ID: <gbZFHWIqYBQmjV8YpZgsqe_BfPAgyz6zA11igHLo_9c=.b3b61288-6287-401e-803a-753ab99652d6@github.com>
On Fri, 21 Jun 2024 16:07:28 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Recent versions of gtest provide a means to skip tests [1]. Unfortunately, the test result parsing in the build counts them as errors and fails the build.
>
> [1] https://google.github.io/googletest/advanced.html#skipping-test-execution
This pull request has now been integrated.
Changeset: a7c5d8a9
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/a7c5d8a96e62b56218384ebc6a531f3bdc12a56f
Stats: 23 lines in 1 file changed: 18 ins; 3 del; 2 mod
8334681: GenShen: Do not use gtest skip test feature
Reviewed-by: kdnilsen
-------------
PR: https://git.openjdk.org/shenandoah/pull/452
From wkemper at openjdk.org Fri Jun 21 17:08:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 21 Jun 2024 17:08:41 GMT
Subject: Integrated: 8334681: GenShen: Do not use gtest skip test feature
Message-ID: <LcL1fFrGmayr30OCF3NUMmMl7TU5zaFdtXIW3ostLWo=.793a08b2-6891-4b5b-bf15-7bc3a9389d6f@github.com>
Clean backport
-------------
Commit messages:
- Backport a7c5d8a96e62b56218384ebc6a531f3bdc12a56f
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/60/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=60&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8334681
Stats: 23 lines in 1 file changed: 18 ins; 3 del; 2 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/60.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/60/head:pull/60
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/60
From wkemper at openjdk.org Fri Jun 21 17:08:41 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 21 Jun 2024 17:08:41 GMT
Subject: Integrated: 8334681: GenShen: Do not use gtest skip test feature
In-Reply-To: <LcL1fFrGmayr30OCF3NUMmMl7TU5zaFdtXIW3ostLWo=.793a08b2-6891-4b5b-bf15-7bc3a9389d6f@github.com>
References: <LcL1fFrGmayr30OCF3NUMmMl7TU5zaFdtXIW3ostLWo=.793a08b2-6891-4b5b-bf15-7bc3a9389d6f@github.com>
Message-ID: <Th9-rcw5hQpzqKKH3mslrDGR2lVRA1qneJjlOYL8eZE=.4dd191ab-04a0-44bf-bab4-39f4a9c5e9d0@github.com>
On Fri, 21 Jun 2024 16:59:26 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Clean backport
This pull request has now been integrated.
Changeset: 0cd58a29
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah-jdk21u/commit/0cd58a2938182261b069c9a0a109024cc27ad961
Stats: 23 lines in 1 file changed: 18 ins; 3 del; 2 mod
8334681: GenShen: Do not use gtest skip test feature
Backport-of: a7c5d8a96e62b56218384ebc6a531f3bdc12a56f
-------------
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/60
From kdnilsen at openjdk.org Sun Jun 23 14:16:55 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sun, 23 Jun 2024 14:16:55 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v20]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <Aa7shSOCE0YAvKl9eVtPSt9AAAvxEIFtFDMzRJBkH2M=.2b8de52e-c117-4b9f-9812-c509e95159f5@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Add instrumentation to debug rare regression
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/7a0c4817..ab4e2015
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=19
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=18-19
Stats: 200 lines in 4 files changed: 200 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Sun Jun 23 22:24:42 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sun, 23 Jun 2024 22:24:42 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v21]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <53FrfJfcRQVWY9g8m_nx2N7CvuOK7eaD9m0V2dQBNnk=.6b62d5d2-ccc1-4caa-a818-f1b7b72cffa1@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
More debugging instrumentation
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/ab4e2015..f21ef0aa
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=20
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=19-20
Stats: 69 lines in 2 files changed: 67 ins; 0 del; 2 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Mon Jun 24 02:12:03 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 24 Jun 2024 02:12:03 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v22]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <zX95cR82GldJLq9jK5_NZgK50XatYxN1Ik495yWt8cw=.735d83eb-ec08-4a88-97a6-ef65c772d1de@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Fix handling of old and young cset regions during freeset rebuild
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/f21ef0aa..d5c2646e
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=21
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=20-21
Stats: 53 lines in 1 file changed: 27 ins; 19 del; 7 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Mon Jun 24 03:51:57 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 24 Jun 2024 03:51:57 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v23]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <bK9AG3WdBPLxapiKhS9MRYCOYAfwUK9v0xQlAr_8vHk=.5f4fcc32-513c-49e9-8519-c8b84fa2e3de@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Fix improper nesteing of #ifdefs
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/d5c2646e..4be5f361
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=22
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=21-22
Stats: 2 lines in 1 file changed: 1 ins; 1 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From shade at openjdk.org Mon Jun 24 08:49:26 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 24 Jun 2024 08:49:26 GMT
Subject: Integrated: 8334594: Generational ZGC: Deadlock after OopMap rewrites
in 8331572
In-Reply-To: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
References: <DqoE230_YKbXqJcM5NGHbjb-6_i6bTbVZ4TrOqCmxr4=.bc7ec720-f392-4adf-b6b4-e96867546220@github.com>
Message-ID: <TxIs6808PD-EuqG5ZxP6JHVByOBfo3WhIQWK0qgmap8=.2a8df327-e20d-4cd3-873e-1efd4a8e25fa@github.com>
On Thu, 20 Jun 2024 09:11:04 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> As shown in the bug, there are cases when acquiring the `ServiceLock` for opportunistic notification leads to deadlock. We can untie the deadlock by checking if `ServiceLock` can be acquired on triggering path, and never blocking otherwise.
>
> Additional testing:
> - [x] Linux x86_64 service fastdebug, `all`
> - [x] Linux AArch64 service fastdebug, `all`
This pull request has now been integrated.
Changeset: 05ff3185
Author: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/05ff3185edd25b381a97f6879f496e97b62dddc2
Stats: 12 lines in 6 files changed: 3 ins; 0 del; 9 mod
8334594: Generational ZGC: Deadlock after OopMap rewrites in 8331572
Reviewed-by: stefank, eosterlund, coleenp, zgu
-------------
PR: https://git.openjdk.org/jdk/pull/19800
From shade at openjdk.org Mon Jun 24 09:07:25 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 24 Jun 2024 09:07:25 GMT
Subject: [jdk23] RFR: 8334594: Generational ZGC: Deadlock after OopMap rewrites
in 8331572
Message-ID: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
Clean backport to fix a deadlock.
-------------
Commit messages:
- Backport 05ff3185edd25b381a97f6879f496e97b62dddc2
Changes: https://git.openjdk.org/jdk/pull/19851/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19851&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8334594
Stats: 12 lines in 6 files changed: 3 ins; 0 del; 9 mod
Patch: https://git.openjdk.org/jdk/pull/19851.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19851/head:pull/19851
PR: https://git.openjdk.org/jdk/pull/19851
From kdnilsen at openjdk.org Mon Jun 24 14:24:59 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 24 Jun 2024 14:24:59 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v24]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <I3cvNyfURgT3L1qGhjawa_1v1hIZdGsSrKyY2WaELYQ=.6ae89a6d-7acc-403c-86ab-6df952de7251@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Relax certain assertions after changing trashed-region handling
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/4be5f361..8a71e29d
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=23
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=22-23
Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From duke at openjdk.org Mon Jun 24 16:23:15 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Mon, 24 Jun 2024 16:23:15 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v3]
In-Reply-To: <KSVOZh0Op3yfFGg0C0Vg_eMtkEulKnAkDpUjDCY6If4=.6dbf481a-2f27-47dd-8dab-ba348bc447e2@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<ctEDifLb23ZsU_23p-SnLB_pCKlje-oG1EyjXlfj9PE=.15b3a1ba-3250-478a-bf5d-839469f12116@github.com>
<uV5j6XMZbbELdQJST5lcP2EWDCGnQhTTcatptCYC_CU=.a55282d4-1691-430d-9af1-85b97d9f01fd@github.com>
<-GBU6zDyLGsnEyT3yEMfW_TnA-stpR1tLeqfvfeVcQ0=.c4b13824-35a4-4e22-8132-721a668a0425@github.com>
<SM6Tj6DXxNSXZbkM-daZA4gB8baJiPtwcua4JKOcaUg=.2bd3dffb-143c-4dc5-b0ef-166536c0d848@github.com>
<Q0Pyyx9YaLS55a7KmCZX6tEA5arxJ0qxVKXW1VAPWKE=.d09724f3-796b-4d20-bfc4-f39a1deb35ea@github.com>
<YsEp4fgCBxneEuJnfidh98QfR0LAUV_H6GVN6PzPqys=.bf6c999d-4cab-4176-be79-10f23d385a7d@github.com>
<KSVOZh0Op3yfFGg0C0Vg_eMtkEulKnAkDpUjDCY6If4=.6dbf481a-2f27-47dd-8dab-ba348bc447e2@github.com>
Message-ID: <xWH2-P1xiVsfFQy7gX0QzGDpuANLlJVeY6-hOCpyPPQ=.8a630987-e60a-4dc3-b017-5b45aa9a68b9@github.com>
On Fri, 7 Jun 2024 20:14:01 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
>>> Having said that, perhaps we should file a separate issue to remove that check, since it seems to fix an actual bug, while I guess this was meant more as an optimization. What do you think?
>>
>> FTR, I don't mind executing cross-modify-fence unconditionally. I do mind going into deopts too often. I do also think that we want to stay on performance-positive side for at least an easy variant of fix, and do potentially regressing things separately. The initial motivation for this work was to resolve an issue in a service workload that runs many threads with similar stacks, and get something that we are sure about for a prompt backport.
>>
>> To that end, we can continue working out the final shape of the patch here, while we mitigate our current service problems with picking up a limited version of this patch with [JDK-8333716](https://bugs.openjdk.org/browse/JDK-8333716) -- it resolves only Shenandoah parts of it, though. Or, we can integrate this patch in its current form, resolving the issue on both Shenandoah and ZGC paths, and work out the check removal as the follow up of [JDK-8310239](https://bugs.openjdk.org/browse/JDK-8310239).
>>
>> I think the latter alternative is more pragmatic.
>
>> > Having said that, perhaps we should file a separate issue to remove that check, since it seems to fix an actual bug, while I guess this was meant more as an optimization. What do you think?
>>
>> FTR, I don't mind executing cross-modify-fence unconditionally. I do mind going into deopts too often. I do also think that we want to stay on performance-positive side for at least an easy variant of fix, and do potentially regressing things separately. The initial motivation for this work was to resolve an issue in a service workload that runs many threads with similar stacks, and get something that we are sure about for a prompt backport.
>
> Fair enough. For what it's worth, aside for the deopt stressing option with arbitrary frequency we can update, we will not deopt more. We just perform an extra cross modifying fence when racingly entering an nmethod concurrently being disarmed. Not performing it might be slightly faster, but is a bug. But I see your point.
>
>> To that end, we can continue working out the final shape of the patch here, while we mitigate our current service problems with picking up a limited version of this patch with [JDK-8333716](https://bugs.openjdk.org/browse/JDK-8333716) -- it resolves only Shenandoah parts of it, though. Or, we can integrate this patch in its current form, resolving the issue on both Shenandoah and ZGC paths, and work out the check removal as the follow up of [JDK-8310239](https://bugs.openjdk.org/browse/JDK-8310239).
>>
>> I think the latter alternative is more pragmatic.
>
> I'm okay with approving this patch, and we fix the actual bug separately. Sounds good? Then this is a refactoring and optimization, without the bug fix.
@fisk I just merged the latest changes. Do I need approval on the merge commit or can I integrate?
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2186951944
From zgu at openjdk.org Mon Jun 24 17:07:36 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 24 Jun 2024 17:07:36 GMT
Subject: RFR: 8334769: Shenandoah: Move CodeCache_lock close to its use in
ShenandoahConcurrentNMethodIterator
Message-ID: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
ShenandoahConcurrentNMethodIterator requires CodeCache_lock at begin and end of iteration.
Currently, the CodeCache_lock is acquired by their callers, which is not very obvious and readable. Let's move the lock inside the methods.
-------------
Commit messages:
- 8334769: Shenandoah: Move CodeCache_lock close to its use in ShenandoahConcurrentNMethodIterator
Changes: https://git.openjdk.org/jdk/pull/19859/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19859&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8334769
Stats: 13 lines in 3 files changed: 3 ins; 8 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19859.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19859/head:pull/19859
PR: https://git.openjdk.org/jdk/pull/19859
From shade at openjdk.org Mon Jun 24 17:37:12 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 24 Jun 2024 17:37:12 GMT
Subject: RFR: 8334769: Shenandoah: Move CodeCache_lock close to its use in
ShenandoahConcurrentNMethodIterator
In-Reply-To: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
References: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
Message-ID: <oFH6dGmj6UFbNkQiJBfY8sXQROnTWSRc--G-7cXUXs8=.c8a58514-9b8f-491a-9347-519473b59a70@github.com>
On Mon, 24 Jun 2024 13:46:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> ShenandoahConcurrentNMethodIterator requires CodeCache_lock at begin and end of iteration.
>
> Currently, the CodeCache_lock is acquired by their callers, which is not very obvious and readable. Let's move the lock inside the methods.
Agree, it looks cleaner that way, and has a symmetry with how `ShenandoahCodeRootsIterator` does it.
@kdnilsen @earthling-amzn might need to take a look.
-------------
Marked as reviewed by shade (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19859#pullrequestreview-2136396732
PR Comment: https://git.openjdk.org/jdk/pull/19859#issuecomment-2187077643
From eosterlund at openjdk.org Mon Jun 24 17:41:14 2024
From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Mon, 24 Jun 2024 17:41:14 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v5]
In-Reply-To: <MFHmR3Lcd4AcIe1HIW41xRDsC17zhantIIdw-P921pk=.de168ec2-1f4f-4d54-8d60-ed8502f80e9d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<MFHmR3Lcd4AcIe1HIW41xRDsC17zhantIIdw-P921pk=.de168ec2-1f4f-4d54-8d60-ed8502f80e9d@github.com>
Message-ID: <aNgJBgyvAbPTXhyhU9CEOwHBeqvS_CxZ2PINreOcIDg=.60fda4b9-b525-457a-be61-69c74fbc7e35@github.com>
On Mon, 17 Jun 2024 16:31:52 GMT, Neethu Prasad <duke at openjdk.org> wrote:
>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>>
>> **Verification**
>>
>> Shenendoah
>>
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>>
>> ZGC
>>
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>>
>> **Issue**
>> https://bugs.openjdk.org/browse/JDK-8331911
>
> Neethu Prasad has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits:
>
> - Merge branch 'openjdk:master' into JDK-8331911
> - 8331911: Remove is_armed check from callers
> - Update full name
> - Address feedback on whitespace and comments
> - 8331911: Fix whitespace
> - 8331911: Reconsider locking for recently disarmed nmethods
Looks good. Will you file a follow-up regarding the dangerous early filtering that we discussed?
-------------
Marked as reviewed by eosterlund (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19285#pullrequestreview-2136405507
From kdnilsen at openjdk.org Mon Jun 24 17:41:59 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 24 Jun 2024 17:41:59 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v25]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <9kNBWxPaM0edZl_14edSbSEO_8SBW9JmTX4dL94P87o=.763e7678-dc4d-4c26-98a0-7d73cc865741@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Arrange for verifier to omit trashed regions from span
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/8a71e29d..5bf12f69
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=24
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=23-24
Stats: 10 lines in 2 files changed: 5 ins; 0 del; 5 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From duke at openjdk.org Mon Jun 24 19:09:14 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Mon, 24 Jun 2024 19:09:14 GMT
Subject: RFR: 8331911: Reconsider locking for recently disarmed nmethods
[v5]
In-Reply-To: <MFHmR3Lcd4AcIe1HIW41xRDsC17zhantIIdw-P921pk=.de168ec2-1f4f-4d54-8d60-ed8502f80e9d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
<MFHmR3Lcd4AcIe1HIW41xRDsC17zhantIIdw-P921pk=.de168ec2-1f4f-4d54-8d60-ed8502f80e9d@github.com>
Message-ID: <vjPzF1DqDhc_0a00_8Uw-mEM09PvWaSTek7Ore_WOCs=.1ed01027-c69d-4f5f-9b7f-f539de940f3d@github.com>
On Mon, 17 Jun 2024 16:31:52 GMT, Neethu Prasad <duke at openjdk.org> wrote:
>> **Notes**
>> We are spending significant time on acquiring the per-nmethod as all the
>> threads are in same nmethod.
>> Adding double-check lock by calling is_armed before lock acquisition.
>>
>> **Verification**
>>
>> Shenendoah
>>
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>>
>> ZGC
>>
>>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>>
>> **Issue**
>> https://bugs.openjdk.org/browse/JDK-8331911
>
> Neethu Prasad has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits:
>
> - Merge branch 'openjdk:master' into JDK-8331911
> - 8331911: Remove is_armed check from callers
> - Update full name
> - Address feedback on whitespace and comments
> - 8331911: Fix whitespace
> - 8331911: Reconsider locking for recently disarmed nmethods
Thanks for the review & approval.
I've created follow up bug -
https://bugs.openjdk.org/browse/JDK-8334890
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19285#issuecomment-2187224670
From xpeng at openjdk.org Tue Jun 25 01:04:39 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Jun 2024 01:04:39 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v3]
In-Reply-To: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
Message-ID: <jPeiRdHjPoLNiAt3biXuQuZ5VlcpOAOf17RLj5VwGDM=.e94f106d-12e7-410b-97bb-a5f1e1cbe7e7@github.com>
> ### Notes
> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>
> #### Test code
>
> public class Alloc {
> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
> static final Object[] sinks = new Object[64*THREADS];
> static volatile boolean start;
> static volatile boolean stop;
>
> public static void main(String... args) throws Throwable {
> for (int t = 0; t < THREADS; t++) {
> int ft = t;
> new Thread(() -> work(ft * 64)).start();
> }
>
> Thread.sleep(1000);
> start = true;
> Thread.sleep(30_000);
> stop = true;
> }
>
> public static void work(int idx) {
> while (!start) { Thread.onSpinWait(); }
> while (!stop) {
> sinks[idx] = new byte[128];
> }
> }
> }
>
>
> Run it like this and observe TTSP times:
>
>
> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>
>
> #### Metrics from tests(TTSP, allocation rate)
> ##### Heavy contention(1280 threads, 32 per CPU core)
> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | -------- | ----- |
> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>
>
> ##### Light contention(40 threads, 1 per CPU core)
> | Spins | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | ------- | ---- |
> | baseline | ...
Xiaolong Peng has updated the pull request incrementally with 15 additional commits since the last revision:
- Coe format and comments
- Fix typo
- Polish and clean up
- Simplify ShenandoahLock::contended_lock_internal
- A bit more comments
- Update comments
- Fix typo in comments
- _threads_at_sp was not removed completely
- Remove the code for using Semaphore to block JavaThread when SP is pending
- Try not to Semaphore to reach SP faster
- ... and 5 more: https://git.openjdk.org/jdk/compare/f388b344...c309886c
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19570/files
- new: https://git.openjdk.org/jdk/pull/19570/files/f388b344..c309886c
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=02
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=01-02
Stats: 61 lines in 2 files changed: 18 ins; 20 del; 23 mod
Patch: https://git.openjdk.org/jdk/pull/19570.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19570/head:pull/19570
PR: https://git.openjdk.org/jdk/pull/19570
From xpeng at openjdk.org Tue Jun 25 02:07:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Jun 2024 02:07:20 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v4]
In-Reply-To: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
Message-ID: <E91jjLBhAfEjL-DI0wTEKG4l9HFWwXb5FThGAJr2nH0=.01ee3cee-5f82-43ec-b743-9112e07134c8@github.com>
> ### Notes
> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>
> #### Test code
>
> public class Alloc {
> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
> static final Object[] sinks = new Object[64*THREADS];
> static volatile boolean start;
> static volatile boolean stop;
>
> public static void main(String... args) throws Throwable {
> for (int t = 0; t < THREADS; t++) {
> int ft = t;
> new Thread(() -> work(ft * 64)).start();
> }
>
> Thread.sleep(1000);
> start = true;
> Thread.sleep(30_000);
> stop = true;
> }
>
> public static void work(int idx) {
> while (!start) { Thread.onSpinWait(); }
> while (!stop) {
> sinks[idx] = new byte[128];
> }
> }
> }
>
>
> Run it like this and observe TTSP times:
>
>
> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>
>
> #### Metrics from tests(TTSP, allocation rate)
> ##### Heavy contention(1280 threads, 32 per CPU core)
> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | -------- | ----- |
> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>
>
> ##### Light contention(40 threads, 1 per CPU core)
> | Spins | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | ------- | ---- |
> | baseline | ...
Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
Fix linux-x64-hs-zero build failure
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19570/files
- new: https://git.openjdk.org/jdk/pull/19570/files/c309886c..b1692ba0
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=03
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=02-03
Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/jdk/pull/19570.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19570/head:pull/19570
PR: https://git.openjdk.org/jdk/pull/19570
From kdnilsen at openjdk.org Tue Jun 25 04:45:10 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 04:45:10 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v26]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <gO67Bp4SGNJqP3VZz2zgwfMb7iZ8aw1_xVwsmxY8IK4=.318298b2-39b6-4119-a2b1-2d121d8675c8@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Remove instrumentation and deprecated code
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/5bf12f69..5400c9c0
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=25
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=24-25
Stats: 273 lines in 5 files changed: 0 ins; 273 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From duke at openjdk.org Tue Jun 25 07:11:15 2024
From: duke at openjdk.org (Neethu Prasad)
Date: Tue, 25 Jun 2024 07:11:15 GMT
Subject: Integrated: 8331911: Reconsider locking for recently disarmed nmethods
In-Reply-To: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
References: <bLiLebeHdscfUXiEKuDvtslPV6wfvGgpCgEkGfUtqHY=.78aea279-e70a-4b5c-8bb6-7ae46b66c60d@github.com>
Message-ID: <Ck0dew3XKGzbmWBOrcBLxq8A38NGGpQ8nqzz1PXsjY0=.e9f6ae03-dd5a-40db-8990-b399951abde9@github.com>
On Fri, 17 May 2024 13:51:57 GMT, Neethu Prasad <duke at openjdk.org> wrote:
> **Notes**
> We are spending significant time on acquiring the per-nmethod as all the
> threads are in same nmethod.
> Adding double-check lock by calling is_armed before lock acquisition.
>
> **Verification**
>
> Shenendoah
>
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
>> [0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
>> [0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
>> [0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
>> [0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
>> [1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
>> [1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
>> [1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms
>
> ZGC
>
>> % /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
>> [0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
>> [0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
>> [0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
>> [0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
>> [0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
>> [1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
>> [1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
>> [1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
>> [1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
>> [1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
>> [1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms
>
> **Issue**
> https://bugs.openjdk.org/browse/JDK-8331911
This pull request has now been integrated.
Changeset: c30e0403
Author: Neethu Prasad <neethp at amazon.com>
Committer: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/c30e040342c69a213bdff321fdcb0d27ff740489
Stats: 30 lines in 4 files changed: 22 ins; 2 del; 6 mod
8331911: Reconsider locking for recently disarmed nmethods
Reviewed-by: shade, eosterlund
-------------
PR: https://git.openjdk.org/jdk/pull/19285
From shade at openjdk.org Tue Jun 25 07:19:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 25 Jun 2024 07:19:14 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v4]
In-Reply-To: <E91jjLBhAfEjL-DI0wTEKG4l9HFWwXb5FThGAJr2nH0=.01ee3cee-5f82-43ec-b743-9112e07134c8@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<E91jjLBhAfEjL-DI0wTEKG4l9HFWwXb5FThGAJr2nH0=.01ee3cee-5f82-43ec-b743-9112e07134c8@github.com>
Message-ID: <eb8VKznkh0UIcnFmcSW5lxOlvmP0nQtzAgqPtgeDRWE=.90b4d43d-4748-427f-83ac-5779ee1687c0@github.com>
On Tue, 25 Jun 2024 02:07:20 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Fix linux-x64-hs-zero build failure
Good work! I have only stylistic nits.
src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 37:
> 35: void ShenandoahLock::contended_lock(bool allow_block_for_safepoint) {
> 36: Thread* thread = Thread::current();
> 37: if (thread->is_Java_thread() && allow_block_for_safepoint) {
Please flip `&&` back. It is easier to short-cut test after a boolean test we know is likely to constant-fold.
src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 49:
> 47: // Spin this much on multi-processor, do not spin on multi-processor.
> 48: int ctr = os::is_MP() ? 0x1F : 0;
> 49: // Apply TTAS to avoid more expenseive CAS calls if the lock is still held by other thread.
`expenseive` -> `expensive`
src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 51:
> 49: // Apply TTAS to avoid more expenseive CAS calls if the lock is still held by other thread.
> 50: while (Atomic::load(&_state) == locked ||
> 51: Atomic::cmpxchg(&_state, unlocked, locked) != unlocked) {
Please indent it as the old code: `Atomic::` should align.
src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 71:
> 69: // VM thread to arm the poll sooner.
> 70: while (SafepointSynchronize::is_synchronizing() &&
> 71: !SafepointMechanism::local_poll_armed(java_thread)) {
Same: indent so that `SafepointSynchronize::` and `!SafepointMechanism` align.
src/hotspot/share/gc/shenandoah/shenandoahLock.hpp line 59:
> 57: if (Atomic::cmpxchg(&_state, unlocked, locked) != unlocked) {
> 58: contended_lock(allow_block_for_safepoint);
> 59: }
I am thinking we can increase the chances for inline if we collapse both branches like:
// Dive into the contended locking handling if there is a safepoint pending and we can
// block the thread, or the fast-path locking have failed.
if ((allow_block_for_safepoint && SafepointSynchronize:is_synchronizing()) ||
(Atomic::cmpxchg(&_state, unlocked, locked) != unlocked)) {
contended_lock(allow_block_for_safepoint);
}
-------------
PR Review: https://git.openjdk.org/jdk/pull/19570#pullrequestreview-2137588815
PR Review Comment: https://git.openjdk.org/jdk/pull/19570#discussion_r1652118100
PR Review Comment: https://git.openjdk.org/jdk/pull/19570#discussion_r1652118951
PR Review Comment: https://git.openjdk.org/jdk/pull/19570#discussion_r1652119621
PR Review Comment: https://git.openjdk.org/jdk/pull/19570#discussion_r1652120574
PR Review Comment: https://git.openjdk.org/jdk/pull/19570#discussion_r1652128318
From xpeng at openjdk.org Tue Jun 25 08:01:42 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Jun 2024 08:01:42 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v5]
In-Reply-To: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
Message-ID: <mBB4A9WXL4FmybTCDbBHj4RXn3WarlkWYfSd2en4DR8=.1cafe82c-76f4-4e03-af32-d822a07f38b4@github.com>
> ### Notes
> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>
> #### Test code
>
> public class Alloc {
> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
> static final Object[] sinks = new Object[64*THREADS];
> static volatile boolean start;
> static volatile boolean stop;
>
> public static void main(String... args) throws Throwable {
> for (int t = 0; t < THREADS; t++) {
> int ft = t;
> new Thread(() -> work(ft * 64)).start();
> }
>
> Thread.sleep(1000);
> start = true;
> Thread.sleep(30_000);
> stop = true;
> }
>
> public static void work(int idx) {
> while (!start) { Thread.onSpinWait(); }
> while (!stop) {
> sinks[idx] = new byte[128];
> }
> }
> }
>
>
> Run it like this and observe TTSP times:
>
>
> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>
>
> #### Metrics from tests(TTSP, allocation rate)
> ##### Heavy contention(1280 threads, 32 per CPU core)
> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | -------- | ----- |
> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>
>
> ##### Light contention(40 threads, 1 per CPU core)
> | Spins | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | ------- | ---- |
> | baseline | ...
Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
Adress review comments
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19570/files
- new: https://git.openjdk.org/jdk/pull/19570/files/b1692ba0..aaeffc25
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=04
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=03-04
Stats: 14 lines in 2 files changed: 2 ins; 5 del; 7 mod
Patch: https://git.openjdk.org/jdk/pull/19570.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19570/head:pull/19570
PR: https://git.openjdk.org/jdk/pull/19570
From xpeng at openjdk.org Tue Jun 25 08:06:49 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Jun 2024 08:06:49 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v6]
In-Reply-To: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
Message-ID: <CLpm_KsuVp-y2w5dmXSiL3FnV-mAqp8Zs_CLM_QBDRo=.c2f9e7ea-e3f7-4c8d-b5b7-15a27083ad8a@github.com>
> ### Notes
> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>
> #### Test code
>
> public class Alloc {
> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
> static final Object[] sinks = new Object[64*THREADS];
> static volatile boolean start;
> static volatile boolean stop;
>
> public static void main(String... args) throws Throwable {
> for (int t = 0; t < THREADS; t++) {
> int ft = t;
> new Thread(() -> work(ft * 64)).start();
> }
>
> Thread.sleep(1000);
> start = true;
> Thread.sleep(30_000);
> stop = true;
> }
>
> public static void work(int idx) {
> while (!start) { Thread.onSpinWait(); }
> while (!stop) {
> sinks[idx] = new byte[128];
> }
> }
> }
>
>
> Run it like this and observe TTSP times:
>
>
> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>
>
> #### Metrics from tests(TTSP, allocation rate)
> ##### Heavy contention(1280 threads, 32 per CPU core)
> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | -------- | ----- |
> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>
>
> ##### Light contention(40 threads, 1 per CPU core)
> | Spins | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | ------- | ---- |
> | baseline | ...
Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
- Add parentheses for better alignment
- Increase spin pauses to 0xFF to address performance regression in lightly contended scenarios
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19570/files
- new: https://git.openjdk.org/jdk/pull/19570/files/aaeffc25..73e58d74
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=05
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=04-05
Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19570.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19570/head:pull/19570
PR: https://git.openjdk.org/jdk/pull/19570
From xpeng at openjdk.org Tue Jun 25 08:06:49 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Jun 2024 08:06:49 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v4]
In-Reply-To: <eb8VKznkh0UIcnFmcSW5lxOlvmP0nQtzAgqPtgeDRWE=.90b4d43d-4748-427f-83ac-5779ee1687c0@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<E91jjLBhAfEjL-DI0wTEKG4l9HFWwXb5FThGAJr2nH0=.01ee3cee-5f82-43ec-b743-9112e07134c8@github.com>
<eb8VKznkh0UIcnFmcSW5lxOlvmP0nQtzAgqPtgeDRWE=.90b4d43d-4748-427f-83ac-5779ee1687c0@github.com>
Message-ID: <-tzuAl0Glx4ckaoc7v5KooEItZNxDuVUT-0I-GyzdNw=.3c3d85c2-cd9a-4dd7-aba5-2fb682045536@github.com>
On Tue, 25 Jun 2024 07:11:04 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Fix linux-x64-hs-zero build failure
>
> src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 37:
>
>> 35: void ShenandoahLock::contended_lock(bool allow_block_for_safepoint) {
>> 36: Thread* thread = Thread::current();
>> 37: if (thread->is_Java_thread() && allow_block_for_safepoint) {
>
> Please flip `&&` back. It is easier to short-cut test after a boolean test we know is likely to constant-fold.
Nice catch, thanks!
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19570#discussion_r1652200709
From shade at openjdk.org Tue Jun 25 08:11:13 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 25 Jun 2024 08:11:13 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v6]
In-Reply-To: <CLpm_KsuVp-y2w5dmXSiL3FnV-mAqp8Zs_CLM_QBDRo=.c2f9e7ea-e3f7-4c8d-b5b7-15a27083ad8a@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<CLpm_KsuVp-y2w5dmXSiL3FnV-mAqp8Zs_CLM_QBDRo=.c2f9e7ea-e3f7-4c8d-b5b7-15a27083ad8a@github.com>
Message-ID: <To9EhSlFT17L4ag_09O1J3ALCS_DBdTs2Dxd0HnW-bY=.972a0441-8351-4e74-81ae-6707aab82019@github.com>
On Tue, 25 Jun 2024 08:06:49 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Add parentheses for better alignment
> - Increase spin pauses to 0xFF to address performance regression in lightly contended scenarios
I like this. @kdnilsen and/or @earthling-amzn should take a look as well.
src/hotspot/share/gc/shenandoah/shenandoahLock.cpp line 57:
> 55: ctr--;
> 56: } else {
> 57: if (ALLOW_BLOCK) {
One other minor nit, let's stack it lower:
if (ctr > 0 ...) {
...
} else if (ALLOW_BLOCK) {
...
} else {
...
}
-------------
Marked as reviewed by shade (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19570#pullrequestreview-2137810261
PR Review Comment: https://git.openjdk.org/jdk/pull/19570#discussion_r1652206857
From xpeng at openjdk.org Tue Jun 25 08:11:13 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Jun 2024 08:11:13 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v6]
In-Reply-To: <CLpm_KsuVp-y2w5dmXSiL3FnV-mAqp8Zs_CLM_QBDRo=.c2f9e7ea-e3f7-4c8d-b5b7-15a27083ad8a@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<CLpm_KsuVp-y2w5dmXSiL3FnV-mAqp8Zs_CLM_QBDRo=.c2f9e7ea-e3f7-4c8d-b5b7-15a27083ad8a@github.com>
Message-ID: <aWwgFmkMu1F9BcgRIuESNS2cWmwlWZZ9H9JoqiKCr3I=.2dc0c5bf-6efe-4ccb-8738-6a4010ca9831@github.com>
On Tue, 25 Jun 2024 08:06:49 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Add parentheses for better alignment
> - Increase spin pauses to 0xFF to address performance regression in lightly contended scenarios
Also increased spin pauses to 255 times based on the test result, based on the test it should be better balanced in all cases, we should not sacrifice the performance of regular cases for the best performances of extreme cases.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19570#issuecomment-2188253223
From xpeng at openjdk.org Tue Jun 25 08:20:43 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Tue, 25 Jun 2024 08:20:43 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
Message-ID: <KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
> ### Notes
> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>
> #### Test code
>
> public class Alloc {
> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
> static final Object[] sinks = new Object[64*THREADS];
> static volatile boolean start;
> static volatile boolean stop;
>
> public static void main(String... args) throws Throwable {
> for (int t = 0; t < THREADS; t++) {
> int ft = t;
> new Thread(() -> work(ft * 64)).start();
> }
>
> Thread.sleep(1000);
> start = true;
> Thread.sleep(30_000);
> stop = true;
> }
>
> public static void work(int idx) {
> while (!start) { Thread.onSpinWait(); }
> while (!stop) {
> sinks[idx] = new byte[128];
> }
> }
> }
>
>
> Run it like this and observe TTSP times:
>
>
> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>
>
> #### Metrics from tests(TTSP, allocation rate)
> ##### Heavy contention(1280 threads, 32 per CPU core)
> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | -------- | ----- |
> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>
>
> ##### Light contention(40 threads, 1 per CPU core)
> | Spins | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | ------- | ---- |
> | baseline | ...
Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
Simplify code with less stacks
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19570/files
- new: https://git.openjdk.org/jdk/pull/19570/files/73e58d74..6fd8205d
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=06
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19570&range=05-06
Stats: 21 lines in 1 file changed: 2 ins; 4 del; 15 mod
Patch: https://git.openjdk.org/jdk/pull/19570.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19570/head:pull/19570
PR: https://git.openjdk.org/jdk/pull/19570
From shade at openjdk.org Tue Jun 25 11:37:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 25 Jun 2024 11:37:14 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
Message-ID: <M7PtADesYq4HcMdioTqC-E4Ky7PjiPCfYa6B5mbjbI8=.32faf78c-e87e-4f2d-a489-610aa18cde19@github.com>
On Tue, 25 Jun 2024 08:20:43 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Simplify code with less stacks
Marked as reviewed by shade (Reviewer).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19570#pullrequestreview-2138376912
From kdnilsen at openjdk.org Tue Jun 25 15:31:35 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 15:31:35 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <2wDffC_YF6eu9FcvWaM8Y4KoSNkzwCslrzT9RbfcKhY=.b53f155e-1575-4281-881c-e88c99ce979b@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
<AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
<2wDffC_YF6eu9FcvWaM8Y4KoSNkzwCslrzT9RbfcKhY=.b53f155e-1575-4281-881c-e88c99ce979b@github.com>
Message-ID: <O03gGa408hn2eYsI6w21_1pjpAk_7zMhf786Tw7KZyc=.64341a29-f4e1-4961-914b-41a72e40d561@github.com>
On Wed, 19 Jun 2024 20:56:04 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> Hmm, interesting, it looks like this does indeed increment only at a full gc, or at least that's the intent. Should check that this is also respected in the generational case. Vide:
>>
>>
>> product(uintx, ShenandoahNoProgressThreshold, 5, EXPERIMENTAL, \
>> "After this number of consecutive Full GCs fail to make " \
>> "progress, Shenandoah will raise out of memory errors. Note " \
>> "that progress is determined by ShenandoahCriticalFreeThreshold") \
>> \
>
> I'm testing now a configuration that honors the intent of this comment, to only increment gc_no_progress count if "consecutive Full GCs" fail to make progress. This is not how it was implemented before, because our implementation has been also incrementing gc_no_progress count if degenerated fails to make progress, and we have not been resetting the count when we experience a productive concurrent GC. My interpretation of the intent is that a concurrent GC happening between two unproductive Full GCs does not count as two "consecutive" unproductive full GCs.
>
> If this change behaves well, we may be able to remove the ShenandoahNoProgressThreshold override on TestThreadFailure#generational.
In general, this new configuration works well and passes all GHA and all CI/CD pipeline tests. However, I tried removing the ShenandoanNoProgressThreshold override from this test, and it fails. In one execution that I carefully analyzed, the behavior is:
1. For NastyThread-0 through NastyThread-12, we perform a FullGC (which has good progress) but the good progress is not enough to satisfy the failed allocation request so we throw OOM.
2. With NastyThread-12, we do not fail fast. GC(127) is concurrent young. GC(128) through GC(132) are Full GCs, each with Bad Progress, but each yielding enough free memory to satisfy at least one additional allocation by NastyThread-12.
3. GC(133) is a full GC also with bad progress. This time, the bad progress is not enough to satisfy the pending alloc request (for 4112 bytes), so we thrown OOM.
4. At this point, we have experienced 5 (Default value of ShenandoaohNoProgressThreshold) consecutive full GCs with no progress, so when the main thread attempts to allocate NastyThread-13 after joining with NastyThread-12, it does not even bother to attempt a Full GC. It just immediately throws OOM.
5. This causes the test to fail, because main is not "supposed" to experience OOM.
Another complication is that the failure doesn't always happen with NastyThread-13. Sometimes it happens with NastyThread-5. GC degradation is not cumulative. Each NastyThread is supposed to start with a clean slate (after a Full GC reclaims all previously allocated memory).
And finally, I have observed that this test will still occasionally fail even with the ShenandoahNoProgressThreshold=24 override.
So I'm puzzling a bit over why GenShen inherently needs a larger value of ShenandoahNoProgressThreshold than traditional Shenandoah in order to pass this test. I think the explanation is that GenShen introduces more "heap fragmentation" between OLD and YOUNG generations. Full GC can help sift out these fragments of memory so that "smaller" allocation requests can still succeed, even though Full GC reports "bad progress".
My current thought is to apply one more tweak to the GenShen behavior: If a pending allocation succeeds following a Full GC, I am inclined to count this as "good progress", regardless of what the other metrics think about progress. That we were able to allocate following Full GC and were not able to allocate before Full GC is the ultimate measure of "good progress". Will experiment with this.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653045650
From kdnilsen at openjdk.org Tue Jun 25 16:18:59 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 16:18:59 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Improve consistency of OOM in GenShen mode
Treat any full GC that enables a previously blocked allocation as good
progress.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/5400c9c0..91c3ab07
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=26
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=25-26
Stats: 13 lines in 2 files changed: 5 ins; 4 del; 4 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From kdnilsen at openjdk.org Tue Jun 25 16:44:31 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 16:44:31 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v8]
In-Reply-To: <O03gGa408hn2eYsI6w21_1pjpAk_7zMhf786Tw7KZyc=.64341a29-f4e1-4961-914b-41a72e40d561@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<aVwWZEoNt6NK9Aq0HrN_1DxfFXrXhRM9VY-VO-SZhu4=.029cfe30-4d27-451e-af86-ff9f8997e87d@github.com>
<9-LisOfX9PRmv4-a5VWu5jtsDhTfdJRPEOx_CxUZ5J4=.6918e7e2-0af8-4c37-8691-725df2199d48@github.com>
<AdFKDn3IEpElWwJ7yimTDClc9rGJaoipiAZg6K2SYGw=.3af19f12-dc77-4631-8e78-bdf889166a5a@github.com>
<2wDffC_YF6eu9FcvWaM8Y4KoSNkzwCslrzT9RbfcKhY=.b53f155e-1575-4281-881c-e88c99ce979b@github.com>
<O03gGa408hn2eYsI6w21_1pjpAk_7zMhf786Tw7KZyc=.64341a29-f4e1-4961-914b-41a72e40d561@github.com>
Message-ID: <NK6Uv2ezIn1QKkxXm4XKDB0K-ylYZbonjPTFqzZ-Uwc=.b13d3119-31cf-4ca9-bb24-25ec1bc221d0@github.com>
On Tue, 25 Jun 2024 15:28:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> I'm testing now a configuration that honors the intent of this comment, to only increment gc_no_progress count if "consecutive Full GCs" fail to make progress. This is not how it was implemented before, because our implementation has been also incrementing gc_no_progress count if degenerated fails to make progress, and we have not been resetting the count when we experience a productive concurrent GC. My interpretation of the intent is that a concurrent GC happening between two unproductive Full GCs does not count as two "consecutive" unproductive full GCs.
>>
>> If this change behaves well, we may be able to remove the ShenandoahNoProgressThreshold override on TestThreadFailure#generational.
>
> In general, this new configuration works well and passes all GHA and all CI/CD pipeline tests. However, I tried removing the ShenandoanNoProgressThreshold override from this test, and it fails. In one execution that I carefully analyzed, the behavior is:
>
> 1. For NastyThread-0 through NastyThread-11, we perform a FullGC (which has good progress) but the good progress is not enough to satisfy the failed allocation request so we throw OOM.
>
> 2. With NastyThread-12, we do not fail fast. GC(127) is concurrent young. GC(128) through GC(132) are Full GCs, each with Bad Progress, but each yielding enough free memory to satisfy at least one additional allocation by NastyThread-12.
>
> 3. GC(133) is a full GC also with bad progress. This time, the bad progress is not enough to satisfy the pending alloc request (for 4112 bytes), so we throw OOM.
>
> 4. At this point, we have experienced 5 (Default value of ShenandoaohNoProgressThreshold) consecutive full GCs with no progress, so when the main thread attempts to allocate NastyThread-13 after joining with NastyThread-12, it does not even bother to attempt a Full GC. It just immediately throws OOM.
>
> 5. This causes the test to fail, because main is not "supposed" to experience OOM.
>
> Another complication is that the failure doesn't always happen with NastyThread-13. Sometimes it happens with NastyThread-5. GC degradation is not cumulative. Each NastyThread is supposed to start with a clean slate (after a Full GC reclaims all previously allocated memory).
>
> And finally, I have observed that this test will still occasionally fail even with the ShenandoahNoProgressThreshold=24 override.
>
> So I'm puzzling a bit over why GenShen inherently needs a larger value of ShenandoahNoProgressThreshold than traditional Shenandoah in order to pass this test. I think the explanation is that GenShen introduces more "heap fragmentation" between OLD and YOUNG generations. Full GC can help sift out these fragments of memory so that "smaller" allocation requests can still succeed, even though Full GC reports "bad progress".
>
> My current thought is to apply one more tweak to the GenShen behavior: If a pending allocation succeeds following a Full GC, I am inclined to count this as "good progress", regardless of what the other metrics think about progress. That we were able to allocate following Full GC and were not able to allocate before Full GC is the ultimate measure of "good progress". Will experiment with this.
With this change, I got 100 consecutive successful runs of the TestThreadFailure.java test without having an override on ShenandoahNoProgressThreshold in the GenShen configuration of the test. I believe this is preferrable to the use of the override. It makes GenShen behave more similar to the way Shenandoah behaves.
I am now testing on CI/CD pipeline to see if this change introduces any other performance or correctness regressions.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653173995
From nprasad at openjdk.org Tue Jun 25 16:48:30 2024
From: nprasad at openjdk.org (Neethu Prasad)
Date: Tue, 25 Jun 2024 16:48:30 GMT
Subject: [jdk23] RFR: 8333716: Shenandoah: Check for disarmed method before
taking the nmethod lock
Message-ID: <3jRU7LDd5urGNq-_dR4ozwbqZfQHgp-mDmMIG4ZCh4E=.0ab05240-e071-4ca1-85e4-d83d66514876@github.com>
This pull request contains a backport of commit [18e7d7b5](https://github.com/openjdk/jdk/commit/18e7d7b5e710b24e49b995777906a197e35795e6) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository.
-------------
Commit messages:
- Backport 18e7d7b5e710b24e49b995777906a197e35795e6
Changes: https://git.openjdk.org/jdk/pull/19888/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19888&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8333716
Stats: 8 lines in 1 file changed: 6 ins; 0 del; 2 mod
Patch: https://git.openjdk.org/jdk/pull/19888.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19888/head:pull/19888
PR: https://git.openjdk.org/jdk/pull/19888
From kcr at openjdk.org Tue Jun 25 18:19:13 2024
From: kcr at openjdk.org (Kevin Rushforth)
Date: Tue, 25 Jun 2024 18:19:13 GMT
Subject: [jdk23] RFR: 8333716: Shenandoah: Check for disarmed method before
taking the nmethod lock
In-Reply-To: <3jRU7LDd5urGNq-_dR4ozwbqZfQHgp-mDmMIG4ZCh4E=.0ab05240-e071-4ca1-85e4-d83d66514876@github.com>
References: <3jRU7LDd5urGNq-_dR4ozwbqZfQHgp-mDmMIG4ZCh4E=.0ab05240-e071-4ca1-85e4-d83d66514876@github.com>
Message-ID: <OzZjEbOrs5We6QIg2bgmr71ZsUjA4ugrIZcd8GIjxto=.1b2752fc-c96f-45cd-99bb-49b534931e6a@github.com>
On Tue, 25 Jun 2024 16:43:10 GMT, Neethu Prasad <nprasad at openjdk.org> wrote:
> This pull request contains a backport of commit [18e7d7b5](https://github.com/openjdk/jdk/commit/18e7d7b5e710b24e49b995777906a197e35795e6) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository.
@neethu-prasad As a P4 Enhancement, this does not meet the criteria for inclusion into JDK 23 during RDP1. You might consider it for jdk23u instead.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19888#issuecomment-2189663720
From kcr at openjdk.org Tue Jun 25 18:27:13 2024
From: kcr at openjdk.org (Kevin Rushforth)
Date: Tue, 25 Jun 2024 18:27:13 GMT
Subject: [jdk23] RFR: 8334594: Generational ZGC: Deadlock after OopMap
rewrites in 8331572
In-Reply-To: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
References: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
Message-ID: <3yzDRbN8qzDoS-J0YiDADvLLh_XCQIpHRPZCkrDVlfI=.7e3577e9-555e-458a-868b-fa06014c698b@github.com>
On Mon, 24 Jun 2024 09:02:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Clean backport to fix a deadlock.
@shipilev Is the priority (P4) of this bug correct? If so, then it doesn't seem to meet the criteria for JDK 23 during RDP1. If the priority is wrong, please update it in JBS.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19851#issuecomment-2189682009
From ysr at openjdk.org Tue Jun 25 20:54:42 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 25 Jun 2024 20:54:42 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
Message-ID: <nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
On Tue, 25 Jun 2024 16:18:59 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Improve consistency of OOM in GenShen mode
>
> Treat any full GC that enables a previously blocked allocation as good
> progress.
A few questions/comments, but (delta of) changes look good to me.
src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 329:
> 327: if (gc.collect(cause)) {
> 328: // Cycle is complete
> 329: heap->notify_gc_progress();
I am trying to understand this. Is the following a reasonable summary of the underlying reasoning?
Any concurrent cycle that completes without degenerating is considered progress irrespective of whether it reclaimed anything. The idea is that if there were allocations during that concurrent cycle, they will have succeeded because, if not we'd have degenerated and not reached here: "collect()" would have returned "false".
src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 307:
> 305: // because that probably means the heap is overloaded and/or fragmented.
> 306: if (!metrics.is_good_progress()) {
> 307: heap->notify_gc_no_progress();
Again, let me make sure I understand the reasoning here:
We are already going to cancel gc and degenerate, so the concurrent collect will return failure, and the control thread will not record progress.
But I don't see anything that actually records lack of progress.
I assume then that a degenerated GC in and of itself isn't indicative of lack of progress, and that lack of progress shouldn't be recorded here, but rather after having worked harder, perhaps in `op_degenerated_futile()`?
It would be good to leave some clarifying comments for when we read this code many months later and have forgotten the context. May be something like:
// We haven't made progress; we'll try harder by canceling GC and upgrading to a full gc
// to see if we make more progress.
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1032:
> 1030: result = allocate_memory_under_lock(req, in_new_region);
> 1031: }
> 1032: if ((result != nullptr) && mode()->is_generational()) {
I am wondering why we are using a `mode()->is_generational()` filter here.
Would this not work in exactly the same way for the non-generational case as well?
If so, I'd extract this fix out more generally and upstream it separately as its own ticket for fixing the OOM'ing behavior for "no progress OOMs".
src/hotspot/share/gc/shenandoah/shenandoahMmuTracker.cpp line 129:
> 127: double now = os::elapsedTime();
> 128: double duration = now - _most_recent_timestamp;
> 129:
Is this change just to remove spurious differences from upstream openjdk/jdk repo?
-------------
Marked as reviewed by ysr (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/440#pullrequestreview-2139790908
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653520690
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653529666
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653543616
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653544682
From wkemper at openjdk.org Tue Jun 25 21:42:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Jun 2024 21:42:33 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
Message-ID: <Gh9XMH_71pPNF5hOrAEKkWex0-kSekrUszrvV0V6_kQ=.f098c636-0a97-4b3f-b2c1-abbf8d26343e@github.com>
On Tue, 25 Jun 2024 16:18:59 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Improve consistency of OOM in GenShen mode
>
> Treat any full GC that enables a previously blocked allocation as good
> progress.
The other side of the oom issue is test failures and timeouts caused by Shenandoah not giving up soon enough. If I recall correctly, there were three tests that would fail because they did _not_ raise an OOME when expected. Could we also run:
make test TEST="runtime/ClassInitErrors/TestOutOfMemoryDuringInit.java compiler/uncommontrap/TestDeoptOOM.java runtime/reflect/ReflectOutOfMemoryError.java" TEST_VM_OPTS="-Xlog:gc:file=/tmp/shen.log -XX:+UseShenandoahGC"
and for genshen:
make test TEST="runtime/ClassInitErrors/TestOutOfMemoryDuringInit.java compiler/uncommontrap/TestDeoptOOM.java runtime/reflect/ReflectOutOfMemoryError.java" TEST_VM_OPTS="-Xlog:gc:file=/tmp/shen.log -XX:+UseShenandoahGC -XX:ShenandoahGCMode=generational"
-------------
PR Comment: https://git.openjdk.org/shenandoah/pull/440#issuecomment-2190015491
From wkemper at openjdk.org Tue Jun 25 21:42:33 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Jun 2024 21:42:33 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
<nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
Message-ID: <RlCwEzWEWrlA5z2tjM1lYgn_DCrYWZZIadRwu1J-aGU=.a50e2652-714a-4ee9-a368-0d20d1faeca9@github.com>
On Tue, 25 Jun 2024 20:38:37 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Improve consistency of OOM in GenShen mode
>>
>> Treat any full GC that enables a previously blocked allocation as good
>> progress.
>
> src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 307:
>
>> 305: // because that probably means the heap is overloaded and/or fragmented.
>> 306: if (!metrics.is_good_progress()) {
>> 307: heap->notify_gc_no_progress();
>
> Again, let me make sure I understand the reasoning here:
>
> We are already going to cancel gc and degenerate, so the concurrent collect will return failure, and the control thread will not record progress.
>
> But I don't see anything that actually records lack of progress.
>
> I assume then that a degenerated GC in and of itself isn't indicative of lack of progress, and that lack of progress shouldn't be recorded here, but rather after having worked harder, perhaps in `op_degenerated_futile()`?
>
> It would be good to leave some clarifying comments for when we read this code many months later and have forgotten the context. May be something like:
>
>
> // We haven't made progress; we'll try harder by canceling GC and upgrading to a full gc
> // to see if we make more progress.
I think that's intentional. We only want to record no-progress after a full GC.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653605043
From wkemper at openjdk.org Tue Jun 25 21:58:15 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Jun 2024 21:58:15 GMT
Subject: RFR: 8333925: Shenandoah: Heuristics should have an option to
ignore abbreviated cycles [v2]
In-Reply-To: <RJwopj5bf7i9sI0-cKmY7A0VwQbBsIrLJmul7OhLZLo=.3276b9a4-3bdf-46ec-aa36-f8cfd2c53721@github.com>
References: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
<RJwopj5bf7i9sI0-cKmY7A0VwQbBsIrLJmul7OhLZLo=.3276b9a4-3bdf-46ec-aa36-f8cfd2c53721@github.com>
Message-ID: <1ZtOGAMyK1WWtQHbbVtXRwZuBh0p4w3tPKCrbL8Tgc0=.45e0c4c5-403a-4dae-b093-0785ff49f6cc@github.com>
On Tue, 11 Jun 2024 22:36:26 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> After concurrent marking is complete, Shenandoah will skip the evacuation phase if a sufficient amount of garbage is found in regions that contain _no live objects_. These abbreviated cycles are much shorter than a cycle that performs evacuation and update references and tend to lower the average cycle time used by the heuristic to predict cycle times. This may cause the heuristic to wait too long to initiate a cycle and may lead to degenerated cycles. This change has the heuristic ignore abbreviated cycle times by default, with an option to have the heuristic count them as it does now.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>
> Rename option to ShenandoahAdaptiveIgnoreAbbreviated, remove unnecessary Amazon copyright from header
Withdrawing this PR because I cannot reproduce the results that compelled us to make this change in the `shenandoah` repo.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19640#issuecomment-2190036867
From wkemper at openjdk.org Tue Jun 25 21:58:16 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Jun 2024 21:58:16 GMT
Subject: Withdrawn: 8333925: Shenandoah: Heuristics should have an option to
ignore abbreviated cycles
In-Reply-To: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
References: <H5A2cxXehobmIc30anqG650Y0-b7EuGkRG16K7PuDfQ=.a46516e5-755a-4e61-97ee-9ef19185c90d@github.com>
Message-ID: <rRbLSrwyKH-4L5EAvhxlZbj_9h4yvdC8jZ2AlQ3ZREs=.5b0ec5f1-6afb-4df6-b75c-03f0c99191ce@github.com>
On Mon, 10 Jun 2024 22:34:00 GMT, William Kemper <wkemper at openjdk.org> wrote:
> After concurrent marking is complete, Shenandoah will skip the evacuation phase if a sufficient amount of garbage is found in regions that contain _no live objects_. These abbreviated cycles are much shorter than a cycle that performs evacuation and update references and tend to lower the average cycle time used by the heuristic to predict cycle times. This may cause the heuristic to wait too long to initiate a cycle and may lead to degenerated cycles. This change has the heuristic ignore abbreviated cycle times by default, with an option to have the heuristic count them as it does now.
This pull request has been closed without being integrated.
-------------
PR: https://git.openjdk.org/jdk/pull/19640
From kdnilsen at openjdk.org Tue Jun 25 22:05:32 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 22:05:32 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
<nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
Message-ID: <DMXHCm2W8o_0V28JeG9K6V2G2Kx99nRBnZfFVEDxbkQ=.c53b8c4e-41db-413a-ac82-a5bea7cce2ac@github.com>
On Tue, 25 Jun 2024 20:29:12 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Improve consistency of OOM in GenShen mode
>>
>> Treat any full GC that enables a previously blocked allocation as good
>> progress.
>
> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 329:
>
>> 327: if (gc.collect(cause)) {
>> 328: // Cycle is complete
>> 329: heap->notify_gc_progress();
>
> I am trying to understand this. Is the following a reasonable summary of the underlying reasoning?
>
> Any concurrent cycle that completes without degenerating is considered progress irrespective of whether it reclaimed anything. The idea is that if there were allocations during that concurrent cycle, they will have succeeded because, if not we'd have degenerated and not reached here: "collect()" would have returned "false".
Right. The idea is that we completed concurrent GC without a failed allocation, and thus concurrent GC was not cancelled...
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653628062
From kdnilsen at openjdk.org Tue Jun 25 22:05:32 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 22:05:32 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <DMXHCm2W8o_0V28JeG9K6V2G2Kx99nRBnZfFVEDxbkQ=.c53b8c4e-41db-413a-ac82-a5bea7cce2ac@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
<nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
<DMXHCm2W8o_0V28JeG9K6V2G2Kx99nRBnZfFVEDxbkQ=.c53b8c4e-41db-413a-ac82-a5bea7cce2ac@github.com>
Message-ID: <bfTD76upJPOArVLinv1yAvydzhARdUpzZtqslpIBL8U=.515b9783-f21a-40e3-8e88-e740d9b74a17@github.com>
On Tue, 25 Jun 2024 22:02:15 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 329:
>>
>>> 327: if (gc.collect(cause)) {
>>> 328: // Cycle is complete
>>> 329: heap->notify_gc_progress();
>>
>> I am trying to understand this. Is the following a reasonable summary of the underlying reasoning?
>>
>> Any concurrent cycle that completes without degenerating is considered progress irrespective of whether it reclaimed anything. The idea is that if there were allocations during that concurrent cycle, they will have succeeded because, if not we'd have degenerated and not reached here: "collect()" would have returned "false".
>
> Right. The idea is that we completed concurrent GC without a failed allocation, and thus concurrent GC was not cancelled...
I suppose we could be more "careful" about how we characterize progress...
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653628390
From kdnilsen at openjdk.org Tue Jun 25 22:11:25 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 22:11:25 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <RlCwEzWEWrlA5z2tjM1lYgn_DCrYWZZIadRwu1J-aGU=.a50e2652-714a-4ee9-a368-0d20d1faeca9@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
<nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
<RlCwEzWEWrlA5z2tjM1lYgn_DCrYWZZIadRwu1J-aGU=.a50e2652-714a-4ee9-a368-0d20d1faeca9@github.com>
Message-ID: <WPfTyLxxB4H0ZtGr_sNJo0kt7sRtkKgI4FK4vlqiZkM=.90a4def6-4675-4d2b-b086-3dc6ba55c0ae@github.com>
On Tue, 25 Jun 2024 21:39:58 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 307:
>>
>>> 305: // because that probably means the heap is overloaded and/or fragmented.
>>> 306: if (!metrics.is_good_progress()) {
>>> 307: heap->notify_gc_no_progress();
>>
>> Again, let me make sure I understand the reasoning here:
>>
>> We are already going to cancel gc and degenerate, so the concurrent collect will return failure, and the control thread will not record progress.
>>
>> But I don't see anything that actually records lack of progress.
>>
>> I assume then that a degenerated GC in and of itself isn't indicative of lack of progress, and that lack of progress shouldn't be recorded here, but rather after having worked harder, perhaps in `op_degenerated_futile()`?
>>
>> It would be good to leave some clarifying comments for when we read this code many months later and have forgotten the context. May be something like:
>>
>>
>> // We haven't made progress; we'll try harder by canceling GC and upgrading to a full gc
>> // to see if we make more progress.
>
> I think that's intentional. We only want to record no-progress after a full GC.
The description of ShenandoahNoProgressThreshold speaks only in terms of Full GCs. In the case that degenerated fails to make progress, we will escalate to full GC. If that Full GC fails to make progress, it will notify_gc_no_progress(). It's really part of the same GC cycle. I don't think it's appropriate to count no progress twice for this situation. If the Full GC does make progress, then it will reset the no-progress counter to zero, and there was no value in incrementing it here...
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653630570
From kdnilsen at openjdk.org Tue Jun 25 22:11:25 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 22:11:25 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
<nWFFk2wXpHXf-Ap_IsEj6kbyg42IeKXXrYABkvHZuZc=.9095f4eb-fb72-40e7-a70b-7d98f5d20632@github.com>
Message-ID: <g_fHvemoKkqK3NRn_0XBiBvZBoxuLGkoYpYvlIENSF0=.546170c7-a7b7-43ae-8fed-575886dbe7de@github.com>
On Tue, 25 Jun 2024 20:49:25 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Improve consistency of OOM in GenShen mode
>>
>> Treat any full GC that enables a previously blocked allocation as good
>> progress.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1032:
>
>> 1030: result = allocate_memory_under_lock(req, in_new_region);
>> 1031: }
>> 1032: if ((result != nullptr) && mode()->is_generational()) {
>
> I am wondering why we are using a `mode()->is_generational()` filter here.
>
> Would this not work in exactly the same way for the non-generational case as well?
>
> If so, I'd extract this fix out more generally and upstream it separately as its own ticket for fixing the OOM'ing behavior for "no progress OOMs".
This would work "equally well" for non-generational mode. I was reluctant to "change the behavior" of non-gen mode, especially because the problem we're facing seems to only manifest in generational mode.
I don't think it would hurt non-gen mode at all. But perhaps some rare actual customer of non-gen might actually be counting on the existing behavior... (?)
> src/hotspot/share/gc/shenandoah/shenandoahMmuTracker.cpp line 129:
>
>> 127: double now = os::elapsedTime();
>> 128: double duration = now - _most_recent_timestamp;
>> 129:
>
> Is this change just to remove spurious differences from upstream openjdk/jdk repo?
I had inserted and then removed some debugging instrumentation here. I will restore the blank lines to "original" form.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653631654
PR Review Comment: https://git.openjdk.org/shenandoah/pull/440#discussion_r1653632523
From kdnilsen at openjdk.org Tue Jun 25 23:11:36 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 23:11:36 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <Gh9XMH_71pPNF5hOrAEKkWex0-kSekrUszrvV0V6_kQ=.f098c636-0a97-4b3f-b2c1-abbf8d26343e@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
<Gh9XMH_71pPNF5hOrAEKkWex0-kSekrUszrvV0V6_kQ=.f098c636-0a97-4b3f-b2c1-abbf8d26343e@github.com>
Message-ID: <pBz1ZIfwkRjeFPItg9_tL9YqXDgBEN0a60l_kOsSnnw=.31943603-921d-482f-b17d-d946edfe22fb@github.com>
On Tue, 25 Jun 2024 21:38:49 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Improve consistency of OOM in GenShen mode
>>
>> Treat any full GC that enables a previously blocked allocation as good
>> progress.
>
> The other side of the oom issue is test failures and timeouts caused by Shenandoah not giving up soon enough. If I recall correctly, there were three tests that would fail because they did _not_ raise an OOME when expected. Could we also run:
>
> make test TEST="runtime/ClassInitErrors/TestOutOfMemoryDuringInit.java compiler/uncommontrap/TestDeoptOOM.java runtime/reflect/ReflectOutOfMemoryError.java" TEST_VM_OPTS="-Xlog:gc:file=/tmp/shen.log -XX:+UseShenandoahGC"
>
> and for genshen:
>
> make test TEST="runtime/ClassInitErrors/TestOutOfMemoryDuringInit.java compiler/uncommontrap/TestDeoptOOM.java runtime/reflect/ReflectOutOfMemoryError.java" TEST_VM_OPTS="-Xlog:gc:file=/tmp/shen.log -XX:+UseShenandoahGC -XX:ShenandoahGCMode=generational"
@earthling-amzn I've tested both command lines with REPEAT_COUNT=10 and all tests pass. (Had to add -XX:+UnlockExperimentalVMOptions to second command)
-------------
PR Comment: https://git.openjdk.org/shenandoah/pull/440#issuecomment-2190166092
From wkemper at openjdk.org Tue Jun 25 23:32:26 2024
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 25 Jun 2024 23:32:26 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v27]
In-Reply-To: <BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
<BcbB0FZ0yHh3ryW7852zGL9yWwYEA9aStpNkPBegy3Q=.ccf2acc5-9e99-4842-a7c7-ff4a72d60fea@github.com>
Message-ID: <3vIklLC-jiYcDFqzMwfxbGwBY9fLzJEshwiJdicIBUQ=.a0cefc2c-67e5-4740-9552-a350cf2e1073@github.com>
On Tue, 25 Jun 2024 16:18:59 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
>> into Generational Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>
> Improve consistency of OOM in GenShen mode
>
> Treat any full GC that enables a previously blocked allocation as good
> progress.
Marked as reviewed by wkemper (Committer).
Thanks for running additional tests.
-------------
PR Review: https://git.openjdk.org/shenandoah/pull/440#pullrequestreview-2140122452
PR Comment: https://git.openjdk.org/shenandoah/pull/440#issuecomment-2190198387
From kdnilsen at openjdk.org Tue Jun 25 23:42:54 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 25 Jun 2024 23:42:54 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation
of FreeSet into GenShen [v28]
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <-aQpprVakY0xyN1b73BdGsz23Gtfkri8MtYQ0j1miW0=.90a46d2b-0cf0-41f5-8023-0cfe06740139@github.com>
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
Back out accidental blank line deletion
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/440/files
- new: https://git.openjdk.org/shenandoah/pull/440/files/91c3ab07..88e585c1
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=27
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=440&range=26-27
Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/shenandoah/pull/440.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/440/head:pull/440
PR: https://git.openjdk.org/shenandoah/pull/440
From nprasad at openjdk.org Tue Jun 25 23:54:14 2024
From: nprasad at openjdk.org (Neethu Prasad)
Date: Tue, 25 Jun 2024 23:54:14 GMT
Subject: [jdk23] RFR: 8333716: Shenandoah: Check for disarmed method before
taking the nmethod lock
In-Reply-To: <OzZjEbOrs5We6QIg2bgmr71ZsUjA4ugrIZcd8GIjxto=.1b2752fc-c96f-45cd-99bb-49b534931e6a@github.com>
References: <3jRU7LDd5urGNq-_dR4ozwbqZfQHgp-mDmMIG4ZCh4E=.0ab05240-e071-4ca1-85e4-d83d66514876@github.com>
<OzZjEbOrs5We6QIg2bgmr71ZsUjA4ugrIZcd8GIjxto=.1b2752fc-c96f-45cd-99bb-49b534931e6a@github.com>
Message-ID: <u50dPDvtfnFcXBzgDEUOnkdWkFLgtp_pfChBimwOwls=.aea65283-a04f-48fa-8bb1-cc7108305ed3@github.com>
On Tue, 25 Jun 2024 18:16:21 GMT, Kevin Rushforth <kcr at openjdk.org> wrote:
>> This pull request contains a backport of commit [18e7d7b5](https://github.com/openjdk/jdk/commit/18e7d7b5e710b24e49b995777906a197e35795e6) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository.
>
> @neethu-prasad As a P4 Enhancement, this does not meet the criteria for inclusion into JDK 23 during RDP1. You might consider it for jdk23u instead.
@kevinrushforth thanks for the feedback. I'll close this PR.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19888#issuecomment-2190219674
From nprasad at openjdk.org Tue Jun 25 23:54:15 2024
From: nprasad at openjdk.org (Neethu Prasad)
Date: Tue, 25 Jun 2024 23:54:15 GMT
Subject: [jdk23] Withdrawn: 8333716: Shenandoah: Check for disarmed method
before taking the nmethod lock
In-Reply-To: <3jRU7LDd5urGNq-_dR4ozwbqZfQHgp-mDmMIG4ZCh4E=.0ab05240-e071-4ca1-85e4-d83d66514876@github.com>
References: <3jRU7LDd5urGNq-_dR4ozwbqZfQHgp-mDmMIG4ZCh4E=.0ab05240-e071-4ca1-85e4-d83d66514876@github.com>
Message-ID: <ki1wzLB1DD9dDe3eV6MCDCbpFASBSfs7pGPJy76-DmI=.b48ab8b7-4127-4c94-ad31-46d36faee0ff@github.com>
On Tue, 25 Jun 2024 16:43:10 GMT, Neethu Prasad <nprasad at openjdk.org> wrote:
> This pull request contains a backport of commit [18e7d7b5](https://github.com/openjdk/jdk/commit/18e7d7b5e710b24e49b995777906a197e35795e6) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository.
This pull request has been closed without being integrated.
-------------
PR: https://git.openjdk.org/jdk/pull/19888
From kdnilsen at openjdk.org Wed Jun 26 01:05:34 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 26 Jun 2024 01:05:34 GMT
Subject: Integrated: 8327000: GenShen: Integrate updated Shenandoah
implementation of FreeSet into GenShen
In-Reply-To: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
References: <JuWmglvlw9Pr8Cbxt5RpDzs-ctTgAGXXFNWFW1qbr84=.3c950663-ccab-4c6a-afcd-736090fd3f13@github.com>
Message-ID: <ZyVU2xbk2wc4CEvo0PFDIrIqWxMWlpk7PkuntYdlNTI=.c6789aa7-1cc6-48bf-b81b-a763197ca079@github.com>
On Sat, 25 May 2024 13:57:39 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
> The mainline implementation of ShenandoahFreeSet was recently updated. This PR integrates the upstream changes
> into Generational Shenandoah.
This pull request has now been integrated.
Changeset: 9d2712dd
Author: Kelvin Nilsen <kdnilsen at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/9d2712ddf39160a618c9c839c46d2510063f45df
Stats: 1865 lines in 19 files changed: 970 ins; 263 del; 632 mod
8327000: GenShen: Integrate updated Shenandoah implementation of FreeSet into GenShen
Reviewed-by: wkemper, ysr
-------------
PR: https://git.openjdk.org/shenandoah/pull/440
From kbarrett at openjdk.org Wed Jun 26 05:21:29 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Wed, 26 Jun 2024 05:21:29 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort
In-Reply-To: <sAlRFyLsVXfzvSC5kK8CbH46w9fth7KwOx9LyDZsP8g=.c85611be-6375-4ef1-a2eb-97083a64c94f@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
<X23eTytvPT0yKjFjbPYduZ4epuSWQ3aj0D1gdIqsjF4=.613e2c3b-de99-46de-a19d-b267619204b0@github.com>
<nl2ACgjveuSPgnZ_zFS_POtPWwnpzMBDkxqZkT5cgoo=.b7f84bd9-bf42-4efd-8b24-d7ba725a174a@github.com>
<sAlRFyLsVXfzvSC5kK8CbH46w9fth7KwOx9LyDZsP8g=.c85611be-6375-4ef1-a2eb-97083a64c94f@github.com>
Message-ID: <A8JTc0JisqTaBqigOqOrCV5u1fjBxpezQsxbAzvuuTw=.297a8bc6-93e2-4651-9370-72e7d2ea3528@github.com>
On Mon, 17 Jun 2024 12:53:33 GMT, Florian Weimer <fweimer at openjdk.org> wrote:
> > > It does not provide any such thing. All the flag does is prevent swapping of
> > > equivalent elements, which doesn't give us any interesting additional ordering
> > > property.
> >
> >
> > I only meant the sort order of the equivalent elements would be maintained.
>
> I think the partitioning phase swaps inequal elements based on comparison with the pivot, and this can move elements equivalent to the pivot past the pivot, with or without that additional equality check.
Yes. I think what @dholmes-ora describes is "stability", which this option does not provide.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19464#issuecomment-2190723303
From kbarrett at openjdk.org Wed Jun 26 05:21:29 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Wed, 26 Jun 2024 05:21:29 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort [v3]
In-Reply-To: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
Message-ID: <ZdfvLep4NbjPX2RqM8cBIa8Z9AV4OpRpwHiut4iLaM4=.3a56b7ed-d2a0-467e-81fb-d68e55dec7e4@github.com>
> The "idempotent" argument is removed from that function, with associated
> simplifications to the implementation. Callers are updated to remove that
> argument. Callers that were providing a false value are unaffected in their
> behavior. The 3 callers that were providing a true value to request the
> associated feature are also unaffected (other than by being made faster),
> because the arrays involved don't contain any equivalent pairs.
>
> There are also some miscellaneous cleanups, including using the swap utility
> and fixing some comments.
>
> Testing: mach5 tier1-3
Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
- Merge branch 'master' into no-idempotent-quicksort
- Merge branch 'master' into no-idempotent-quicksort
- improve find_pivot description
- remove idempotent
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19464/files
- new: https://git.openjdk.org/jdk/pull/19464/files/5cee3b81..07dd7040
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19464&range=02
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19464&range=01-02
Stats: 73160 lines in 1648 files changed: 45921 ins; 20473 del; 6766 mod
Patch: https://git.openjdk.org/jdk/pull/19464.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19464/head:pull/19464
PR: https://git.openjdk.org/jdk/pull/19464
From kbarrett at openjdk.org Wed Jun 26 05:21:29 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Wed, 26 Jun 2024 05:21:29 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort [v2]
In-Reply-To: <vEw_CJ5GwZgEBnk3P27bxURjZlWH0oWUCyK8KPtRg5Y=.97e5b0e0-ae39-4976-a536-a6348093c068@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
<vEw_CJ5GwZgEBnk3P27bxURjZlWH0oWUCyK8KPtRg5Y=.97e5b0e0-ae39-4976-a536-a6348093c068@github.com>
Message-ID: <rLJAxE5vWanYFbyWiBGCvDwICnioeUtR1W6_on76_M0=.63d55e07-b7b8-4da3-b3ee-b042e95c46ad@github.com>
On Wed, 19 Jun 2024 08:41:24 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> The "idempotent" argument is removed from that function, with associated
>> simplifications to the implementation. Callers are updated to remove that
>> argument. Callers that were providing a false value are unaffected in their
>> behavior. The 3 callers that were providing a true value to request the
>> associated feature are also unaffected (other than by being made faster),
>> because the arrays involved don't contain any equivalent pairs.
>>
>> There are also some miscellaneous cleanups, including using the swap utility
>> and fixing some comments.
>>
>> Testing: mach5 tier1-3
>
> Kim Barrett has updated the pull request incrementally with one additional commit since the last revision:
>
> improve find_pivot description
Thanks for reviews.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19464#issuecomment-2190724041
From kbarrett at openjdk.org Wed Jun 26 05:21:29 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Wed, 26 Jun 2024 05:21:29 GMT
Subject: RFR: 8333133: Simplify QuickSort::sort [v3]
In-Reply-To: <nYycWzpNJUUKPhesxmPR2PUcsUXCwkn0aoVcgcosz70=.0031103d-b722-4daf-9cf1-469a37c129cb@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
<nYycWzpNJUUKPhesxmPR2PUcsUXCwkn0aoVcgcosz70=.0031103d-b722-4daf-9cf1-469a37c129cb@github.com>
Message-ID: <0mYXOm6IaLlZALCjMW1QqB7qWAVALTu7p726WGgODpA=.e39360ec-fc40-4ba0-a35a-e7d461dfc344@github.com>
On Tue, 11 Jun 2024 05:32:03 GMT, David Holmes <dholmes at openjdk.org> wrote:
>> Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
>>
>> - Merge branch 'master' into no-idempotent-quicksort
>> - Merge branch 'master' into no-idempotent-quicksort
>> - improve find_pivot description
>> - remove idempotent
>
> src/hotspot/share/utilities/quickSort.hpp line 43:
>
>> 41: // We swap these three values into the right place in the array. This
>> 42: // means that this method not only returns the index of the pivot
>> 43: // element. It also alters the array so that:
>
> Pre-existing nit: this should be one sentence: "... element, it also ..."
I ended up doing a rewrite of the description.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19464#discussion_r1654047792
From kbarrett at openjdk.org Wed Jun 26 05:21:30 2024
From: kbarrett at openjdk.org (Kim Barrett)
Date: Wed, 26 Jun 2024 05:21:30 GMT
Subject: Integrated: 8333133: Simplify QuickSort::sort
In-Reply-To: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
References: <R8DaxsGPw4s4RTFxgGgVHUkxqTP7WYRUlNYDdxQV9OA=.ceb28776-108e-4e0b-aeae-c7909496c667@github.com>
Message-ID: <SSM9mJxAp2xn7-zrspzRcmyGkBfoItIp5RqYBKesdsw=.6ab1a030-f1af-45fc-ae96-4cd9ebae8640@github.com>
On Wed, 29 May 2024 18:52:03 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
> The "idempotent" argument is removed from that function, with associated
> simplifications to the implementation. Callers are updated to remove that
> argument. Callers that were providing a false value are unaffected in their
> behavior. The 3 callers that were providing a true value to request the
> associated feature are also unaffected (other than by being made faster),
> because the arrays involved don't contain any equivalent pairs.
>
> There are also some miscellaneous cleanups, including using the swap utility
> and fixing some comments.
>
> Testing: mach5 tier1-3
This pull request has now been integrated.
Changeset: 25c3845b
Author: Kim Barrett <kbarrett at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/25c3845be270462388ee5e7330cc7315e5c738df
Stats: 130 lines in 11 files changed: 5 ins; 97 del; 28 mod
8333133: Simplify QuickSort::sort
Reviewed-by: shade, dholmes
-------------
PR: https://git.openjdk.org/jdk/pull/19464
From stefank at openjdk.org Wed Jun 26 07:41:11 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 26 Jun 2024 07:41:11 GMT
Subject: [jdk23] RFR: 8334594: Generational ZGC: Deadlock after OopMap
rewrites in 8331572
In-Reply-To: <3yzDRbN8qzDoS-J0YiDADvLLh_XCQIpHRPZCkrDVlfI=.7e3577e9-555e-458a-868b-fa06014c698b@github.com>
References: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
<3yzDRbN8qzDoS-J0YiDADvLLh_XCQIpHRPZCkrDVlfI=.7e3577e9-555e-458a-868b-fa06014c698b@github.com>
Message-ID: <goFJIuxq4GCT_dGr3nAmdAbiV3caOTOQTXu3vgUi3C8=.c5cb3f1d-bc75-4880-be71-5d20e54e4722@github.com>
On Tue, 25 Jun 2024 18:25:02 GMT, Kevin Rushforth <kcr at openjdk.org> wrote:
> @shipilev Is the priority (P4) of this bug correct? If so, then it doesn't seem to meet the criteria for JDK 23 during RDP1. If the priority is wrong, please update it in JBS.
The status is not correct. For GC bugs we tend to *not* set the priority when bugs are created (so they are left at the default, P4 [which is an unfortunate default, IMHO]), and then the GC triage will assign the correct priority. This bug was fixed before GC triage got to triaging the bug, and hence it didn't get the appropriate priority set. I'll update the priority to a P2.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19851#issuecomment-2191026927
From stefank at openjdk.org Wed Jun 26 07:45:09 2024
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 26 Jun 2024 07:45:09 GMT
Subject: [jdk23] RFR: 8334594: Generational ZGC: Deadlock after OopMap
rewrites in 8331572
In-Reply-To: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
References: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
Message-ID: <RDzohliHnrmaxhzFjiRYaGog1vlvn5eNf2JGNNigGCs=.eccfff59-798a-4711-9457-36f94a0f067e@github.com>
On Mon, 24 Jun 2024 09:02:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Clean backport to fix a deadlock.
Marked as reviewed by stefank (Reviewer).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19851#pullrequestreview-2140999606
From shade at openjdk.org Wed Jun 26 07:49:16 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 26 Jun 2024 07:49:16 GMT
Subject: [jdk23] RFR: 8334594: Generational ZGC: Deadlock after OopMap
rewrites in 8331572
In-Reply-To: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
References: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
Message-ID: <YKcXv4dtX1YiFYqmWF8QoxWz3FlTnceytC7ELrpEHg4=.907bda86-6d00-4b98-ba98-34430016acdb@github.com>
On Mon, 24 Jun 2024 09:02:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Clean backport to fix a deadlock.
Yes, I agree this is a P2 bug.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19851#issuecomment-2191038488
From shade at openjdk.org Wed Jun 26 07:49:17 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 26 Jun 2024 07:49:17 GMT
Subject: [jdk23] Integrated: 8334594: Generational ZGC: Deadlock after OopMap
rewrites in 8331572
In-Reply-To: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
References: <uPpyjaKbX6g0j1GlcptLoXWwBGw00FgXqocK1YdC9jE=.57aea726-fe19-41be-88bf-a559913a493c@github.com>
Message-ID: <X3RXFtw8n4ZtpBzazB2Z29-RHo8DCOb1Nzl0--o7cxk=.09b2ad52-3a03-45a9-bf9f-1c06d944403d@github.com>
On Mon, 24 Jun 2024 09:02:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Clean backport to fix a deadlock.
This pull request has now been integrated.
Changeset: d1510505
Author: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/d1510505c1fb0c31063e7a1ba3bb6ead4cdd7568
Stats: 12 lines in 6 files changed: 3 ins; 0 del; 9 mod
8334594: Generational ZGC: Deadlock after OopMap rewrites in 8331572
Reviewed-by: stefank
Backport-of: 05ff3185edd25b381a97f6879f496e97b62dddc2
-------------
PR: https://git.openjdk.org/jdk/pull/19851
From wkemper at openjdk.org Wed Jun 26 16:52:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Jun 2024 16:52:08 GMT
Subject: RFR: 8334769: Shenandoah: Move CodeCache_lock close to its use in
ShenandoahConcurrentNMethodIterator
In-Reply-To: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
References: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
Message-ID: <NUW-nXZED1eUvxoA3havXxPF-12AVuyttDoQsDrfGQo=.d6a3351a-1ef0-45e1-a34b-f907d24fb995@github.com>
On Mon, 24 Jun 2024 13:46:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> ShenandoahConcurrentNMethodIterator requires CodeCache_lock at begin and end of iteration.
>
> Currently, the CodeCache_lock is acquired by their callers, which is not very obvious and readable. Let's move the lock inside the methods.
Marked as reviewed by wkemper (Committer).
Looks good to me.
-------------
PR Review: https://git.openjdk.org/jdk/pull/19859#pullrequestreview-2142452683
PR Comment: https://git.openjdk.org/jdk/pull/19859#issuecomment-2192193004
From kdnilsen at openjdk.org Wed Jun 26 17:01:15 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 26 Jun 2024 17:01:15 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
Message-ID: <LjdSksbf5yoPYdadGI469ObtM0MqiptZC-rNH6x8INE=.ddc44cfb-8d71-43f6-b292-68a61562bc6f@github.com>
On Tue, 25 Jun 2024 08:20:43 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Simplify code with less stacks
Thanks for doing this fix.
-------------
Marked as reviewed by kdnilsen (Author).
PR Review: https://git.openjdk.org/jdk/pull/19570#pullrequestreview-2142476247
From kdnilsen at openjdk.org Wed Jun 26 17:02:09 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 26 Jun 2024 17:02:09 GMT
Subject: RFR: 8334769: Shenandoah: Move CodeCache_lock close to its use in
ShenandoahConcurrentNMethodIterator
In-Reply-To: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
References: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
Message-ID: <pblSHH8mCxNH9jWK2z3SbEeuGadi3cvuZ1JILfooUKg=.da8a9df0-d515-4c97-9363-241de32a1830@github.com>
On Mon, 24 Jun 2024 13:46:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> ShenandoahConcurrentNMethodIterator requires CodeCache_lock at begin and end of iteration.
>
> Currently, the CodeCache_lock is acquired by their callers, which is not very obvious and readable. Let's move the lock inside the methods.
Thanks.
-------------
Marked as reviewed by kdnilsen (Author).
PR Review: https://git.openjdk.org/jdk/pull/19859#pullrequestreview-2142481062
From wkemper at openjdk.org Wed Jun 26 17:07:14 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Jun 2024 17:07:14 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
Message-ID: <w7LTJachy0SEPvijLtyFL3RIaNuSr0RgzWiJWQN2JrA=.0f87f0fc-ecbf-4c0f-b5a2-76235d2fd51d@github.com>
On Tue, 25 Jun 2024 08:20:43 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Simplify code with less stacks
Are we trying to minimize TTSP or SP Polls? Does SP mean Safepoint or SpinPause here? If we're trying to minimize TTSP, The result table suggests 0x0F would be better?
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19570#issuecomment-2192226292
From xpeng at openjdk.org Wed Jun 26 17:22:10 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Jun 2024 17:22:10 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <w7LTJachy0SEPvijLtyFL3RIaNuSr0RgzWiJWQN2JrA=.0f87f0fc-ecbf-4c0f-b5a2-76235d2fd51d@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
<w7LTJachy0SEPvijLtyFL3RIaNuSr0RgzWiJWQN2JrA=.0f87f0fc-ecbf-4c0f-b5a2-76235d2fd51d@github.com>
Message-ID: <isjZrIjkoo2-einQGJZRKnKDtGov7lEKK4p3OcHtkiE=.cf6d9f29-3096-4f22-9b17-fd86a317e38c@github.com>
On Wed, 26 Jun 2024 17:05:01 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Are we trying to minimize TTSP or SP Polls? Does SP mean Safepoint or SpinPause here? If we're trying to minimize TTSP, The result table suggests 0x0F would be better?
Thank you William for looking reviewing this.
SP means Safepoint here.
I have discussed the reason for the choice in the description. Yes lower spin pauses does give much better result for the designed scenarios w/ TLAB disabled causing extremely heavily contended heap lock, less than 0xFF spin pauses seem to be overly optimized and causing regression for the regular/normal case in which TLAB is likely/always enabled.
We also observed some regressions in the metrics from shenandoah's test farm with 0x1F spin pauses, e.g. Spring latency, which is gone after I increased the spin pauses to 0xFF.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19570#issuecomment-2192249413
From wkemper at openjdk.org Wed Jun 26 17:29:12 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Jun 2024 17:29:12 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
Message-ID: <hhOPngKXYWLv_4U07fBrgofamoqQRgUsxLFz2haTjjE=.d5d82a27-b265-4d8d-9535-de50cb621154@github.com>
On Tue, 25 Jun 2024 08:20:43 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Simplify code with less stacks
Okay. I don't understand the data in the third table for: `java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:+UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java`. Are those values also SP Polls? or are they GC counts? At any rate, isn't lower better? or did you chose 0xFF because it showed the best performance on dacapo testing?
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19570#issuecomment-2192260401
From kdnilsen at openjdk.org Wed Jun 26 17:59:17 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 26 Jun 2024 17:59:17 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
Message-ID: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
1. Throw OOM after failed allocation request following a Full GC (rather
than retrying as long as Full GC makes good progress because
repeatedly retrying the allocation request creates brown-out behavior
with no identified benefits on real-world workloads)
2. Count a successful allocation following a blocking
handle_allocation_failure() request to be good GC progress.
Otherwise, we increment gc_no_progress_count in full GCs that
have bad progress but successful allocations, and this causes
unwanted failure to even try a full GC in a different thread after
an out-of-memory condition might have been resolved in this thread.
3. Count a completed concurrent GC cycle as good progress, regardless
of how much memory it might have been able to reclaim. The fact that
concurrent GC succeeded without allocation failure and without
degeneration is considered good progress. Successful concurrent
GCs between Full GCs will reset the gc_no_progress_count to zero.
4. Do not count degenerated cycles as having no-progress. If a
degenerated cycle has no progress, it will upgrade to full GC.
The upgraded full GC will evaluate its own progress. We don't
want to count this "same [upgraded] cycle" twice.
These changes have been tested over a variety of workloads and standard tests. These changes have also been tested with the generational mode of Shenandoah. It appears these changes provide more robust and consistent handling across a diversity of scenarios than the original implementation.
-------------
Commit messages:
- Improve OOM handling
- Merge branch 'openjdk:master' into master
- Revert "Make GC logging less verbose"
- Make GC logging less verbose
- Merge branch 'openjdk:master' into master
- Merge branch 'openjdk:master' into master
- Merge branch 'openjdk:master' into master
- Merge branch 'openjdk:master' into master
- Merge branch 'openjdk:master' into master
- Merge branch 'openjdk:master' into master
- ... and 10 more: https://git.openjdk.org/jdk/compare/c66f785f...439d394c
Changes: https://git.openjdk.org/jdk/pull/19912/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19912&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8335126
Stats: 47 lines in 3 files changed: 27 ins; 1 del; 19 mod
Patch: https://git.openjdk.org/jdk/pull/19912.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19912/head:pull/19912
PR: https://git.openjdk.org/jdk/pull/19912
From kdnilsen at openjdk.org Wed Jun 26 17:59:18 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 26 Jun 2024 17:59:18 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
Message-ID: <V-D6wiIs0dNxqwoNYhgRDjU_MAD3k4Iuwku_TzmsCKY=.c5634031-83f6-4b2a-a0ef-b953073e08ad@github.com>
On Wed, 26 Jun 2024 17:51:36 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
> 1. Throw OOM after failed allocation request following a Full GC (rather
> than retrying as long as Full GC makes good progress because
> repeatedly retrying the allocation request creates brown-out behavior
> with no identified benefits on real-world workloads)
>
> 2. Count a successful allocation following a blocking
> handle_allocation_failure() request to be good GC progress.
> Otherwise, we increment gc_no_progress_count in full GCs that
> have bad progress but successful allocations, and this causes
> unwanted failure to even try a full GC in a different thread after
> an out-of-memory condition might have been resolved in this thread.
>
> 3. Count a completed concurrent GC cycle as good progress, regardless
> of how much memory it might have been able to reclaim. The fact that
> concurrent GC succeeded without allocation failure and without
> degeneration is considered good progress. Successful concurrent
> GCs between Full GCs will reset the gc_no_progress_count to zero.
>
> 4. Do not count degenerated cycles as having no-progress. If a
> degenerated cycle has no progress, it will upgrade to full GC.
> The upgraded full GC will evaluate its own progress. We don't
> want to count this "same [upgraded] cycle" twice.
>
> These changes have been tested over a variety of workloads and standard tests. These changes have also been tested with the generational mode of Shenandoah. It appears these changes provide more robust and consistent handling across a diversity of scenarios than the original implementation.
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 969:
> 967: // So the second thread experiences OOMError even through another GC would have reclaimed the memory it wanted
> 968: // to allocate.
> 969: // 2. A GLOBAL GC won't necessarily reclaim all garbage. Following a concurrent Generational GLOBAL GC, we may
Hmmm. I let Generational concepts slip into this comment. Maybe harmless. Maybe inappropriate.
Reviewers: feel free to ask that I remove this mention...
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19912#discussion_r1655305604
From xpeng at openjdk.org Wed Jun 26 18:00:14 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Jun 2024 18:00:14 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <hhOPngKXYWLv_4U07fBrgofamoqQRgUsxLFz2haTjjE=.d5d82a27-b265-4d8d-9535-de50cb621154@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
<hhOPngKXYWLv_4U07fBrgofamoqQRgUsxLFz2haTjjE=.d5d82a27-b265-4d8d-9535-de50cb621154@github.com>
Message-ID: <fkMgyRVLONM2xRDol9U-6afkYFn1DqZlvoXK_SJwHKc=.d372434e-41a4-4bdb-ba4b-19fa275e9eb0@github.com>
On Wed, 26 Jun 2024 17:26:47 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Okay. I don't understand the data in the third table for: `java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:+UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java`. Are those values also SP Polls? or are they GC counts? At any rate, isn't lower better? or did you chose 0xFF because it showed the best performance on dacapo testing?
These are GC counts I collected from gc log, run the same test code with command line java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:+UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java , with TLAB enabled, so they are close to real world cases.
Basically I was using GC count as a proxy of allocation rate, same test within same amount of time, more GCs counts, better throughput and performance.
The test indicates less than 0xFF spin pauses causes regression in throughput except the designed test to mimic extreme contention we ran in the pull request.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19570#issuecomment-2192311250
From wkemper at openjdk.org Wed Jun 26 18:09:13 2024
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 26 Jun 2024 18:09:13 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
Message-ID: <6GGaL73H3vuWAjxX07mpjwC0SAHgbEJxWJy3BUFKhb0=.c8bddea5-950c-41d7-ae2f-56a55550477a@github.com>
On Tue, 25 Jun 2024 08:20:43 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Simplify code with less stacks
Marked as reviewed by wkemper (Committer).
Got it, thank you!
-------------
PR Review: https://git.openjdk.org/jdk/pull/19570#pullrequestreview-2142617151
PR Comment: https://git.openjdk.org/jdk/pull/19570#issuecomment-2192325686
From xpeng at openjdk.org Wed Jun 26 19:22:11 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Jun 2024 19:22:11 GMT
Subject: RFR: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock [v7]
In-Reply-To: <KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
<KPUn9jn_NPrXlohBqd-CjZVI6bwvwaG5HJvOn7mNCKw=.dc0c5dc5-4212-44de-acc5-c994cf9a2446@github.com>
Message-ID: <domniHCsV-XGtMy2VVdYeVcHhTZDZAubh2KG2WvrRSo=.a35f72b0-e5a0-4868-94b9-041eb6859861@github.com>
On Tue, 25 Jun 2024 08:20:43 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> ### Notes
>> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>>
>> #### Test code
>>
>> public class Alloc {
>> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
>> static final Object[] sinks = new Object[64*THREADS];
>> static volatile boolean start;
>> static volatile boolean stop;
>>
>> public static void main(String... args) throws Throwable {
>> for (int t = 0; t < THREADS; t++) {
>> int ft = t;
>> new Thread(() -> work(ft * 64)).start();
>> }
>>
>> Thread.sleep(1000);
>> start = true;
>> Thread.sleep(30_000);
>> stop = true;
>> }
>>
>> public static void work(int idx) {
>> while (!start) { Thread.onSpinWait(); }
>> while (!stop) {
>> sinks[idx] = new byte[128];
>> }
>> }
>> }
>>
>>
>> Run it like this and observe TTSP times:
>>
>>
>> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>>
>>
>> #### Metrics from tests(TTSP, allocation rate)
>> ##### Heavy contention(1280 threads, 32 per CPU core)
>> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
>> | -------- | -------- | ------------ | ----------- | -------- | ----- |
>> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
>> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
>> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
>> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
>> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
>> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
>> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
>> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
>> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>>
>>
>> ##### Light contention(40 threads, 1 per CPU core)
>> | Spins | SP polls | Average TTSP | 2% T...
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Simplify code with less stacks
Thank you all for the review and suggestions, I'll start integration.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19570#issuecomment-2192463897
From xpeng at openjdk.org Wed Jun 26 19:28:14 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Jun 2024 19:28:14 GMT
Subject: Integrated: 8331411: Shenandoah: Reconsider spinning duration in
ShenandoahLock
In-Reply-To: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
References: <hiB50h80rds91qKQ5QSeNRjv3K00DKeaGKXX62PAMuc=.0de98e1a-7d1d-44a1-8f73-b634602f9776@github.com>
Message-ID: <vljkhKGys7DV2XbvpIEvG9baJuNzM0MVO4-M0CY9t-E=.b0fcc79c-75c8-4451-aea9-d106561116ab@github.com>
On Wed, 5 Jun 2024 23:44:48 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
> ### Notes
> While doing CAS to get the lock, original implementation sleep/yield once after spinning 0xFFF times, and do these over and over again until get the lock successfully, it is like ```(N spins + sleep/yield) loop ```, based on test results, it seems doing more spins results in worse performance, we decided to change the algorithm to ```(N spins) + (yield loop)```, meanwhile block thread immediately if Safepoint is pending. But still need to determine the best N value for spins, tested multiple possible values: 0, 0x01, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF, and compare the results with the baseline data(original implementation).
>
> #### Test code
>
> public class Alloc {
> static final int THREADS = 1280; //32 threads per CPU core, 40 cores
> static final Object[] sinks = new Object[64*THREADS];
> static volatile boolean start;
> static volatile boolean stop;
>
> public static void main(String... args) throws Throwable {
> for (int t = 0; t < THREADS; t++) {
> int ft = t;
> new Thread(() -> work(ft * 64)).start();
> }
>
> Thread.sleep(1000);
> start = true;
> Thread.sleep(30_000);
> stop = true;
> }
>
> public static void work(int idx) {
> while (!start) { Thread.onSpinWait(); }
> while (!stop) {
> sinks[idx] = new byte[128];
> }
> }
> }
>
>
> Run it like this and observe TTSP times:
>
>
> java -Xms256m -Xmx256m -XX:+UseShenandoahGC -XX:-UseTLAB -Xlog:gc -Xlog:safepoint Alloc.java
>
>
> #### Metrics from tests(TTSP, allocation rate)
> ##### Heavy contention(1280 threads, 32 per CPU core)
> | Test | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | -------- | ----- |
> | baseline | 18 | 3882361 | 3882361 | 43310117 | 49197 |
> | 0x00 | 168 | 861677 | 589036 | 46937732 | 44005 |
> | 0x01 | 164 | 627056 | 572697 | 10004767 | 55472 |
> | 0x07 | 163 | 650578 | 625329 | 5312631 | 53734 |
> | 0x0F | 164 | 590398 | 557325 | 6481761 | 56794 |
> | 0x1F | 144 | 814400 | 790089 | 5024881 | 56041 |
> | 0x3F | 137 | 830288 | 801192 | 5533538 | 54982 |
> | 0x7F | 132 | 1101625 | 845626 | 35425614 | 57492 |
> | 0xFF | 125 | 1005433 | 970988 | 6193342 | 54362 |
>
>
> ##### Light contention(40 threads, 1 per CPU core)
> | Spins | SP polls | Average TTSP | 2% TRIMMEAN | MAX | MIN |
> | -------- | -------- | ------------ | ----------- | ------- | ---- |
> | baseline | ...
This pull request has now been integrated.
Changeset: 817edcb6
Author: Xiaolong Peng <xpeng at openjdk.org>
Committer: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/817edcb697cbb8c608c9292cdc4b99db4f5844dc
Stats: 47 lines in 2 files changed: 22 ins; 11 del; 14 mod
8331411: Shenandoah: Reconsider spinning duration in ShenandoahLock
Reviewed-by: shade, kdnilsen, wkemper
-------------
PR: https://git.openjdk.org/jdk/pull/19570
From ysr at openjdk.org Wed Jun 26 20:22:10 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Jun 2024 20:22:10 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
Message-ID: <oe5sQkzNeuZttxwtxx79i4nhndo8biEcsYZNRj2lT3I=.13be581e-f4fd-4b39-a183-1111d618cbaf@github.com>
On Wed, 26 Jun 2024 17:51:36 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
> 1. Throw OOM after failed allocation request following a Full GC (rather
> than retrying as long as Full GC makes good progress because
> repeatedly retrying the allocation request creates brown-out behavior
> with no identified benefits on real-world workloads)
>
> 2. Count a successful allocation following a blocking
> handle_allocation_failure() request to be good GC progress.
> Otherwise, we increment gc_no_progress_count in full GCs that
> have bad progress but successful allocations, and this causes
> unwanted failure to even try a full GC in a different thread after
> an out-of-memory condition might have been resolved in this thread.
>
> 3. Count a completed concurrent GC cycle as good progress, regardless
> of how much memory it might have been able to reclaim. The fact that
> concurrent GC succeeded without allocation failure and without
> degeneration is considered good progress. Successful concurrent
> GCs between Full GCs will reset the gc_no_progress_count to zero.
>
> 4. Do not count degenerated cycles as having no-progress. If a
> degenerated cycle has no progress, it will upgrade to full GC.
> The upgraded full GC will evaluate its own progress. We don't
> want to count this "same [upgraded] cycle" twice.
>
> These changes have been tested over a variety of workloads and standard tests. These changes have also been tested with the generational mode of Shenandoah. It appears these changes provide more robust and consistent handling across a diversity of scenarios than the original implementation.
Re yr PR description, are the terms "good progress", "bad progress" and "no progress" described precisely somewhere in the documentation?
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19912#issuecomment-2192555502
From zgu at openjdk.org Wed Jun 26 20:27:13 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Wed, 26 Jun 2024 20:27:13 GMT
Subject: RFR: 8334769: Shenandoah: Move CodeCache_lock close to its use in
ShenandoahConcurrentNMethodIterator
In-Reply-To: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
References: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
Message-ID: <iLf_knX6ACwN85c68pkUuzO3zpaP9Ge7EbHRqqDa0og=.fdf2a654-cb1c-4676-9486-e26af7eae4ba@github.com>
On Mon, 24 Jun 2024 13:46:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> ShenandoahConcurrentNMethodIterator requires CodeCache_lock at begin and end of iteration.
>
> Currently, the CodeCache_lock is acquired by their callers, which is not very obvious and readable. Let's move the lock inside the methods.
Thanks, @shipilev @earthling-amzn and @kdnilsen
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19859#issuecomment-2192562583
From zgu at openjdk.org Wed Jun 26 20:27:13 2024
From: zgu at openjdk.org (Zhengyu Gu)
Date: Wed, 26 Jun 2024 20:27:13 GMT
Subject: Integrated: 8334769: Shenandoah: Move CodeCache_lock close to its use
in ShenandoahConcurrentNMethodIterator
In-Reply-To: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
References: <At48QD_UTo2BRzkjZRXsw4oyd106FpHQw4ksPRCOrZo=.cd05b8f8-371d-401a-9a5b-8dd1740e5872@github.com>
Message-ID: <Aa_qTvXrtbG9A1WiByEEyFBKS59fNRW7fziiOvOraEQ=.06009d81-405d-431a-bd60-ffba7c153fbe@github.com>
On Mon, 24 Jun 2024 13:46:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> ShenandoahConcurrentNMethodIterator requires CodeCache_lock at begin and end of iteration.
>
> Currently, the CodeCache_lock is acquired by their callers, which is not very obvious and readable. Let's move the lock inside the methods.
This pull request has now been integrated.
Changeset: 4ebb7712
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/4ebb77120af5a4ccbfde63b24cb50e05a3161f16
Stats: 13 lines in 3 files changed: 3 ins; 8 del; 2 mod
8334769: Shenandoah: Move CodeCache_lock close to its use in ShenandoahConcurrentNMethodIterator
Reviewed-by: shade, wkemper, kdnilsen
-------------
PR: https://git.openjdk.org/jdk/pull/19859
From ysr at openjdk.org Wed Jun 26 20:30:09 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Jun 2024 20:30:09 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <oe5sQkzNeuZttxwtxx79i4nhndo8biEcsYZNRj2lT3I=.13be581e-f4fd-4b39-a183-1111d618cbaf@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
<oe5sQkzNeuZttxwtxx79i4nhndo8biEcsYZNRj2lT3I=.13be581e-f4fd-4b39-a183-1111d618cbaf@github.com>
Message-ID: <f8mocJO_S6adGiaIkX7pUi51eSZDMNvjUtZjsReEuwE=.bbf1ad31-bb01-4d01-b8bd-8230a335b98a@github.com>
On Wed, 26 Jun 2024 20:19:25 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
> Re yr PR description, are the terms "good progress", "bad progress" and "no progress" described precisely somewhere in the documentation?
To me it sounded like the intention is:
1. good progress: all's good, we are swimming, no need to worry
2. bad progress: things may not be going well, but there is some progress and we need to be watchful
3. no progress: we have been in a bad progress state for too long, and we want to give up and throw an OOM because we are more or less stuck and moving too slowly.
Is that the sense in which these terms are being used in yr PR description? Is something like that written down somewhere in the documentation comments? Thanks!
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19912#issuecomment-2192568117
From xpeng at openjdk.org Wed Jun 26 22:45:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Wed, 26 Jun 2024 22:45:20 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
Message-ID: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Hi all,
This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
Basically it moves the ShenandoahHeapLocker into ShenandoahFreeSet::log_status and only acquire heap lock when log is actually enabled. Since the lock is not a reentrant lock, re-arrangement of code related to ShenandoahFreeSet::rebuild is needed.
The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine.
Additional test:
- [x] `make test TEST=hotspot_gc_shenandoah`
Test summary
==============================
TEST TOTAL PASS FAIL ERROR
jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
==============================
TEST SUCCESS
Best,
Xiaolong.
-------------
Commit messages:
- Add shenandoah_assert_not_heaplocked
- 8334147: Shenandoah: Avoid taking lock for free set logging
- Move free_set()->log_status() out of ShenandoahFullGC::phase5_epilog
- code format
- Restructure code of ShenandoahFreeSet::finish_rebuild to move log_status out of it to avoid re-entering heap lock
- Shenandoah: Avoid taking lock for free set logging
Changes: https://git.openjdk.org/jdk/pull/19915/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8334147
Stats: 27 lines in 5 files changed: 6 ins; 10 del; 11 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From ysr at openjdk.org Wed Jun 26 23:28:09 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Jun 2024 23:28:09 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
Message-ID: <7IsBGRT5ClFO6hJAv-B-BIPpPmYjNQbWZzk2t2YgsY4=.c885efb2-f519-48d3-8c64-acb555d4ead0@github.com>
On Wed, 26 Jun 2024 17:51:36 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
> 1. Throw OOM after failed allocation request following a Full GC (rather
> than retrying as long as Full GC makes good progress because
> repeatedly retrying the allocation request creates brown-out behavior
> with no identified benefits on real-world workloads)
>
> 2. Count a successful allocation following a blocking
> handle_allocation_failure() request to be good GC progress.
> Otherwise, we increment gc_no_progress_count in full GCs that
> have bad progress but successful allocations, and this causes
> unwanted failure to even try a full GC in a different thread after
> an out-of-memory condition might have been resolved in this thread.
>
> 3. Count a completed concurrent GC cycle as good progress, regardless
> of how much memory it might have been able to reclaim. The fact that
> concurrent GC succeeded without allocation failure and without
> degeneration is considered good progress. Successful concurrent
> GCs between Full GCs will reset the gc_no_progress_count to zero.
>
> 4. Do not count degenerated cycles as having no-progress. If a
> degenerated cycle has no progress, it will upgrade to full GC.
> The upgraded full GC will evaluate its own progress. We don't
> want to count this "same [upgraded] cycle" twice.
>
> These changes have been tested over a variety of workloads and standard tests. These changes have also been tested with the generational mode of Shenandoah. It appears these changes provide more robust and consistent handling across a diversity of scenarios than the original implementation.
Reviewed. The code changes look good to me, so I am approving this.
Will discuss the terminology questions I raised earlier in our discussion tomorrow.
I think we want to move some of the big comments out of line and into a block comment at the start of this method. We also want to clearly document the intent of the terms "good_progress", "no_progress", etc. somewhere in the code, probably in the header files for the methods that are named `*_progress_*`.
-------------
Marked as reviewed by ysr (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19912#pullrequestreview-2143124067
From ysr at openjdk.org Wed Jun 26 23:28:10 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 26 Jun 2024 23:28:10 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <V-D6wiIs0dNxqwoNYhgRDjU_MAD3k4Iuwku_TzmsCKY=.c5634031-83f6-4b2a-a0ef-b953073e08ad@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
<V-D6wiIs0dNxqwoNYhgRDjU_MAD3k4Iuwku_TzmsCKY=.c5634031-83f6-4b2a-a0ef-b953073e08ad@github.com>
Message-ID: <Q5KZ4yjvPzeYBC3TODmBrv3woB3wiZ00ygtP15vbI9I=.b05be9f7-9ef4-4f6e-b497-09dda5dd7da0@github.com>
On Wed, 26 Jun 2024 17:56:28 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> 1. Throw OOM after failed allocation request following a Full GC (rather
>> than retrying as long as Full GC makes good progress because
>> repeatedly retrying the allocation request creates brown-out behavior
>> with no identified benefits on real-world workloads)
>>
>> 2. Count a successful allocation following a blocking
>> handle_allocation_failure() request to be good GC progress.
>> Otherwise, we increment gc_no_progress_count in full GCs that
>> have bad progress but successful allocations, and this causes
>> unwanted failure to even try a full GC in a different thread after
>> an out-of-memory condition might have been resolved in this thread.
>>
>> 3. Count a completed concurrent GC cycle as good progress, regardless
>> of how much memory it might have been able to reclaim. The fact that
>> concurrent GC succeeded without allocation failure and without
>> degeneration is considered good progress. Successful concurrent
>> GCs between Full GCs will reset the gc_no_progress_count to zero.
>>
>> 4. Do not count degenerated cycles as having no-progress. If a
>> degenerated cycle has no progress, it will upgrade to full GC.
>> The upgraded full GC will evaluate its own progress. We don't
>> want to count this "same [upgraded] cycle" twice.
>>
>> These changes have been tested over a variety of workloads and standard tests. These changes have also been tested with the generational mode of Shenandoah. It appears these changes provide more robust and consistent handling across a diversity of scenarios than the original implementation.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 969:
>
>> 967: // So the second thread experiences OOMError even through another GC would have reclaimed the memory it wanted
>> 968: // to allocate.
>> 969: // 2. A GLOBAL GC won't necessarily reclaim all garbage. Following a concurrent Generational GLOBAL GC, we may
>
> Hmmm. I let Generational concepts slip into this comment. Maybe harmless. Maybe inappropriate.
> Reviewers: feel free to ask that I remove this mention...
I think this is OK for now because we expect to integrate generational shenandoah soon, but am fine with defering it to GenShen integration in the future if other reviewers feel that would be better.
One question I had was about the policy for the treatment/reclamation of soft references, and whether that changes and/or escalates to "clear all" upon reaching a "less than good progress" state prior to an OOM (reflecting the best efforts promise prior to OOM).
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19912#discussion_r1655626477
From ysr at openjdk.org Thu Jun 27 01:24:08 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Jun 2024 01:24:08 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <aOhuQkveeX1BTAuYkKkarXGO7V3eEhzOPhxmGG7C0Bw=.bed69f11-b1d9-442b-857a-97fd80fff496@github.com>
On Wed, 26 Jun 2024 21:57:54 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it moves the ShenandoahHeapLocker into ShenandoahFreeSet::log_status and only acquire heap lock when log is actually enabled. Since the lock is not a reentrant lock, re-arrangement of code related to ShenandoahFreeSet::rebuild is needed.
>
> The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine.
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
This _may be_ cleaner and would largely avoid relocating the calls in some cases like you did; _may be_ ....
void log_status() {
if (logging enabled) {
lock heap;
log_status_work_with_lock();
}
}
where the (now) private work method `log_status_work_with_lock()` asserts that logging is enabled and lock is held, then does the logging work without checking if logging is enabled like the old `log_status()` did.
For the other callers who call from within contexts where the lock is held, you can have a variant that calls `log_status_with_lock()` which does:
void log_status_with_lock() {
assert that heap lock is held;
if (logging enabled) {
log_status_work_with_lock();
}
}
Not sure if this is worthwhile though.
Will let others more familiar with Shenandoah code opine/advise on that.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2192876959
From xpeng at openjdk.org Thu Jun 27 02:07:41 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 02:07:41 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v2]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <kxu3HKCmzcXuPT4rnCE3I3xx7yC1ml9LdA3MkkkHLzM=.7f657a69-8549-4761-a343-c4f07f4ad021@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it moves the ShenandoahHeapLocker into ShenandoahFreeSet::log_status and only acquire heap lock when log is actually enabled. Since the lock is not a reentrant lock, re-arrangement of code related to ShenandoahFreeSet::rebuild is needed.
>
> The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine.
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
Add method log_status_with_heap_lock and make log_status private
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/597ded38..a8c07f96
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=01
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=00-01
Stats: 26 lines in 6 files changed: 16 ins; 5 del; 5 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From xpeng at openjdk.org Thu Jun 27 02:14:41 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 02:14:41 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v3]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <DvSWxLvtOdGtKR8jytwOrwXoEmpcapDWN-I5Jebd7v0=.79286ae4-9ef5-426a-863e-ec3beff0566e@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
Missed one log
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/a8c07f96..d794e364
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=02
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=01-02
Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From xpeng at openjdk.org Thu Jun 27 02:23:36 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 02:23:36 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v4]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <Uzdj_6GLPbQDrHB9WKaGS4LljL72HHThwcfDznh9bK8=.3ff0836e-83c9-4285-9740-8be97419f40e@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
- code format
- Renaming
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/d794e364..e6895d60
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=03
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=02-03
Stats: 10 lines in 4 files changed: 0 ins; 1 del; 9 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From xpeng at openjdk.org Thu Jun 27 02:28:14 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 02:28:14 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v4]
In-Reply-To: <Uzdj_6GLPbQDrHB9WKaGS4LljL72HHThwcfDznh9bK8=.3ff0836e-83c9-4285-9740-8be97419f40e@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Uzdj_6GLPbQDrHB9WKaGS4LljL72HHThwcfDznh9bK8=.3ff0836e-83c9-4285-9740-8be97419f40e@github.com>
Message-ID: <VdCj6fjA20RaWurlxLTLBfr5XvVgHgX_SKAGPxObi7Q=.6770ac94-ae90-4f68-98c4-331e00a3eff7@github.com>
On Thu, 27 Jun 2024 02:23:36 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - code format
> - Renaming
Thank you Ramki for taking time to review and provide suggestion.
Yes, it is definitely much cleaner if we create another private method to print the logs with the assumption that log is already acquired, and it won't have the issue I mentioned.
I have updated the PR, only difference is naming, I named the private method ```log_status_with_heap_locked```.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2192941242
From xpeng at openjdk.org Thu Jun 27 04:12:09 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 04:12:09 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v4]
In-Reply-To: <Uzdj_6GLPbQDrHB9WKaGS4LljL72HHThwcfDznh9bK8=.3ff0836e-83c9-4285-9740-8be97419f40e@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Uzdj_6GLPbQDrHB9WKaGS4LljL72HHThwcfDznh9bK8=.3ff0836e-83c9-4285-9740-8be97419f40e@github.com>
Message-ID: <zblmB2PF6K9DIAa63razo9Mo119Z4NKN9EMz-uIAOm4=.92dd61d4-b759-42f2-b17e-0fd27f55be1f@github.com>
On Thu, 27 Jun 2024 02:23:36 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - code format
> - Renaming
Reran hotspot_gc_shenandoah test:
Test summary
==============================
TEST TOTAL PASS FAIL ERROR
jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
==============================
TEST SUCCESS
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2193599350
From xpeng at openjdk.org Thu Jun 27 05:35:42 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 05:35:42 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v5]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <eVuWRAxAGKUKXsijr6hvzirIWvRosVGL1EyArS-N6Pc=.26050ce4-5ca8-4950-a260-4295e5079d18@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:
- Merge branch 'openjdk:master' into JDK-8334147
- code format
- Renaming
- Missed one log
- Add method log_status_with_heap_lock and make log_status private
- Add shenandoah_assert_not_heaplocked
- 8334147: Shenandoah: Avoid taking lock for free set logging
- Move free_set()->log_status() out of ShenandoahFullGC::phase5_epilog
- code format
- Restructure code of ShenandoahFreeSet::finish_rebuild to move log_status out of it to avoid re-entering heap lock
- ... and 1 more: https://git.openjdk.org/jdk/compare/78c43e3e...8ea36214
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/e6895d60..8ea36214
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=04
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=03-04
Stats: 1378 lines in 20 files changed: 1016 ins; 215 del; 147 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From xpeng at openjdk.org Thu Jun 27 07:37:36 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 07:37:36 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
Make log_status_with_heap_locked public
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/8ea36214..d9073c3e
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=05
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=04-05
Stats: 3 lines in 1 file changed: 1 ins; 2 del; 0 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From xpeng at openjdk.org Thu Jun 27 07:43:10 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 07:43:10 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
Message-ID: <5xfbU4dk9uH7DjvFzuWeqFXcJD8aMdCuP_wctq_G75Q=.41c96800-5aa9-484d-89f8-bbac106f0778@github.com>
On Thu, 27 Jun 2024 07:37:36 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Make log_status_with_heap_locked public
There is one test failure which I can't really understand how it is possible, but making log_status_with_heap_locked public instead private seems working, making it even weirder.
compiler.interpreter.Test6833129.java on linux-x86
Error: [0.582s][error][gc,verify] Object 0xa6d80010 has a null klass
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xf452b5c7, pid=12043, tid=12066
#
# JRE version: OpenJDK Runtime Environment (24.0) (fastdebug build 24-internal-pengxiaolong-8ea36214a538412f1a304ce0968c74ac120a5b45)
# Java VM: OpenJDK Server VM (fastdebug 24-internal-pengxiaolong-8ea36214a538412f1a304ce0968c74ac120a5b45, mixed mode, sharing, tiered, g1 gc, linux-x86)
# Problematic frame:
# V [libjvm.so+0x52b5c7] oopDesc::size_given_klass(Klass*)+0x17
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2194010253
From shade at openjdk.org Thu Jun 27 08:00:11 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 08:00:11 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <5xfbU4dk9uH7DjvFzuWeqFXcJD8aMdCuP_wctq_G75Q=.41c96800-5aa9-484d-89f8-bbac106f0778@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
<5xfbU4dk9uH7DjvFzuWeqFXcJD8aMdCuP_wctq_G75Q=.41c96800-5aa9-484d-89f8-bbac106f0778@github.com>
Message-ID: <my8ZywmdcZDG9ZAV1bS5IiuhZKy7U_L62JVjC_jFCd4=.be6beff6-a996-4224-a629-f6144e759975@github.com>
On Thu, 27 Jun 2024 07:40:25 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
> Java VM: OpenJDK Server VM (fastdebug 24-internal-pengxiaolong-8ea36214a538412f1a304ce0968c74ac120a5b45, mixed mode, sharing, tiered, g1 gc, linux-x86)
Note "g1 gc", this is a G1 bug, not Shenandoah one. We should still bisect that one and report it.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2194040118
From shade at openjdk.org Thu Jun 27 08:26:11 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 08:26:11 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
Message-ID: <xQegh_cDKff89O8trhKvFiqivh1fcJhu2LR81BMtddE=.b01e87d0-59a8-48c4-a8f2-f7d5d7f10a83@github.com>
On Thu, 27 Jun 2024 07:37:36 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Make log_status_with_heap_locked public
It looks good to me, with minor nits. I also added "disabled" to the issue synopsis to reflect that we really disabling the lock taking when logging is not requested.
I agree pulling `update_capacity_and_used_at_gc` and `record_whole_heap_examined_timestamp` out of the heap-locked region is sane.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1139:
> 1137: #ifdef ASSERT
> 1138: || LogTarget(Debug, gc, free)::is_enabled()
> 1139: #endif
OK, I see it matches what `log_status_with_heap_locked` is already doing.
For future reference, a shorter variant of this would be:
if (LogTarget(Info, gc, free)::is_enabled() DEBUG_ONLY(|| LogTarget(Debug, gc, free)::is_enabled()) {
...but this variant is also good.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 344:
> 342: void recycle_trash();
> 343: void log_status();
> 344: void log_status_with_heap_locked();
Suggestion: this can be just `log_status_under_lock`. We already have other methods with `*_under_lock`.
-------------
Marked as reviewed by shade (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19915#pullrequestreview-2144624252
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1656686636
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1656689076
From wkemper at openjdk.org Thu Jun 27 14:21:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Jun 2024 14:21:37 GMT
Subject: RFR: Merge openjdk/jdk21u-dev:master
Message-ID: <giboUA0EXRDsi8yOPN2DZQeP3hlnXS11U6O55VwjIK8=.d7a07eb2-0d7c-486d-9aca-314e0d00f01f@github.com>
Merges tag jdk-21.0.4+6
-------------
Commit messages:
- 8334441: Mark tests in jdk_security_infra group as manual
The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/61/files
Stats: 156 lines in 8 files changed: 5 ins; 0 del; 151 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/61.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/61/head:pull/61
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/61
From xpeng at openjdk.org Thu Jun 27 15:40:36 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 15:40:36 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v7]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <fPun0-xX8Bc9hGpjjDQNM8TqHovqNn9BNjNTO58FXUc=.eae2006d-3e07-41bf-a746-e8d2c19c74fb@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
Polish the code a little bit
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/d9073c3e..2747696b
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=06
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=05-06
Stats: 8 lines in 2 files changed: 0 ins; 3 del; 5 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From shade at openjdk.org Thu Jun 27 15:48:12 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 15:48:12 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v7]
In-Reply-To: <fPun0-xX8Bc9hGpjjDQNM8TqHovqNn9BNjNTO58FXUc=.eae2006d-3e07-41bf-a746-e8d2c19c74fb@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<fPun0-xX8Bc9hGpjjDQNM8TqHovqNn9BNjNTO58FXUc=.eae2006d-3e07-41bf-a746-e8d2c19c74fb@github.com>
Message-ID: <VWl9kLC0M9wqbym2zz3u2R06IxvUDxGV3gxXhYslTTU=.0d8bf3c8-6186-4b04-8a13-da05640e4c3f@github.com>
On Thu, 27 Jun 2024 15:40:36 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Polish the code a little bit
Marked as reviewed by shade (Reviewer).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19915#pullrequestreview-2145792218
From shade at openjdk.org Thu Jun 27 15:48:13 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 15:48:13 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <my8ZywmdcZDG9ZAV1bS5IiuhZKy7U_L62JVjC_jFCd4=.be6beff6-a996-4224-a629-f6144e759975@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
<5xfbU4dk9uH7DjvFzuWeqFXcJD8aMdCuP_wctq_G75Q=.41c96800-5aa9-484d-89f8-bbac106f0778@github.com>
<my8ZywmdcZDG9ZAV1bS5IiuhZKy7U_L62JVjC_jFCd4=.be6beff6-a996-4224-a629-f6144e759975@github.com>
Message-ID: <W3qP9tSbAWXo8BVervi8gnxTY0aS-eBsfEHsmIXEpDI=.a04feb38-a9cd-4032-8082-ed6af8829070@github.com>
On Thu, 27 Jun 2024 07:57:50 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Note "g1 gc", this is a G1 bug, not Shenandoah one. We should still bisect that one and report it.
Just caught it separately, I'll follow up myself.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195055502
From xpeng at openjdk.org Thu Jun 27 15:48:14 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 15:48:14 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <xQegh_cDKff89O8trhKvFiqivh1fcJhu2LR81BMtddE=.b01e87d0-59a8-48c4-a8f2-f7d5d7f10a83@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
<xQegh_cDKff89O8trhKvFiqivh1fcJhu2LR81BMtddE=.b01e87d0-59a8-48c4-a8f2-f7d5d7f10a83@github.com>
Message-ID: <vgc9iS_Qwc1TORSw_qRv-qK8QgKQQL5MOk7InUFUqKs=.786bbc68-38c0-48a3-8c6b-fdcd684c4b60@github.com>
On Thu, 27 Jun 2024 08:22:54 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Make log_status_with_heap_locked public
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 344:
>
>> 342: void recycle_trash();
>> 343: void log_status();
>> 344: void log_status_with_heap_locked();
>
> Suggestion: this can be just `log_status_under_lock`. We already have other methods with `*_under_lock`.
Checked some code in this name pattern:
HeapWord* ShenandoahHeap::allocate_memory_under_lock(ShenandoahAllocRequest& req, bool& in_new_region) {
// If we are dealing with mutator allocation, then we may need to block for safepoint.
// We cannot block for safepoint for GC allocations, because there is a high chance
// we are already running at safepoint or from stack watermark machinery, and we cannot
// block again.
ShenandoahHeapLocker locker(lock(), req.is_mutator_alloc());
return _free_set->allocate(req, in_new_region);
}
It assumes lock is not acquired, lock is acquired in the method, I can change it to this pattern to align the naming conversions/patterns, but won't be a simple renaming.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657366587
From shade at openjdk.org Thu Jun 27 15:48:14 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 15:48:14 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <vgc9iS_Qwc1TORSw_qRv-qK8QgKQQL5MOk7InUFUqKs=.786bbc68-38c0-48a3-8c6b-fdcd684c4b60@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
<xQegh_cDKff89O8trhKvFiqivh1fcJhu2LR81BMtddE=.b01e87d0-59a8-48c4-a8f2-f7d5d7f10a83@github.com>
<vgc9iS_Qwc1TORSw_qRv-qK8QgKQQL5MOk7InUFUqKs=.786bbc68-38c0-48a3-8c6b-fdcd684c4b60@github.com>
Message-ID: <zn16cdpOvV4Dtiy6QrVbhRhiDHhxu0ZvIVaPI1or-8Q=.6148cb96-2eb3-4e4b-a98d-260d85d80887@github.com>
On Thu, 27 Jun 2024 15:43:07 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 344:
>>
>>> 342: void recycle_trash();
>>> 343: void log_status();
>>> 344: void log_status_with_heap_locked();
>>
>> Suggestion: this can be just `log_status_under_lock`. We already have other methods with `*_under_lock`.
>
> Checked some code in this name pattern:
>
> HeapWord* ShenandoahHeap::allocate_memory_under_lock(ShenandoahAllocRequest& req, bool& in_new_region) {
> // If we are dealing with mutator allocation, then we may need to block for safepoint.
> // We cannot block for safepoint for GC allocations, because there is a high chance
> // we are already running at safepoint or from stack watermark machinery, and we cannot
> // block again.
> ShenandoahHeapLocker locker(lock(), req.is_mutator_alloc());
> return _free_set->allocate(req, in_new_region);
> }
>
>
> It assumes lock is not acquired, lock is acquired in the method, I can change it to this pattern to align the naming conversions/patterns, but won't be a simple renaming.
Ah right. Okay, nevermind, whatever works.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657369504
From xpeng at openjdk.org Thu Jun 27 15:55:35 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 15:55:35 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v8]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
- Accidentally removed a empty line
- Method renaming
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/2747696b..3044aec2
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=07
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=06-07
Stats: 8 lines in 3 files changed: 1 ins; 0 del; 7 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From xpeng at openjdk.org Thu Jun 27 15:55:36 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 15:55:36 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <xQegh_cDKff89O8trhKvFiqivh1fcJhu2LR81BMtddE=.b01e87d0-59a8-48c4-a8f2-f7d5d7f10a83@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
<xQegh_cDKff89O8trhKvFiqivh1fcJhu2LR81BMtddE=.b01e87d0-59a8-48c4-a8f2-f7d5d7f10a83@github.com>
Message-ID: <9hXJSaU1YWR7_5q505Gr9T-fKH4YhbvElGRkvNodhUo=.c145135c-b888-4861-92f0-a068815ea0bb@github.com>
On Thu, 27 Jun 2024 08:21:12 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Make log_status_with_heap_locked public
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1139:
>
>> 1137: #ifdef ASSERT
>> 1138: || LogTarget(Debug, gc, free)::is_enabled()
>> 1139: #endif
>
> OK, I see it matches what `log_status_with_heap_locked` is already doing.
>
> For future reference, a shorter variant of this would be:
>
>
> if (LogTarget(Info, gc, free)::is_enabled() DEBUG_ONLY(|| LogTarget(Debug, gc, free)::is_enabled()) {
>
>
> ...but this variant is also good.
Updated the code to shorter variant.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657385526
From xpeng at openjdk.org Thu Jun 27 15:55:36 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 15:55:36 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <zn16cdpOvV4Dtiy6QrVbhRhiDHhxu0ZvIVaPI1or-8Q=.6148cb96-2eb3-4e4b-a98d-260d85d80887@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
<xQegh_cDKff89O8trhKvFiqivh1fcJhu2LR81BMtddE=.b01e87d0-59a8-48c4-a8f2-f7d5d7f10a83@github.com>
<vgc9iS_Qwc1TORSw_qRv-qK8QgKQQL5MOk7InUFUqKs=.786bbc68-38c0-48a3-8c6b-fdcd684c4b60@github.com>
<zn16cdpOvV4Dtiy6QrVbhRhiDHhxu0ZvIVaPI1or-8Q=.6148cb96-2eb3-4e4b-a98d-260d85d80887@github.com>
Message-ID: <Yz1o4-k2nAhUadarsapnQdnvJzwZZtn30ZmotZQ1iKc=.2a0f9dcf-5635-4a94-8721-fc9de0beeb44@github.com>
On Thu, 27 Jun 2024 15:45:18 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Checked some code in this name pattern:
>>
>> HeapWord* ShenandoahHeap::allocate_memory_under_lock(ShenandoahAllocRequest& req, bool& in_new_region) {
>> // If we are dealing with mutator allocation, then we may need to block for safepoint.
>> // We cannot block for safepoint for GC allocations, because there is a high chance
>> // we are already running at safepoint or from stack watermark machinery, and we cannot
>> // block again.
>> ShenandoahHeapLocker locker(lock(), req.is_mutator_alloc());
>> return _free_set->allocate(req, in_new_region);
>> }
>>
>>
>> It assumes lock is not acquired, lock is acquired in the method, I can change it to this pattern to align the naming conversions/patterns, but won't be a simple renaming.
>
> Ah right. Okay, nevermind, whatever works.
I have fixed it, thanks!
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657383519
From xpeng at openjdk.org Thu Jun 27 15:58:11 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 15:58:11 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <W3qP9tSbAWXo8BVervi8gnxTY0aS-eBsfEHsmIXEpDI=.a04feb38-a9cd-4032-8082-ed6af8829070@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
<5xfbU4dk9uH7DjvFzuWeqFXcJD8aMdCuP_wctq_G75Q=.41c96800-5aa9-484d-89f8-bbac106f0778@github.com>
<my8ZywmdcZDG9ZAV1bS5IiuhZKy7U_L62JVjC_jFCd4=.be6beff6-a996-4224-a629-f6144e759975@github.com>
<W3qP9tSbAWXo8BVervi8gnxTY0aS-eBsfEHsmIXEpDI=.a04feb38-a9cd-4032-8082-ed6af8829070@github.com>
Message-ID: <DP0wWyemYRwHRRprOxxCwnbLBu2JzsdebtEp4fXAbeM=.93957b21-c718-4d4c-a84f-ac2d84c5d67c@github.com>
On Thu, 27 Jun 2024 15:45:38 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> > Note "g1 gc", this is a G1 bug, not Shenandoah one. We should still bisect that one and report it.
>
> Just caught it separately, I'll follow up myself.
Have you already created bug for it? Just want to watch on it and see what is the root cause, didn't get chance to dig into it, but it seem super weird.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195085998
From shade at openjdk.org Thu Jun 27 16:07:12 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 16:07:12 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for free set logging
[v6]
In-Reply-To: <DP0wWyemYRwHRRprOxxCwnbLBu2JzsdebtEp4fXAbeM=.93957b21-c718-4d4c-a84f-ac2d84c5d67c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<Hun0VYhmjs-y6TnAaoRSc1uMLAiWAqofl4Oc7YXRqCw=.e452bfe2-5622-44a7-8efc-f25eb07f82f7@github.com>
<5xfbU4dk9uH7DjvFzuWeqFXcJD8aMdCuP_wctq_G75Q=.41c96800-5aa9-484d-89f8-bbac106f0778@github.com>
<my8ZywmdcZDG9ZAV1bS5IiuhZKy7U_L62JVjC_jFCd4=.be6beff6-a996-4224-a629-f6144e759975@github.com>
<W3qP9tSbAWXo8BVervi8gnxTY0aS-eBsfEHsmIXEpDI=.a04feb38-a9cd-4032-8082-ed6af8829070@github.com>
<DP0wWyemYRwHRRprOxxCwnbLBu2JzsdebtEp4fXAbeM=.93957b21-c718-4d4c-a84f-ac2d84c5d67c@github.com>
Message-ID: <JkVZJQwAVwdcVcEbAcicNWgpsv3y8qMq7i8Z8j6pzwQ=.69d0a22d-d2b3-4278-9d32-daadcb3308aa@github.com>
On Thu, 27 Jun 2024 15:55:25 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
> Have you already created bug for it? Just want to watch on it and see what is the root cause, didn't get chance to dig into it, but it seem super weird.
https://bugs.openjdk.org/browse/JDK-8335266
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195102870
From shade at openjdk.org Thu Jun 27 16:17:12 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 16:17:12 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <yMzhLKAVC7LU_lFHTtgULPKxHjnHOwf722uwoPALGc4=.a5559f9e-f5d5-469c-8dbe-d4b92ab7684f@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
This looks fine too.
-------------
Marked as reviewed by shade (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19915#pullrequestreview-2145890260
From xpeng at openjdk.org Thu Jun 27 16:36:11 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 16:36:11 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <xG_VgKXV0ro92CezHbkz0HaiOJcih-0plvJ5ocnlv64=.e804821a-0e51-4a19-8c9c-a2f3194341f7@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
Thank you Ramki and Aleksey!
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195174633
From shade at openjdk.org Thu Jun 27 16:36:12 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 16:36:12 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <egwRCryezSh9oUIt7270JyovgHz4ItoedA8-0UzAZzo=.db372fe5-dc60-49fd-8899-3dc71cd49d87@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
As long as Ramki actually reviews... :)
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195177648
From shade at openjdk.org Thu Jun 27 17:00:12 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 17:00:12 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <l2yNCt28lIMMV0mU4FscN4JJGMlsTO39PjLMGhttw1E=.2fe3febb-813f-4f6f-bac5-78b839a568c7@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
Also, this reminds me we should add the test that actually goes in with `-Xlog:gc+free=info` to see if we fail any of the asserts or break anything _before_ we actually enable the logging in prod. I suggest a simple test: just an empty main, but run with `-XX:ShenandoahGCHeuristics=aggressive`. Put it here: https://github.com/openjdk/jdk/tree/master/test/hotspot/jtreg/gc/shenandoah/options
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195218434
From duke at openjdk.org Thu Jun 27 17:12:43 2024
From: duke at openjdk.org (duke)
Date: Thu, 27 Jun 2024 17:12:43 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <-VNwa-ybRjqfqtBERZFTh7VwYldUwLPrvxSuIoY9jkA=.c1ea515f-e7d5-410b-9fe0-2e7f1d08e7ac@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
@pengxiaolong
Your change (at version 3044aec2962baa5718cef2818bdca93aea295824) is now ready to be sponsored by a Committer.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195175891
From wkemper at openjdk.org Thu Jun 27 18:32:22 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Jun 2024 18:32:22 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
Message-ID: <FV-uuJWWMaGnjfOne7_zXBtrp1A8lGF0kjJvAQZTsv8=.4b738590-003e-43cd-a15f-c4320f908b18@github.com>
On Wed, 26 Jun 2024 17:51:36 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
> 1. Throw OOM after failed allocation request following a Full GC (rather
> than retrying as long as Full GC makes good progress because
> repeatedly retrying the allocation request creates brown-out behavior
> with no identified benefits on real-world workloads)
>
> 2. Count a successful allocation following a blocking
> handle_allocation_failure() request to be good GC progress.
> Otherwise, we increment gc_no_progress_count in full GCs that
> have bad progress but successful allocations, and this causes
> unwanted failure to even try a full GC in a different thread after
> an out-of-memory condition might have been resolved in this thread.
>
> 3. Count a completed concurrent GC cycle as good progress, regardless
> of how much memory it might have been able to reclaim. The fact that
> concurrent GC succeeded without allocation failure and without
> degeneration is considered good progress. Successful concurrent
> GCs between Full GCs will reset the gc_no_progress_count to zero.
>
> 4. Do not count degenerated cycles as having no-progress. If a
> degenerated cycle has no progress, it will upgrade to full GC.
> The upgraded full GC will evaluate its own progress. We don't
> want to count this "same [upgraded] cycle" twice.
>
> These changes have been tested over a variety of workloads and standard tests. These changes have also been tested with the generational mode of Shenandoah. It appears these changes provide more robust and consistent handling across a diversity of scenarios than the original implementation.
Marked as reviewed by wkemper (Committer).
-------------
PR Review: https://git.openjdk.org/jdk/pull/19912#pullrequestreview-2146215417
From xpeng at openjdk.org Thu Jun 27 18:45:19 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 18:45:19 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <zvseW_IdcMHAWGvMSx5vkGY9Bh7H3oQGjJTp9B87wgg=.0d689678-272c-4112-b9d8-f413c7631d4f@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
I have run test ```make test TEST=jtreg/gc/shenandoah/TestWithLogLevel.java```, verified the GC log, I can find log like this:
[0.188s][info][gc,free ] Free: 667M, Max: 512K regular, 667M humongous, Frag: 0% external, 0% internal; Used: 0B, Mutator Free: 1334 Collector Reserve: 52736K, Max: 512K; Used: 0B
[0.188s][info][gc,start ] GC(0) Concurrent reset
[0.188s][info][gc,task ] GC(0) Using 3 of 6 workers for concurrent reset
[0.188s][info][gc,ergo ] GC(0) Pacer for Reset. Non-Taxable: 1024M
[0.192s][info][gc ] GC(0) Concurrent reset 4.207ms
[0.197s][info][gc,start ] GC(0) Pause Init Mark (unload classes)
[0.197s][info][gc,task ] GC(0) Using 6 of 6 workers for init marking
[0.198s][info][gc,ergo ] GC(0) Pacer for Mark. Expected Live: 102M, Free: 667M, Non-Taxable: 68300K, Alloc Tax Rate: 0.2x
[0.198s][info][gc ] GC(0) Pause Init Mark (unload classes) 1.247ms
[0.198s][info][gc,start ] GC(0) Concurrent marking roots
[0.198s][info][gc,task ] GC(0) Using 3 of 6 workers for concurrent marking roots
[0.198s][info][gc ] GC(0) Concurrent marking roots 0.171ms
[0.198s][info][gc,start ] GC(0) Concurrent marking (unload classes)
[0.198s][info][gc,task ] GC(0) Using 3 of 6 workers for concurrent marking
[0.293s][info][gc ] Cancelling GC: Stopping VM
[0.566s][info][gc ] GC(0) Concurrent marking (unload classes) 367.435ms
[0.566s][info][gc,free ] Free: 427M, Max: 512K regular, 427M humongous, Frag: 0% external, 0% internal; Used: 0B, Mutator Free: 854 Collector Reserve: 52736K, Max: 512K; Used: 0B
We don't need to add another test for it.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195445625
From shade at openjdk.org Thu Jun 27 18:49:20 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 18:49:20 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging
In-Reply-To: <aOhuQkveeX1BTAuYkKkarXGO7V3eEhzOPhxmGG7C0Bw=.bed69f11-b1d9-442b-857a-97fd80fff496@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<aOhuQkveeX1BTAuYkKkarXGO7V3eEhzOPhxmGG7C0Bw=.bed69f11-b1d9-442b-857a-97fd80fff496@github.com>
Message-ID: <PWfBmKg_RojRfT0tN72G5gEBRuvdF6tAk_9Ypv5g5-c=.d232e2bd-5714-438c-9961-2ccc70b17ef0@github.com>
On Thu, 27 Jun 2024 01:18:06 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> This _may be_ cleaner and would largely avoid relocating the calls in some cases like you did; _may be_ ....
>
>
> void log_status() {
> if (logging enabled) {
> lock heap;
> log_status_work_with_lock();
> }
> }
>
>
> where the (now) private work method `log_status_work_with_lock()` asserts that logging is enabled and lock is held, then does the logging work without checking if logging is enabled like the old `log_status()` did.
>
> For the other callers who call from within contexts where the lock is held, you can have a variant that calls `log_status_with_lock()` which does:
>
>
> void log_status_with_lock() {
> assert that heap lock is held;
> if (logging enabled) {
> log_status_work_with_lock();
> }
> }
>
>
> Not sure if this is worthwhile though.
>
> Will let others more familiar with Shenandoah code opine/advise on that.
@ysramakrishna, please approve and sponsor?
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195453115
From ysr at openjdk.org Thu Jun 27 18:53:19 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Jun 2024 18:53:19 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <Q5KZ4yjvPzeYBC3TODmBrv3woB3wiZ00ygtP15vbI9I=.b05be9f7-9ef4-4f6e-b497-09dda5dd7da0@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
<V-D6wiIs0dNxqwoNYhgRDjU_MAD3k4Iuwku_TzmsCKY=.c5634031-83f6-4b2a-a0ef-b953073e08ad@github.com>
<Q5KZ4yjvPzeYBC3TODmBrv3woB3wiZ00ygtP15vbI9I=.b05be9f7-9ef4-4f6e-b497-09dda5dd7da0@github.com>
Message-ID: <8oYCucrhGyGRCZzPbgEI4ISE_gBa2p1-JjBSrOlMuek=.9447e6ad-26bd-4293-b5a7-76a9fd5ea7f5@github.com>
On Wed, 26 Jun 2024 23:19:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 969:
>>
>>> 967: // So the second thread experiences OOMError even through another GC would have reclaimed the memory it wanted
>>> 968: // to allocate.
>>> 969: // 2. A GLOBAL GC won't necessarily reclaim all garbage. Following a concurrent Generational GLOBAL GC, we may
>>
>> Hmmm. I let Generational concepts slip into this comment. Maybe harmless. Maybe inappropriate.
>> Reviewers: feel free to ask that I remove this mention...
>
> I think this is OK for now because we expect to integrate generational shenandoah soon, but am fine with defering it to GenShen integration in the future if other reviewers feel that would be better.
>
> One question I had was about the policy for the treatment/reclamation of soft references, and whether that changes and/or escalates to "clear all" upon reaching a "less than good progress" state prior to an OOM (reflecting the best efforts promise prior to OOM).
My soft ref clearing policy question above was cleared up in the code walkthrough today. We do clear all soft refs upon an allocation failure and therefore, _a fortiori_, before throwing an OOM.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19912#discussion_r1657661550
From kdnilsen at openjdk.org Thu Jun 27 19:04:20 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Jun 2024 19:04:20 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <8oYCucrhGyGRCZzPbgEI4ISE_gBa2p1-JjBSrOlMuek=.9447e6ad-26bd-4293-b5a7-76a9fd5ea7f5@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
<V-D6wiIs0dNxqwoNYhgRDjU_MAD3k4Iuwku_TzmsCKY=.c5634031-83f6-4b2a-a0ef-b953073e08ad@github.com>
<Q5KZ4yjvPzeYBC3TODmBrv3woB3wiZ00ygtP15vbI9I=.b05be9f7-9ef4-4f6e-b497-09dda5dd7da0@github.com>
<8oYCucrhGyGRCZzPbgEI4ISE_gBa2p1-JjBSrOlMuek=.9447e6ad-26bd-4293-b5a7-76a9fd5ea7f5@github.com>
Message-ID: <YLxGeTkUEfzqQZ-pA4Vf-8hBA6IA2TIzEX38UIM1a3U=.dbcaef4d-5b53-45e9-82bf-33285a01b049@github.com>
On Thu, 27 Jun 2024 18:50:28 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> I think this is OK for now because we expect to integrate generational shenandoah soon, but am fine with defering it to GenShen integration in the future if other reviewers feel that would be better.
>>
>> One question I had was about the policy for the treatment/reclamation of soft references, and whether that changes and/or escalates to "clear all" upon reaching a "less than good progress" state prior to an OOM (reflecting the best efforts promise prior to OOM).
>
> My soft ref clearing policy question above was cleared up in the code walkthrough today. We do clear all soft refs upon an allocation failure and therefore, _a fortiori_, before throwing an OOM.
In ShenanoahControlThread::run_service(), we set set_should_clear_all_soft_refs(true) if alloc_failure_pending. This has the effect of expediting soft-ref processing even before we escalate to full gc. ShenandoahFullGC::phase1_mark_heap() also sets soft_reference_policy(true), to "forcefully purge all soft references".
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19912#discussion_r1657678985
From shade at openjdk.org Thu Jun 27 19:09:18 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 19:09:18 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
Message-ID: <45BHtt51LihYveREtNv8vM0saAiwbeqWrSP8cyxm1EM=.39941e81-f71c-4333-b47c-516bb69375fb@github.com>
On Wed, 26 Jun 2024 17:51:36 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
> 1. Throw OOM after failed allocation request following a Full GC (rather
> than retrying as long as Full GC makes good progress because
> repeatedly retrying the allocation request creates brown-out behavior
> with no identified benefits on real-world workloads)
>
> 2. Count a successful allocation following a blocking
> handle_allocation_failure() request to be good GC progress.
> Otherwise, we increment gc_no_progress_count in full GCs that
> have bad progress but successful allocations, and this causes
> unwanted failure to even try a full GC in a different thread after
> an out-of-memory condition might have been resolved in this thread.
>
> 3. Count a completed concurrent GC cycle as good progress, regardless
> of how much memory it might have been able to reclaim. The fact that
> concurrent GC succeeded without allocation failure and without
> degeneration is considered good progress. Successful concurrent
> GCs between Full GCs will reset the gc_no_progress_count to zero.
>
> 4. Do not count degenerated cycles as having no-progress. If a
> degenerated cycle has no progress, it will upgrade to full GC.
> The upgraded full GC will evaluate its own progress. We don't
> want to count this "same [upgraded] cycle" twice.
>
> These changes have been tested over a variety of workloads and standard tests. These changes have also been tested with the generational mode of Shenandoah. It appears these changes provide more robust and consistent handling across a diversity of scenarios than the original implementation.
All right, this makes sense to me, as long as experiments show this is a good idea as well. But I think the comment should be shorter.
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 960:
> 958: // b) We experienced at least one Full GC (whether or not it had good progress)
> 959: //
> 960: // TODO: Rather than require a Full GC before throwing OOMError, it might be more appropriate for handle_alloc_failure()
Pro-tip: If you find yourself writing a large TODO comment, it should probably be transplanted straight into a new issue.
-------------
Marked as reviewed by shade (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19912#pullrequestreview-2146288690
PR Review Comment: https://git.openjdk.org/jdk/pull/19912#discussion_r1657679558
From ysr at openjdk.org Thu Jun 27 20:06:20 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Jun 2024 20:06:20 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging
In-Reply-To: <aOhuQkveeX1BTAuYkKkarXGO7V3eEhzOPhxmGG7C0Bw=.bed69f11-b1d9-442b-857a-97fd80fff496@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<aOhuQkveeX1BTAuYkKkarXGO7V3eEhzOPhxmGG7C0Bw=.bed69f11-b1d9-442b-857a-97fd80fff496@github.com>
Message-ID: <hsAdZheJUfPEcTsA4tNr_Mw77Nhr8znmCRC3GEwHRwo=.73411712-6c43-4dd0-bad1-f738f48c6b06@github.com>
On Thu, 27 Jun 2024 01:18:06 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> This _may be_ cleaner and would largely avoid relocating the calls in some cases like you did; _may be_ ....
>
>
> void log_status() {
> if (logging enabled) {
> lock heap;
> log_status_work_with_lock();
> }
> }
>
>
> where the (now) private work method `log_status_work_with_lock()` asserts that logging is enabled and lock is held, then does the logging work without checking if logging is enabled like the old `log_status()` did.
>
> For the other callers who call from within contexts where the lock is held, you can have a variant that calls `log_status_with_lock()` which does:
>
>
> void log_status_with_lock() {
> assert that heap lock is held;
> if (logging enabled) {
> log_status_work_with_lock();
> }
> }
>
>
> Not sure if this is worthwhile though.
>
> Will let others more familiar with Shenandoah code opine/advise on that.
> @ysramakrishna, please approve and sponsor?
Yes, I'll review and sponsor. Thanks for the reminder!
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195578965
From ysr at openjdk.org Thu Jun 27 20:35:09 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Jun 2024 20:35:09 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v19]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <l-EsCg6_o0IpB4b0psA54A89wLsi5hlhnK1iPD9Eknw=.0a37c061-fa45-44ae-8978-4b93acd0a16f@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 49 commits:
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Revert debugging detritus.
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Code called from LRB cannot rely on SGH::_gc_generation and must instead
use the _active_generation field.
Still need to aduit remaining _gc_generation uses for safety.
Removed too strong assert introduced for debugging in previous commit.
- Merge branch 'master' into active_generation
- Some asserts to catch a tricky race. These assertions may be too strong
in general but would help with debugging a rare crash.
- Merge branch 'master' into active_generation
- ... and 39 more: https://git.openjdk.org/shenandoah/compare/9d2712dd...b12e17cb
-------------
Changes: https://git.openjdk.org/shenandoah/pull/407/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=18
Stats: 166 lines in 16 files changed: 133 ins; 1 del; 32 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From ysr at openjdk.org Thu Jun 27 20:41:19 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Jun 2024 20:41:19 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <LSNTaDu-ipbjBCmUhj2TlzY0MXed4L8w1N14E7QCKsg=.5611f6ef-aec3-43d5-922f-4a454d0b4e9d@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
Looks good, modulo suggestion for documentation line in header file. No need for rereview. If you ping me, I'll sponsor, or @shipilev can.
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 344:
> 342: void recycle_trash();
> 343: void log_status();
> 344: void log_status_under_lock();
Add a brief 1-line documentation comment for each of the methods, mentioning the preconditions & what they do.
-------------
Marked as reviewed by ysr (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19915#pullrequestreview-2146477228
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657789734
From kdnilsen at openjdk.org Thu Jun 27 20:43:50 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 27 Jun 2024 20:43:50 GMT
Subject: RFR: 8327000: GenShen: Integrate updated Shenandoah implementation of
FreeSet into GenShen
Message-ID: <GZrg8aGOHJs3UgWJX72Lr4asgn5Mpyw-Iz77MJ89beo=.f1ec7d2f-484b-4a80-873e-1192600c4538@github.com>
This pull request contains a backport of commit 9d2712dd from the openjdk/shenandoah repository.
The commit being backported was authored by Kelvin Nilsen on 26 Jun 2024 and was reviewed by William Kemper and Y. Srinivas Ramakrishna.
-------------
Commit messages:
- Backport 9d2712ddf39160a618c9c839c46d2510063f45df
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/62/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=62&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8327000
Stats: 2425 lines in 22 files changed: 1531 ins; 263 del; 631 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/62.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/62/head:pull/62
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/62
From xpeng at openjdk.org Thu Jun 27 20:55:50 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 20:55:50 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v9]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <GXpGtWxo8BCKfiioZzX-wYhPHEwTvq8QURIV7oez3qY=.63098fdb-2279-4703-b596-29e12e892176@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request incrementally with three additional commits since the last revision:
- typo
- Fix indenting
- Add one line comments for log_status and log_status_under_lock
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/3044aec2..c3c1c44b
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=08
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=07-08
Stats: 3 lines in 2 files changed: 2 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From shade at openjdk.org Thu Jun 27 20:55:50 2024
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 27 Jun 2024 20:55:50 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <QKsF-B9_6dzslG4CC5RPL-fsiERh4rjrrT1ZP2C3eXA=.cff8b7d2-84b1-4613-921f-bb273a267177@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
Marked as reviewed by shade (Reviewer).
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1139:
> 1137: DEBUG_ONLY(|| LogTarget(Debug, gc, free)::is_enabled())) {
> 1138: ShenandoahHeapLocker locker(_heap->lock());
> 1139: log_status();
If you do the changes, fix up the indenting here as well. I think one space is missing here in 2-space indent.
-------------
PR Review: https://git.openjdk.org/jdk/pull/19915#pullrequestreview-2146487316
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657797499
From ysr at openjdk.org Thu Jun 27 20:55:51 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Jun 2024 20:55:51 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
Message-ID: <EouoCv9Fa5qy7JcueQ2xlUjidR1MAuwjxAD4b0pKTxE=.3bc41e1d-e85a-4969-9b86-8c5567fca192@github.com>
On Thu, 27 Jun 2024 15:55:35 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>
> - Accidentally removed a empty line
> - Method renaming
Another comment.
-------------
PR Review: https://git.openjdk.org/jdk/pull/19915#pullrequestreview-2146486712
From ysr at openjdk.org Thu Jun 27 20:55:51 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Jun 2024 20:55:51 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v9]
In-Reply-To: <GXpGtWxo8BCKfiioZzX-wYhPHEwTvq8QURIV7oez3qY=.63098fdb-2279-4703-b596-29e12e892176@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<GXpGtWxo8BCKfiioZzX-wYhPHEwTvq8QURIV7oez3qY=.63098fdb-2279-4703-b596-29e12e892176@github.com>
Message-ID: <wum_zFc9JcaLDCCpEewGeZdhlFXgbrmqV3agnwwhTq8=.3380677f-2685-4133-a3b5-e7bd8d87ff66@github.com>
On Thu, 27 Jun 2024 20:52:11 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with three additional commits since the last revision:
>
> - typo
> - Fix indenting
> - Add one line comments for log_status and log_status_under_lock
src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 343:
> 341:
> 342: void recycle_trash();
> 343: // log status, assuming lock has alredy been acquired by the caller.
"already". Also see previous comment about removing enablement check in `log_status` and asserting. Will then need to check other callers and retest.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657801256
From ysr at openjdk.org Thu Jun 27 20:55:51 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 27 Jun 2024 20:55:51 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <LSNTaDu-ipbjBCmUhj2TlzY0MXed4L8w1N14E7QCKsg=.5611f6ef-aec3-43d5-922f-4a454d0b4e9d@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
<LSNTaDu-ipbjBCmUhj2TlzY0MXed4L8w1N14E7QCKsg=.5611f6ef-aec3-43d5-922f-4a454d0b4e9d@github.com>
Message-ID: <Zwa1clq7cOD5N-JTuEUXzhmFcAs7xPBWnjUvBsrBJfs=.498f5eca-3b0d-455a-9041-026654eb68d2@github.com>
On Thu, 27 Jun 2024 20:37:12 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Accidentally removed a empty line
>> - Method renaming
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 344:
>
>> 342: void recycle_trash();
>> 343: void log_status();
>> 344: void log_status_under_lock();
>
> Add a brief 1-line documentation comment for each of the methods, mentioning the preconditions & what they do.
I forgot. Shouldn't `log_status` remove the enablement check, and assert that logging is enabled?
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657797139
From xpeng at openjdk.org Thu Jun 27 20:55:51 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 20:55:51 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <QKsF-B9_6dzslG4CC5RPL-fsiERh4rjrrT1ZP2C3eXA=.cff8b7d2-84b1-4613-921f-bb273a267177@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
<QKsF-B9_6dzslG4CC5RPL-fsiERh4rjrrT1ZP2C3eXA=.cff8b7d2-84b1-4613-921f-bb273a267177@github.com>
Message-ID: <IHzROGWPtlo8C9XmukzptdZGtQjfKVuPAJ-IGzZlO3E=.6f1b0cb1-c195-4b6b-b338-f7c99c053c13@github.com>
On Thu, 27 Jun 2024 20:43:50 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Xiaolong Peng has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Accidentally removed a empty line
>> - Method renaming
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1139:
>
>> 1137: DEBUG_ONLY(|| LogTarget(Debug, gc, free)::is_enabled())) {
>> 1138: ShenandoahHeapLocker locker(_heap->lock());
>> 1139: log_status();
>
> If you do the changes, fix up the indenting here as well. I think one space is missing here in 2-space indent.
I thinks is the line 1140 having indent issue, fixed.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657803880
From xpeng at openjdk.org Thu Jun 27 20:58:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 20:58:20 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v9]
In-Reply-To: <wum_zFc9JcaLDCCpEewGeZdhlFXgbrmqV3agnwwhTq8=.3380677f-2685-4133-a3b5-e7bd8d87ff66@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<GXpGtWxo8BCKfiioZzX-wYhPHEwTvq8QURIV7oez3qY=.63098fdb-2279-4703-b596-29e12e892176@github.com>
<wum_zFc9JcaLDCCpEewGeZdhlFXgbrmqV3agnwwhTq8=.3380677f-2685-4133-a3b5-e7bd8d87ff66@github.com>
Message-ID: <VnZhXyRsmoJOnRq6VaMmHzcs_y1f9DaxQpXJTuFALhQ=.1e191282-9b41-4e58-a794-c15e9c603dc6@github.com>
On Thu, 27 Jun 2024 20:47:53 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> Xiaolong Peng has updated the pull request incrementally with three additional commits since the last revision:
>>
>> - typo
>> - Fix indenting
>> - Add one line comments for log_status and log_status_under_lock
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 343:
>
>> 341:
>> 342: void recycle_trash();
>> 343: // log status, assuming lock has alredy been acquired by the caller.
>
> "already". Also see previous comment about removing enablement check in `log_status` and asserting. Will then need to check other callers and retest.
We should keep it, finish_rebuild also call log_status since it already holds the lock, removing enablement check will affect the log behavior in finish_rebuild.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657809001
From xpeng at openjdk.org Thu Jun 27 22:21:47 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 22:21:47 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v10]
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
Make log_status private
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/19915/files
- new: https://git.openjdk.org/jdk/pull/19915/files/c3c1c44b..f944809f
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=09
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=19915&range=08-09
Stats: 5 lines in 1 file changed: 3 ins; 2 del; 0 mod
Patch: https://git.openjdk.org/jdk/pull/19915.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/19915/head:pull/19915
PR: https://git.openjdk.org/jdk/pull/19915
From xpeng at openjdk.org Thu Jun 27 22:32:19 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 22:32:19 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v10]
In-Reply-To: <VnZhXyRsmoJOnRq6VaMmHzcs_y1f9DaxQpXJTuFALhQ=.1e191282-9b41-4e58-a794-c15e9c603dc6@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<GXpGtWxo8BCKfiioZzX-wYhPHEwTvq8QURIV7oez3qY=.63098fdb-2279-4703-b596-29e12e892176@github.com>
<wum_zFc9JcaLDCCpEewGeZdhlFXgbrmqV3agnwwhTq8=.3380677f-2685-4133-a3b5-e7bd8d87ff66@github.com>
<VnZhXyRsmoJOnRq6VaMmHzcs_y1f9DaxQpXJTuFALhQ=.1e191282-9b41-4e58-a794-c15e9c603dc6@github.com>
Message-ID: <zZeVABrjDY2dY9KUIkYCQbhS0JUMBJwx1Hns6VaGHrY=.19fb092b-46c8-490c-9b52-b555d9eae452@github.com>
On Thu, 27 Jun 2024 20:55:44 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 343:
>>
>>> 341:
>>> 342: void recycle_trash();
>>> 343: // log status, assuming lock has alredy been acquired by the caller.
>>
>> "already". Also see previous comment about removing enablement check in `log_status` and asserting. Will then need to check other callers and retest.
>
> We should keep it, finish_rebuild also call log_status since it already holds the lock, removing enablement check will affect the log behavior in finish_rebuild.
I got your point, but we have two enablement checks, one for Info one for Debug, we can't remove it from log_status.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657881647
From xpeng at openjdk.org Thu Jun 27 22:58:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 22:58:20 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v10]
In-Reply-To: <h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
Message-ID: <EQbU_f1FxuYiP4LD2AUAm6a_KKKvL_uOR-pesII1EqU=.8bbfebf5-d86a-4c93-8d94-894055cbc7eb@github.com>
On Thu, 27 Jun 2024 22:21:47 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Make log_status private
Ran shenandoah test again after log_status private:
```
Test summary
==============================
TEST TOTAL PASS FAIL ERROR
jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
==============================
TEST SUCCESS
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2195794535
From xpeng at openjdk.org Thu Jun 27 22:58:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Thu, 27 Jun 2024 22:58:20 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v8]
In-Reply-To: <Zwa1clq7cOD5N-JTuEUXzhmFcAs7xPBWnjUvBsrBJfs=.498f5eca-3b0d-455a-9041-026654eb68d2@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<K7fwbGce3VGHsgbIOYy6_xDPbkfZLNXRzGC52o7Ynpk=.4b4f34e1-d02c-4b46-b4a4-fae9c29ccd3c@github.com>
<LSNTaDu-ipbjBCmUhj2TlzY0MXed4L8w1N14E7QCKsg=.5611f6ef-aec3-43d5-922f-4a454d0b4e9d@github.com>
<Zwa1clq7cOD5N-JTuEUXzhmFcAs7xPBWnjUvBsrBJfs=.498f5eca-3b0d-455a-9041-026654eb68d2@github.com>
Message-ID: <e5Dz8RtIsombTgkoJwMoTD0FEi5f39HQGr7Ngl6IYT8=.88fa80e6-a3a3-4a99-99b5-e68daff3e1c0@github.com>
On Thu, 27 Jun 2024 20:43:24 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp line 344:
>>
>>> 342: void recycle_trash();
>>> 343: void log_status();
>>> 344: void log_status_under_lock();
>>
>> Add a brief 1-line documentation comment for each of the methods, mentioning the preconditions & what they do.
>
> I forgot. Shouldn't `log_status` remove the enablement check, and assert that logging is enabled?
Discussed it in another conversation.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19915#discussion_r1657900503
From wkemper at openjdk.org Thu Jun 27 23:06:59 2024
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 27 Jun 2024 23:06:59 GMT
Subject: RFR: 8335289: GenShen: Whitebox breakpoint GC requests may cause
assertions
Message-ID: <Jpr9OhH2-G-jZRRVjiHPiaaZBAPxfZg9OJL6Arz-uTU=.85c16410-fc87-48da-b19e-c7164264c326@github.com>
When a test requests a concurrent GC breakpoint, the calling thread arranges for itself to block until the concurrent GC thread notifies it that the GC has reached the requested breakpoint (phase). The code that handles the whitebox breakpoint request should therefore not block the caller. An attempt was made to do this, but the request just has the caller thread run in a busy loop without waiting. What's more, this loop resets the requested gc cause on every iteration, which may lead to gc cycles with a wb_breakpoint cause, but no breakpoint set - which violates assertions.
-------------
Commit messages:
- Do not busy loop when handling whitebox breakpoint request
Changes: https://git.openjdk.org/shenandoah/pull/453/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=453&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8335289
Stats: 26 lines in 2 files changed: 20 ins; 4 del; 2 mod
Patch: https://git.openjdk.org/shenandoah/pull/453.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/453/head:pull/453
PR: https://git.openjdk.org/shenandoah/pull/453
From ysr at openjdk.org Fri Jun 28 03:34:22 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 28 Jun 2024 03:34:22 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v10]
In-Reply-To: <h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
Message-ID: <i8PInE3AZPLDa7JTzQmCYeyREuP3oAW2kV_FC4j36Co=.0a3b22eb-d553-48e6-a7bf-a88163841a59@github.com>
On Thu, 27 Jun 2024 22:21:47 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Make log_status private
LGTM.
Looks like your GHA testing tickled pre-existing a G1 and a Shenandoah bug each, but those can't be caused by your changes.
Reviewed. ?
I or Aleksey can /sponsor once you integrate. Sorry for the delay in getting this done sooner.
-------------
Marked as reviewed by ysr (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19915#pullrequestreview-2146954778
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2196048620
From xpeng at openjdk.org Fri Jun 28 03:38:20 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 28 Jun 2024 03:38:20 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v10]
In-Reply-To: <h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
Message-ID: <hNC13Q8LWTT8ynMmQ0sXRwteqEFVWSoA7DbnMHreGNo=.79737f9a-c13b-461e-af64-635f836c84bb@github.com>
On Thu, 27 Jun 2024 22:21:47 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Make log_status private
No worries! Thank you Ramki! I also noticed the bug in Shenandoah, will create a bug in system for it!
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2196066052
From duke at openjdk.org Fri Jun 28 03:38:20 2024
From: duke at openjdk.org (duke)
Date: Fri, 28 Jun 2024 03:38:20 GMT
Subject: RFR: 8334147: Shenandoah: Avoid taking lock for disabled free set
logging [v10]
In-Reply-To: <h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
<h_52gen-e695tx-8rOuSHl-Pa867AmT47_gNH0t1vG0=.c8c0863a-9f03-45ec-910f-e5c65879b649@github.com>
Message-ID: <5c6iYVaDCt2RvODNrHVqLMXN92KkO9UsFCpvAQGxhXQ=.0df5788d-8ec3-44e8-9d9a-2453ce29feba@github.com>
On Thu, 27 Jun 2024 22:21:47 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
>> Hi all,
>> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>>
>>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>>
>> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>>
>> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>>
>> Additional test:
>> - [x] `make test TEST=hotspot_gc_shenandoah`
>>
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
>> ==============================
>> TEST SUCCESS
>>
>>
>> Best,
>> Xiaolong.
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
>
> Make log_status private
@pengxiaolong
Your change (at version f944809fa40f5f6260b2890a1eb7862adf64ca71) is now ready to be sponsored by a Committer.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/19915#issuecomment-2196068516
From xpeng at openjdk.org Fri Jun 28 06:22:25 2024
From: xpeng at openjdk.org (Xiaolong Peng)
Date: Fri, 28 Jun 2024 06:22:25 GMT
Subject: Integrated: 8334147: Shenandoah: Avoid taking lock for disabled free
set logging
In-Reply-To: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
References: <XtpyoA_ftdjR4wgxHvTiAXgQlN-QCnHucQ00Tpcgyhs=.78efab05-19bc-4a0a-a89b-51fbbbb5b29f@github.com>
Message-ID: <Tz5OUEZs16K9VX2SYGzcM0xPNye-W9Xt_tD8VVJxSzI=.21a0b3df-c26b-45e0-9e5c-109c2b3d833f@github.com>
On Wed, 26 Jun 2024 21:57:54 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:
> Hi all,
> This pull request propose a fix for the issue https://bugs.openjdk.org/browse/JDK-8334147
>
>> There are multiple places in Shenandoah where we take heap lock for potential free set diagnostics. There is no point in taking that lock if we do not report anything. We should at very least take the lock only when logging is actually needed.
>
> Basically it adds a public method ```ShenandoahFreeSet::log_status_with_heap_lock``` which acquire lock when there is need to print logs, also make the ```log_status``` private since not expect it to be called out of ShenandoahFreeSet class.
>
> <s>The change for ShenandoahFreeSet::rebuild is probably debatable, ShenandoahFreeSet::log_status will acquire lock again after ShenandoahFreeSet::rebuild is executed, hence the metrics/information in log may not be always consistent, but it might be fine. </s>
>
> Additional test:
> - [x] `make test TEST=hotspot_gc_shenandoah`
>
> Test summary
> ==============================
> TEST TOTAL PASS FAIL ERROR
> jtreg:test/hotspot/jtreg:hotspot_gc_shenandoah 259 259 0 0
> ==============================
> TEST SUCCESS
>
>
> Best,
> Xiaolong.
This pull request has now been integrated.
Changeset: c47a0e00
Author: Xiaolong Peng <xpeng at openjdk.org>
Committer: Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/c47a0e005e54551e42ee1ae33d7169417a5f86d4
Stats: 33 lines in 4 files changed: 14 ins; 9 del; 10 mod
8334147: Shenandoah: Avoid taking lock for disabled free set logging
Reviewed-by: shade, ysr
-------------
PR: https://git.openjdk.org/jdk/pull/19915
From rkennke at openjdk.org Fri Jun 28 14:41:27 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 28 Jun 2024 14:41:27 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <YLxGeTkUEfzqQZ-pA4Vf-8hBA6IA2TIzEX38UIM1a3U=.dbcaef4d-5b53-45e9-82bf-33285a01b049@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
<V-D6wiIs0dNxqwoNYhgRDjU_MAD3k4Iuwku_TzmsCKY=.c5634031-83f6-4b2a-a0ef-b953073e08ad@github.com>
<Q5KZ4yjvPzeYBC3TODmBrv3woB3wiZ00ygtP15vbI9I=.b05be9f7-9ef4-4f6e-b497-09dda5dd7da0@github.com>
<8oYCucrhGyGRCZzPbgEI4ISE_gBa2p1-JjBSrOlMuek=.9447e6ad-26bd-4293-b5a7-76a9fd5ea7f5@github.com>
<YLxGeTkUEfzqQZ-pA4Vf-8hBA6IA2TIzEX38UIM1a3U=.dbcaef4d-5b53-45e9-82bf-33285a01b049@github.com>
Message-ID: <SE-_gmtEsT3L30kq2thUQKLT6aRQXNMii3wi5KrgeRE=.fc453cc0-8b82-4db0-8453-4b11e0ccd0b1@github.com>
On Thu, 27 Jun 2024 19:01:30 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
>> My soft ref clearing policy question above was cleared up in the code walkthrough today. We do clear all soft refs upon an allocation failure and therefore, _a fortiori_, before throwing an OOM.
>
> In ShenanoahControlThread::run_service(), we set set_should_clear_all_soft_refs(true) if alloc_failure_pending. This has the effect of expediting soft-ref processing even before we escalate to full gc. ShenandoahFullGC::phase1_mark_heap() also sets soft_reference_policy(true), to "forcefully purge all soft references".
I think it is ok because we expect generational Shenandoah Real Soon Now. ;-)
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/19912#discussion_r1658859347
From rkennke at openjdk.org Fri Jun 28 14:41:26 2024
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 28 Jun 2024 14:41:26 GMT
Subject: RFR: 8335126: Shenandoah: Improve OOM handling
In-Reply-To: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
References: <rvYIY7TB1AprxPCM762Jvv7I-2uzCE7no3OEBvp9w48=.e206a517-b895-4675-9f65-b43dd19b79dc@github.com>
Message-ID: <uGIbnoJOio2AsDo6mU2WGwWkcoR8BwzpWP9wE_AuleY=.9df4688f-e98d-4737-a604-00abe7acd831@github.com>
On Wed, 26 Jun 2024 17:51:36 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:
> 1. Throw OOM after failed allocation request following a Full GC (rather
> than retrying as long as Full GC makes good progress because
> repeatedly retrying the allocation request creates brown-out behavior
> with no identified benefits on real-world workloads)
>
> 2. Count a successful allocation following a blocking
> handle_allocation_failure() request to be good GC progress.
> Otherwise, we increment gc_no_progress_count in full GCs that
> have bad progress but successful allocations, and this causes
> unwanted failure to even try a full GC in a different thread after
> an out-of-memory condition might have been resolved in this thread.
>
> 3. Count a completed concurrent GC cycle as good progress, regardless
> of how much memory it might have been able to reclaim. The fact that
> concurrent GC succeeded without allocation failure and without
> degeneration is considered good progress. Successful concurrent
> GCs between Full GCs will reset the gc_no_progress_count to zero.
>
> 4. Do not count degenerated cycles as having no-progress. If a
> degenerated cycle has no progress, it will upgrade to full GC.
> The upgraded full GC will evaluate its own progress. We don't
> want to count this "same [upgraded] cycle" twice.
>
> These changes have been tested over a variety of workloads and standard tests. These changes have also been tested with the generational mode of Shenandoah. It appears these changes provide more robust and consistent handling across a diversity of scenarios than the original implementation.
Looks ok to me.
-------------
Marked as reviewed by rkennke (Reviewer).
PR Review: https://git.openjdk.org/jdk/pull/19912#pullrequestreview-2148214783
From kdnilsen at openjdk.org Fri Jun 28 16:25:33 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 28 Jun 2024 16:25:33 GMT
Subject: RFR: 8335289: GenShen: Whitebox breakpoint GC requests may cause
assertions
In-Reply-To: <Jpr9OhH2-G-jZRRVjiHPiaaZBAPxfZg9OJL6Arz-uTU=.85c16410-fc87-48da-b19e-c7164264c326@github.com>
References: <Jpr9OhH2-G-jZRRVjiHPiaaZBAPxfZg9OJL6Arz-uTU=.85c16410-fc87-48da-b19e-c7164264c326@github.com>
Message-ID: <2XAHMhDMgIq2yFn9KptXaBc9fe2pNCy2bgwk3pMc2as=.b0d20117-9b11-479a-8355-7db9f1462e6d@github.com>
On Thu, 27 Jun 2024 23:02:41 GMT, William Kemper <wkemper at openjdk.org> wrote:
> When a test requests a concurrent GC breakpoint, the calling thread arranges for itself to block until the concurrent GC thread notifies it that the GC has reached the requested breakpoint (phase). The code that handles the whitebox breakpoint request should therefore not block the caller. An attempt was made to do this, but the request just has the caller thread run in a busy loop without waiting. What's more, this loop resets the requested gc cause on every iteration, which may lead to gc cycles with a wb_breakpoint cause, but no breakpoint set - which violates assertions.
Marked as reviewed by kdnilsen (Committer).
-------------
PR Review: https://git.openjdk.org/shenandoah/pull/453#pullrequestreview-2148427119
From wkemper at openjdk.org Fri Jun 28 16:26:58 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Jun 2024 16:26:58 GMT
Subject: RFR: 8335347: GenShen: Revert change that has adaptive heuristic
ignore abbreviated cycles
Message-ID: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
Additional performance testing has shown no benefit from this feature. In fact, latency is degraded on the extremem benchmark with this feature enabled.
-------------
Commit messages:
- Merge remote-tracking branch 'shenandoah/master' into revert-ignore-short-cycles
- Do not ignore abbreviated cycles
Changes: https://git.openjdk.org/shenandoah/pull/455/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=455&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8335347
Stats: 26 lines in 10 files changed: 1 ins; 14 del; 11 mod
Patch: https://git.openjdk.org/shenandoah/pull/455.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/455/head:pull/455
PR: https://git.openjdk.org/shenandoah/pull/455
From kdnilsen at openjdk.org Fri Jun 28 16:36:46 2024
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 28 Jun 2024 16:36:46 GMT
Subject: RFR: 8335347: GenShen: Revert change that has adaptive heuristic
ignore abbreviated cycles
In-Reply-To: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
References: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
Message-ID: <_Dkbss2wmpuS0Tf4EFlj8MyHaHzyBWcgwX7s7Xh3Ono=.899685c7-4e0c-4a17-b5e3-fd4276e4f8fb@github.com>
On Fri, 28 Jun 2024 16:22:03 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Additional performance testing has shown no benefit from this feature. In fact, latency is degraded on the extremem benchmark with this feature enabled.
Thanks.
-------------
Marked as reviewed by kdnilsen (Committer).
PR Review: https://git.openjdk.org/shenandoah/pull/455#pullrequestreview-2148457407
From wkemper at openjdk.org Fri Jun 28 18:13:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Jun 2024 18:13:37 GMT
Subject: Integrated: 8335289: GenShen: Whitebox breakpoint GC requests may
cause assertions
In-Reply-To: <Jpr9OhH2-G-jZRRVjiHPiaaZBAPxfZg9OJL6Arz-uTU=.85c16410-fc87-48da-b19e-c7164264c326@github.com>
References: <Jpr9OhH2-G-jZRRVjiHPiaaZBAPxfZg9OJL6Arz-uTU=.85c16410-fc87-48da-b19e-c7164264c326@github.com>
Message-ID: <zUKTSg8ksbSZnMw04xw6FnLcvS2lDonzbJzzhsOa-M8=.e4b7607b-d477-44e6-9a12-49e5ba287b8e@github.com>
On Thu, 27 Jun 2024 23:02:41 GMT, William Kemper <wkemper at openjdk.org> wrote:
> When a test requests a concurrent GC breakpoint, the calling thread arranges for itself to block until the concurrent GC thread notifies it that the GC has reached the requested breakpoint (phase). The code that handles the whitebox breakpoint request should therefore not block the caller. An attempt was made to do this, but the request just has the caller thread run in a busy loop without waiting. What's more, this loop resets the requested gc cause on every iteration, which may lead to gc cycles with a wb_breakpoint cause, but no breakpoint set - which violates assertions.
This pull request has now been integrated.
Changeset: 71182e24
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/71182e240ce4f4a6e3a8773f61be6b091e2d65e9
Stats: 26 lines in 2 files changed: 20 ins; 4 del; 2 mod
8335289: GenShen: Whitebox breakpoint GC requests may cause assertions
Reviewed-by: kdnilsen
-------------
PR: https://git.openjdk.org/shenandoah/pull/453
From wkemper at openjdk.org Fri Jun 28 18:13:37 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Jun 2024 18:13:37 GMT
Subject: Integrated: 8335347: GenShen: Revert change that has adaptive
heuristic ignore abbreviated cycles
In-Reply-To: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
References: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
Message-ID: <DBLMVd71ESBScwvjALKzacFArDO6l4chjc_LH-Z9ayQ=.f7d305f5-3e39-47ca-a0dc-2140ea5c5655@github.com>
On Fri, 28 Jun 2024 16:22:03 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Additional performance testing has shown no benefit from this feature. In fact, latency is degraded on the extremem benchmark with this feature enabled.
This pull request has now been integrated.
Changeset: 7542e909
Author: William Kemper <wkemper at openjdk.org>
URL: https://git.openjdk.org/shenandoah/commit/7542e909d1b4fb8a9a29ab5caf61a6be792f0f03
Stats: 26 lines in 10 files changed: 1 ins; 14 del; 11 mod
8335347: GenShen: Revert change that has adaptive heuristic ignore abbreviated cycles
Reviewed-by: kdnilsen
-------------
PR: https://git.openjdk.org/shenandoah/pull/455
From ysr at openjdk.org Fri Jun 28 19:02:45 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 28 Jun 2024 19:02:45 GMT
Subject: RFR: 8335347: GenShen: Revert change that has adaptive heuristic
ignore abbreviated cycles
In-Reply-To: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
References: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
Message-ID: <YFDjXCtP5KIpLu_lgdLwpihcl5E3dIGNgSyqhc0GsoM=.c56bbbe0-9a2a-4df1-8663-143e23059a02@github.com>
On Fri, 28 Jun 2024 16:22:03 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Additional performance testing has shown no benefit from this feature. In fact, latency is degraded on the extremem benchmark with this feature enabled.
Did we compare degenerated cycle count for more spiky or variable load benchmarks, where the original fix was supposed to help.
I also wonder if instead of the "bang, bang" control (completely ignore abbreviated as was the previous fix, now withdrawn vs treat abbreviated numbers on par with non-abbreviated cycles), I wonder if you might maintain two decaying averages for the two types separately, then toss a coin whose probability is based on the decaying ratio of the two types of events, and use the decaying average value of the corresponding type or the other based on the result of the coin toss. In effect, you are simulating and trying to predict the probability of a long or short cycle, and using the appropriate measure. Not sure how that would behave since the value seems to be continuously calculated, it would probably effectively revert to the same value as the decaying average gives without doing this more complex simulation. :-)
Anyway, was just thinking out loud here...
-------------
PR Comment: https://git.openjdk.org/shenandoah/pull/455#issuecomment-2197470115
From wkemper at openjdk.org Fri Jun 28 20:37:46 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Jun 2024 20:37:46 GMT
Subject: RFR: 8335347: GenShen: Revert change that has adaptive heuristic
ignore abbreviated cycles
In-Reply-To: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
References: <BzQ00kwqlnl-5l77P8FsmNUHGEogZ7ETCeTYYDVIPrQ=.cc3fde7b-e7bf-4fa2-9d95-f6a9857fc32d@github.com>
Message-ID: <bA7UiyOhrPArTyOr17SHMAs3FkLIoGN7EwIbc4Ll7_g=.d580c1d1-d1bd-43ae-a83f-668409260f77@github.com>
On Fri, 28 Jun 2024 16:22:03 GMT, William Kemper <wkemper at openjdk.org> wrote:
> Additional performance testing has shown no benefit from this feature. In fact, latency is degraded on the extremem benchmark with this feature enabled.
We tried every benchmark in the pipeline (dacapo, specjvm2008, specjbb2015, heapothesys, extremem, dilluvian). I think this attempt at optimization didn't work because an application is going to either have many abbreviated cycles, in which case, the heuristic lives in ignorance and runs based on the time for the rare regular length cycle (making it too aggressive). Or, the application may have very few abbreviated cycles, in which case this optimization doesn't matter. I'm sure we could craft a workload that created a sequence of abbreviated cycles, and then caused a non-abbreviated cycle, but this seems uncommon in practice. In those situations, if a degenerated cycle did occur, the late penalties would make the heuristic more aggressive. Your idea is interesting and perhaps something we could experiment with in the fullness of time.
-------------
PR Comment: https://git.openjdk.org/shenandoah/pull/455#issuecomment-2197622038
From wkemper at openjdk.org Fri Jun 28 22:05:08 2024
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 28 Jun 2024 22:05:08 GMT
Subject: RFR: 8335289: GenShen: Whitebox breakpoint GC requests may cause
assertions
Message-ID: <4mqg5wQ8EsOCwR_d7styAkrBVlxfdmm7v9o-B4G_BjI=.19d03445-aa39-4c8c-9eed-11ed356439ac@github.com>
Clean backport, low risk. Fixes intermittent test failure.
-------------
Commit messages:
- Backport 71182e240ce4f4a6e3a8773f61be6b091e2d65e9
Changes: https://git.openjdk.org/shenandoah-jdk21u/pull/63/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah-jdk21u&pr=63&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8335289
Stats: 26 lines in 2 files changed: 20 ins; 4 del; 2 mod
Patch: https://git.openjdk.org/shenandoah-jdk21u/pull/63.diff
Fetch: git fetch https://git.openjdk.org/shenandoah-jdk21u.git pull/63/head:pull/63
PR: https://git.openjdk.org/shenandoah-jdk21u/pull/63
From ysr at openjdk.org Sat Jun 29 05:09:12 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 29 Jun 2024 05:09:12 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v20]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <Bv0PYMWArBnxoxOF_JOGQ8oVeip3YicjIN1S7GYxmJg=.34a968bb-7c10-416a-96b3-e94df7bdacb5@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 50 commits:
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Revert debugging detritus.
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Merge branch 'master' into active_generation
- Code called from LRB cannot rely on SGH::_gc_generation and must instead
use the _active_generation field.
Still need to aduit remaining _gc_generation uses for safety.
Removed too strong assert introduced for debugging in previous commit.
- Merge branch 'master' into active_generation
- Some asserts to catch a tricky race. These assertions may be too strong
in general but would help with debugging a rare crash.
- ... and 40 more: https://git.openjdk.org/shenandoah/compare/71182e24...cbb60008
-------------
Changes: https://git.openjdk.org/shenandoah/pull/407/files
Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=19
Stats: 166 lines in 16 files changed: 133 ins; 1 del; 32 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From ysr at openjdk.org Sat Jun 29 05:17:31 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 29 Jun 2024 05:17:31 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v4]
In-Reply-To: <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
<DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>
<E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
Message-ID: <tYKSh5a7BNfuyYzgu784Znc7HofwAWm0-UsBbBEasrQ=.c5e24cb1-675e-41e7-baaf-6636594a899e@github.com>
On Thu, 23 May 2024 20:49:06 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Small clean-ups.
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationalEvacuationTask.cpp line 134:
>
>> 132: {
>> 133: const size_t old_garbage_threshold = (ShenandoahHeapRegion::region_size_bytes() * ShenandoahOldGarbageThreshold) / 100;
>> 134: assert(_heap->gc_generation()->is_mark_complete(), "sanity");
>
> Could use `_heap->gc_generation()->complete_marking_context()` on line 129, which just asserts that `is_mark_complete` before returning marking context.
Done.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1659594828
From ysr at openjdk.org Sat Jun 29 05:17:33 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 29 Jun 2024 05:17:33 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v16]
In-Reply-To: <W0hbPypmFWrIJy5janVs8N4TW9Bg4PDaJImstnIc0AI=.b28ef645-d7c4-4807-ac6c-febe1ea066a4@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
<7T_n4NkvVNHUDrI_jc_TzkIzd9rOvuKlhfdYxIQ7Ks4=.bc308f63-77e4-40ab-b893-063c99a3e56c@github.com>
<W0hbPypmFWrIJy5janVs8N4TW9Bg4PDaJImstnIc0AI=.b28ef645-d7c4-4807-ac6c-febe1ea066a4@github.com>
Message-ID: <Uq6RYcRslyvQJf1JZNeskUleQMoDfPcyq4JcSZz6Y7E=.04755dfb-4992-4fa6-8dbc-0b316dc89734@github.com>
On Mon, 17 Jun 2024 22:10:48 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits:
>>
>> - Merge branch 'master' into active_generation
>> - Merge branch 'master' into active_generation
>> - Merge branch 'master' into active_generation
>> - Code called from LRB cannot rely on SGH::_gc_generation and must instead
>> use the _active_generation field.
>>
>> Still need to aduit remaining _gc_generation uses for safety.
>>
>> Removed too strong assert introduced for debugging in previous commit.
>> - Merge branch 'master' into active_generation
>> - Some asserts to catch a tricky race. These assertions may be too strong
>> in general but would help with debugging a rare crash.
>> - Merge branch 'master' into active_generation
>> - Remove vestigial thread() method in ShenandoahController.
>> - cosmetic
>> - jcheck white-space
>> - ... and 36 more: https://git.openjdk.org/shenandoah/compare/2abf4cb4...b513abae
>
> src/hotspot/share/gc/shenandoah/shenandoahGenerationalHeap.cpp line 796:
>
>> 794: ShenandoahHeapRegion* r = _regions->next();
>> 795: // We update references for global, old, and young collections.
>> 796: ShenandoahGeneration* const gc_generation = _heap->gc_generation();
>
> It would certainly involve more changes, but we could pass the correct generation from `shConcurrentGC` in this case. I think we could do this in most cases (save for the barriers, but you've established a safe pattern for them to access the active generation already).
Let'd defer that cleanup for later.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1659595970
From ysr at openjdk.org Sat Jun 29 05:25:39 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 29 Jun 2024 05:25:39 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v4]
In-Reply-To: <E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
<DF5i_KW9r7Mk72OfZ0paVceeJ62TZaKRqMCdMechfy4=.8b1e1247-de68-4ca0-8815-f63af1d108a0@github.com>
<E2VLSYiMvOviDWW8XlisygLwPUFGRS8uOtIYpfrmeRM=.8905ea8e-63ed-4228-b1a5-f5c9abb1774e@github.com>
Message-ID: <ReNggYVrHGjw5SeFH0w7KuSmqRLfJIO_gyFb9OHHJhk=.1bf972c9-d2ee-48e5-85c9-fc65a667e115@github.com>
On Thu, 23 May 2024 20:54:49 GMT, William Kemper <wkemper at openjdk.org> wrote:
>> Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Small clean-ups.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 376:
>
>> 374:
>> 375: assert(is_in(obj), "only check if is in active generation for objects (" PTR_FORMAT ") in heap", p2i(obj));
>> 376: assert(gen == (ShenandoahGeneration*)old_generation() ||
>
> These casts shouldn't be necessary any more.
Done... Seems more patently tautological now in the new form, but I'll let it stay in this form for now. Let me know if it can or should be removed entirely.
-------------
PR Review Comment: https://git.openjdk.org/shenandoah/pull/407#discussion_r1659605122
From ysr at openjdk.org Sat Jun 29 05:31:05 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 29 Jun 2024 05:31:05 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v21]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <ZhM4aS5z0W9mXKkSOf_j-tXFjqxlTSVbpo1rGixNmQs=.4819e17a-2958-472c-aa50-92eb5bd8b91e@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
Fix for the few remaining review comments.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/407/files
- new: https://git.openjdk.org/shenandoah/pull/407/files/cbb60008..af57394c
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=20
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=19-20
Stats: 3 lines in 2 files changed: 0 ins; 1 del; 2 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407
From ysr at openjdk.org Sat Jun 29 06:44:13 2024
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sat, 29 Jun 2024 06:44:13 GMT
Subject: RFR: 8328235: GenShen: Robustify ShenandoahGCSession and fix
missing use [v22]
In-Reply-To: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
References: <ViHIdrG0sja7rFJOMLnOkso6t--zQrnoQSnsrkLz4B0=.859112c8-2064-4a51-912c-0ce5667af37d@github.com>
Message-ID: <n_QXKj7Y_93aHY8Nbds6JmdgU9lKwuhrpIGPoFHTX44=.a2002865-fdbb-4023-910e-a7333d2475e0@github.com>
> ShenandoahGCSession is intended to create a scope where the ShenandoahHeap's _gc_cause and _gc_generation field reflect the current gc cycle. We now check that we do not overwrite existing non-default settings (respectively _no_gc and nullptr). The destructor of the scope/stack object also resets these fields to their default settings, ensuring intended uses. This uncovered a situation where the scope was not entered when it should have been, which we have now fixed.
>
> A case of flickering of active_generation() was identified when used concurrently by mutators while it was being modified by the controller thread. To deal with this, we have carefully gone through the setting and use of the field, and found that an expedient fix for the race is to split the variable into two:
> - _gc_generation is set & cleared by the controller thread whenever it enters and exits a GC scope, and services concurrent gc cycles for young or old generations.
> - _active_generation is set to the value in _gc_generation at the start of each Shenandoah GC safepoint operation so that mutator threads and load barriers always see a consistent value between safepoints.
>
> Asserts check the protocol for setting and clearing the variables.
>
> An alternative approach is to not use a global variable for the _gc_generation indirected through the heap, but rather to pass it into the closures that do the work. This would work as well, but the changes would potentially touch more code. We would still have to have set the variable that is consulted by the load barriers, in a mutator-safe fashion at a safepoint, like we do today. This or other alternative approaches may be investigated in the future to potentially make this protocol more self-contained and robust rather than leaking as it does today into many places in the code.
>
> *Testing*:
> - [x] code pipeline
> - [x] specjbb testing
> - [ ] specjbb performance
> - [x] jtreg:hotspot_gc and jtreg:hotspot:tier1 w/fastdebug
> - [x] GHA
Y. Srinivas Ramakrishna has updated the pull request incrementally with one additional commit since the last revision:
Disallow mutator threads from reading the asynchronously updated
_gc_generation field of ShHeap.
-------------
Changes:
- all: https://git.openjdk.org/shenandoah/pull/407/files
- new: https://git.openjdk.org/shenandoah/pull/407/files/af57394c..c79fea11
Webrevs:
- full: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=21
- incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=407&range=20-21
Stats: 4 lines in 1 file changed: 3 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/shenandoah/pull/407.diff
Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/407/head:pull/407
PR: https://git.openjdk.org/shenandoah/pull/407