From rwestrel at redhat.com Mon Nov 2 08:32:22 2020
From: rwestrel at redhat.com (Roland Westrelin)
Date: Mon, 02 Nov 2020 09:32:22 +0100
Subject: shenandoah c2 compiler crash in jdk8
In-Reply-To: <CAEyj=Y7Gjk8N8iNTz766cgtWcxnOW-MY=J4EyN0iKr4UeA6oQQ@mail.gmail.com>
References: <CAEyj=Y5W1D-w=kLP26M02MkdOnP1WiPwVyXkqsNSdu6kT8SqaQ@mail.gmail.com>
<CAEyj=Y7Gjk8N8iNTz766cgtWcxnOW-MY=J4EyN0iKr4UeA6oQQ@mail.gmail.com>
Message-ID: <87a6w0duqx.fsf@redhat.com>
Hi,
> It stops crashing if I add `-XX:MaxNodeLimit=100000` option.. Is it true
> that Shenandoah needs substantially more nodes compared to CMS? If so,
> default (75000) probably needs to be adjusted... as crash in release
> version has less than obvious stacktrace
>
> Pls confirm it's not a bug :)
I'm not sure bumping MaxNodeLimit is the best way to fix this. Would
there be a way for us to reproduce the crash?
Roland.
From stefank at openjdk.java.net Mon Nov 2 11:40:04 2020
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Mon, 2 Nov 2020 11:40:04 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address
In-Reply-To: <gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
Message-ID: <wwqY3sh8_y7Z4vBJyiI6hvFIc_D7Kc30Qxul2wG_E5w=.6d0863e3-c849-4f09-9366-c4be641299cf@github.com>
On Mon, 2 Nov 2020 08:25:28 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> src/hotspot/share/prims/jvmtiTagMap.cpp line 126:
>
>> 124: // concurrent GCs. So fix it here once we have a lock or are
>> 125: // at a safepoint.
>> 126: // SetTag and GetTag should not post events!
>
> I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case.
Maybe even move this comment to the set_tag/get_tag code.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From stefank at openjdk.java.net Mon Nov 2 11:40:04 2020
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Mon, 2 Nov 2020 11:40:04 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Commented on nits, and reviewed GC code and tag map code. Didn't look closely on the hashmap changes.
src/hotspot/share/gc/shared/oopStorageSet.hpp line 41:
> 39: // Must be updated when new OopStorages are introduced
> 40: static const uint strong_count = 4 JVMTI_ONLY(+ 1);
> 41: static const uint weak_count = 5 JVMTI_ONLY(+1) JFR_ONLY(+ 1);
All other uses `+ 1` instead of `+1`.
src/hotspot/share/gc/shared/weakProcessorPhaseTimes.hpp line 49:
> 47: double _phase_times_sec[1];
> 48: size_t _phase_dead_items[1];
> 49: size_t _phase_total_items[1];
This should be removed and the associated reset_items
src/hotspot/share/gc/z/zOopClosures.hpp line 64:
> 62: };
> 63:
> 64: class ZPhantomKeepAliveOopClosure : public ZRootsIteratorClosure {
Seems like you flipped the location of these two. Maybe revert?
src/hotspot/share/prims/jvmtiExport.hpp line 405:
> 403:
> 404: // Delete me and all my callers!
> 405: static void weak_oops_do(BoolObjectClosure* b, OopClosure* f) {}
Maybe delete?
src/hotspot/share/prims/jvmtiTagMap.cpp line 126:
> 124: // concurrent GCs. So fix it here once we have a lock or are
> 125: // at a safepoint.
> 126: // SetTag and GetTag should not post events!
I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case.
src/hotspot/share/prims/jvmtiTagMap.cpp line 131:
> 129: // Operating on the hashmap must always be locked, since concurrent GC threads may
> 130: // notify while running through a safepoint.
> 131: assert(is_locked(), "checking");
Maybe move this to the top of the function to make it very clear.
src/hotspot/share/prims/jvmtiTagMap.cpp line 133:
> 131: assert(is_locked(), "checking");
> 132: if (post_events && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) {
> 133: log_info(jvmti, table)("TagMap table needs posting before heap walk");
Not sure about the "before heap walk" since this is also done from GetObjectsWithTags, which does *not* do a heap walk but still requires posting.
src/hotspot/share/prims/jvmtiTagMap.cpp line 140:
> 138: hashmap()->rehash();
> 139: _needs_rehashing = false;
> 140: }
It's not clear to me that it's correct to rehash *after* posting. I think it is, because unlink_and_post will use load barriers to fixup old pointers.
src/hotspot/share/prims/jvmtiTagMap.cpp line 146:
> 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed.
> 145: void JvmtiTagMap::check_hashmaps_for_heapwalk() {
> 146:
Extra white space. (Also double whitespace after this function)
src/hotspot/share/prims/jvmtiTagMap.cpp line 144:
> 142:
> 143: // This checks for posting and rehashing and is called from the heap walks.
> 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed.
Should this comment be moved down to the lock taking?
src/hotspot/share/prims/jvmtiTagMap.cpp line 377:
> 375: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag);
> 376:
> 377: // Check if we have to processing for concurrent GCs.
Sentence seems to be missing a few words.
src/hotspot/share/prims/jvmtiTagMap.cpp line 954:
> 952: o->klass()->external_name());
> 953: return;
> 954: }
Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch?
src/hotspot/share/prims/jvmtiTagMap.cpp line 1152:
> 1150: void JvmtiTagMap::unlink_and_post_locked() {
> 1151: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag);
> 1152: log_info(jvmti, table)("TagMap table needs posting before GetObjectTags");
There's no function called GetObjectTags. This log line needs to be adjusted.
src/hotspot/share/prims/jvmtiTagMap.cpp line 1162:
> 1160: VMOp_Type type() const { return VMOp_Cleanup; }
> 1161: void doit() {
> 1162: _tag_map->unlink_and_post_locked();
Either inline unlink_and_post_locked() or updated gc_notification to use it?
src/hotspot/share/prims/jvmtiTagMap.cpp line 1279:
> 1277: // Can't post ObjectFree events here from a JavaThread, so this
> 1278: // will race with the gc_notification thread in the tiny
> 1279: // window where the oop is not marked but hasn't been notified that
Please don't use "oop" when referring to "objects".
src/hotspot/share/prims/jvmtiTagMap.cpp line 2975:
> 2973: }
> 2974:
> 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved
oops => objects
src/hotspot/share/prims/jvmtiTagMap.cpp line 2977:
> 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved
> 2976: // and have their new addresses, the table can be rehashed.
> 2977: void JvmtiTagMap::set_needs_processing() {
Maybe rename to set_needs_rehashing?
src/hotspot/share/prims/jvmtiTagMap.cpp line 2985:
> 2983:
> 2984: JvmtiEnvIterator it;
> 2985: for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
The iterator seems fast enough, so it seems unnecessary to have the environments_might_exist check.
src/hotspot/share/prims/jvmtiTagMap.cpp line 2998:
> 2996: // thread creation and before VMThread creation (1 thread); initial GC
> 2997: // verification can happen in that window which gets to here.
> 2998: if (!JvmtiEnv::environments_might_exist()) { return; }
I don't know what this comment is saying, and why the code is needed.
src/hotspot/share/prims/jvmtiTagMap.cpp line 3020:
> 3018: JvmtiTagMap* tag_map = env->tag_map_acquire();
> 3019: if (tag_map != NULL && !tag_map->is_empty()) {
> 3020: if (num_dead_entries > 0) {
The other num_dead_entries check for != 0. Maybe use the same in the two branches?
src/hotspot/share/prims/jvmtiTagMap.cpp line 3023:
> 3021: tag_map->hashmap()->unlink_and_post(tag_map->env());
> 3022: }
> 3023: tag_map->_needs_rehashing = true;
Maybe add a small comment why this is deferred.
src/hotspot/share/prims/jvmtiTagMap.hpp line 56:
> 54: void entry_iterate(JvmtiTagMapEntryClosure* closure);
> 55: void post_dead_object_on_vm_thread();
> 56: public:
Looked nicer when there was a blank line before public. Now it looks like public "relates" more to the code before than after.
src/hotspot/share/prims/jvmtiTagMap.hpp line 114:
> 112: static void check_hashmaps_for_heapwalk();
> 113: static void set_needs_processing() NOT_JVMTI_RETURN;
> 114: static void gc_notification(size_t num_dead_entries) NOT_JVMTI_RETURN;
Have you verified that this builds without JVMTI?
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 50:
> 48: // A subsequent oop_load without AS_NO_KEEPALIVE (the object() accessor)
> 49: // keeps the oop alive before doing so.
> 50: return literal().peek();
I'm not sure we should be talking about the low-level Access names. Maybe reword in terms of WeakHandle operations?
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 81:
> 79: void JvmtiTagMapTable::free_entry(JvmtiTagMapEntry* entry) {
> 80: unlink_entry(entry);
> 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage
release *to* OopStorage?
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 82:
> 80: unlink_entry(entry);
> 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage
> 82: FREE_C_HEAP_ARRAY(char, entry); // C_Heap free.
Seems excessively redundant:
// C_Heap free.
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 98:
> 96:
> 97: // The obj is in the table as a target already
> 98: if (target != NULL && target == obj) {
Wonder if we could assert that obj is not NULL at the entry of this function, and then change this to simply target == obj?
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 122:
> 120: int index = hash_to_index(hash);
> 121: // One was added while acquiring the lock
> 122: JvmtiTagMapEntry* entry = find(index, hash, obj);
Should this be done inside ASSERT?
test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 64:
> 62: static jclass klass = NULL;
> 63: static jobject testedObject = NULL;
> 64: const jlong TESTED_TAG_VALUE = (5555555L);
Remove parenthesis?
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From ihse at openjdk.java.net Mon Nov 2 12:30:56 2020
From: ihse at openjdk.java.net (Magnus Ihse Bursie)
Date: Mon, 2 Nov 2020 12:30:56 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <4yShPUjSyZSPGnbVh-jr58lGT9psXrg88HZIk5Wa-40=.c2f77dae-d582-4fb4-9f6e-bdcb5ced63b0@github.com>
On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Build changes look good. Not reviewed hotspot changes.
-------------
Marked as reviewed by ihse (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Mon Nov 2 12:56:12 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 2 Nov 2020 12:56:12 GMT
Subject: RFR: 8254315: Shenandoah: Concurrent weak reference processing
[v27]
In-Reply-To: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
References: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
Message-ID: <nX_W9GiE5jJ-z92qIA3erWWhtqg7IiYHQxqdnS-S1ZE=.2db706bb-d9bd-46e5-b3f4-fd6137a91a1a@github.com>
> Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.
>
> There are 3 main items that contribute to pause time linear to number of references, or worse:
> - We need to scan and consider each reference on the various 'discovered' lists.
> - We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
> - Finally, all no-longer-reachable references need to be enqueued in the 'pending list'
>
> The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.
>
> The solution to this is two-fold:
> 1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
> 2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.
>
> Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 84 commits:
- Adopt ShenandoahReferenceBarrier to recent changes in LRB runtime impl
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Invert strong/weak in marking tasks and related code
- Fix merge mistake
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Pass marking-strength through chunked arrays
- Rename mark_final -> mark_weak and several cleanups (by shade)
- Some more ShMarkTask cleanups
- Call into native-LRB on unknown oop strenght (i.e. reflection) too
- Put in comment about API impedence mismatch around interpreter native LRB
- ... and 74 more: https://git.openjdk.java.net/jdk/compare/eb66418b...c09fda9a
-------------
Changes: https://git.openjdk.java.net/jdk/pull/505/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=26
Stats: 2416 lines in 55 files changed: 1643 ins; 565 del; 208 mod
Patch: https://git.openjdk.java.net/jdk/pull/505.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/505/head:pull/505
PR: https://git.openjdk.java.net/jdk/pull/505
From roland at openjdk.java.net Mon Nov 2 13:07:08 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Mon, 2 Nov 2020 13:07:08 GMT
Subject: RFR: 8255400: Shenandoah: C2 failures after JDK-8255000
Message-ID: <Zv9hoMVjvTsiGoUPkZKHKXmtEN1CSvYRAuVaSuJ7BQw=.33b7028a-4261-4812-bfed-f41b4c97a07a@github.com>
At barrier expansion time, the IR graph may contain a Halt node whose
control is a region. In that case, code that wires raw memory creates
a memory Phi at the region. But that Phi has no use because the Halt
node doesn't consume any memory. That dead Phi causes the assert to
trigger. I propose some adjustments so a Phi is not created in that
case.
-------------
Commit messages:
- jcheck
- more test
- fix
- test
Changes: https://git.openjdk.java.net/jdk/pull/1000/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1000&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255400
Stats: 87 lines in 2 files changed: 81 ins; 0 del; 6 mod
Patch: https://git.openjdk.java.net/jdk/pull/1000.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1000/head:pull/1000
PR: https://git.openjdk.java.net/jdk/pull/1000
From coleenp at openjdk.java.net Mon Nov 2 13:22:15 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 2 Nov 2020 13:22:15 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address
In-Reply-To: <gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
Message-ID: <Otlfd7ZOdIvKTY4H7EALKzsBlB-tBvJgP7_H0TODW6w=.9b0ba2a9-8436-45fd-91dd-dae750ee3261@github.com>
On Mon, 2 Nov 2020 08:08:53 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> src/hotspot/share/gc/shared/oopStorageSet.hpp line 41:
>
>> 39: // Must be updated when new OopStorages are introduced
>> 40: static const uint strong_count = 4 JVMTI_ONLY(+ 1);
>> 41: static const uint weak_count = 5 JVMTI_ONLY(+1) JFR_ONLY(+ 1);
>
> All other uses `+ 1` instead of `+1`.
Fixed, although I think the space looks strange there but I'll go along.
> src/hotspot/share/gc/shared/weakProcessorPhaseTimes.hpp line 49:
>
>> 47: double _phase_times_sec[1];
>> 48: size_t _phase_dead_items[1];
>> 49: size_t _phase_total_items[1];
>
> This should be removed and the associated reset_items
Removed.
> src/hotspot/share/gc/z/zOopClosures.hpp line 64:
>
>> 62: };
>> 63:
>> 64: class ZPhantomKeepAliveOopClosure : public ZRootsIteratorClosure {
>
> Seems like you flipped the location of these two. Maybe revert?
Reverted. There was a rebasing conflict here so this was unintentional.
> src/hotspot/share/prims/jvmtiExport.hpp line 405:
>
>> 403:
>> 404: // Delete me and all my callers!
>> 405: static void weak_oops_do(BoolObjectClosure* b, OopClosure* f) {}
>
> Maybe delete?
Yes, meant to do that.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 131:
>
>> 129: // Operating on the hashmap must always be locked, since concurrent GC threads may
>> 130: // notify while running through a safepoint.
>> 131: assert(is_locked(), "checking");
>
> Maybe move this to the top of the function to make it very clear.
ok.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 133:
>
>> 131: assert(is_locked(), "checking");
>> 132: if (post_events && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) {
>> 133: log_info(jvmti, table)("TagMap table needs posting before heap walk");
>
> Not sure about the "before heap walk" since this is also done from GetObjectsWithTags, which does *not* do a heap walk but still requires posting.
I don't call check_hashmap for GetObjectsWithTags.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 140:
>
>> 138: hashmap()->rehash();
>> 139: _needs_rehashing = false;
>> 140: }
>
> It's not clear to me that it's correct to rehash *after* posting. I think it is, because unlink_and_post will use load barriers to fixup old pointers.
I think it's better that the rehashing doesn't encounter null entries and the WeakHandle.peek() operation is used for both so I hope it would get the same answer. If not, which seems bad, the last answer should be what we hash on.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 144:
>
>> 142:
>> 143: // This checks for posting and rehashing and is called from the heap walks.
>> 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed.
>
> Should this comment be moved down to the lock taking?
ok, I also made it singular since Erik pointed out that we don't need the other lock.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 146:
>
>> 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed.
>> 145: void JvmtiTagMap::check_hashmaps_for_heapwalk() {
>> 146:
>
> Extra white space. (Also double whitespace after this function)
? I removed the double whitespace after the function and put the whitespace here. It needs some whitespace.
// This checks for posting and rehashing and is called from the heap walks.
void JvmtiTagMap::check_hashmaps_for_heapwalk() {
assert(SafepointSynchronize::is_at_safepoint(), "called from safepoints");
// Verify that the tag map tables are valid and unconditionally post events
// that are expected to be posted before gc_notification.
JvmtiEnvIterator it;
> src/hotspot/share/prims/jvmtiTagMap.cpp line 377:
>
>> 375: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag);
>> 376:
>> 377: // Check if we have to processing for concurrent GCs.
>
> Sentence seems to be missing a few words.
removed the sentence, because non concurrent GCs also defer rehashing to next use.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 954:
>
>> 952: o->klass()->external_name());
>> 953: return;
>> 954: }
>
> Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch?
Because it crashed with my changes and didn't without. I cannot recollect why.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 1152:
>
>> 1150: void JvmtiTagMap::unlink_and_post_locked() {
>> 1151: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag);
>> 1152: log_info(jvmti, table)("TagMap table needs posting before GetObjectTags");
>
> There's no function called GetObjectTags. This log line needs to be adjusted.
GetObjectsWithTags fixed.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 1162:
>
>> 1160: VMOp_Type type() const { return VMOp_Cleanup; }
>> 1161: void doit() {
>> 1162: _tag_map->unlink_and_post_locked();
>
> Either inline unlink_and_post_locked() or updated gc_notification to use it?
I thought of trying to share it but one logs and the other doesn't and it only saves 1 lines of code.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 1279:
>
>> 1277: // Can't post ObjectFree events here from a JavaThread, so this
>> 1278: // will race with the gc_notification thread in the tiny
>> 1279: // window where the oop is not marked but hasn't been notified that
>
> Please don't use "oop" when referring to "objects".
fixed.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 2975:
>
>> 2973: }
>> 2974:
>> 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved
>
> oops => objects
fixed.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 2977:
>
>> 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved
>> 2976: // and have their new addresses, the table can be rehashed.
>> 2977: void JvmtiTagMap::set_needs_processing() {
>
> Maybe rename to set_needs_rehashing?
Since I went back and forth about what this function did (it posted events at one time), I thought the generic _processing name would be better. GC callers shouldn't really have to know what processing we're doing here. Hopefully it won't change from rehashing. That's why I like processing.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 2985:
>
>> 2983:
>> 2984: JvmtiEnvIterator it;
>> 2985: for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
>
> The iterator seems fast enough, so it seems unnecessary to have the environments_might_exist check.
yes, it looks like it does the same thing. I'll remove it.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 2998:
>
>> 2996: // thread creation and before VMThread creation (1 thread); initial GC
>> 2997: // verification can happen in that window which gets to here.
>> 2998: if (!JvmtiEnv::environments_might_exist()) { return; }
>
> I don't know what this comment is saying, and why the code is needed.
I've spent tons of time trying to understand this comment too. I think gc verification used to call oops do on the tagmap table. This comments obsolete now, and I'll remove it.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 3020:
>
>> 3018: JvmtiTagMap* tag_map = env->tag_map_acquire();
>> 3019: if (tag_map != NULL && !tag_map->is_empty()) {
>> 3020: if (num_dead_entries > 0) {
>
> The other num_dead_entries check for != 0. Maybe use the same in the two branches?
ok.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 3023:
>
>> 3021: tag_map->hashmap()->unlink_and_post(tag_map->env());
>> 3022: }
>> 3023: tag_map->_needs_rehashing = true;
>
> Maybe add a small comment why this is deferred.
// Later GC code will relocate the oops, so defer rehashing until then.
?
> src/hotspot/share/prims/jvmtiTagMap.hpp line 56:
>
>> 54: void entry_iterate(JvmtiTagMapEntryClosure* closure);
>> 55: void post_dead_object_on_vm_thread();
>> 56: public:
>
> Looked nicer when there was a blank line before public. Now it looks like public "relates" more to the code before than after.
ok
> src/hotspot/share/prims/jvmtiTagMap.hpp line 114:
>
>> 112: static void check_hashmaps_for_heapwalk();
>> 113: static void set_needs_processing() NOT_JVMTI_RETURN;
>> 114: static void gc_notification(size_t num_dead_entries) NOT_JVMTI_RETURN;
>
> Have you verified that this builds without JVMTI?
I will do (might have already done) that. Building non-oracle platforms builds minimal vm.
> src/hotspot/share/prims/jvmtiTagMapTable.cpp line 50:
>
>> 48: // A subsequent oop_load without AS_NO_KEEPALIVE (the object() accessor)
>> 49: // keeps the oop alive before doing so.
>> 50: return literal().peek();
>
> I'm not sure we should be talking about the low-level Access names. Maybe reword in terms of WeakHandle operations?
I'm going to say:
// Just peek at the object without keeping it alive.
> src/hotspot/share/prims/jvmtiTagMapTable.cpp line 81:
>
>> 79: void JvmtiTagMapTable::free_entry(JvmtiTagMapEntry* entry) {
>> 80: unlink_entry(entry);
>> 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage
>
> release *to* OopStorage?
fixed
> src/hotspot/share/prims/jvmtiTagMapTable.cpp line 98:
>
>> 96:
>> 97: // The obj is in the table as a target already
>> 98: if (target != NULL && target == obj) {
>
> Wonder if we could assert that obj is not NULL at the entry of this function, and then change this to simply target == obj?
makes sense.
> src/hotspot/share/prims/jvmtiTagMapTable.cpp line 122:
>
>> 120: int index = hash_to_index(hash);
>> 121: // One was added while acquiring the lock
>> 122: JvmtiTagMapEntry* entry = find(index, hash, obj);
>
> Should this be done inside ASSERT?
yes.
> test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 64:
>
>> 62: static jclass klass = NULL;
>> 63: static jobject testedObject = NULL;
>> 64: const jlong TESTED_TAG_VALUE = (5555555L);
>
> Remove parenthesis?
I copied this from some other place that had parenthesis.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Mon Nov 2 13:22:01 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 2 Nov 2020 13:22:01 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <g4Yzu6YAIY0IWKDX0GGcL4qD_vRGPrnUQt4nJKDdIhM=.981f28f1-6b4c-4014-baef-b8b162ee7ff4@github.com>
On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
I think I addressed your comments, retesting now. Thank you!
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Mon Nov 2 13:22:15 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 2 Nov 2020 13:22:15 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address
In-Reply-To: <wwqY3sh8_y7Z4vBJyiI6hvFIc_D7Kc30Qxul2wG_E5w=.6d0863e3-c849-4f09-9366-c4be641299cf@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
<wwqY3sh8_y7Z4vBJyiI6hvFIc_D7Kc30Qxul2wG_E5w=.6d0863e3-c849-4f09-9366-c4be641299cf@github.com>
Message-ID: <AqYzgbnx8MN7P51EeIFg9YQjdPl0P787Gz-qrzFXjTc=.d6d74b88-1e0d-4191-b55c-c600c001f942@github.com>
On Mon, 2 Nov 2020 08:34:17 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:
>> src/hotspot/share/prims/jvmtiTagMap.cpp line 126:
>>
>>> 124: // concurrent GCs. So fix it here once we have a lock or are
>>> 125: // at a safepoint.
>>> 126: // SetTag and GetTag should not post events!
>>
>> I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case.
>
> Maybe even move this comment to the set_tag/get_tag code.
I was trying to explain why there's a boolean there but I can put this comment at both get_tag and set_tag.
// Check if we have to processing for concurrent GCs.
// GetTag should not post events because the JavaThread has to
// transition to native for the callback and this cannot stop for
// safepoints with the hashmap lock held.
check_hashmap(false);
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Mon Nov 2 13:24:57 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 2 Nov 2020 13:24:57 GMT
Subject: RFR: 8255400: Shenandoah: C2 failures after JDK-8255000
In-Reply-To: <Zv9hoMVjvTsiGoUPkZKHKXmtEN1CSvYRAuVaSuJ7BQw=.33b7028a-4261-4812-bfed-f41b4c97a07a@github.com>
References: <Zv9hoMVjvTsiGoUPkZKHKXmtEN1CSvYRAuVaSuJ7BQw=.33b7028a-4261-4812-bfed-f41b4c97a07a@github.com>
Message-ID: <6L73HWSfNYZTr5AT1zuVXK0U6f8QVXw11drbIee3Q-A=.564585b7-5532-4699-b502-7b1d940cfb86@github.com>
On Mon, 2 Nov 2020 09:51:01 GMT, Roland Westrelin <roland at openjdk.org> wrote:
> At barrier expansion time, the IR graph may contain a Halt node whose
> control is a region. In that case, code that wires raw memory creates
> a memory Phi at the region. But that Phi has no use because the Halt
> node doesn't consume any memory. That dead Phi causes the assert to
> trigger. I propose some adjustments so a Phi is not created in that
> case.
Looks good to me! Thank you!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1000
From coleenp at openjdk.java.net Mon Nov 2 13:28:59 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 2 Nov 2020 13:28:59 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address
In-Reply-To: <OPBOkOwYCpaoMic1YZnMz4mWBhWihywNcJInzM_UW2o=.aaac99d7-c097-4e1f-882c-f373edcfd262@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<OPBOkOwYCpaoMic1YZnMz4mWBhWihywNcJInzM_UW2o=.aaac99d7-c097-4e1f-882c-f373edcfd262@github.com>
Message-ID: <lTK2sK1xsFz4zJUYDAhWM2ax0qjLAcOP2pIzf-v0u9I=.731f57ca-b1fd-400a-b102-793b3d092fc0@github.com>
On Fri, 30 Oct 2020 20:46:31 GMT, Erik Joelsson <erikj at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Build changes look ok.
There should be a thank you emoji that doesn't send email except maybe to the person reviewing the code. Thank you @erikj79 and @magicus for reviewing the build changes. There should also be a 'fixed' emoji.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From zgu at openjdk.java.net Mon Nov 2 13:48:02 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 2 Nov 2020 13:48:02 GMT
Subject: RFR: 8255691: Shenandoah: Invoke native-LRB only on non-strong
refs [v3]
In-Reply-To: <2wI5SLQjP_SmJOstjPHk3ct5b1Pr3nZEvi97NWCbo-E=.39961cac-4a3b-48c4-b608-425f024020b0@github.com>
References: <2O7ZS_-HyPZpm5m2sMTf-dZv8Qkw3tB47hMgORkckoU=.a58d7945-a2ea-4f76-9e01-540521bd77ea@github.com>
<2wI5SLQjP_SmJOstjPHk3ct5b1Pr3nZEvi97NWCbo-E=.39961cac-4a3b-48c4-b608-425f024020b0@github.com>
Message-ID: <0OzLgZFXNCorTwEdj9a2zFvR8Dj10pxCA8FiYp_tLdM=.edd21518-28c5-40ad-b2ee-a9e770e878b2@github.com>
On Fri, 30 Oct 2020 19:39:07 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> The way that current native LRB is implemented is wrong (but non-fatal) and misleading. It's purpose is to prevent resurrection of unreachable non-strong references, and it should only be invoked on non-strong references, not all native references. This distinction will become even more important once we get concurrent reference processing: then we also want to invoke this barrier on referent-loads.
>>
>> This changes the runtime-part of native-LRB so that it is only invoked when it's invoked with non-strong reference decorator. Otherwise it acts as regular LRB.
>>
>> Testing: hotspot_gc_shenandoah
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Rename LRB-native -> LRB-weak
Marked as reviewed by zgu (Reviewer).
src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp line 974:
> 972: CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier);
> 973:
> 974: address calladdr = is_native ? CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_weak)
Please rename is_native -> is_weak.
-------------
PR: https://git.openjdk.java.net/jdk/pull/961
From shade at openjdk.java.net Mon Nov 2 14:04:03 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 2 Nov 2020 14:04:03 GMT
Subject: RFR: 8255760: Shenandoah: match constants style in ShenandoahMarkTask
fallback
Message-ID: <SzzYSN9QffEIGQQDiZV1-Lns6PSaKSzUy0pNtP2t6GM=.d950ac04-958d-4a66-9303-137fff56927b@github.com>
JDK-8255457 cleaned up the 64-bit version of ShenandoahMarkTask. However, there is also the 32-bit fallback without any tricky mechanics. Still, it should match the style of the optimized version by using `static const`-s instead of enums. I missed that during the JDK-8255457 work.
Additional testing:
- [x] Linux x86_32 `hotspot_gc_shenandoah`
-------------
Commit messages:
- 8255760: Shenandoah: match constants style in ShenandoahMarkTask fallback
Changes: https://git.openjdk.java.net/jdk/pull/1006/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1006&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255760
Stats: 7 lines in 1 file changed: 1 ins; 0 del; 6 mod
Patch: https://git.openjdk.java.net/jdk/pull/1006.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1006/head:pull/1006
PR: https://git.openjdk.java.net/jdk/pull/1006
From zgu at openjdk.java.net Mon Nov 2 14:12:57 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 2 Nov 2020 14:12:57 GMT
Subject: RFR: 8255760: Shenandoah: match constants style in
ShenandoahMarkTask fallback
In-Reply-To: <SzzYSN9QffEIGQQDiZV1-Lns6PSaKSzUy0pNtP2t6GM=.d950ac04-958d-4a66-9303-137fff56927b@github.com>
References: <SzzYSN9QffEIGQQDiZV1-Lns6PSaKSzUy0pNtP2t6GM=.d950ac04-958d-4a66-9303-137fff56927b@github.com>
Message-ID: <SXxkP2PEgMBbfczgp51piO5e7HX1oxkLcnQgbtTx-FI=.9533b297-82de-443a-9e1e-feec488b6a34@github.com>
On Mon, 2 Nov 2020 13:55:59 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> JDK-8255457 cleaned up the 64-bit version of ShenandoahMarkTask. However, there is also the 32-bit fallback without any tricky mechanics. Still, it should match the style of the optimized version by using `static const`-s instead of enums. I missed that during the JDK-8255457 work.
>
> Additional testing:
> - [x] Linux x86_32 `hotspot_gc_shenandoah`
Marked as reviewed by zgu (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/1006
From shade at openjdk.java.net Mon Nov 2 14:16:16 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 2 Nov 2020 14:16:16 GMT
Subject: RFR: 8255691: Shenandoah: Invoke native-LRB only on non-strong
refs [v3]
In-Reply-To: <2wI5SLQjP_SmJOstjPHk3ct5b1Pr3nZEvi97NWCbo-E=.39961cac-4a3b-48c4-b608-425f024020b0@github.com>
References: <2O7ZS_-HyPZpm5m2sMTf-dZv8Qkw3tB47hMgORkckoU=.a58d7945-a2ea-4f76-9e01-540521bd77ea@github.com>
<2wI5SLQjP_SmJOstjPHk3ct5b1Pr3nZEvi97NWCbo-E=.39961cac-4a3b-48c4-b608-425f024020b0@github.com>
Message-ID: <zKjjBR1VcMBoyk3tWqQM9KfrqTGQs3ziJz_F5kPp27g=.5e812e6f-c12e-4628-95dd-3470c1af6930@github.com>
On Fri, 30 Oct 2020 19:39:07 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> The way that current native LRB is implemented is wrong (but non-fatal) and misleading. It's purpose is to prevent resurrection of unreachable non-strong references, and it should only be invoked on non-strong references, not all native references. This distinction will become even more important once we get concurrent reference processing: then we also want to invoke this barrier on referent-loads.
>>
>> This changes the runtime-part of native-LRB so that it is only invoked when it's invoked with non-strong reference decorator. Otherwise it acts as regular LRB.
>>
>> Testing: hotspot_gc_shenandoah
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Rename LRB-native -> LRB-weak
Generally looks good, some minor nits.
src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 1063:
> 1061: Node* in2 = n->in(2);
> 1062:
> 1063: // If one input is NULL, then step over the barriers (except LRB native) on the other input
Should be `weak LRB`, not `LRB native` now? Probably text search for `native` elsewhere?
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.cpp line 98:
> 96: }
> 97:
> 98: return ((decorators & IN_NATIVE) != 0) && ((decorators & ON_STRONG_OOP_REF) == 0);
This would change with conc ref processing, right? Currently this only accepts "native" + "weak" LRBs.
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 105:
> 103: inline oop ShenandoahBarrierSet::load_reference_barrier(oop obj, T* load_addr) {
> 104:
> 105: // Prevent resurrection of unreachable non-strorg references.
Typo: "non-strorg"
-------------
Changes requested by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/961
From rkennke at openjdk.java.net Mon Nov 2 14:16:14 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 2 Nov 2020 14:16:14 GMT
Subject: RFR: 8255691: Shenandoah: Invoke native-LRB only on non-strong
refs [v4]
In-Reply-To: <2O7ZS_-HyPZpm5m2sMTf-dZv8Qkw3tB47hMgORkckoU=.a58d7945-a2ea-4f76-9e01-540521bd77ea@github.com>
References: <2O7ZS_-HyPZpm5m2sMTf-dZv8Qkw3tB47hMgORkckoU=.a58d7945-a2ea-4f76-9e01-540521bd77ea@github.com>
Message-ID: <iHynBAY0-kyHDUBUj-QDmXOiRypLdZh_D2MTTM9JcV8=.91ac0170-73c5-4384-98fa-7dcb535ef010@github.com>
> The way that current native LRB is implemented is wrong (but non-fatal) and misleading. It's purpose is to prevent resurrection of unreachable non-strong references, and it should only be invoked on non-strong references, not all native references. This distinction will become even more important once we get concurrent reference processing: then we also want to invoke this barrier on referent-loads.
>
> This changes the runtime-part of native-LRB so that it is only invoked when it's invoked with non-strong reference decorator. Otherwise it acts as regular LRB.
>
> Testing: hotspot_gc_shenandoah
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Rename argument is_native -> is_weak
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/961/files
- new: https://git.openjdk.java.net/jdk/pull/961/files/86c80228..2db8a9d5
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=961&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=961&range=02-03
Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod
Patch: https://git.openjdk.java.net/jdk/pull/961.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/961/head:pull/961
PR: https://git.openjdk.java.net/jdk/pull/961
From rkennke at openjdk.java.net Mon Nov 2 14:16:17 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 2 Nov 2020 14:16:17 GMT
Subject: Integrated: 8255691: Shenandoah: Invoke native-LRB only on non-strong
refs
In-Reply-To: <2O7ZS_-HyPZpm5m2sMTf-dZv8Qkw3tB47hMgORkckoU=.a58d7945-a2ea-4f76-9e01-540521bd77ea@github.com>
References: <2O7ZS_-HyPZpm5m2sMTf-dZv8Qkw3tB47hMgORkckoU=.a58d7945-a2ea-4f76-9e01-540521bd77ea@github.com>
Message-ID: <tpNxpulSVMrBzugMUI9HAIJVPMVMYNcxM0fP1FpgC3E=.9b2ea54b-822e-4a2d-964c-66309fef545e@github.com>
On Fri, 30 Oct 2020 18:12:18 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> The way that current native LRB is implemented is wrong (but non-fatal) and misleading. It's purpose is to prevent resurrection of unreachable non-strong references, and it should only be invoked on non-strong references, not all native references. This distinction will become even more important once we get concurrent reference processing: then we also want to invoke this barrier on referent-loads.
>
> This changes the runtime-part of native-LRB so that it is only invoked when it's invoked with non-strong reference decorator. Otherwise it acts as regular LRB.
>
> Testing: hotspot_gc_shenandoah
This pull request has now been integrated.
Changeset: 1019581c
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/1019581c
Stats: 84 lines in 14 files changed: 4 ins; 3 del; 77 mod
8255691: Shenandoah: Invoke native-LRB only on non-strong refs
Reviewed-by: zgu
-------------
PR: https://git.openjdk.java.net/jdk/pull/961
From rkennke at openjdk.java.net Mon Nov 2 14:41:55 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 2 Nov 2020 14:41:55 GMT
Subject: RFR: 8255760: Shenandoah: match constants style in
ShenandoahMarkTask fallback
In-Reply-To: <SzzYSN9QffEIGQQDiZV1-Lns6PSaKSzUy0pNtP2t6GM=.d950ac04-958d-4a66-9303-137fff56927b@github.com>
References: <SzzYSN9QffEIGQQDiZV1-Lns6PSaKSzUy0pNtP2t6GM=.d950ac04-958d-4a66-9303-137fff56927b@github.com>
Message-ID: <PVwtlMgnYyMOR8X3pSYtp2lIt5iT743CFoHyQpL9TIQ=.3c29c07f-035c-4dea-bcf7-07641f112317@github.com>
On Mon, 2 Nov 2020 13:55:59 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> JDK-8255457 cleaned up the 64-bit version of ShenandoahMarkTask. However, there is also the 32-bit fallback without any tricky mechanics. Still, it should match the style of the optimized version by using `static const`-s instead of enums. I missed that during the JDK-8255457 work.
>
> Additional testing:
> - [x] Linux x86_32 `hotspot_gc_shenandoah`
Looks good to me!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1006
From zgu at openjdk.java.net Mon Nov 2 14:53:13 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 2 Nov 2020 14:53:13 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into separate
classes
Message-ID: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
First step, I would like to split STW and concurrent mark, so that:
1) Code has to special case for STW and concurrent mark.
2) STW mark does not need to rendezvous workers between root mark and the rest of mark
3) STW mark does not need to activate SATB barrier and drain SATB buffers.
4) STW mark does not need to remark some of roots.
The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
A few changes:
1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
-------------
Commit messages:
- Make ShenandoahMarkCompact stack allocated
- Split finish mark and pre-evaculation
- 8255019: Shenandoah: Split STW and concurrent mark into separate classes
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255019
Stats: 2474 lines in 21 files changed: 1299 ins; 989 del; 186 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From rkennke at openjdk.java.net Mon Nov 2 15:28:09 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 2 Nov 2020 15:28:09 GMT
Subject: RFR: 8254315: Shenandoah: Concurrent weak reference processing
[v28]
In-Reply-To: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
References: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
Message-ID: <ESIoaB7uavIH8VJsj3BGIGuvmjz00PLbNAl21OJqRkA=.f2a5fb59-7cbe-463a-9d01-be0ca92875b3@github.com>
> Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.
>
> There are 3 main items that contribute to pause time linear to number of references, or worse:
> - We need to scan and consider each reference on the various 'discovered' lists.
> - We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
> - Finally, all no-longer-reachable references need to be enqueued in the 'pending list'
>
> The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.
>
> The solution to this is two-fold:
> 1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
> 2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.
>
> Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 86 commits:
- Invoke interpreter weak-LRB on weak-refs too (fixes merge mistake)
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Adopt ShenandoahReferenceBarrier to recent changes in LRB runtime impl
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Invert strong/weak in marking tasks and related code
- Fix merge mistake
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Pass marking-strength through chunked arrays
- Rename mark_final -> mark_weak and several cleanups (by shade)
- Some more ShMarkTask cleanups
- ... and 76 more: https://git.openjdk.java.net/jdk/compare/4c66b158...2c2579d2
-------------
Changes: https://git.openjdk.java.net/jdk/pull/505/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=27
Stats: 2409 lines in 55 files changed: 1632 ins; 564 del; 213 mod
Patch: https://git.openjdk.java.net/jdk/pull/505.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/505/head:pull/505
PR: https://git.openjdk.java.net/jdk/pull/505
From roland at openjdk.java.net Mon Nov 2 15:45:58 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Mon, 2 Nov 2020 15:45:58 GMT
Subject: RFR: 8255401: Shenandoah: Allow oldval and newval registers to
overlap in cmpxchg_oop()
In-Reply-To: <aYoSNZYvC131cCrit7cbFauLZnSq0MMxdA909g_WA-w=.70cf7b94-6f49-482f-9aca-dfcb8c2d639b@github.com>
References: <aYoSNZYvC131cCrit7cbFauLZnSq0MMxdA909g_WA-w=.70cf7b94-6f49-482f-9aca-dfcb8c2d639b@github.com>
Message-ID: <_EufJHEmM7yEZd_RS0vzeafF8BFqVA3nm_J4n-B9IeM=.94b48189-7853-4154-9a1f-5409f2414717@github.com>
On Mon, 26 Oct 2020 20:50:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> We encountered a failure in testing:
>
> Internal Error (/home/jenkins/workspace/nightly/jdk-jdk/src/hotspot/share/asm/register.hpp:141), pid=15470, tid=15611
> assert(a != b && a != c && a != d && b != c && b != d && c != d) failed: registers must be different: a=0x0000000000000000, b=0x0000000000000000, c=0x000000000000000b, d=0x000000000000000a
>
> in:
>
> Stack: [0x00007fb8fa2e3000,0x00007fb8fa3e4000], sp=0x00007fb8fa3deca0, free space=1007k
> Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V [libjvm.so+0x156890e] ShenandoahBarrierSetAssembler::cmpxchg_oop(MacroAssembler*, RegisterImpl*, Address, RegisterImpl*, RegisterImpl*, bool, RegisterImpl*, RegisterImpl*)+0xde
> V [libjvm.so+0x3ec1d1] compareAndSwapN_shenandoahNode::emit(CodeBuffer&, PhaseRegAlloc*) const+0x571
>
> It seems to appear very rarely.
>
> The failure is that both newval and oldval are the same register (rax). I believe it is ok for the two registers to overlap:
> - It is not expected that newval is preserved across the cmpxchg
> - The CAS will override newval, but:
> - The first CAS is unaffected by the overlap
> - The retry-loop is only entered when previous-value == old-value, and thus newval will still hold the same value
>
> For aarch64 it matters even less, because newval is never overridden.
>
> Testing: hotspot_gc_shenandoah (x86 & aarch64).
Looks good to me
-------------
Marked as reviewed by roland (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/871
From roland at openjdk.java.net Mon Nov 2 15:53:07 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Mon, 2 Nov 2020 15:53:07 GMT
Subject: Integrated: 8255400: Shenandoah: C2 failures after JDK-8255000
In-Reply-To: <Zv9hoMVjvTsiGoUPkZKHKXmtEN1CSvYRAuVaSuJ7BQw=.33b7028a-4261-4812-bfed-f41b4c97a07a@github.com>
References: <Zv9hoMVjvTsiGoUPkZKHKXmtEN1CSvYRAuVaSuJ7BQw=.33b7028a-4261-4812-bfed-f41b4c97a07a@github.com>
Message-ID: <ISEW3iuhLuD_XdG1xfsSfECg7qEYOjPbHpeAWJWOlDo=.873e36cb-f996-4e24-9bc0-6b33c4a9290d@github.com>
On Mon, 2 Nov 2020 09:51:01 GMT, Roland Westrelin <roland at openjdk.org> wrote:
> At barrier expansion time, the IR graph may contain a Halt node whose
> control is a region. In that case, code that wires raw memory creates
> a memory Phi at the region. But that Phi has no use because the Halt
> node doesn't consume any memory. That dead Phi causes the assert to
> trigger. I propose some adjustments so a Phi is not created in that
> case.
This pull request has now been integrated.
Changeset: a3aad119
Author: Roland Westrelin <roland at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/a3aad119
Stats: 87 lines in 2 files changed: 81 ins; 0 del; 6 mod
8255400: Shenandoah: C2 failures after JDK-8255000
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1000
From zgu at openjdk.java.net Mon Nov 2 15:59:12 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 2 Nov 2020 15:59:12 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v2]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <-vhPpP79vJJaPn9vvmaj3jcQKuJ3y2e5CPwqpd4pGow=.5a2aa1e5-e600-444a-a9c9-03fe0ece90b3@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Make ShenandoahMarkCompact stack allocated
- Split finish mark and pre-evaculation
- 8255019: Shenandoah: Split STW and concurrent mark into separate classes
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1009/files
- new: https://git.openjdk.java.net/jdk/pull/1009/files/6b75cf0d..a616a46d
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=00-01
Stats: 4314 lines in 190 files changed: 2371 ins; 800 del; 1143 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From coleenp at openjdk.java.net Mon Nov 2 15:58:15 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 2 Nov 2020 15:58:15 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
Code review comments from StefanK.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/534326da..cb4c83e0
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=00-01
Stats: 75 lines in 9 files changed: 12 ins; 48 del; 15 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From zgu at openjdk.java.net Mon Nov 2 16:07:02 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 2 Nov 2020 16:07:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
Message-ID: <Mqn6N5-CfXZq_M0PVh2u3KsMHA4BQIU6Ae05lj-xRfE=.12cf1ecf-a31f-476b-9257-3effa4c3b398@github.com>
On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> Code review comments from StefanK.
Shenandoah part looks good.
-------------
Marked as reviewed by zgu (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Mon Nov 2 16:08:56 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 2 Nov 2020 16:08:56 GMT
Subject: Integrated: 8255401: Shenandoah: Allow oldval and newval registers to
overlap in cmpxchg_oop()
In-Reply-To: <aYoSNZYvC131cCrit7cbFauLZnSq0MMxdA909g_WA-w=.70cf7b94-6f49-482f-9aca-dfcb8c2d639b@github.com>
References: <aYoSNZYvC131cCrit7cbFauLZnSq0MMxdA909g_WA-w=.70cf7b94-6f49-482f-9aca-dfcb8c2d639b@github.com>
Message-ID: <mgDVMXY-qoxeI1xukxTimNqF_fCxulKKNqtan2q47Eo=.b3f77b62-36e6-4562-9d54-e6a8d31514e4@github.com>
On Mon, 26 Oct 2020 20:50:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> We encountered a failure in testing:
>
> Internal Error (/home/jenkins/workspace/nightly/jdk-jdk/src/hotspot/share/asm/register.hpp:141), pid=15470, tid=15611
> assert(a != b && a != c && a != d && b != c && b != d && c != d) failed: registers must be different: a=0x0000000000000000, b=0x0000000000000000, c=0x000000000000000b, d=0x000000000000000a
>
> in:
>
> Stack: [0x00007fb8fa2e3000,0x00007fb8fa3e4000], sp=0x00007fb8fa3deca0, free space=1007k
> Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V [libjvm.so+0x156890e] ShenandoahBarrierSetAssembler::cmpxchg_oop(MacroAssembler*, RegisterImpl*, Address, RegisterImpl*, RegisterImpl*, bool, RegisterImpl*, RegisterImpl*)+0xde
> V [libjvm.so+0x3ec1d1] compareAndSwapN_shenandoahNode::emit(CodeBuffer&, PhaseRegAlloc*) const+0x571
>
> It seems to appear very rarely.
>
> The failure is that both newval and oldval are the same register (rax). I believe it is ok for the two registers to overlap:
> - It is not expected that newval is preserved across the cmpxchg
> - The CAS will override newval, but:
> - The first CAS is unaffected by the overlap
> - The retry-loop is only entered when previous-value == old-value, and thus newval will still hold the same value
>
> For aarch64 it matters even less, because newval is never overridden.
>
> Testing: hotspot_gc_shenandoah (x86 & aarch64).
This pull request has now been integrated.
Changeset: 0e19ded9
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/0e19ded9
Stats: 4 lines in 2 files changed: 2 ins; 0 del; 2 mod
8255401: Shenandoah: Allow oldval and newval registers to overlap in cmpxchg_oop()
Reviewed-by: roland
-------------
PR: https://git.openjdk.java.net/jdk/pull/871
From eosterlund at openjdk.java.net Mon Nov 2 16:14:06 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Mon, 2 Nov 2020 16:14:06 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
Message-ID: <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com>
On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> Code review comments from StefanK.
Looks great in general. Great work Coleen, and thanks again for fixing this. I like all the red lines in the GC code. I added a few nits/questions.
test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 656:
> 654: result = NSK_FALSE;
> 655:
> 656: printf("Object free events %d\n", ObjectFreeEventsCount);
Is this old debug info you forgot to remove? Other code seems to use NSK_DISPLAY macros instead.
src/hotspot/share/prims/jvmtiTagMap.cpp line 345:
> 343:
> 344: // Check if we have to process for concurrent GCs.
> 345: check_hashmap(false);
Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap.
src/hotspot/share/prims/jvmtiTagMap.cpp line 3009:
> 3007: // Lock each hashmap from concurrent posting and cleaning
> 3008: MutexLocker ml(tag_map->lock(), Mutex::_no_safepoint_check_flag);
> 3009: tag_map->hashmap()->unlink_and_post(tag_map->env());
This could call unlink_and_post_locked instead of manually locking.
-------------
Changes requested by eosterlund (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/967
From eosterlund at openjdk.java.net Mon Nov 2 16:14:06 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Mon, 2 Nov 2020 16:14:06 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <Otlfd7ZOdIvKTY4H7EALKzsBlB-tBvJgP7_H0TODW6w=.9b0ba2a9-8436-45fd-91dd-dae750ee3261@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
<Otlfd7ZOdIvKTY4H7EALKzsBlB-tBvJgP7_H0TODW6w=.9b0ba2a9-8436-45fd-91dd-dae750ee3261@github.com>
Message-ID: <avkoSFPhSuga7cXc2XsOwWyIrgJLGAVyJx5mYEqLjHU=.36725611-1e29-43ba-9227-ad8919750625@github.com>
On Mon, 2 Nov 2020 12:50:23 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> src/hotspot/share/prims/jvmtiTagMap.cpp line 954:
>>
>>> 952: o->klass()->external_name());
>>> 953: return;
>>> 954: }
>>
>> Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch?
>
> Because it crashed with my changes and didn't without. I cannot recollect why.
I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Mon Nov 2 16:27:03 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 2 Nov 2020 16:27:03 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
<0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com>
Message-ID: <OCdd9qJRaD2E7hiMko36IC65sROz04nLdGZn1t4F7YA=.8aa8c60c-9a01-438f-a6e9-8f3424e7ed97@github.com>
On Mon, 2 Nov 2020 15:18:43 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Code review comments from StefanK.
>
> test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 656:
>
>> 654: result = NSK_FALSE;
>> 655:
>> 656: printf("Object free events %d\n", ObjectFreeEventsCount);
>
> Is this old debug info you forgot to remove? Other code seems to use NSK_DISPLAY macros instead.
yes, removed it.
> src/hotspot/share/prims/jvmtiTagMap.cpp line 345:
>
>> 343:
>> 344: // Check if we have to process for concurrent GCs.
>> 345: check_hashmap(false);
>
> Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap.
Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before?
> src/hotspot/share/prims/jvmtiTagMap.cpp line 3009:
>
>> 3007: // Lock each hashmap from concurrent posting and cleaning
>> 3008: MutexLocker ml(tag_map->lock(), Mutex::_no_safepoint_check_flag);
>> 3009: tag_map->hashmap()->unlink_and_post(tag_map->env());
>
> This could call unlink_and_post_locked instead of manually locking.
Ok, 2 requests. I can call that then and move the logging.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Mon Nov 2 16:57:14 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 2 Nov 2020 16:57:14 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <LrQJ_xP1ZhhN8kP7g3BhU5v2dd1YbDhg83MRWcDtwwA=.d1650ac8-5fa3-489e-ad4c-eca2091cbd9d@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
<Otlfd7ZOdIvKTY4H7EALKzsBlB-tBvJgP7_H0TODW6w=.9b0ba2a9-8436-45fd-91dd-dae750ee3261@github.com>
<avkoSFPhSuga7cXc2XsOwWyIrgJLGAVyJx5mYEqLjHU=.36725611-1e29-43ba-9227-ad8919750625@github.com>
<LrQJ_xP1ZhhN8kP7g3BhU5v2dd1YbDhg83MRWcDtwwA=.d1650ac8-5fa3-489e-ad4c-eca2091cbd9d@github.com>
Message-ID: <RmyhkBGmEMwFDN_2j8uETIsLk6-G-t9DHJuc8WJMLv0=.a3d71b15-7090-43f4-ad6e-f11f3bb9dca0@github.com>
On Mon, 2 Nov 2020 16:52:02 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference.
>
> Now I remember. I added an assert in JvmtiTagMapTable::find() for oop != NULL which didn't exist in the current hashmap code. The current hashmap code just didn't find a null oop. I tracked it down to the fact that we're finding dormant objects whose class hasn't been loaded yet.
So I think we do load the archived heap from CDS. The heap walker capabilities can be added dynamically.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Mon Nov 2 16:57:13 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 2 Nov 2020 16:57:13 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <avkoSFPhSuga7cXc2XsOwWyIrgJLGAVyJx5mYEqLjHU=.36725611-1e29-43ba-9227-ad8919750625@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
<Otlfd7ZOdIvKTY4H7EALKzsBlB-tBvJgP7_H0TODW6w=.9b0ba2a9-8436-45fd-91dd-dae750ee3261@github.com>
<avkoSFPhSuga7cXc2XsOwWyIrgJLGAVyJx5mYEqLjHU=.36725611-1e29-43ba-9227-ad8919750625@github.com>
Message-ID: <LrQJ_xP1ZhhN8kP7g3BhU5v2dd1YbDhg83MRWcDtwwA=.d1650ac8-5fa3-489e-ad4c-eca2091cbd9d@github.com>
On Mon, 2 Nov 2020 15:45:18 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
>> Because it crashed with my changes and didn't without. I cannot recollect why.
>
> I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference.
Now I remember. I added an assert in JvmtiTagMapTable::find() for oop != NULL which didn't exist in the current hashmap code. The current hashmap code just didn't find a null oop. I tracked it down to the fact that we're finding dormant objects whose class hasn't been loaded yet.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From shade at openjdk.java.net Mon Nov 2 17:40:57 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 2 Nov 2020 17:40:57 GMT
Subject: Integrated: 8255760: Shenandoah: match constants style in
ShenandoahMarkTask fallback
In-Reply-To: <SzzYSN9QffEIGQQDiZV1-Lns6PSaKSzUy0pNtP2t6GM=.d950ac04-958d-4a66-9303-137fff56927b@github.com>
References: <SzzYSN9QffEIGQQDiZV1-Lns6PSaKSzUy0pNtP2t6GM=.d950ac04-958d-4a66-9303-137fff56927b@github.com>
Message-ID: <vZ6fc3XUbF1whP-uLbXKbyh9LXIfe3Gr97eZ3IGx4os=.2a856203-9911-4f50-ba93-ed03f9a50f41@github.com>
On Mon, 2 Nov 2020 13:55:59 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> JDK-8255457 cleaned up the 64-bit version of ShenandoahMarkTask. However, there is also the 32-bit fallback without any tricky mechanics. Still, it should match the style of the optimized version by using `static const`-s instead of enums. I missed that during the JDK-8255457 work.
>
> Additional testing:
> - [x] Linux x86_32 `hotspot_gc_shenandoah`
This pull request has now been integrated.
Changeset: d93e3a7d
Author: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/d93e3a7d
Stats: 7 lines in 1 file changed: 1 ins; 0 del; 6 mod
8255760: Shenandoah: match constants style in ShenandoahMarkTask fallback
Reviewed-by: zgu, rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1006
From nevgeniev at gmail.com Mon Nov 2 20:02:41 2020
From: nevgeniev at gmail.com (Nick Evgeniev)
Date: Mon, 2 Nov 2020 14:02:41 -0600
Subject: shenandoah c2 compiler crash in jdk8
In-Reply-To: <87a6w0duqx.fsf@redhat.com>
References: <CAEyj=Y5W1D-w=kLP26M02MkdOnP1WiPwVyXkqsNSdu6kT8SqaQ@mail.gmail.com>
<CAEyj=Y7Gjk8N8iNTz766cgtWcxnOW-MY=J4EyN0iKr4UeA6oQQ@mail.gmail.com>
<87a6w0duqx.fsf@redhat.com>
Message-ID: <CAEyj=Y5S4+rfkaAOfr12_0v-5gOM3=wYRHAjY-GXTOa6bM+fYw@mail.gmail.com>
hi,
I'll try to come up with sample code, not sure to what degree compilation
'context' is critical. But if you have some build to try (with more
logging/tracing) i'm more than willing to help... Also i've tried to bump
up in steps 75k->80k->90k->100k. It fails everywhere in between. for 80k it
fails in Node::req()
I think that the qn that needs to be answered is whether there is a `leak`
in nodes or you just need more of them. Does that make sense? If so, is
there an option to dump c2 node usage over time?
also just in case there is original stack trace (from release build w/o
assertions):
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
V [libjvm.so+0x4b6eae] Compile::final_graph_reshaping()+0x1e
V [libjvm.so+0x4b923e] Compile::Optimize()+0xd8e
V [libjvm.so+0x4bb217] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*,
int, bool, bool, bool)+0x11c7
V [libjvm.so+0x40fd8c] C2Compiler::compile_method(ciEnv*, ciMethod*,
int)+0x20c
V [libjvm.so+0x4c2096]
CompileBroker::invoke_compiler_on_method(CompileTask*)+0xd46
V [libjvm.so+0x4c4977] CompileBroker::compiler_thread_loop()+0x657
V [libjvm.so+0xb22981] JavaThread::thread_main_inner()+0xf1
V [libjvm.so+0x9674a2] java_start(Thread*)+0x132
C [libpthread.so.0+0x7ea5] start_thread+0xc5
On Mon, 2 Nov 2020 at 02:32, Roland Westrelin <rwestrel at redhat.com> wrote:
>
> Hi,
>
> > It stops crashing if I add `-XX:MaxNodeLimit=100000` option.. Is it true
> > that Shenandoah needs substantially more nodes compared to CMS? If so,
> > default (75000) probably needs to be adjusted... as crash in release
> > version has less than obvious stacktrace
> >
> > Pls confirm it's not a bug :)
>
> I'm not sure bumping MaxNodeLimit is the best way to fix this. Would
> there be a way for us to reproduce the crash?
>
> Roland.
>
>
From nevgeniev at gmail.com Mon Nov 2 22:49:23 2020
From: nevgeniev at gmail.com (Nick Evgeniev)
Date: Mon, 2 Nov 2020 16:49:23 -0600
Subject: shenandoah c2 compiler crash in jdk8
In-Reply-To: <CAEyj=Y5S4+rfkaAOfr12_0v-5gOM3=wYRHAjY-GXTOa6bM+fYw@mail.gmail.com>
References: <CAEyj=Y5W1D-w=kLP26M02MkdOnP1WiPwVyXkqsNSdu6kT8SqaQ@mail.gmail.com>
<CAEyj=Y7Gjk8N8iNTz766cgtWcxnOW-MY=J4EyN0iKr4UeA6oQQ@mail.gmail.com>
<87a6w0duqx.fsf@redhat.com>
<CAEyj=Y5S4+rfkaAOfr12_0v-5gOM3=wYRHAjY-GXTOa6bM+fYw@mail.gmail.com>
Message-ID: <CAEyj=Y5saNKTbuab_0NLUFAohnfKJ28VQm-DB8nEZHGCwD5KeQ@mail.gmail.com>
hi,
Just a bit more info -- w/o Shenandoah I can push that limit safely to just
3k before replay starts crashing.. So, substantially is a weak word...
unless it's a bug Shenandoah needs 30x more nodes. Hope it helps
On Mon, 2 Nov 2020 at 14:02, Nick Evgeniev <nevgeniev at gmail.com> wrote:
> hi,
>
> I'll try to come up with sample code, not sure to what degree compilation
> 'context' is critical. But if you have some build to try (with more
> logging/tracing) i'm more than willing to help... Also i've tried to bump
> up in steps 75k->80k->90k->100k. It fails everywhere in between. for 80k it
> fails in Node::req()
>
> I think that the qn that needs to be answered is whether there is a `leak`
> in nodes or you just need more of them. Does that make sense? If so, is
> there an option to dump c2 node usage over time?
>
> also just in case there is original stack trace (from release build w/o
> assertions):
>
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> V [libjvm.so+0x4b6eae] Compile::final_graph_reshaping()+0x1e
> V [libjvm.so+0x4b923e] Compile::Optimize()+0xd8e
> V [libjvm.so+0x4bb217] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*,
> int, bool, bool, bool)+0x11c7
> V [libjvm.so+0x40fd8c] C2Compiler::compile_method(ciEnv*, ciMethod*,
> int)+0x20c
> V [libjvm.so+0x4c2096]
> CompileBroker::invoke_compiler_on_method(CompileTask*)+0xd46
> V [libjvm.so+0x4c4977] CompileBroker::compiler_thread_loop()+0x657
> V [libjvm.so+0xb22981] JavaThread::thread_main_inner()+0xf1
> V [libjvm.so+0x9674a2] java_start(Thread*)+0x132
> C [libpthread.so.0+0x7ea5] start_thread+0xc5
>
>
> On Mon, 2 Nov 2020 at 02:32, Roland Westrelin <rwestrel at redhat.com> wrote:
>
>>
>> Hi,
>>
>> > It stops crashing if I add `-XX:MaxNodeLimit=100000` option.. Is it true
>> > that Shenandoah needs substantially more nodes compared to CMS? If so,
>> > default (75000) probably needs to be adjusted... as crash in release
>> > version has less than obvious stacktrace
>> >
>> > Pls confirm it's not a bug :)
>>
>> I'm not sure bumping MaxNodeLimit is the best way to fix this. Would
>> there be a way for us to reproduce the crash?
>>
>> Roland.
>>
>>
From rkennke at openjdk.java.net Mon Nov 2 22:52:04 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 2 Nov 2020 22:52:04 GMT
Subject: RFR: 8255762: Shenandoah: Consolidate/streamline interpreter LRBs
Message-ID: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
We currently have two LRB implementations in interpreter: one normal and one for native/weak LRB. We should consolidate them into one.
The main issue here is that the two implementations follow different approaches:
- the normal LRB calls through the shenandoah_lrb stub which does additional null- and cset-checking
- the weak LRB calls to runtime directly and must not do cset-checking
The reason for calling through the stub is that it gives more freedom to allocate two registers that are required for the cset check. However, we can invert the cset addressing like we did in JDK-8245465 and save a register. We can also eliminate the null-check and let the cset-check subsume it (like we do everywhere else). Allocating a single register for the cset-check is easy, and we can do so in-line without the extra jump through the stub. The runtime call through the stub has also been very costly: it dumps 2KB of register data on the stack at each call, that is very excessive. save_xmm_registers() should be more than enough (in-fact, I am almost certain that this is excessive too, and we should only need to save/restore xmm0 - but not in this patch). Not needing to generate the call-stub is also helpful for backportability, because in jdk8-shenandoah we cannot do that.
Testing: hotspot_gc_shenandoah (x86_64, x86_32)
-------------
Commit messages:
- Improve register shuffling in interpreter LRB/aarch64
- Consolidate/streamline interpreter LRBs/aarch64 part
- Fix x86_32 build
- 8255762: Shenandoah: Consolidate/streamline interpreter LRBs
Changes: https://git.openjdk.java.net/jdk/pull/1010/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1010&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255762
Stats: 424 lines in 4 files changed: 45 ins; 312 del; 67 mod
Patch: https://git.openjdk.java.net/jdk/pull/1010.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1010/head:pull/1010
PR: https://git.openjdk.java.net/jdk/pull/1010
From shade at openjdk.java.net Tue Nov 3 09:10:56 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 3 Nov 2020 09:10:56 GMT
Subject: RFR: 8255762: Shenandoah: Consolidate/streamline interpreter LRBs
In-Reply-To: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
References: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
Message-ID: <LrPpzTBJnKIofhd_X1Xsv1m6Jf88Q732c5_adVjCwu0=.ee79aada-bf35-409d-a390-5d56b67e197e@github.com>
On Mon, 2 Nov 2020 15:55:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> We currently have two LRB implementations in interpreter: one normal and one for native/weak LRB. We should consolidate them into one.
>
> The main issue here is that the two implementations follow different approaches:
> - the normal LRB calls through the shenandoah_lrb stub which does additional null- and cset-checking
> - the weak LRB calls to runtime directly and must not do cset-checking
>
> The reason for calling through the stub is that it gives more freedom to allocate two registers that are required for the cset check. However, we can invert the cset addressing like we did in JDK-8245465 and save a register. We can also eliminate the null-check and let the cset-check subsume it (like we do everywhere else). Allocating a single register for the cset-check is easy, and we can do so in-line without the extra jump through the stub. The runtime call through the stub has also been very costly: it dumps 2KB of register data on the stack at each call, that is very excessive. save_xmm_registers() should be more than enough (in-fact, I am almost certain that this is excessive too, and we should only need to save/restore xmm0 - but not in this patch). Not needing to generate the call-stub is also helpful for backportability, because in jdk8-shenandoah we cannot do that.
>
> Testing: hotspot_gc_shenandoah (x86_64, x86_32)
This looks nice. Please run `hotspot_gc_shenandoah` on `aarch64`?
src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 239:
> 237:
> 238: // Check for heap stability
> 239: __ tbz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, not_fwded);
`not_fwded` reads as if the object is not forwarded. May it should be `heap_stable`?
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 307:
> 305: assert(tmp1 != dst, "");
> 306: assert(tmp1 != src.base(), "");
> 307: assert(tmp1 != src.index(), "");
Is this just `assert_different_registers(tmp1, dst, src.base(), src.index())`, or am I missing something?
-------------
Changes requested by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1010
From eosterlund at openjdk.java.net Tue Nov 3 10:25:00 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Tue, 3 Nov 2020 10:25:00 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <OCdd9qJRaD2E7hiMko36IC65sROz04nLdGZn1t4F7YA=.8aa8c60c-9a01-438f-a6e9-8f3424e7ed97@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
<0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com>
<OCdd9qJRaD2E7hiMko36IC65sROz04nLdGZn1t4F7YA=.8aa8c60c-9a01-438f-a6e9-8f3424e7ed97@github.com>
Message-ID: <E2A90KxG02EeNA0yEpcTjp33ZB7VIz1QU2OkjB9CPQY=.ec6654ad-4d0b-442d-8296-885b254462c9@github.com>
On Mon, 2 Nov 2020 16:22:57 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> src/hotspot/share/prims/jvmtiTagMap.cpp line 345:
>>
>>> 343:
>>> 344: // Check if we have to process for concurrent GCs.
>>> 345: check_hashmap(false);
>>
>> Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap.
>
> Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before?
ZGC people passionately place the comment after the argument value.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From stefank at openjdk.java.net Tue Nov 3 10:36:57 2020
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Tue, 3 Nov 2020 10:36:57 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <Otlfd7ZOdIvKTY4H7EALKzsBlB-tBvJgP7_H0TODW6w=.9b0ba2a9-8436-45fd-91dd-dae750ee3261@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
<Otlfd7ZOdIvKTY4H7EALKzsBlB-tBvJgP7_H0TODW6w=.9b0ba2a9-8436-45fd-91dd-dae750ee3261@github.com>
Message-ID: <vf5jHhZkN1s3Rb6bdMKnJYp_GE5bFWPo17dQATU1_w0=.9ba84e38-e76e-4210-94cc-e84465da1cfc@github.com>
On Mon, 2 Nov 2020 12:58:49 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
> GC callers shouldn't really have to know what processing we're doing here.
I completely disagree with this. It's extremely important that the GC and Runtime code agrees on what this code does and where the GC *must* call it. Knowing the details allows us to skip calling this after mark end, but forces us to call it in relocate start, when objects should start to move. Though, I don't want to block this review because of this point, so if you still thinks that a non-descriptive name is better then we can argue that separately.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From stefank at openjdk.java.net Tue Nov 3 10:44:05 2020
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Tue, 3 Nov 2020 10:44:05 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
Message-ID: <0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com>
On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> Code review comments from StefanK.
Some more nit-picking to make the code more consistent.
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 52:
> 50: : Hashtable<WeakHandle, mtServiceability>(_table_size, sizeof(JvmtiTagMapEntry)) {}
> 51:
> 52:
Double whitespace
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 185:
> 183: // Serially remove unused oops from the table, and notify jvmti.
> 184: void JvmtiTagMapTable::unlink_and_post(JvmtiEnv* env) {
> 185:
Stray newline
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 224:
> 222: // Rehash oops in the table
> 223: void JvmtiTagMapTable::rehash() {
> 224:
Stray newline
src/hotspot/share/prims/jvmtiTagMapTable.hpp line 75:
> 73:
> 74: void resize_if_needed();
> 75: public:
Newline between
src/hotspot/share/prims/jvmtiTagMapTable.hpp line 100:
> 98: };
> 99:
> 100:
Double newline
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 258:
> 256: int rehash_len = moved_entries.length();
> 257: // Now add back in the entries that were removed.
> 258: for (int i = 0; i < moved_entries.length(); i++) {
rehash_len is read, but not used in for loop condition.
src/hotspot/share/prims/jvmtiTagMapTable.cpp line 165:
> 163: }
> 164: }
> 165: const int _resize_load_trigger = 5; // load factor that will trigger the resize
Newline between
-------------
Marked as reviewed by stefank (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Tue Nov 3 11:11:11 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 11:11:11 GMT
Subject: RFR: 8255762: Shenandoah: Consolidate/streamline interpreter LRBs
[v2]
In-Reply-To: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
References: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
Message-ID: <ucaEJx7KuRh1F8ixJ0EJbDbXIXJwjdCg6CD6C2mskrI=.32e4fe10-a20a-4978-b7d3-f4cce1116e2d@github.com>
> We currently have two LRB implementations in interpreter: one normal and one for native/weak LRB. We should consolidate them into one.
>
> The main issue here is that the two implementations follow different approaches:
> - the normal LRB calls through the shenandoah_lrb stub which does additional null- and cset-checking
> - the weak LRB calls to runtime directly and must not do cset-checking
>
> The reason for calling through the stub is that it gives more freedom to allocate two registers that are required for the cset check. However, we can invert the cset addressing like we did in JDK-8245465 and save a register. We can also eliminate the null-check and let the cset-check subsume it (like we do everywhere else). Allocating a single register for the cset-check is easy, and we can do so in-line without the extra jump through the stub. The runtime call through the stub has also been very costly: it dumps 2KB of register data on the stack at each call, that is very excessive. save_xmm_registers() should be more than enough (in-fact, I am almost certain that this is excessive too, and we should only need to save/restore xmm0 - but not in this patch). Not needing to generate the call-stub is also helpful for backportability, because in jdk8-shenandoah we cannot do that.
>
> Testing: hotspot_gc_shenandoah (x86_64, x86_32)
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
A few touch-ups
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1010/files
- new: https://git.openjdk.java.net/jdk/pull/1010/files/6573511d..12382e6c
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1010&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1010&range=00-01
Stats: 9 lines in 2 files changed: 0 ins; 1 del; 8 mod
Patch: https://git.openjdk.java.net/jdk/pull/1010.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1010/head:pull/1010
PR: https://git.openjdk.java.net/jdk/pull/1010
From rkennke at openjdk.java.net Tue Nov 3 11:15:57 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 11:15:57 GMT
Subject: RFR: 8255762: Shenandoah: Consolidate/streamline interpreter LRBs
[v2]
In-Reply-To: <LrPpzTBJnKIofhd_X1Xsv1m6Jf88Q732c5_adVjCwu0=.ee79aada-bf35-409d-a390-5d56b67e197e@github.com>
References: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
<LrPpzTBJnKIofhd_X1Xsv1m6Jf88Q732c5_adVjCwu0=.ee79aada-bf35-409d-a390-5d56b67e197e@github.com>
Message-ID: <CuVrPmP6ATEasPQHjwuoJodQXdraipRPluuAZ5RUJWk=.962a7ce6-e342-499e-86aa-49ee6ddb22c4@github.com>
On Tue, 3 Nov 2020 09:04:46 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>>
>> A few touch-ups
>
> src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 307:
>
>> 305: assert(tmp1 != dst, "");
>> 306: assert(tmp1 != src.base(), "");
>> 307: assert(tmp1 != src.index(), "");
>
> Is this just `assert_different_registers(tmp1, dst, src.base(), src.index())`, or am I missing something?
Not quite. dst can still legitimately be src.base(). Also, assert_different_registers() is somewhat unpractical because it fails in the wrong place. Anyhow, I rewrote it to be more succinct.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1010
From shade at openjdk.java.net Tue Nov 3 12:16:01 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 3 Nov 2020 12:16:01 GMT
Subject: RFR: 8255606: Enable concurrent stack processing on x86_32
platforms [v2]
In-Reply-To: <THq75VMpdQw4i0lW6Cnp_HjWbas5NBF1lqb31WzNr54=.7cfd7a18-8633-4ff9-9170-b61301763a47@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
<THq75VMpdQw4i0lW6Cnp_HjWbas5NBF1lqb31WzNr54=.7cfd7a18-8633-4ff9-9170-b61301763a47@github.com>
Message-ID: <3UnxY32uP-ERCn8odu787AX2OqPH8soXglsanBwpz3Q=.8eb97050-dd9e-4294-861c-aa5c3f3a90e2@github.com>
On Fri, 30 Oct 2020 12:14:55 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> 8255606: Enable concurrent stack processing on x86_32 platforms
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>
> Fix jump direction
I have minor comments.
src/hotspot/cpu/x86/x86_32.ad line 656:
> 654: st->print_cr("POPL EBP"); st->print("\t");
> 655: if (do_polling() && C->is_method_compilation()) {
> 656: st->print("cmpptr rsp, poll_offset[thread] \n\t"
Let's say `CMPL` here? I don't think we reference `cmpptr` in outputs here. Also, note every instruction is capitalized in `x86_32.ad`.
src/hotspot/cpu/x86/x86_64.ad line 933:
> 931: if (do_polling() && C->is_method_compilation()) {
> 932: st->print("\t");
> 933: st->print_cr("cmpptr rsp, poll_offset[r15_thread] \n\t"
Ditto, leave `cmpq` here.
-------------
Changes requested by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/945
From coleenp at openjdk.java.net Tue Nov 3 12:22:03 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 12:22:03 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <E2A90KxG02EeNA0yEpcTjp33ZB7VIz1QU2OkjB9CPQY=.ec6654ad-4d0b-442d-8296-885b254462c9@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
<0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com>
<OCdd9qJRaD2E7hiMko36IC65sROz04nLdGZn1t4F7YA=.8aa8c60c-9a01-438f-a6e9-8f3424e7ed97@github.com>
<E2A90KxG02EeNA0yEpcTjp33ZB7VIz1QU2OkjB9CPQY=.ec6654ad-4d0b-442d-8296-885b254462c9@github.com>
Message-ID: <nBYobUxzHYjt3ChOAldb5aNH0p7r7K_Gj4ysjG8ZhbI=.727864e9-0604-4de6-906c-883f1c780566@github.com>
On Tue, 3 Nov 2020 10:22:09 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
>> Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before?
>
> ZGC people passionately place the comment after the argument value.
I see that but in the non-zgc code it's the opposite and this is non-zgc code.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From shade at openjdk.java.net Tue Nov 3 12:23:59 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 3 Nov 2020 12:23:59 GMT
Subject: RFR: 8255762: Shenandoah: Consolidate/streamline interpreter LRBs
[v2]
In-Reply-To: <ucaEJx7KuRh1F8ixJ0EJbDbXIXJwjdCg6CD6C2mskrI=.32e4fe10-a20a-4978-b7d3-f4cce1116e2d@github.com>
References: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
<ucaEJx7KuRh1F8ixJ0EJbDbXIXJwjdCg6CD6C2mskrI=.32e4fe10-a20a-4978-b7d3-f4cce1116e2d@github.com>
Message-ID: <Pf2M0zxJcZsJHfJ15nmNVCTmuV0wOA9GnJ0kMgYX5fE=.b35cadde-ff44-4b11-bb5d-4fd64cf985fc@github.com>
On Tue, 3 Nov 2020 11:11:11 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> We currently have two LRB implementations in interpreter: one normal and one for native/weak LRB. We should consolidate them into one.
>>
>> The main issue here is that the two implementations follow different approaches:
>> - the normal LRB calls through the shenandoah_lrb stub which does additional null- and cset-checking
>> - the weak LRB calls to runtime directly and must not do cset-checking
>>
>> The reason for calling through the stub is that it gives more freedom to allocate two registers that are required for the cset check. However, we can invert the cset addressing like we did in JDK-8245465 and save a register. We can also eliminate the null-check and let the cset-check subsume it (like we do everywhere else). Allocating a single register for the cset-check is easy, and we can do so in-line without the extra jump through the stub. The runtime call through the stub has also been very costly: it dumps 2KB of register data on the stack at each call, that is very excessive. save_xmm_registers() should be more than enough (in-fact, I am almost certain that this is excessive too, and we should only need to save/restore xmm0 - but not in this patch). Not needing to generate the call-stub is also helpful for backportability, because in jdk8-shenandoah we cannot do that.
>>
>> Testing: hotspot_gc_shenandoah (x86_64, x86_32, aarch64)
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> A few touch-ups
Looks good, apart from two nits.
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 301:
> 299: Register r = as_Register(i);
> 300: if (r != rsp && r != rbp && r != dst && r != src.base() && r != src.index()) {
> 301: tmp1 = r;
Minor nit: looks like you just want to `break` from here, and not check `tmp1->is_valid()` in the loop predicate?
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 335:
> 333: assert(slot == 0, "must use all slots");
> 334:
> 335: Register tmp2 = dst == rsi ? rdx : rsi;
Please do parentheses around `(dst == rsi)`.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1010
From coleenp at openjdk.java.net Tue Nov 3 12:36:01 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 12:36:01 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com>
<0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com>
Message-ID: <pGfumblNLzkK84RZc5FS4syl4tP6lixrSumlPTasjcU=.cdbfc2e8-6f54-48b1-9b70-6b33cc99df6a@github.com>
On Tue, 3 Nov 2020 10:36:21 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Code review comments from StefanK.
>
> src/hotspot/share/prims/jvmtiTagMapTable.cpp line 185:
>
>> 183: // Serially remove unused oops from the table, and notify jvmti.
>> 184: void JvmtiTagMapTable::unlink_and_post(JvmtiEnv* env) {
>> 185:
>
> Stray newline
that's a useful newline.
> src/hotspot/share/prims/jvmtiTagMapTable.cpp line 258:
>
>> 256: int rehash_len = moved_entries.length();
>> 257: // Now add back in the entries that were removed.
>> 258: for (int i = 0; i < moved_entries.length(); i++) {
>
> rehash_len is read, but not used in for loop condition.
It's in the logging message below. I'll use it in the loop too.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 12:40:06 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 12:40:06 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2]
In-Reply-To: <vf5jHhZkN1s3Rb6bdMKnJYp_GE5bFWPo17dQATU1_w0=.9ba84e38-e76e-4210-94cc-e84465da1cfc@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
<Otlfd7ZOdIvKTY4H7EALKzsBlB-tBvJgP7_H0TODW6w=.9b0ba2a9-8436-45fd-91dd-dae750ee3261@github.com>
<vf5jHhZkN1s3Rb6bdMKnJYp_GE5bFWPo17dQATU1_w0=.9ba84e38-e76e-4210-94cc-e84465da1cfc@github.com>
Message-ID: <cfiKz0aa5i3EwbVlv8hVI8b9siiT9KFbql7o8vyqh3M=.8f887a33-b7c4-4e33-b9bd-1fb2c8dd8b89@github.com>
On Tue, 3 Nov 2020 10:33:16 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:
>> Since I went back and forth about what this function did (it posted events at one time), I thought the generic _processing name would be better. GC callers shouldn't really have to know what processing we're doing here. Hopefully it won't change from rehashing. That's why I like processing.
>
>> GC callers shouldn't really have to know what processing we're doing here.
>
> I completely disagree with this. It's extremely important that the GC and Runtime code agrees on what this code does and where the GC *must* call it. Knowing the details allows us to skip calling this after mark end, but forces us to call it in relocate start, when objects should start to move. Though, I don't want to block this review because of this point, so if you still thinks that a non-descriptive name is better then we can argue that separately.
Ok, I'll rename it to needs_hashing.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From rwestrel at redhat.com Tue Nov 3 12:42:00 2020
From: rwestrel at redhat.com (Roland Westrelin)
Date: Tue, 03 Nov 2020 13:42:00 +0100
Subject: shenandoah c2 compiler crash in jdk8
In-Reply-To: <CAEyj=Y5S4+rfkaAOfr12_0v-5gOM3=wYRHAjY-GXTOa6bM+fYw@mail.gmail.com>
References: <CAEyj=Y5W1D-w=kLP26M02MkdOnP1WiPwVyXkqsNSdu6kT8SqaQ@mail.gmail.com>
<CAEyj=Y7Gjk8N8iNTz766cgtWcxnOW-MY=J4EyN0iKr4UeA6oQQ@mail.gmail.com>
<87a6w0duqx.fsf@redhat.com>
<CAEyj=Y5S4+rfkaAOfr12_0v-5gOM3=wYRHAjY-GXTOa6bM+fYw@mail.gmail.com>
Message-ID: <87h7q6d33b.fsf@redhat.com>
> also just in case there is original stack trace (from release build w/o
> assertions):
>
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> V [libjvm.so+0x4b6eae] Compile::final_graph_reshaping()+0x1e
> V [libjvm.so+0x4b923e] Compile::Optimize()+0xd8e
> V [libjvm.so+0x4bb217] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*,
> int, bool, bool, bool)+0x11c7
> V [libjvm.so+0x40fd8c] C2Compiler::compile_method(ciEnv*, ciMethod*,
> int)+0x20c
> V [libjvm.so+0x4c2096]
> CompileBroker::invoke_compiler_on_method(CompileTask*)+0xd46
> V [libjvm.so+0x4c4977] CompileBroker::compiler_thread_loop()+0x657
> V [libjvm.so+0xb22981] JavaThread::thread_main_inner()+0xf1
> V [libjvm.so+0x9674a2] java_start(Thread*)+0x132
> C [libpthread.so.0+0x7ea5] start_thread+0xc5
Thanks. I'm surprised the assert and the crash are related. The assert
catches the IR graph growing too big. That in itself shouldn't cause a
crash in the release build. I suppose the release build crash doesn't
reproduce once the node limit is bumped? Maybe that hides the bug rather
than work around it.
One way to reproduce the issues is by using replay compilation. The VM
should have dumped replay_pid files for the 2 crashes. Running:
java -XX:+ReplayCompiles -XX:ReplayDataFile=replay_pidxxx.log -XX:+ReplayIgnoreInitErrors
with the same classpath and compile time option as the actual runs is
often all it takes to reproduce the crash. The VM needs to find every
class that's mentioned in the replay file (it's a text file so you can
look into it).
If that works then and you were willing to share the class files I
should be able to reproduce it as well.
Roland.
From coleenp at openjdk.java.net Tue Nov 3 12:58:22 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 12:58:22 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
More review comments from Stefan and ErikO
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/cb4c83e0..eeaf9aed
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=01-02
Stats: 18 lines in 7 files changed: 3 ins; 6 del; 9 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Tue Nov 3 13:04:06 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 13:04:06 GMT
Subject: RFR: 8255762: Shenandoah: Consolidate/streamline interpreter LRBs
[v3]
In-Reply-To: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
References: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
Message-ID: <Bnz3I5Di8poLQzqAlGwSKyHbnaZR4gOpPAeBK1HnJuQ=.815a8165-bb50-4f41-a493-752672964957@github.com>
> We currently have two LRB implementations in interpreter: one normal and one for native/weak LRB. We should consolidate them into one.
>
> The main issue here is that the two implementations follow different approaches:
> - the normal LRB calls through the shenandoah_lrb stub which does additional null- and cset-checking
> - the weak LRB calls to runtime directly and must not do cset-checking
>
> The reason for calling through the stub is that it gives more freedom to allocate two registers that are required for the cset check. However, we can invert the cset addressing like we did in JDK-8245465 and save a register. We can also eliminate the null-check and let the cset-check subsume it (like we do everywhere else). Allocating a single register for the cset-check is easy, and we can do so in-line without the extra jump through the stub. The runtime call through the stub has also been very costly: it dumps 2KB of register data on the stack at each call, that is very excessive. save_xmm_registers() should be more than enough (in-fact, I am almost certain that this is excessive too, and we should only need to save/restore xmm0 - but not in this patch). Not needing to generate the call-stub is also helpful for backportability, because in jdk8-shenandoah we cannot do that.
>
> Testing: hotspot_gc_shenandoah (x86_64, x86_32, aarch64)
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
A few more touch-ups
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1010/files
- new: https://git.openjdk.java.net/jdk/pull/1010/files/12382e6c..f031885d
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1010&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1010&range=01-02
Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1010.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1010/head:pull/1010
PR: https://git.openjdk.java.net/jdk/pull/1010
From rkennke at openjdk.java.net Tue Nov 3 13:04:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 13:04:07 GMT
Subject: Integrated: 8255762: Shenandoah: Consolidate/streamline interpreter
LRBs
In-Reply-To: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
References: <umrFv01MzeFZgElG5p7XF3MM9xvWE-1FXor28IX8asg=.078d8040-a0d7-42ba-b06f-fcafa07aaffb@github.com>
Message-ID: <N6SVFcij0nws8inlkkoF6tpZ5p6B11s_rj7tYyYnuXA=.22302109-22f3-4a9a-b20f-7b565d68cc60@github.com>
On Mon, 2 Nov 2020 15:55:40 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> We currently have two LRB implementations in interpreter: one normal and one for native/weak LRB. We should consolidate them into one.
>
> The main issue here is that the two implementations follow different approaches:
> - the normal LRB calls through the shenandoah_lrb stub which does additional null- and cset-checking
> - the weak LRB calls to runtime directly and must not do cset-checking
>
> The reason for calling through the stub is that it gives more freedom to allocate two registers that are required for the cset check. However, we can invert the cset addressing like we did in JDK-8245465 and save a register. We can also eliminate the null-check and let the cset-check subsume it (like we do everywhere else). Allocating a single register for the cset-check is easy, and we can do so in-line without the extra jump through the stub. The runtime call through the stub has also been very costly: it dumps 2KB of register data on the stack at each call, that is very excessive. save_xmm_registers() should be more than enough (in-fact, I am almost certain that this is excessive too, and we should only need to save/restore xmm0 - but not in this patch). Not needing to generate the call-stub is also helpful for backportability, because in jdk8-shenandoah we cannot do that.
>
> Testing: hotspot_gc_shenandoah (x86_64, x86_32, aarch64)
This pull request has now been integrated.
Changeset: 93ef0091
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/93ef0091
Stats: 418 lines in 4 files changed: 41 ins; 308 del; 69 mod
8255762: Shenandoah: Consolidate/streamline interpreter LRBs
Reviewed-by: shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1010
From eosterlund at openjdk.java.net Tue Nov 3 13:12:02 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Tue, 3 Nov 2020 13:12:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
Message-ID: <4rQSjnt6O05gF9AFgc-nOxXIPehn_TKL0jJj85fvbr4=.c80aafac-a743-47d0-9d1a-31caa35d54ec@github.com>
On Tue, 3 Nov 2020 12:58:22 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> More review comments from Stefan and ErikO
Marked as reviewed by eosterlund (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Tue Nov 3 13:26:09 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 13:26:09 GMT
Subject: RFR: 8254315: Shenandoah: Concurrent weak reference processing
[v29]
In-Reply-To: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
References: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
Message-ID: <Jv2ZK8CLUhnZwPiIAp6v-R5bOXvFr6VwLW1wc-n2CoQ=.c5590bcf-91dc-4f70-becf-cad9a52265ba@github.com>
> Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.
>
> There are 3 main items that contribute to pause time linear to number of references, or worse:
> - We need to scan and consider each reference on the various 'discovered' lists.
> - We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
> - Finally, all no-longer-reachable references need to be enqueued in the 'pending list'
>
> The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.
>
> The solution to this is two-fold:
> 1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
> 2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.
>
> Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 87 commits:
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Invoke interpreter weak-LRB on weak-refs too (fixes merge mistake)
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Adopt ShenandoahReferenceBarrier to recent changes in LRB runtime impl
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Invert strong/weak in marking tasks and related code
- Fix merge mistake
- Merge branch 'master' into shenandoah-concurrent-weakrefs
- Pass marking-strength through chunked arrays
- Rename mark_final -> mark_weak and several cleanups (by shade)
- ... and 77 more: https://git.openjdk.java.net/jdk/compare/93ef0091...5e4a8f22
-------------
Changes: https://git.openjdk.java.net/jdk/pull/505/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=28
Stats: 2444 lines in 55 files changed: 1638 ins; 576 del; 230 mod
Patch: https://git.openjdk.java.net/jdk/pull/505.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/505/head:pull/505
PR: https://git.openjdk.java.net/jdk/pull/505
From rkennke at openjdk.java.net Tue Nov 3 13:34:17 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 13:34:17 GMT
Subject: RFR: 8254315: Shenandoah: Concurrent weak reference processing
[v30]
In-Reply-To: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
References: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
Message-ID: <e47ISC9yWkC4ypJ7E9S6sqVmEElF044sKxxpY_WiYJU=.0be08a6f-c4c9-4eea-a83e-1c8c044841db@github.com>
> Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.
>
> There are 3 main items that contribute to pause time linear to number of references, or worse:
> - We need to scan and consider each reference on the various 'discovered' lists.
> - We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
> - Finally, all no-longer-reachable references need to be enqueued in the 'pending list'
>
> The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.
>
> The solution to this is two-fold:
> 1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
> 2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.
>
> Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Invert check for access kind (merge mistake)
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/505/files
- new: https://git.openjdk.java.net/jdk/pull/505/files/5e4a8f22..58dead58
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=29
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=28-29
Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod
Patch: https://git.openjdk.java.net/jdk/pull/505.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/505/head:pull/505
PR: https://git.openjdk.java.net/jdk/pull/505
From zgu at openjdk.java.net Tue Nov 3 13:41:11 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 3 Nov 2020 13:41:11 GMT
Subject: RFR: 8255606: Enable concurrent stack processing on x86_32
platforms [v3]
In-Reply-To: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
Message-ID: <qRI016OxhTrE34h_QLWlgQhs4CEfoZfWQ3uvorIOBAE=.f130b6e7-cc74-4347-9877-f238e9076b43@github.com>
> 8255606: Enable concurrent stack processing on x86_32 platforms
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Reverted back to cmpq/cmpl
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/945/files
- new: https://git.openjdk.java.net/jdk/pull/945/files/c946b816..d6a88224
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=945&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=945&range=01-02
Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/945.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/945/head:pull/945
PR: https://git.openjdk.java.net/jdk/pull/945
From zgu at openjdk.java.net Tue Nov 3 13:44:13 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 3 Nov 2020 13:44:13 GMT
Subject: RFR: 8255606: Enable concurrent stack processing on x86_32
platforms [v4]
In-Reply-To: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
Message-ID: <FGzWPusWrI_AYSOeaCoE1ZIjzzAw6ppEsWZNTp2csQU=.59705d2e-bc35-487c-9cb5-45113a6cd7a0@github.com>
> 8255606: Enable concurrent stack processing on x86_32 platforms
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Capitalize instructions in x86_32.ad
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/945/files
- new: https://git.openjdk.java.net/jdk/pull/945/files/d6a88224..28c69c49
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=945&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=945&range=02-03
Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/945.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/945/head:pull/945
PR: https://git.openjdk.java.net/jdk/pull/945
From zgu at openjdk.java.net Tue Nov 3 13:44:15 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 3 Nov 2020 13:44:15 GMT
Subject: RFR: 8255606: Enable concurrent stack processing on x86_32
platforms [v2]
In-Reply-To: <3UnxY32uP-ERCn8odu787AX2OqPH8soXglsanBwpz3Q=.8eb97050-dd9e-4294-861c-aa5c3f3a90e2@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
<THq75VMpdQw4i0lW6Cnp_HjWbas5NBF1lqb31WzNr54=.7cfd7a18-8633-4ff9-9170-b61301763a47@github.com>
<3UnxY32uP-ERCn8odu787AX2OqPH8soXglsanBwpz3Q=.8eb97050-dd9e-4294-861c-aa5c3f3a90e2@github.com>
Message-ID: <5JJIeaZhiTez4B7sDxCW0OYD86cuhCk6jQXGXzPucow=.36b27a50-5e05-4089-ba79-15fc707dda9d@github.com>
On Tue, 3 Nov 2020 12:09:36 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Fix jump direction
>
> src/hotspot/cpu/x86/x86_32.ad line 656:
>
>> 654: st->print_cr("POPL EBP"); st->print("\t");
>> 655: if (do_polling() && C->is_method_compilation()) {
>> 656: st->print("cmpptr rsp, poll_offset[thread] \n\t"
>
> Let's say `CMPL` here? I don't think we reference `cmpptr` in outputs here. Also, note every instruction is capitalized in `x86_32.ad`.
Thanks for the review. Updated according to your suggestions.
-------------
PR: https://git.openjdk.java.net/jdk/pull/945
From nevgeniev at gmail.com Tue Nov 3 14:34:21 2020
From: nevgeniev at gmail.com (Nick Evgeniev)
Date: Tue, 3 Nov 2020 08:34:21 -0600
Subject: shenandoah c2 compiler crash in jdk8
In-Reply-To: <87h7q6d33b.fsf@redhat.com>
References: <CAEyj=Y5W1D-w=kLP26M02MkdOnP1WiPwVyXkqsNSdu6kT8SqaQ@mail.gmail.com>
<CAEyj=Y7Gjk8N8iNTz766cgtWcxnOW-MY=J4EyN0iKr4UeA6oQQ@mail.gmail.com>
<87a6w0duqx.fsf@redhat.com>
<CAEyj=Y5S4+rfkaAOfr12_0v-5gOM3=wYRHAjY-GXTOa6bM+fYw@mail.gmail.com>
<87h7q6d33b.fsf@redhat.com>
Message-ID: <CAEyj=Y74LNeqj16Y2XHdNUg1-QfXH4QLuVY0-svfTuNtCtRvAw@mail.gmail.com>
hi,
yes, replay is what I'm doing (i.e. all my previous posts are based on
-XX:+ReplayCompiles)... crash in prod is hard to reproduce
On Tue, 3 Nov 2020 at 06:42, Roland Westrelin <rwestrel at redhat.com> wrote:
>
> > also just in case there is original stack trace (from release build w/o
> > assertions):
> >
> > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> > code)
> > V [libjvm.so+0x4b6eae] Compile::final_graph_reshaping()+0x1e
> > V [libjvm.so+0x4b923e] Compile::Optimize()+0xd8e
> > V [libjvm.so+0x4bb217] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*,
> > int, bool, bool, bool)+0x11c7
> > V [libjvm.so+0x40fd8c] C2Compiler::compile_method(ciEnv*, ciMethod*,
> > int)+0x20c
> > V [libjvm.so+0x4c2096]
> > CompileBroker::invoke_compiler_on_method(CompileTask*)+0xd46
> > V [libjvm.so+0x4c4977] CompileBroker::compiler_thread_loop()+0x657
> > V [libjvm.so+0xb22981] JavaThread::thread_main_inner()+0xf1
> > V [libjvm.so+0x9674a2] java_start(Thread*)+0x132
> > C [libpthread.so.0+0x7ea5] start_thread+0xc5
>
> Thanks. I'm surprised the assert and the crash are related. The assert
> catches the IR graph growing too big. That in itself shouldn't cause a
> crash in the release build. I suppose the release build crash doesn't
> reproduce once the node limit is bumped? Maybe that hides the bug rather
> than work around it.
>
> One way to reproduce the issues is by using replay compilation. The VM
> should have dumped replay_pid files for the 2 crashes. Running:
>
> java -XX:+ReplayCompiles -XX:ReplayDataFile=replay_pidxxx.log
> -XX:+ReplayIgnoreInitErrors
>
> with the same classpath and compile time option as the actual runs is
> often all it takes to reproduce the crash. The VM needs to find every
> class that's mentioned in the replay file (it's a text file so you can
> look into it).
>
> If that works then and you were willing to share the class files I
> should be able to reproduce it as well.
>
> Roland.
>
>
From rkennke at openjdk.java.net Tue Nov 3 14:36:19 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 14:36:19 GMT
Subject: RFR: 8254315: Shenandoah: Concurrent weak reference processing
[v31]
In-Reply-To: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
References: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
Message-ID: <dCk9aOCRoFijupdllTQk5d_AkpVxCT8QeOFCkxAUW0c=.2398a3f7-6e5f-49ef-9c31-dd7e0ff00f0a@github.com>
> Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.
>
> There are 3 main items that contribute to pause time linear to number of references, or worse:
> - We need to scan and consider each reference on the various 'discovered' lists.
> - We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
> - Finally, all no-longer-reachable references need to be enqueued in the 'pending list'
>
> The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.
>
> The solution to this is two-fold:
> 1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
> 2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.
>
> Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
AArch64 build fixes
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/505/files
- new: https://git.openjdk.java.net/jdk/pull/505/files/58dead58..9165b6e5
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=30
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=29-30
Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
Patch: https://git.openjdk.java.net/jdk/pull/505.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/505/head:pull/505
PR: https://git.openjdk.java.net/jdk/pull/505
From coleenp at openjdk.java.net Tue Nov 3 14:53:12 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 14:53:12 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
- Merge branch 'master' into jvmti-table
- Merge branch 'master' into jvmti-table
- More review comments from Stefan and ErikO
- Code review comments from StefanK.
- 8212879: Make JVMTI TagMap table not hash on oop address
-------------
Changes: https://git.openjdk.java.net/jdk/pull/967/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=03
Stats: 1749 lines in 41 files changed: 627 ins; 1020 del; 102 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Tue Nov 3 16:50:00 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 16:50:00 GMT
Subject: RFR: 8255606: Enable concurrent stack processing on x86_32
platforms [v4]
In-Reply-To: <FGzWPusWrI_AYSOeaCoE1ZIjzzAw6ppEsWZNTp2csQU=.59705d2e-bc35-487c-9cb5-45113a6cd7a0@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
<FGzWPusWrI_AYSOeaCoE1ZIjzzAw6ppEsWZNTp2csQU=.59705d2e-bc35-487c-9cb5-45113a6cd7a0@github.com>
Message-ID: <jHrHo0fm_jB2_sl75r6-jaBBbvWiT5K9BvRXsdFjR9E=.c1d24eef-4e57-4833-b19f-1ca7a74e508e@github.com>
On Tue, 3 Nov 2020 13:44:13 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> 8255606: Enable concurrent stack processing on x86_32 platforms
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>
> Capitalize instructions in x86_32.ad
Looks good to me! Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/945
From eosterlund at openjdk.java.net Tue Nov 3 17:06:56 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Tue, 3 Nov 2020 17:06:56 GMT
Subject: RFR: 8255606: Enable concurrent stack processing on x86_32
platforms [v4]
In-Reply-To: <FGzWPusWrI_AYSOeaCoE1ZIjzzAw6ppEsWZNTp2csQU=.59705d2e-bc35-487c-9cb5-45113a6cd7a0@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
<FGzWPusWrI_AYSOeaCoE1ZIjzzAw6ppEsWZNTp2csQU=.59705d2e-bc35-487c-9cb5-45113a6cd7a0@github.com>
Message-ID: <fqy5El0dYMvAPuQhyBwCn21ZlB1tURZwggtIRk4EFNU=.50061658-2933-4c13-b8f3-aa278f4ae410@github.com>
On Tue, 3 Nov 2020 13:44:13 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> 8255606: Enable concurrent stack processing on x86_32 platforms
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>
> Capitalize instructions in x86_32.ad
Looks good. Nice to see one more platform support this. Hoping that eventually all platforms will support this.
-------------
Marked as reviewed by eosterlund (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/945
From zgu at openjdk.java.net Tue Nov 3 17:12:55 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 3 Nov 2020 17:12:55 GMT
Subject: RFR: 8255606: Enable concurrent stack processing on x86_32
platforms [v4]
In-Reply-To: <fqy5El0dYMvAPuQhyBwCn21ZlB1tURZwggtIRk4EFNU=.50061658-2933-4c13-b8f3-aa278f4ae410@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
<FGzWPusWrI_AYSOeaCoE1ZIjzzAw6ppEsWZNTp2csQU=.59705d2e-bc35-487c-9cb5-45113a6cd7a0@github.com>
<fqy5El0dYMvAPuQhyBwCn21ZlB1tURZwggtIRk4EFNU=.50061658-2933-4c13-b8f3-aa278f4ae410@github.com>
Message-ID: <rut1LJ9fKuvKGneVlODGdkr854KZWlmU11T3iWV7SeQ=.9d77658d-8482-4a80-81b1-633a99177eba@github.com>
On Tue, 3 Nov 2020 17:04:33 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
> Looks good. Nice to see one more platform support this. Hoping that eventually all platforms will support this.
Thanks, Erik.
-------------
PR: https://git.openjdk.java.net/jdk/pull/945
From shade at openjdk.java.net Tue Nov 3 17:28:59 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 3 Nov 2020 17:28:59 GMT
Subject: RFR: 8255606: Enable concurrent stack processing on x86_32
platforms [v4]
In-Reply-To: <FGzWPusWrI_AYSOeaCoE1ZIjzzAw6ppEsWZNTp2csQU=.59705d2e-bc35-487c-9cb5-45113a6cd7a0@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
<FGzWPusWrI_AYSOeaCoE1ZIjzzAw6ppEsWZNTp2csQU=.59705d2e-bc35-487c-9cb5-45113a6cd7a0@github.com>
Message-ID: <EYf_garT33A2mANQ_dit2fTK218EEFlvDeYsm4oIv08=.4390d996-8156-4570-8129-7221294b35e4@github.com>
On Tue, 3 Nov 2020 13:44:13 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> 8255606: Enable concurrent stack processing on x86_32 platforms
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>
> Capitalize instructions in x86_32.ad
Marked as reviewed by shade (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/945
From zgu at openjdk.java.net Tue Nov 3 17:29:00 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 3 Nov 2020 17:29:00 GMT
Subject: Integrated: 8255606: Enable concurrent stack processing on x86_32
platforms
In-Reply-To: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
References: <YPHH-fLZnuS13dMQFi_m1Q2Btp8_2TfaIczqbSSQ4e4=.c09a37c7-e048-4659-9853-d3b9aad61e4d@github.com>
Message-ID: <zm-g0rPGhj1k88XEAGkr_gjenGTXKK2f6j9j6oDKgHI=.41dcad64-c282-4ae7-972b-5060f0ea8bcb@github.com>
On Thu, 29 Oct 2020 20:40:30 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> 8255606: Enable concurrent stack processing on x86_32 platforms
This pull request has now been integrated.
Changeset: 134e22a0
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/134e22a0
Stats: 63 lines in 7 files changed: 34 ins; 14 del; 15 mod
8255606: Enable concurrent stack processing on x86_32 platforms
Reviewed-by: shade, rkennke, eosterlund
-------------
PR: https://git.openjdk.java.net/jdk/pull/945
From kbarrett at openjdk.java.net Tue Nov 3 18:12:10 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Tue, 3 Nov 2020 18:12:10 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
Message-ID: <ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
On Tue, 3 Nov 2020 14:53:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>
> - Merge branch 'master' into jvmti-table
> - Merge branch 'master' into jvmti-table
> - More review comments from Stefan and ErikO
> - Code review comments from StefanK.
> - 8212879: Make JVMTI TagMap table not hash on oop address
src/hotspot/share/prims/jvmtiTagMap.cpp line 3018:
> 3016: }
> 3017: // Later GC code will relocate the oops, so defer rehashing until then.
> 3018: tag_map->_needs_rehashing = true;
This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.)
For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing.
(I think CMS oldgen remark may have been similar, for what that's worth.)
src/hotspot/share/prims/jvmtiTagMap.cpp line 3015:
> 3013: if (tag_map != NULL && !tag_map->is_empty()) {
> 3014: if (num_dead_entries != 0) {
> 3015: tag_map->hashmap()->unlink_and_post(tag_map->env());
Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup.
src/hotspot/share/prims/jvmtiTagMap.cpp line 2979:
> 2977:
> 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved
> 2979: // and have their new addresses, the table can be rehashed.
I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that).
src/hotspot/share/prims/jvmtiTagMap.cpp line 127:
> 125: // The table cleaning, posting and rehashing can race for
> 126: // concurrent GCs. So fix it here once we have a lock or are
> 127: // at a safepoint.
I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time).
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From kbarrett at openjdk.java.net Tue Nov 3 18:12:07 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Tue, 3 Nov 2020 18:12:07 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
Message-ID: <Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
On Tue, 3 Nov 2020 12:58:22 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> More review comments from Stefan and ErikO
src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 41:
> 39: class Iterator;
> 40:
> 41: typedef void (*Processor)(BoolObjectClosure*, OopClosure*);
I think this typedef is to support serial phases and that it is probably no longer used.
src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50:
> 48: };
> 49:
> 50: typedef uint WeakProcessorPhase;
This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes.
But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here:
src/hotspot/share/gc/shared/weakProcessor.inline.hpp
116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase);
117 StorageState* cur_state = _storage_states.par_state(oopstorage_index);
=>
103 StorageState* cur_state = _storage_states.par_state(phase);
I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet.
enum class : uint WeakProcessorPhase {};
ENUMERATOR_RANGE(WeakProcessorPhase,
static_cast<WeakProcessorPhase>(0),
static_cast<WeakProcessorPhase>(OopStorageSet::weak_count));
and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator<WeakProcessorPhase> (which involves more than a type alias).
Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From shade at openjdk.java.net Tue Nov 3 18:24:05 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 3 Nov 2020 18:24:05 GMT
Subject: RFR: 8254315: Shenandoah: Concurrent weak reference processing
[v31]
In-Reply-To: <dCk9aOCRoFijupdllTQk5d_AkpVxCT8QeOFCkxAUW0c=.2398a3f7-6e5f-49ef-9c31-dd7e0ff00f0a@github.com>
References: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
<dCk9aOCRoFijupdllTQk5d_AkpVxCT8QeOFCkxAUW0c=.2398a3f7-6e5f-49ef-9c31-dd7e0ff00f0a@github.com>
Message-ID: <2rSnA-jJHIly2sT0ZMSZbo0hNasGUgeylnJ9jVs5QBU=.aa5c1c5f-1392-43be-97f3-d21eb679467b@github.com>
On Tue, 3 Nov 2020 14:36:19 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.
>>
>> There are 3 main items that contribute to pause time linear to number of references, or worse:
>> - We need to scan and consider each reference on the various 'discovered' lists.
>> - We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
>> - Finally, all no-longer-reachable references need to be enqueued in the 'pending list'
>>
>> The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.
>>
>> The solution to this is two-fold:
>> 1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
>> 2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.
>>
>> Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> AArch64 build fixes
Looks fine, modulo few nits below:
src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp line 56:
> 54: _load_reference_barrier_normal_rt_code_blob(NULL),
> 55: _load_reference_barrier_native_rt_code_blob(NULL),
> 56: _load_reference_barrier_weakref_rt_code_blob(NULL) {}
`weakref` or `weak`? `weakref` looks like new nomenclature to me.
src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 1066:
> 1064: if (in1->bottom_type() == TypePtr::NULL_PTR &&
> 1065: !((in2->Opcode() == Op_ShenandoahLoadReferenceBarrier) &&
> 1066: ((ShenandoahLoadReferenceBarrierNode*)in2)->kind() != ShenandoahBarrierSet::AccessKind::NORMAL)) {
The comment "LRB native" deserves a new wording now? I.e. "then step over normal LRB barriers".
src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.hpp line 53:
> 51:
> 52: template <class T>
> 53: inline void do_chunked_array_start(ShenandoahObjToScanQueue* q, T* cl, oop array, bool strong);
This should be `bool weak`?
src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.hpp line 56:
> 54:
> 55: template <class T>
> 56: inline void do_chunked_array(ShenandoahObjToScanQueue* q, T* cl, oop array, int chunk, int pow, bool strong);
This should be `bool weak`?
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/505
From rkennke at openjdk.java.net Tue Nov 3 19:03:11 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 19:03:11 GMT
Subject: RFR: 8254315: Shenandoah: Concurrent weak reference processing
[v32]
In-Reply-To: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
References: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
Message-ID: <8LETdj0OOts00biRQDWKIr-FWZ2rPQf-DrdOnjxz68s=.6e174122-8317-48a1-8e49-6fd2da1d1230@github.com>
> Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.
>
> There are 3 main items that contribute to pause time linear to number of references, or worse:
> - We need to scan and consider each reference on the various 'discovered' lists.
> - We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
> - Finally, all no-longer-reachable references need to be enqueued in the 'pending list'
>
> The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.
>
> The solution to this is two-fold:
> 1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
> 2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.
>
> Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification
Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
- Merge remote-tracking branch 'origin/shenandoah-concurrent-weakrefs' into shenandoah-concurrent-weakrefs
- Final touch-ups
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/505/files
- new: https://git.openjdk.java.net/jdk/pull/505/files/9165b6e5..9f367a7c
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=31
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=505&range=30-31
Stats: 14 lines in 6 files changed: 0 ins; 0 del; 14 mod
Patch: https://git.openjdk.java.net/jdk/pull/505.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/505/head:pull/505
PR: https://git.openjdk.java.net/jdk/pull/505
From rkennke at openjdk.java.net Tue Nov 3 19:03:12 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 19:03:12 GMT
Subject: Integrated: 8254315: Shenandoah: Concurrent weak reference processing
In-Reply-To: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
References: <8byaPRNFSF4tG_fA2jxtiDwcEbbMS_Zmk39w86ugIV4=.6a942481-9fd0-44f7-a42a-3668b22bea3e@github.com>
Message-ID: <XJXqKOt7ccYpB_L7UKdDLXYj5HeNOUa9bTnExSxi2o0=.304adf26-3e09-4f7a-a6ee-1d8d0eb36209@github.com>
On Mon, 5 Oct 2020 13:42:02 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.
>
> There are 3 main items that contribute to pause time linear to number of references, or worse:
> - We need to scan and consider each reference on the various 'discovered' lists.
> - We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
> - Finally, all no-longer-reachable references need to be enqueued in the 'pending list'
>
> The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.
>
> The solution to this is two-fold:
> 1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
> 2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.
>
> Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification
This pull request has now been integrated.
Changeset: f64a15d6
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/f64a15d6
Stats: 2440 lines in 55 files changed: 1638 ins; 576 del; 226 mod
8254315: Shenandoah: Concurrent weak reference processing
Reviewed-by: zgu, shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/505
From rkennke at openjdk.java.net Tue Nov 3 20:40:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 20:40:07 GMT
Subject: Integrated: 8255852: Shenandoah: Fix incorrect copyright header in
new files added by 8254315
Message-ID: <KCXi0LEJ1Rqw4WhmgBQ1TtKIpg1xvRXHCKVimNrV3OA=.5c355a70-2ec8-47ad-a787-decf075d2e98@github.com>
I apparently pushed files with broken copyright:
src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.cpp
src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.hpp
src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.inline.hpp
src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp
src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.hpp
They are missing the line:
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
Testing: build (x86_64)
-------------
Commit messages:
- 8255852: Shenandoah: Fix incorrect copyright header in new files added by 8254315
Changes: https://git.openjdk.java.net/jdk/pull/1042/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1042&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255852
Stats: 5 lines in 5 files changed: 5 ins; 0 del; 0 mod
Patch: https://git.openjdk.java.net/jdk/pull/1042.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1042/head:pull/1042
PR: https://git.openjdk.java.net/jdk/pull/1042
From zgu at openjdk.java.net Tue Nov 3 20:40:08 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 3 Nov 2020 20:40:08 GMT
Subject: Integrated: 8255852: Shenandoah: Fix incorrect copyright header
in new files added by 8254315
In-Reply-To: <KCXi0LEJ1Rqw4WhmgBQ1TtKIpg1xvRXHCKVimNrV3OA=.5c355a70-2ec8-47ad-a787-decf075d2e98@github.com>
References: <KCXi0LEJ1Rqw4WhmgBQ1TtKIpg1xvRXHCKVimNrV3OA=.5c355a70-2ec8-47ad-a787-decf075d2e98@github.com>
Message-ID: <sQNACZ6nybZ_2yeXtr-hMAxss6CXEkI2wGW21nVbY5I=.85f1419e-db83-404b-9077-ae0b4f326561@github.com>
On Tue, 3 Nov 2020 20:26:26 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> I apparently pushed files with broken copyright:
>
> src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.cpp
> src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.hpp
> src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.inline.hpp
> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp
> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.hpp
>
> They are missing the line:
>
> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>
> Testing: build (x86_64)
Looks good and trivial
-------------
Marked as reviewed by zgu (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1042
From rkennke at openjdk.java.net Tue Nov 3 20:40:08 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 3 Nov 2020 20:40:08 GMT
Subject: Integrated: 8255852: Shenandoah: Fix incorrect copyright header in
new files added by 8254315
In-Reply-To: <KCXi0LEJ1Rqw4WhmgBQ1TtKIpg1xvRXHCKVimNrV3OA=.5c355a70-2ec8-47ad-a787-decf075d2e98@github.com>
References: <KCXi0LEJ1Rqw4WhmgBQ1TtKIpg1xvRXHCKVimNrV3OA=.5c355a70-2ec8-47ad-a787-decf075d2e98@github.com>
Message-ID: <nABWnODiWotgBAXzIEhFP1s3UDRwmmaLyO2SZdiLG8g=.8745a0d1-0b34-4e85-97b4-01e122f550fe@github.com>
On Tue, 3 Nov 2020 20:26:26 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> I apparently pushed files with broken copyright:
>
> src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.cpp
> src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.hpp
> src/hotspot/share/gc/shenandoah/shenandoahMarkBitMap.inline.hpp
> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp
> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.hpp
>
> They are missing the line:
>
> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>
> Testing: build (x86_64)
This pull request has now been integrated.
Changeset: eab99f37
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/eab99f37
Stats: 5 lines in 5 files changed: 5 ins; 0 del; 0 mod
8255852: Shenandoah: Fix incorrect copyright header in new files added by 8254315
Reviewed-by: zgu
-------------
PR: https://git.openjdk.java.net/jdk/pull/1042
From coleenp at openjdk.java.net Tue Nov 3 21:17:05 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 21:17:05 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
Message-ID: <D0tXRBpNv7FmHN9iMZdiHmDpVXcXD42Rhf04Mo7XSs8=.7a75f15b-27b5-4c08-8251-c0d4cb1ba282@github.com>
On Tue, 3 Nov 2020 14:47:35 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>>
>> - Merge branch 'master' into jvmti-table
>> - Merge branch 'master' into jvmti-table
>> - More review comments from Stefan and ErikO
>> - Code review comments from StefanK.
>> - 8212879: Make JVMTI TagMap table not hash on oop address
>
> src/hotspot/share/prims/jvmtiTagMap.cpp line 3018:
>
>> 3016: }
>> 3017: // Later GC code will relocate the oops, so defer rehashing until then.
>> 3018: tag_map->_needs_rehashing = true;
>
> This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.)
>
> For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing.
>
> (I think CMS oldgen remark may have been similar, for what that's worth.)
Ok, so I'm going to need help to know where in all the different GCs to make this call.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From ayang at openjdk.java.net Tue Nov 3 21:17:03 2020
From: ayang at openjdk.java.net (Albert Mingkun Yang)
Date: Tue, 3 Nov 2020 21:17:03 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
Message-ID: <ffo3ekNFPb1U0A9CrXkZfSNbwn09QN3v2bFRuPBHn6c=.eb2d1bcd-ee82-473a-b023-8cbe53626252@github.com>
On Tue, 3 Nov 2020 14:53:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>
> - Merge branch 'master' into jvmti-table
> - Merge branch 'master' into jvmti-table
> - More review comments from Stefan and ErikO
> - Code review comments from StefanK.
> - 8212879: Make JVMTI TagMap table not hash on oop address
src/hotspot/share/utilities/hashtable.cpp line 164:
> 162: }
> 163: }
> 164: return newsize;
It is existing code, but could be made clearer as part of this PR:
int newsize;
for (int i=0; i<arraysize; ++i) {
newsize = primelist[i];
if (newsize >= requested)
break;
}
return newsize;
Additionally, this method could be made `const`, right?
PS: not a review, just a comment in passing
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From daniel.daugherty at oracle.com Tue Nov 3 21:18:23 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 3 Nov 2020 16:18:23 -0500
Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address
In-Reply-To: <AqYzgbnx8MN7P51EeIFg9YQjdPl0P787Gz-qrzFXjTc=.d6d74b88-1e0d-4191-b55c-c600c001f942@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<gwwfs935rEoSy2oPvdKGp65rM1vswSwLBIaGsrdZnAg=.0cd30f1a-517b-4163-8fff-70268b8840cd@github.com>
<wwqY3sh8_y7Z4vBJyiI6hvFIc_D7Kc30Qxul2wG_E5w=.6d0863e3-c849-4f09-9366-c4be641299cf@github.com>
<AqYzgbnx8MN7P51EeIFg9YQjdPl0P787Gz-qrzFXjTc=.d6d74b88-1e0d-4191-b55c-c600c001f942@github.com>
Message-ID: <3fb77c6e-a99b-c90b-9170-1ce5fca6df1b@oracle.com>
On 11/2/20 8:22 AM, Coleen Phillimore wrote:
> On Mon, 2 Nov 2020 08:34:17 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:
>
>>> src/hotspot/share/prims/jvmtiTagMap.cpp line 126:
>>>
>>>> 124: // concurrent GCs. So fix it here once we have a lock or are
>>>> 125: // at a safepoint.
>>>> 126: // SetTag and GetTag should not post events!
>>> I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case.
>> Maybe even move this comment to the set_tag/get_tag code.
> I was trying to explain why there's a boolean there but I can put this comment at both get_tag and set_tag.
>
> // Check if we have to processing for concurrent GCs.
Typo: s/we have to processing/we have to do processing/
> // GetTag should not post events because the JavaThread has to
> // transition to native for the callback and this cannot stop for
> // safepoints with the hashmap lock held.
> check_hashmap(false);
>
> -------------
>
> PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 21:28:08 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 21:28:08 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
Message-ID: <SqMjFCfYm0xIPR4fnCzA70iK3wzDED4aO3dASmeCd6E=.30b2eb69-2ece-4a62-9409-3e91aded8e75@github.com>
On Tue, 3 Nov 2020 14:50:36 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>>
>> - Merge branch 'master' into jvmti-table
>> - Merge branch 'master' into jvmti-table
>> - More review comments from Stefan and ErikO
>> - Code review comments from StefanK.
>> - 8212879: Make JVMTI TagMap table not hash on oop address
>
> src/hotspot/share/prims/jvmtiTagMap.cpp line 3015:
>
>> 3013: if (tag_map != NULL && !tag_map->is_empty()) {
>> 3014: if (num_dead_entries != 0) {
>> 3015: tag_map->hashmap()->unlink_and_post(tag_map->env());
>
> Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup.
The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From dcubed at openjdk.java.net Tue Nov 3 21:28:06 2020
From: dcubed at openjdk.java.net (Daniel D.Daugherty)
Date: Tue, 3 Nov 2020 21:28:06 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <g4Yzu6YAIY0IWKDX0GGcL4qD_vRGPrnUQt4nJKDdIhM=.981f28f1-6b4c-4014-baef-b8b162ee7ff4@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<g4Yzu6YAIY0IWKDX0GGcL4qD_vRGPrnUQt4nJKDdIhM=.981f28f1-6b4c-4014-baef-b8b162ee7ff4@github.com>
Message-ID: <6uV_hLuxf7fu90EgzZckv9LwT-jAVboukH35MKaTtcU=.40efbbd6-fd6e-427a-a940-784ec86ddb15@github.com>
On Mon, 2 Nov 2020 13:19:08 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>>
>> - Merge branch 'master' into jvmti-table
>> - Merge branch 'master' into jvmti-table
>> - More review comments from Stefan and ErikO
>> - Code review comments from StefanK.
>> - 8212879: Make JVMTI TagMap table not hash on oop address
>
> I think I addressed your comments, retesting now. Thank you!
@coleenp - please make sure you hear from someone on the Serviceability team
for this PR...
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 21:33:59 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 21:33:59 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
Message-ID: <Spqq9dORyH2M6ZrkCpGKDFA1jWfFTbiqxl3gNcQEPJk=.c964ac35-062a-4471-829a-856f5ec1c04e@github.com>
On Tue, 3 Nov 2020 16:12:21 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>>
>> - Merge branch 'master' into jvmti-table
>> - Merge branch 'master' into jvmti-table
>> - More review comments from Stefan and ErikO
>> - Code review comments from StefanK.
>> - 8212879: Make JVMTI TagMap table not hash on oop address
>
> src/hotspot/share/prims/jvmtiTagMap.cpp line 2979:
>
>> 2977:
>> 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved
>> 2979: // and have their new addresses, the table can be rehashed.
>
> I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that).
So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case?
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 21:46:04 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 21:46:04 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
Message-ID: <jKyDBy89u-3EdYr0CCKsfGI-SM7FgGWCTFtG9YWk6yw=.196f1a2d-ac5d-4a6d-99cc-48854e747736@github.com>
On Tue, 3 Nov 2020 16:17:58 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>>
>> - Merge branch 'master' into jvmti-table
>> - Merge branch 'master' into jvmti-table
>> - More review comments from Stefan and ErikO
>> - Code review comments from StefanK.
>> - 8212879: Make JVMTI TagMap table not hash on oop address
>
> src/hotspot/share/prims/jvmtiTagMap.cpp line 127:
>
>> 125: // The table cleaning, posting and rehashing can race for
>> 126: // concurrent GCs. So fix it here once we have a lock or are
>> 127: // at a safepoint.
>
> I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time).
The comment is trying to describe the situation like:
1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked)
2. safepoint for heap walk
2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A.
3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened
The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable.
Both operations require the table to be locked.
The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 21:46:02 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 21:46:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <g4Yzu6YAIY0IWKDX0GGcL4qD_vRGPrnUQt4nJKDdIhM=.981f28f1-6b4c-4014-baef-b8b162ee7ff4@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<g4Yzu6YAIY0IWKDX0GGcL4qD_vRGPrnUQt4nJKDdIhM=.981f28f1-6b4c-4014-baef-b8b162ee7ff4@github.com>
Message-ID: <tp6ulT2TOH7d-yku79KX1w8rw4eHsb3kJlV7itN7cFE=.39011947-3605-465d-b643-ff22ff231ea9@github.com>
On Mon, 2 Nov 2020 13:19:08 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>>
>> - Merge branch 'master' into jvmti-table
>> - Merge branch 'master' into jvmti-table
>> - More review comments from Stefan and ErikO
>> - Code review comments from StefanK.
>> - 8212879: Make JVMTI TagMap table not hash on oop address
>
> I think I addressed your comments, retesting now. Thank you!
> @coleenp - please make sure you hear from someone on the Serviceability team
> for this PR...
I've asked @sspitsyn to review this.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 21:46:05 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 21:46:05 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
<Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
Message-ID: <uP1wFQb7tSiU8MRnqMeUhvH1vzIuu4J4f1pVKVfbwzA=.de34b0d0-7c7f-49b1-bd18-8adc945b9284@github.com>
On Tue, 3 Nov 2020 13:43:32 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>>
>> More review comments from Stefan and ErikO
>
> src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 41:
>
>> 39: class Iterator;
>> 40:
>> 41: typedef void (*Processor)(BoolObjectClosure*, OopClosure*);
>
> I think this typedef is to support serial phases and that it is probably no longer used.
ok, removed.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 21:51:03 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 21:51:03 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
<Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
Message-ID: <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com>
On Tue, 3 Nov 2020 13:45:57 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>>
>> More review comments from Stefan and ErikO
>
> src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50:
>
>> 48: };
>> 49:
>> 50: typedef uint WeakProcessorPhase;
>
> This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes.
>
> But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here:
>
> src/hotspot/share/gc/shared/weakProcessor.inline.hpp
> 116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase);
> 117 StorageState* cur_state = _storage_states.par_state(oopstorage_index);
> =>
> 103 StorageState* cur_state = _storage_states.par_state(phase);
>
> I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet.
> enum class : uint WeakProcessorPhase {};
>
> ENUMERATOR_RANGE(WeakProcessorPhase,
> static_cast<WeakProcessorPhase>(0),
> static_cast<WeakProcessorPhase>(OopStorageSet::weak_count));
> and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator<WeakProcessorPhase> (which involves more than a type alias).
>
> Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet.
Ok, so I'm not sure what to do with this:
enum Phase {
// Serial phase.
JVMTI_ONLY(jvmti)
// Additional implicit phase values follow for oopstorages.
`};`
I've removed the only thing in this enum.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From sspitsyn at openjdk.java.net Tue Nov 3 22:23:00 2020
From: sspitsyn at openjdk.java.net (Serguei Spitsyn)
Date: Tue, 3 Nov 2020 22:23:00 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <tp6ulT2TOH7d-yku79KX1w8rw4eHsb3kJlV7itN7cFE=.39011947-3605-465d-b643-ff22ff231ea9@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<g4Yzu6YAIY0IWKDX0GGcL4qD_vRGPrnUQt4nJKDdIhM=.981f28f1-6b4c-4014-baef-b8b162ee7ff4@github.com>
<tp6ulT2TOH7d-yku79KX1w8rw4eHsb3kJlV7itN7cFE=.39011947-3605-465d-b643-ff22ff231ea9@github.com>
Message-ID: <hS6JlWfBo7TKdbJd2w-2Jwxq2fraqxEQFrwtNkL81nI=.0e4f2b18-7b4b-4d85-9de6-bd6ccdf5690e@github.com>
On Tue, 3 Nov 2020 21:41:55 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
> > @coleenp - please make sure you hear from someone on the Serviceability team
> > for this PR...
>
> I've asked @sspitsyn to review this.
Yes, I'm reviewing this. Still need another pass.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 23:41:02 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 23:41:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
<Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
<0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com>
Message-ID: <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com>
On Tue, 3 Nov 2020 21:47:24 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50:
>>
>>> 48: };
>>> 49:
>>> 50: typedef uint WeakProcessorPhase;
>>
>> This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes.
>>
>> But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here:
>>
>> src/hotspot/share/gc/shared/weakProcessor.inline.hpp
>> 116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase);
>> 117 StorageState* cur_state = _storage_states.par_state(oopstorage_index);
>> =>
>> 103 StorageState* cur_state = _storage_states.par_state(phase);
>>
>> I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet.
>> enum class : uint WeakProcessorPhase {};
>>
>> ENUMERATOR_RANGE(WeakProcessorPhase,
>> static_cast<WeakProcessorPhase>(0),
>> static_cast<WeakProcessorPhase>(OopStorageSet::weak_count));
>> and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator<WeakProcessorPhase> (which involves more than a type alias).
>>
>> Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet.
>
> Ok, so I'm not sure what to do with this:
>
> enum Phase {
> // Serial phase.
> JVMTI_ONLY(jvmti)
> // Additional implicit phase values follow for oopstorages.
> `};`
>
> I've removed the only thing in this enum.
>Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet.
This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Tue Nov 3 23:41:02 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 3 Nov 2020 23:41:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <ffo3ekNFPb1U0A9CrXkZfSNbwn09QN3v2bFRuPBHn6c=.eb2d1bcd-ee82-473a-b023-8cbe53626252@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ffo3ekNFPb1U0A9CrXkZfSNbwn09QN3v2bFRuPBHn6c=.eb2d1bcd-ee82-473a-b023-8cbe53626252@github.com>
Message-ID: <V2KwpWSuR98eK7dcTL_IWj8Q0E17qTLVXkzp9GB-0Qo=.7b3b92ff-5fc9-40e3-9c48-3f8b3e93ad2c@github.com>
On Tue, 3 Nov 2020 21:12:49 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
>>
>> - Merge branch 'master' into jvmti-table
>> - Merge branch 'master' into jvmti-table
>> - More review comments from Stefan and ErikO
>> - Code review comments from StefanK.
>> - 8212879: Make JVMTI TagMap table not hash on oop address
>
> src/hotspot/share/utilities/hashtable.cpp line 164:
>
>> 162: }
>> 163: }
>> 164: return newsize;
>
> It is existing code, but could be made clearer as part of this PR:
> int newsize;
> for (int i=0; i<arraysize; ++i) {
> newsize = primelist[i];
> if (newsize >= requested)
> break;
> }
> return newsize;
> Additionally, this method could be made `const`, right?
>
> PS: not a review, just a comment in passing
Yes, that is a lot simpler and better. I'd copied that code from another file without changing it that much.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Wed Nov 4 00:08:10 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 4 Nov 2020 00:08:10 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
Code review comments from Kim and Albert.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/1c3f2e1e..f66ea839
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=04
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=03-04
Stats: 10 lines in 3 files changed: 0 ins; 4 del; 6 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From zgu at openjdk.java.net Wed Nov 4 00:17:59 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 4 Nov 2020 00:17:59 GMT
Subject: RFR: 8255847: Shenandoah: Shenandoah should not mark through weak
roots
Message-ID: <dJvQ4JLRc1zmIW86lSXrk7HU5mjVb5n_1hKm_G8WJQs=.5d0d7443-9391-4e17-80ef-91a988b3f501@github.com>
After moving weak root processing into concurrent phase, Shenandoah should no longer marks through weak roots, even when class unloading is disabled, given weak root processing no longer contributes to latency.
There are a couple of bugs:
1) ShenandoahRootVerifier was not updated to reflect the change.
The problem did not show up due to SH::parallel_cleaning() uses wrong flag to determine if it should cleanup weak roots, and it will be addressed in separate CR.
2) Concurrent roots scanner should not mark through string dedup roots.
Test:
- [x] hotspot_gc_shenandoah
- [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify -XX:-ClassUnloading
- [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify -XX:+UseStringDeduplication
-------------
Commit messages:
- Remove StrDedup root from concurrent root scanner
- 8255847: Shenandoah: Shenandoah root verifier's roots_do() should not include weak roots
Changes: https://git.openjdk.java.net/jdk/pull/1050/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1050&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255847
Stats: 11 lines in 3 files changed: 0 ins; 11 del; 0 mod
Patch: https://git.openjdk.java.net/jdk/pull/1050.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1050/head:pull/1050
PR: https://git.openjdk.java.net/jdk/pull/1050
From sspitsyn at openjdk.java.net Wed Nov 4 02:19:00 2020
From: sspitsyn at openjdk.java.net (Serguei Spitsyn)
Date: Wed, 4 Nov 2020 02:19:00 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5]
In-Reply-To: <FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
Message-ID: <cfXXqQMzEuagcHhmKX6XfJHBafVg4GYn6iS0RmlDuQU=.dd5966d1-3d09-414b-aae2-c9c0aeaacbc7@github.com>
On Wed, 4 Nov 2020 00:08:10 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> Code review comments from Kim and Albert.
Hi Coleen,
Wow, there are a lot of simplifications and code removal with this fix!
It looks great in general, just some nits below.
I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :)
src/hotspot/share/prims/jvmtiTagMap.hpp:
Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? :
`+ void post_dead_object_on_vm_thread();`
src/hotspot/share/prims/jvmtiTagMap.cpp:
Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk:
122 void JvmtiTagMap::check_hashmap(bool post_events) {
. . .
143 // This checks for posting and rehashing and is called from the heap walks.
144 void JvmtiTagMap::check_hashmaps_for_heapwalk() {
I'm just curious how this fragment was added. Did you get any failures in testing? :
1038 // skip if object is a dormant shared object whose mirror hasn't been loaded
1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) {
1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj),
1041 obj->klass()->external_name());
1042 return;
1043 }
Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? :
`1186 bool _some_dead;`
Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts.
2996 if (!is_vm_thread) {
2997 if (num_dead_entries != 0) {
2998 JvmtiEnvIterator it;
2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
3000 JvmtiTagMap* tag_map = env->tag_map_acquire();
3001 if (tag_map != NULL) {
3002 // Lock each hashmap from concurrent posting and cleaning
3003 tag_map->unlink_and_post_locked();
3004 }
3005 }
3006 // there's another callback for needs_rehashing
3007 }
3008 } else {
3009 assert(SafepointSynchronize::is_at_safepoint(), "must be");
3010 JvmtiEnvIterator it;
3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
3012 JvmtiTagMap* tag_map = env->tag_map_acquire();
3013 if (tag_map != NULL && !tag_map->is_empty()) {
3014 if (num_dead_entries != 0) {
3015 tag_map->hashmap()->unlink_and_post(tag_map->env());
3016 }
3017 // Later GC code will relocate the oops, so defer rehashing until then.
3018 tag_map->_needs_rehashing = true;
3019 }
It feels like it can be refactored/simplified, at least, a little bit.
Is it possible to check and just return if (num_dead_entries == 0)?
If not, then, at least, it can be done the same way (except of locking).
Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts.
src/hotspot/share/utilities/hashtable.cpp:
Nit: Need space after the '{' :
`+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;`
src/hotspot/share/prims/jvmtiTagMapTable.cpp:
Nit: Extra space after assert:
`119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");`
Thanks,
Serguei
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From sspitsyn at openjdk.java.net Wed Nov 4 05:40:03 2020
From: sspitsyn at openjdk.java.net (Serguei Spitsyn)
Date: Wed, 4 Nov 2020 05:40:03 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5]
In-Reply-To: <cfXXqQMzEuagcHhmKX6XfJHBafVg4GYn6iS0RmlDuQU=.dd5966d1-3d09-414b-aae2-c9c0aeaacbc7@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
<cfXXqQMzEuagcHhmKX6XfJHBafVg4GYn6iS0RmlDuQU=.dd5966d1-3d09-414b-aae2-c9c0aeaacbc7@github.com>
Message-ID: <J4M3JEytey8V8c_dZRQGDj8YrarZYdpWfGewAyPxXXE=.49047df2-4452-4169-9a26-b4ddf08adce6@github.com>
On Wed, 4 Nov 2020 02:15:52 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Code review comments from Kim and Albert.
>
> Hi Coleen,
>
> Wow, there are a lot of simplifications and code removal with this fix!
> It looks great in general, just some nits below.
> I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :)
>
> src/hotspot/share/prims/jvmtiTagMap.hpp:
>
> Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? :
> `+ void post_dead_object_on_vm_thread();`
>
> src/hotspot/share/prims/jvmtiTagMap.cpp:
>
> Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk:
> 122 void JvmtiTagMap::check_hashmap(bool post_events) {
> . . .
> 143 // This checks for posting and rehashing and is called from the heap walks.
> 144 void JvmtiTagMap::check_hashmaps_for_heapwalk() {
>
> I'm just curious how this fragment was added. Did you get any failures in testing? :
> 1038 // skip if object is a dormant shared object whose mirror hasn't been loaded
> 1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) {
> 1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj),
> 1041 obj->klass()->external_name());
> 1042 return;
> 1043 }
>
> Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? :
> `1186 bool _some_dead;`
>
> Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts.
> 2996 if (!is_vm_thread) {
> 2997 if (num_dead_entries != 0) {
> 2998 JvmtiEnvIterator it;
> 2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
> 3000 JvmtiTagMap* tag_map = env->tag_map_acquire();
> 3001 if (tag_map != NULL) {
> 3002 // Lock each hashmap from concurrent posting and cleaning
> 3003 tag_map->unlink_and_post_locked();
> 3004 }
> 3005 }
> 3006 // there's another callback for needs_rehashing
> 3007 }
> 3008 } else {
> 3009 assert(SafepointSynchronize::is_at_safepoint(), "must be");
> 3010 JvmtiEnvIterator it;
> 3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
> 3012 JvmtiTagMap* tag_map = env->tag_map_acquire();
> 3013 if (tag_map != NULL && !tag_map->is_empty()) {
> 3014 if (num_dead_entries != 0) {
> 3015 tag_map->hashmap()->unlink_and_post(tag_map->env());
> 3016 }
> 3017 // Later GC code will relocate the oops, so defer rehashing until then.
> 3018 tag_map->_needs_rehashing = true;
> 3019 }
> It feels like it can be refactored/simplified, at least, a little bit.
> Is it possible to check and just return if (num_dead_entries == 0)?
> If not, then, at least, it can be done the same way (except of locking).
> Q: Should the _needs_rehashing be set in both contexts?
>
> Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts.
>
> src/hotspot/share/utilities/hashtable.cpp:
>
> Nit: Need space after the '{' :
> `+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;`
>
> src/hotspot/share/prims/jvmtiTagMapTable.cpp:
>
> Nit: Extra space after assert:
> `119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");`
>
> Thanks,
> Serguei
More about possible refactoring of the JvmtiTagMap::gc_notification().
I'm thinking about something like below:
void JvmtiTagMap::unlink_and_post_for_all_envs() {
if (num_dead_entries == 0) {
return; // nothing to unlink and post
}
JvmtiEnvIterator it;
for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
JvmtiTagMap* tag_map = env->tag_map_acquire();
if (tag_map != NULL && !tag_map->is_empty()) {
tag_map->unlink_and_post();
}
}
}
void JvmtiTagMap::gc_notification(size_t num_dead_entries) {
if (Thread::current()->is_VM_thread()) {
assert(SafepointSynchronize::is_at_safepoint(), "must be");
unlink_and_post_for_all_envs();
set_needs_rehashing();
} else {
MutexLocker ml(JvmtiTagMap_lock(), Mutex::_no_safepoint_check_flag);
unlink_and_post_for_all_envs();
// there's another callback for needs_rehashing
}
}
If we still need a lock per each JvmtiTagMap then it is possible to add this fragment to the unlink_and_post_for_all_envs:
bool is_vm_thread = Thread::current()->is_VM_thread()
MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag);
Then the code above could look like below:
void JvmtiTagMap::unlink_and_post_for_all_envs() {
if (num_dead_entries == 0) {
return; // nothing to unlink and post
}
bool is_vm_thread = Thread::current()->is_VM_thread()
JvmtiEnvIterator it;
for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
JvmtiTagMap* tag_map = env->tag_map_acquire();
if (tag_map != NULL && !tag_map->is_empty()) {
MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag);
tag_map->unlink_and_post();
}
}
}
void JvmtiTagMap::gc_notification(size_t num_dead_entries) {
if (Thread::current()->is_VM_thread()) {
assert(SafepointSynchronize::is_at_safepoint(), "must be");
set_needs_rehashing();
}
unlink_and_post_for_all_envs();
}
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From thartmann at openjdk.java.net Wed Nov 4 07:08:07 2020
From: thartmann at openjdk.java.net (Tobias Hartmann)
Date: Wed, 4 Nov 2020 07:08:07 GMT
Subject: RFR: 8255665: C2 should aggressively remove temporary hook nodes
[v2]
In-Reply-To: <w5DqBzO0s7Tw_DJPDmT_pYxXPplJ8BnEHme1y3eeo1A=.628b7868-0d14-4524-aa97-1c75b34b73c7@github.com>
References: <H9yHPRN01lNmqpbvEVHH61K_QslMdNJC1ruWFTbfyg8=.b66f1e8e-ae2a-4787-b409-074c96d6dfc9@github.com>
<wwXoDjiPSlL03IbkqnQel21DE3iXxNa9FLd10rLJado=.e7ceb439-2e55-4e9b-b008-f5a1ad88888b@github.com>
<w5DqBzO0s7Tw_DJPDmT_pYxXPplJ8BnEHme1y3eeo1A=.628b7868-0d14-4524-aa97-1c75b34b73c7@github.com>
Message-ID: <fFPRVhe-Wn0gJDQe03xxfgXBGf_6B275zs3FUG_QakQ=.933f7b5d-8801-44f5-a6fb-3ab7afd90d6f@github.com>
On Mon, 2 Nov 2020 18:41:04 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:
>> Tobias Hartmann has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Restored handling of constant nodes
>
> src/hotspot/share/opto/cfgnode.cpp line 2308:
>
>> 2306: igvn->_worklist.remove(hook);
>> 2307: }
>> 2308: hook->destruct();
>
> I think we should pass PhaseGVN* into destruct() and do removal from worklist there because it looks like repetitive pattern. Also we can take Compile pointer from PhaseGVN instead of calling Compile::current():
> https://github.com/openjdk/jdk/blob/master/src/hotspot/share/opto/node.cpp#L584
Thanks Vladimir, that's a good suggestion. I've updated the PR accordingly.
-------------
PR: https://git.openjdk.java.net/jdk/pull/994
From thartmann at openjdk.java.net Wed Nov 4 07:08:06 2020
From: thartmann at openjdk.java.net (Tobias Hartmann)
Date: Wed, 4 Nov 2020 07:08:06 GMT
Subject: RFR: 8255665: C2 should aggressively remove temporary hook nodes
[v3]
In-Reply-To: <H9yHPRN01lNmqpbvEVHH61K_QslMdNJC1ruWFTbfyg8=.b66f1e8e-ae2a-4787-b409-074c96d6dfc9@github.com>
References: <H9yHPRN01lNmqpbvEVHH61K_QslMdNJC1ruWFTbfyg8=.b66f1e8e-ae2a-4787-b409-074c96d6dfc9@github.com>
Message-ID: <5wM4jpVaL-BlP7FdbWwQkWgpxZZbUlpTVoFw3SRBUqA=.2419c090-2d90-4ff1-8be9-d9167b0dc1ef@github.com>
> C2 often creates temporary "hook" nodes to keep other nodes alive. Although dead, these are sometimes not removed because they don't end up on the IGVN worklist. JDK-8040213 added detection of modified nodes that are not re-processed by IGVN but currently ignores dead nodes.
>
> This patch includes the following changes:
> - Adjust detection of modified nodes such that dead nodes are includes as well. This revealed several locations were dead nodes are not eagerly destructed (or not even added to the worklist for later removal). I've fixed all of these.
> - No need to yank node inputs before calling `destruct`.
> - `kill_dead_code` accidentally re-adds dead nodes to the `_modified_nodes` list. `Compile::remove_modified_node` should be called at the end to avoid this.
> - Some removal of dead code.
>
> Tested with tier1-3, higher tiers are running.
>
> JDK-8255670 will further improve detection.
>
> Thanks,
> Tobias
Tobias Hartmann has updated the pull request incrementally with one additional commit since the last revision:
Pass PhaseValues to Node::destruct
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/994/files
- new: https://git.openjdk.java.net/jdk/pull/994/files/e8899406..21a73e6d
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=994&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=994&range=01-02
Stats: 41 lines in 12 files changed: 3 ins; 20 del; 18 mod
Patch: https://git.openjdk.java.net/jdk/pull/994.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/994/head:pull/994
PR: https://git.openjdk.java.net/jdk/pull/994
From kbarrett at openjdk.java.net Wed Nov 4 09:36:02 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Wed, 4 Nov 2020 09:36:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <gxedU6v8A_Mkqc_lHxakcxetjdDJaM-6M2OZPGy8rlg=.59b52b6b-2c61-4ff9-b830-174bd14dd6d1@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
<Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
<0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com>
<1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com>
<gxedU6v8A_Mkqc_lHxakcxetjdDJaM-6M2OZPGy8rlg=.59b52b6b-2c61-4ff9-b830-174bd14dd6d1@github.com>
Message-ID: <w_eD2tSaxZ05w2Yo3T2fjV00vT_DdUTnVzhrcLdsjfk=.0d35ce07-ca69-4dc9-8ecf-2fb705ebfb9f@github.com>
On Wed, 4 Nov 2020 07:41:39 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>>>Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet.
>>
>> This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting.
>
>> Ok, so I'm not sure what to do with this:
>>
>> enum Phase {
>> // Serial phase.
>> JVMTI_ONLY(jvmti)
>> // Additional implicit phase values follow for oopstorages.
>> `};`
>>
>> I've removed the only thing in this enum.
>
> Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums.
> > Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet.
>
> This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting.
I think the deletion stopped at the wrong place; it either went too far, or not far enough.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From kbarrett at openjdk.java.net Wed Nov 4 09:36:02 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Wed, 4 Nov 2020 09:36:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
<Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
<0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com>
<1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com>
Message-ID: <gxedU6v8A_Mkqc_lHxakcxetjdDJaM-6M2OZPGy8rlg=.59b52b6b-2c61-4ff9-b830-174bd14dd6d1@github.com>
On Tue, 3 Nov 2020 23:38:08 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> Ok, so I'm not sure what to do with this:
>>
>> enum Phase {
>> // Serial phase.
>> JVMTI_ONLY(jvmti)
>> // Additional implicit phase values follow for oopstorages.
>> `};`
>>
>> I've removed the only thing in this enum.
>
>>Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet.
>
> This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting.
> Ok, so I'm not sure what to do with this:
>
> enum Phase {
> // Serial phase.
> JVMTI_ONLY(jvmti)
> // Additional implicit phase values follow for oopstorages.
> `};`
>
> I've removed the only thing in this enum.
Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From kbarrett at openjdk.java.net Wed Nov 4 09:36:03 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Wed, 4 Nov 2020 09:36:03 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <Spqq9dORyH2M6ZrkCpGKDFA1jWfFTbiqxl3gNcQEPJk=.c964ac35-062a-4471-829a-856f5ec1c04e@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<Spqq9dORyH2M6ZrkCpGKDFA1jWfFTbiqxl3gNcQEPJk=.c964ac35-062a-4471-829a-856f5ec1c04e@github.com>
Message-ID: <t4O-bCM6Vmtj_7ho9Lp385qpLAzD7-2vMAWPqlsdHIE=.111c897f-96f9-4281-b28e-88e1d1345057@github.com>
On Tue, 3 Nov 2020 21:31:35 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> src/hotspot/share/prims/jvmtiTagMap.cpp line 2979:
>>
>>> 2977:
>>> 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved
>>> 2979: // and have their new addresses, the table can be rehashed.
>>
>> I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that).
>
> So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case?
I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.)
>> src/hotspot/share/prims/jvmtiTagMap.cpp line 3015:
>>
>>> 3013: if (tag_map != NULL && !tag_map->is_empty()) {
>>> 3014: if (num_dead_entries != 0) {
>>> 3015: tag_map->hashmap()->unlink_and_post(tag_map->env());
>>
>> Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup.
>
> The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table.
I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From kbarrett at openjdk.java.net Wed Nov 4 09:36:04 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Wed, 4 Nov 2020 09:36:04 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5]
In-Reply-To: <FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
Message-ID: <PRK7_SkeC4IUY2_R8RUGx4LPDRNQnEN6iO788E1kfW4=.f6871609-5e81-478c-9269-50824b166c40@github.com>
On Wed, 4 Nov 2020 00:08:10 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> Code review comments from Kim and Albert.
src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36:
> 34: class JvmtiTagMapEntryClosure;
> 35:
> 36: class JvmtiTagMapEntry : public HashtableEntry<WeakHandle, mtServiceability> {
By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%).
It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From kbarrett at openjdk.java.net Wed Nov 4 09:36:04 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Wed, 4 Nov 2020 09:36:04 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <t4O-bCM6Vmtj_7ho9Lp385qpLAzD7-2vMAWPqlsdHIE=.111c897f-96f9-4281-b28e-88e1d1345057@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<Spqq9dORyH2M6ZrkCpGKDFA1jWfFTbiqxl3gNcQEPJk=.c964ac35-062a-4471-829a-856f5ec1c04e@github.com>
<t4O-bCM6Vmtj_7ho9Lp385qpLAzD7-2vMAWPqlsdHIE=.111c897f-96f9-4281-b28e-88e1d1345057@github.com>
Message-ID: <w2jGKoh-kG4Cxy42LGV4RImf9dUqZNX1mjK1ZWemPMU=.906a6ba1-2e5d-4836-b255-4e77ae41aa4b@github.com>
On Wed, 4 Nov 2020 07:52:12 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case?
>
> I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.)
It used to be that jvmti tagmap processing was all-in-one (in GC weak reference processing, with the weak clearing, dead table entry removal, and rehashing all done in one pass. This change has split that up, with the weak clearing happening in a different place (still as part of the GC's weak reference processing) than the others (which I claim can now be part of the mutator, whether further separated or not).
"Concurrent GC" has nothing to do with whether tagmaps need rehashing. Any copying collector needs to do so. A non-copying collector (whether concurrent or not) would not. (We don't have any of those in HotSpot today.) And weak reference clearing (whether concurrent or not) has nothing to do with whether objects have been moved and so the hashing has been invalidated.
There's also a "well known" issue with address-based hashing and generational or similar collectors, where a simple boolean "objects have moved" flag can be problematic, and which tagmaps seem likely to be prone to. The old do_weak_oops tries to mitigate it by recognizing when the object didn't move.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From kbarrett at openjdk.java.net Wed Nov 4 10:08:00 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Wed, 4 Nov 2020 10:08:00 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <jKyDBy89u-3EdYr0CCKsfGI-SM7FgGWCTFtG9YWk6yw=.196f1a2d-ac5d-4a6d-99cc-48854e747736@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<jKyDBy89u-3EdYr0CCKsfGI-SM7FgGWCTFtG9YWk6yw=.196f1a2d-ac5d-4a6d-99cc-48854e747736@github.com>
Message-ID: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com>
On Tue, 3 Nov 2020 21:40:39 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> src/hotspot/share/prims/jvmtiTagMap.cpp line 127:
>>
>>> 125: // The table cleaning, posting and rehashing can race for
>>> 126: // concurrent GCs. So fix it here once we have a lock or are
>>> 127: // at a safepoint.
>>
>> I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time).
>
> The comment is trying to describe the situation like:
> 1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked)
> 2. safepoint for heap walk
> 2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A.
> 3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened
>
> The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable.
>
> Both operations require the table to be locked.
>
> The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints.
For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing, an
d so still effectively be a serial subtask after all the parallel subtasks are done or mostly done.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Wed Nov 4 12:21:12 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 4 Nov 2020 12:21:12 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5]
In-Reply-To: <J4M3JEytey8V8c_dZRQGDj8YrarZYdpWfGewAyPxXXE=.49047df2-4452-4169-9a26-b4ddf08adce6@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
<cfXXqQMzEuagcHhmKX6XfJHBafVg4GYn6iS0RmlDuQU=.dd5966d1-3d09-414b-aae2-c9c0aeaacbc7@github.com>
<J4M3JEytey8V8c_dZRQGDj8YrarZYdpWfGewAyPxXXE=.49047df2-4452-4169-9a26-b4ddf08adce6@github.com>
Message-ID: <PxXWCCjhTfSfilMs54eI6McosfePnGZKwRlYRVys9LA=.72788810-0c02-4886-8791-87104d51864e@github.com>
On Wed, 4 Nov 2020 05:37:00 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:
>> Hi Coleen,
>>
>> Wow, there are a lot of simplifications and code removal with this fix!
>> It looks great in general, just some nits below.
>> I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :)
>>
>> src/hotspot/share/prims/jvmtiTagMap.hpp:
>>
>> Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? :
>> `+ void post_dead_object_on_vm_thread();`
>>
>> src/hotspot/share/prims/jvmtiTagMap.cpp:
>>
>> Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk:
>> 122 void JvmtiTagMap::check_hashmap(bool post_events) {
>> . . .
>> 143 // This checks for posting and rehashing and is called from the heap walks.
>> 144 void JvmtiTagMap::check_hashmaps_for_heapwalk() {
>>
>> I'm just curious how this fragment was added. Did you get any failures in testing? :
>> 1038 // skip if object is a dormant shared object whose mirror hasn't been loaded
>> 1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) {
>> 1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj),
>> 1041 obj->klass()->external_name());
>> 1042 return;
>> 1043 }
>>
>> Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? :
>> `1186 bool _some_dead;`
>>
>> Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts.
>> 2996 if (!is_vm_thread) {
>> 2997 if (num_dead_entries != 0) {
>> 2998 JvmtiEnvIterator it;
>> 2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
>> 3000 JvmtiTagMap* tag_map = env->tag_map_acquire();
>> 3001 if (tag_map != NULL) {
>> 3002 // Lock each hashmap from concurrent posting and cleaning
>> 3003 tag_map->unlink_and_post_locked();
>> 3004 }
>> 3005 }
>> 3006 // there's another callback for needs_rehashing
>> 3007 }
>> 3008 } else {
>> 3009 assert(SafepointSynchronize::is_at_safepoint(), "must be");
>> 3010 JvmtiEnvIterator it;
>> 3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
>> 3012 JvmtiTagMap* tag_map = env->tag_map_acquire();
>> 3013 if (tag_map != NULL && !tag_map->is_empty()) {
>> 3014 if (num_dead_entries != 0) {
>> 3015 tag_map->hashmap()->unlink_and_post(tag_map->env());
>> 3016 }
>> 3017 // Later GC code will relocate the oops, so defer rehashing until then.
>> 3018 tag_map->_needs_rehashing = true;
>> 3019 }
>> It feels like it can be refactored/simplified, at least, a little bit.
>> Is it possible to check and just return if (num_dead_entries == 0)?
>> If not, then, at least, it can be done the same way (except of locking).
>> Q: Should the _needs_rehashing be set in both contexts?
>>
>> Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts.
>>
>> src/hotspot/share/utilities/hashtable.cpp:
>>
>> Nit: Need space after the '{' :
>> `+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;`
>>
>> src/hotspot/share/prims/jvmtiTagMapTable.cpp:
>>
>> Nit: Extra space after assert:
>> `119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");`
>>
>> Thanks,
>> Serguei
>
> More about possible refactoring of the JvmtiTagMap::gc_notification().
> I'm thinking about something like below:
>
> void JvmtiTagMap::unlink_and_post_for_all_envs() {
> if (num_dead_entries == 0) {
> return; // nothing to unlink and post
> }
> JvmtiEnvIterator it;
> for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
> JvmtiTagMap* tag_map = env->tag_map_acquire();
> if (tag_map != NULL && !tag_map->is_empty()) {
> tag_map->unlink_and_post();
> }
> }
> }
>
> void JvmtiTagMap::gc_notification(size_t num_dead_entries) {
> if (Thread::current()->is_VM_thread()) {
> assert(SafepointSynchronize::is_at_safepoint(), "must be");
> unlink_and_post_for_all_envs();
> set_needs_rehashing();
> } else {
> MutexLocker ml(JvmtiTagMap_lock(), Mutex::_no_safepoint_check_flag);
> unlink_and_post_for_all_envs();
> // there's another callback for needs_rehashing
> }
> }
>
> If we still need a lock per each JvmtiTagMap then it is possible to add this fragment to the unlink_and_post_for_all_envs:
> bool is_vm_thread = Thread::current()->is_VM_thread()
> MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag);
>
> Then the code above could look like below:
>
> void JvmtiTagMap::unlink_and_post_for_all_envs() {
> if (num_dead_entries == 0) {
> return; // nothing to unlink and post
> }
> bool is_vm_thread = Thread::current()->is_VM_thread()
> JvmtiEnvIterator it;
> for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
> JvmtiTagMap* tag_map = env->tag_map_acquire();
> if (tag_map != NULL && !tag_map->is_empty()) {
> MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag);
> tag_map->unlink_and_post();
> }
> }
> }
>
> void JvmtiTagMap::gc_notification(size_t num_dead_entries) {
> if (Thread::current()->is_VM_thread()) {
> assert(SafepointSynchronize::is_at_safepoint(), "must be");
> set_needs_rehashing();
> }
> unlink_and_post_for_all_envs();
> }
@sspitsyn Thank you for reviewing the code. The gc_notification refactoring is awkward because each table needs a lock and not a global lock. If we find another place in the GCs to call set_needs_rehashing() it might be possible to make gc_notification call two functions with a boolean to decide whether to take the lock. We're still working on @kimbarrett comments so maybe the notification will change to some new thread and be refactored that way if necessary. I fixed the code for your other comments.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Wed Nov 4 12:21:12 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 4 Nov 2020 12:21:12 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <rrqQkn7YRMjol0rpPIDSBsbXZXqFULDSRxMlMPJ5cjs=.f10aea2a-10db-4555-ba82-8664a472e393@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision:
- Add back WeakProcessorPhases::Phase enum.
- Serguei 1.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/f66ea839..7d3fdf68
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=05
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=04-05
Stats: 18 lines in 5 files changed: 7 ins; 0 del; 11 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Wed Nov 4 12:38:05 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 4 Nov 2020 12:38:05 GMT
Subject: RFR: 8255886: Shenandoah: Avoid register clash when calling
LRB-runtime from interpreter
Message-ID: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
JDK-8255762 caused test failure on Windows because of overlapping argument registers in the LRB runtime call. The problem is more general, though, but hasn't manifested anywhere else.
Testing: hotspot_gc_shenandoah (linux: x86_64, x86_32, windows: x86_64)
-------------
Commit messages:
- 8255886: Shenandoah: Avoid register clash when calling LRB-runtime from interpreter
Changes: https://git.openjdk.java.net/jdk/pull/1054/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255886
Stats: 18 lines in 1 file changed: 10 ins; 0 del; 8 mod
Patch: https://git.openjdk.java.net/jdk/pull/1054.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1054/head:pull/1054
PR: https://git.openjdk.java.net/jdk/pull/1054
From rwestrel at redhat.com Wed Nov 4 14:06:16 2020
From: rwestrel at redhat.com (Roland Westrelin)
Date: Wed, 04 Nov 2020 15:06:16 +0100
Subject: shenandoah c2 compiler crash in jdk8
In-Reply-To: <CAEyj=Y5S4+rfkaAOfr12_0v-5gOM3=wYRHAjY-GXTOa6bM+fYw@mail.gmail.com>
References: <CAEyj=Y5W1D-w=kLP26M02MkdOnP1WiPwVyXkqsNSdu6kT8SqaQ@mail.gmail.com>
<CAEyj=Y7Gjk8N8iNTz766cgtWcxnOW-MY=J4EyN0iKr4UeA6oQQ@mail.gmail.com>
<87a6w0duqx.fsf@redhat.com>
<CAEyj=Y5S4+rfkaAOfr12_0v-5gOM3=wYRHAjY-GXTOa6bM+fYw@mail.gmail.com>
Message-ID: <87361pcj3b.fsf@redhat.com>
> I think that the qn that needs to be answered is whether there is a `leak`
> in nodes or you just need more of them. Does that make sense? If so, is
> there an option to dump c2 node usage over time?
The assert failure you posted happens at barrier expansion
time. Shenandoah's barriers are initially inserted as macro instructions
in the c2 IR and as a final step in the optimization process, they are
expanded. It seems that what happens is that the c2 IR gets close to its
limit size before barrier expansion and then expansion pushes the IR
size over the limit.
It makes sense that bumping MaxNodeLimit works around the problem but I
don't see that as a robust fix. With a large MaxNodeLimit, C2 would have
more head room to transform the IR before barrier expansion. So nothing
says that in some cases, c2 wouldn't end up at barrier expansion with a
much larger IR that would cause expansion to fail in the same way.
I don't see how a leak of nodes would happen in this scenario. During
expansion, barriers are expanded one by one. There's no creation of new
barriers. I don't expect "node usage over time" to give a picture that's
clear enough of what's going on either.
Roland.
From rkennke at openjdk.java.net Wed Nov 4 15:03:10 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 4 Nov 2020 15:03:10 GMT
Subject: RFR: 8255886: Shenandoah: Avoid register clash when calling
LRB-runtime from interpreter [v2]
In-Reply-To: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
References: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
Message-ID: <yn6cWbkig9ZAUTy_KjiNg03EHVS2UFALLhH_3b9L_vU=.ff618a1b-754d-4832-b70a-116ea1736bc3@github.com>
> JDK-8255762 caused test failure on Windows because of overlapping argument registers in the LRB runtime call. The problem is more general, though, but hasn't manifested anywhere else.
>
> Testing: hotspot_gc_shenandoah (linux: x86_64, x86_32, windows: x86_64)
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Generate regular call_VM_leaf for non-weak LRB
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1054/files
- new: https://git.openjdk.java.net/jdk/pull/1054/files/7418c712..339dde59
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=00-01
Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
Patch: https://git.openjdk.java.net/jdk/pull/1054.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1054/head:pull/1054
PR: https://git.openjdk.java.net/jdk/pull/1054
From rkennke at openjdk.java.net Wed Nov 4 15:17:06 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 4 Nov 2020 15:17:06 GMT
Subject: RFR: 8255886: Shenandoah: Avoid register clash when calling
LRB-runtime from interpreter [v3]
In-Reply-To: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
References: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
Message-ID: <72TjNYQ1Sc9N-3mA0bxvN7fdTMN5JApffKyr2-EZw2E=.1cf707a3-b951-4840-8166-b5bed966da6b@github.com>
> JDK-8255762 caused test failure on Windows because of overlapping argument registers in the LRB runtime call. The problem is more general, though, but hasn't manifested anywhere else.
>
> Testing: hotspot_gc_shenandoah (linux: x86_64, x86_32, windows: x86_64)
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
- Merge branch 'master' into JDK-8255886
- Generate regular call_VM_leaf for non-weak LRB
- 8255886: Shenandoah: Avoid register clash when calling LRB-runtime from interpreter
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1054/files
- new: https://git.openjdk.java.net/jdk/pull/1054/files/339dde59..504e56ae
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=01-02
Stats: 10 lines in 3 files changed: 1 ins; 0 del; 9 mod
Patch: https://git.openjdk.java.net/jdk/pull/1054.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1054/head:pull/1054
PR: https://git.openjdk.java.net/jdk/pull/1054
From zgu at openjdk.java.net Wed Nov 4 15:26:05 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 4 Nov 2020 15:26:05 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v3]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <xTmAMrHnJIMg-7SczG7bAK3yZ7Co5D9KTLfw4n6Y3CY=.974b2820-926d-4060-9e6e-4c4c021baf13@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:
- Merge branch 'JDK-8255019-sh-mark' of github.com:zhengyu123/jdk into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Merge branch 'master' into JDK-8255019-sh-mark
- Make ShenandoahMarkCompact stack allocated
- Split finish mark and pre-evaculation
- 8255019: Shenandoah: Split STW and concurrent mark into separate classes
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=02
Stats: 1967 lines in 21 files changed: 1073 ins; 736 del; 158 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From zgu at openjdk.java.net Wed Nov 4 16:50:10 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 4 Nov 2020 16:50:10 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v4]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <agxU16AFnNat3IsTGjJWHAj-VTyMXdf-6HOrS4vbWXc=.b4132dad-77f1-45a1-bb3b-1d543a2875db@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Move weak reference processing out of STWMark and fix its timings
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1009/files
- new: https://git.openjdk.java.net/jdk/pull/1009/files/b4efedbd..bb72deab
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=02-03
Stats: 10 lines in 4 files changed: 2 ins; 5 del; 3 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From rkennke at openjdk.java.net Wed Nov 4 17:23:04 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 4 Nov 2020 17:23:04 GMT
Subject: RFR: 8255886: Shenandoah: Avoid register clash when calling
LRB-runtime from interpreter [v4]
In-Reply-To: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
References: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
Message-ID: <wV8UrLMsHREE6--tHCXWYhk3tEWUvk3dArol0An7OF8=.e88add93-1cc1-4053-af06-788784cec8cf@github.com>
> JDK-8255762 caused test failure on Windows because of overlapping argument registers in the LRB runtime call. The problem is more general, though, but hasn't manifested anywhere else.
>
> Testing: hotspot_gc_shenandoah (linux: x86_64, x86_32, windows: x86_64)
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Use 2-register form of cset-check, MacOSX doesn't allocate cset-table in location for 32bit-addressing
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1054/files
- new: https://git.openjdk.java.net/jdk/pull/1054/files/504e56ae..e83cae28
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=02-03
Stats: 11 lines in 1 file changed: 7 ins; 0 del; 4 mod
Patch: https://git.openjdk.java.net/jdk/pull/1054.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1054/head:pull/1054
PR: https://git.openjdk.java.net/jdk/pull/1054
From shade at openjdk.java.net Wed Nov 4 17:45:57 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 4 Nov 2020 17:45:57 GMT
Subject: RFR: 8255886: Shenandoah: Avoid register clash when calling
LRB-runtime from interpreter [v4]
In-Reply-To: <wV8UrLMsHREE6--tHCXWYhk3tEWUvk3dArol0An7OF8=.e88add93-1cc1-4053-af06-788784cec8cf@github.com>
References: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
<wV8UrLMsHREE6--tHCXWYhk3tEWUvk3dArol0An7OF8=.e88add93-1cc1-4053-af06-788784cec8cf@github.com>
Message-ID: <njBf6vV24x-NQPm0I8FXVHuxrWappAnGqtgNG9rT6pM=.bc900e1e-0cca-4380-9af6-a20f6f81f060@github.com>
On Wed, 4 Nov 2020 17:23:04 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> JDK-8255762 caused test failure on Windows because of overlapping argument registers in the LRB runtime call. The problem is more general, though, but hasn't manifested anywhere else.
>>
>> Testing: hotspot_gc_shenandoah (linux: x86_64, x86_32, windows: x86_64)
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Use 2-register form of cset-check, MacOSX doesn't allocate cset-table in location for 32bit-addressing
Minor nits
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 297:
> 295: if (kind == ShenandoahBarrierSet::AccessKind::NORMAL) {
> 296: // Test for object in cset
> 297: // Allocate tmp-reg.
"Allocate temporary registers" now.
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 312:
> 310: __ push(tmp2);
> 311: assert_different_registers(tmp1, src.base(), src.index());
> 312: assert_different_registers(tmp1, dst);
Let's do:
assert(tmp1 != noreg, "tmp1 allocated");
assert(tmp2 != noreg, "tmp2 allocated");
assert_different_registers(tmp1, tmp2, src.base(), src.index());
assert_different_registers(tmp1, tmp2, dst);
__ push(tmp1);
__ push(tmp2);
-------------
Changes requested by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1054
From rkennke at openjdk.java.net Wed Nov 4 18:30:08 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 4 Nov 2020 18:30:08 GMT
Subject: RFR: 8255886: Shenandoah: Avoid register clash when calling
LRB-runtime from interpreter [v5]
In-Reply-To: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
References: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
Message-ID: <BcwoSV27jqFtB1rzZ6P1jdGnv8x475WhDWUs3kNLXz0=.4f3b816c-e58b-4089-b2ad-2cc41a7ee8d8@github.com>
> JDK-8255762 caused test failure on Windows because of overlapping argument registers in the LRB runtime call. The problem is more general, though, but hasn't manifested anywhere else.
>
> Testing: hotspot_gc_shenandoah (linux: x86_64, x86_32, windows: x86_64)
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Some touch-ups
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1054/files
- new: https://git.openjdk.java.net/jdk/pull/1054/files/e83cae28..607b98f7
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=04
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1054&range=03-04
Stats: 8 lines in 1 file changed: 5 ins; 2 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1054.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1054/head:pull/1054
PR: https://git.openjdk.java.net/jdk/pull/1054
From shade at openjdk.java.net Wed Nov 4 18:32:59 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 4 Nov 2020 18:32:59 GMT
Subject: RFR: 8255886: Shenandoah: Avoid register clash when calling
LRB-runtime from interpreter [v5]
In-Reply-To: <BcwoSV27jqFtB1rzZ6P1jdGnv8x475WhDWUs3kNLXz0=.4f3b816c-e58b-4089-b2ad-2cc41a7ee8d8@github.com>
References: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
<BcwoSV27jqFtB1rzZ6P1jdGnv8x475WhDWUs3kNLXz0=.4f3b816c-e58b-4089-b2ad-2cc41a7ee8d8@github.com>
Message-ID: <7q8itHvAjJ4U8j8KwY-xeP-TP2HKx-k1FOXetEs1gBg=.e1c3b029-6ab9-44a3-a52e-d257c7127059@github.com>
On Wed, 4 Nov 2020 18:30:08 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> JDK-8255762 caused test failure on Windows because of overlapping argument registers in the LRB runtime call. The problem is more general, though, but hasn't manifested anywhere else.
>>
>> Testing: hotspot_gc_shenandoah (linux: x86_64, x86_32, windows: x86_64)
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Some touch-ups
Looks good. Please rename the bug and PR to capture the cset check changes. Suggestion: "Shenandoah: Resolve cset address truncation and register clash in interpreter LRB"
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1054
From kvn at openjdk.java.net Wed Nov 4 19:03:58 2020
From: kvn at openjdk.java.net (Vladimir Kozlov)
Date: Wed, 4 Nov 2020 19:03:58 GMT
Subject: RFR: 8255665: C2 should aggressively remove temporary hook nodes
[v3]
In-Reply-To: <5wM4jpVaL-BlP7FdbWwQkWgpxZZbUlpTVoFw3SRBUqA=.2419c090-2d90-4ff1-8be9-d9167b0dc1ef@github.com>
References: <H9yHPRN01lNmqpbvEVHH61K_QslMdNJC1ruWFTbfyg8=.b66f1e8e-ae2a-4787-b409-074c96d6dfc9@github.com>
<5wM4jpVaL-BlP7FdbWwQkWgpxZZbUlpTVoFw3SRBUqA=.2419c090-2d90-4ff1-8be9-d9167b0dc1ef@github.com>
Message-ID: <aotwvmbaohOOvZhWXENl-PEcZWxAZYn9rHbqEg1f778=.4975299f-1344-47e9-b786-e8ba4775e6f2@github.com>
On Wed, 4 Nov 2020 07:08:06 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:
>> C2 often creates temporary "hook" nodes to keep other nodes alive. Although dead, these are sometimes not removed because they don't end up on the IGVN worklist. JDK-8040213 added detection of modified nodes that are not re-processed by IGVN but currently ignores dead nodes.
>>
>> This patch includes the following changes:
>> - Adjust detection of modified nodes such that dead nodes are includes as well. This revealed several locations were dead nodes are not eagerly destructed (or not even added to the worklist for later removal). I've fixed all of these.
>> - No need to yank node inputs before calling `destruct`.
>> - `kill_dead_code` accidentally re-adds dead nodes to the `_modified_nodes` list. `Compile::remove_modified_node` should be called at the end to avoid this.
>> - Some removal of dead code.
>>
>> Tested with tier1-3, higher tiers are running.
>>
>> JDK-8255670 will further improve detection.
>>
>> Thanks,
>> Tobias
>
> Tobias Hartmann has updated the pull request incrementally with one additional commit since the last revision:
>
> Pass PhaseValues to Node::destruct
Good.
-------------
Marked as reviewed by kvn (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/994
From rkennke at openjdk.java.net Wed Nov 4 21:33:56 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 4 Nov 2020 21:33:56 GMT
Subject: Integrated: 8255886: Shenandoah: Resolve cset address truncation and
register clash in interpreter LRB
In-Reply-To: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
References: <ySb7PSSCla7FrSTYNv2F_L-dd6Xu2t32nDCQ8fsyIIM=.0fb44af4-6106-4024-a73b-b5d835ac0817@github.com>
Message-ID: <nK1ZON4WU66uoGO0PQPnz1znR1GvKfUmATTd6Jfc2sw=.e853d3cd-eb48-453f-b55e-8882f3325a23@github.com>
On Wed, 4 Nov 2020 12:28:23 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> JDK-8255762 caused test failure on Windows because of overlapping argument registers in the LRB runtime call. The problem is more general, though, but hasn't manifested anywhere else.
>
> Testing: hotspot_gc_shenandoah (linux: x86_64, x86_32, windows: x86_64)
This pull request has now been integrated.
Changeset: 29db1dcd
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/29db1dcd
Stats: 36 lines in 1 file changed: 21 ins; 1 del; 14 mod
8255886: Shenandoah: Resolve cset address truncation and register clash in interpreter LRB
Reviewed-by: shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1054
From rkennke at openjdk.java.net Wed Nov 4 21:40:56 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 4 Nov 2020 21:40:56 GMT
Subject: RFR: 8255847: Shenandoah: Shenandoah should not mark through weak
roots
In-Reply-To: <dJvQ4JLRc1zmIW86lSXrk7HU5mjVb5n_1hKm_G8WJQs=.5d0d7443-9391-4e17-80ef-91a988b3f501@github.com>
References: <dJvQ4JLRc1zmIW86lSXrk7HU5mjVb5n_1hKm_G8WJQs=.5d0d7443-9391-4e17-80ef-91a988b3f501@github.com>
Message-ID: <94-imIwm3FHENBFcksLSvUK9ndfKUYfssdzeHf-yq0I=.ff46f281-6321-4a05-8ced-381c997b7d46@github.com>
On Wed, 4 Nov 2020 00:09:13 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> After moving weak root processing into concurrent phase, Shenandoah should no longer marks through weak roots, even when class unloading is disabled, given weak root processing no longer contributes to latency.
>
> There are a couple of bugs:
> 1) ShenandoahRootVerifier was not updated to reflect the change.
> The problem did not show up due to SH::parallel_cleaning() uses wrong flag to determine if it should cleanup weak roots, and it will be addressed in separate CR.
>
> 2) Concurrent roots scanner should not mark through string dedup roots.
>
> Test:
> - [x] hotspot_gc_shenandoah
> - [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify -XX:-ClassUnloading
> - [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify -XX:+UseStringDeduplication
Looks good. Make sure to merge latest tip to get full pre-submit testing in GitHub actions.
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1050
From sspitsyn at openjdk.java.net Wed Nov 4 22:12:06 2020
From: sspitsyn at openjdk.java.net (Serguei Spitsyn)
Date: Wed, 4 Nov 2020 22:12:06 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6]
In-Reply-To: <rrqQkn7YRMjol0rpPIDSBsbXZXqFULDSRxMlMPJ5cjs=.f10aea2a-10db-4555-ba82-8664a472e393@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<rrqQkn7YRMjol0rpPIDSBsbXZXqFULDSRxMlMPJ5cjs=.f10aea2a-10db-4555-ba82-8664a472e393@github.com>
Message-ID: <TgRmyCL8pDACuMxxHfdsqN9mOVxyZaKNRpfXbssEl94=.69114d11-53a5-4220-b5ba-e9ecdfe96f26@github.com>
On Wed, 4 Nov 2020 12:21:12 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision:
>
> - Add back WeakProcessorPhases::Phase enum.
> - Serguei 1.
Thank you for the update, Coleen!
I leave it for you to decide to refactor the gc_notification or not.
Thanks,
Serguei
-------------
Marked as reviewed by sspitsyn (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/967
From zgu at openjdk.java.net Thu Nov 5 01:57:15 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 5 Nov 2020 01:57:15 GMT
Subject: RFR: 8255847: Shenandoah: Shenandoah should not mark through weak
roots [v2]
In-Reply-To: <dJvQ4JLRc1zmIW86lSXrk7HU5mjVb5n_1hKm_G8WJQs=.5d0d7443-9391-4e17-80ef-91a988b3f501@github.com>
References: <dJvQ4JLRc1zmIW86lSXrk7HU5mjVb5n_1hKm_G8WJQs=.5d0d7443-9391-4e17-80ef-91a988b3f501@github.com>
Message-ID: <uHF41WQ7bRL4yie_9jiwQxjn5pj7pFecYsHrzV8qJLw=.b06750b5-42c0-4d5c-84ac-aa46e8596a47@github.com>
> After moving weak root processing into concurrent phase, Shenandoah should no longer marks through weak roots, even when class unloading is disabled, given weak root processing no longer contributes to latency.
>
> There are a couple of bugs:
> 1) ShenandoahRootVerifier was not updated to reflect the change.
> The problem did not show up due to SH::parallel_cleaning() uses wrong flag to determine if it should cleanup weak roots, and it will be addressed in separate CR.
>
> 2) Concurrent roots scanner should not mark through string dedup roots.
>
> Test:
> - [x] hotspot_gc_shenandoah
> - [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify -XX:-ClassUnloading
> - [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify -XX:+UseStringDeduplication
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
- Merge branch 'master' into JDK-8255847-root-verifier
- Remove StrDedup root from concurrent root scanner
- 8255847: Shenandoah: Shenandoah root verifier's roots_do() should not include weak roots
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1050/files
- new: https://git.openjdk.java.net/jdk/pull/1050/files/2dc92484..fca66171
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1050&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1050&range=00-01
Stats: 6612 lines in 443 files changed: 3686 ins; 1935 del; 991 mod
Patch: https://git.openjdk.java.net/jdk/pull/1050.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1050/head:pull/1050
PR: https://git.openjdk.java.net/jdk/pull/1050
From thartmann at openjdk.java.net Thu Nov 5 07:25:56 2020
From: thartmann at openjdk.java.net (Tobias Hartmann)
Date: Thu, 5 Nov 2020 07:25:56 GMT
Subject: RFR: 8255665: C2 should aggressively remove temporary hook nodes
[v3]
In-Reply-To: <aotwvmbaohOOvZhWXENl-PEcZWxAZYn9rHbqEg1f778=.4975299f-1344-47e9-b786-e8ba4775e6f2@github.com>
References: <H9yHPRN01lNmqpbvEVHH61K_QslMdNJC1ruWFTbfyg8=.b66f1e8e-ae2a-4787-b409-074c96d6dfc9@github.com>
<5wM4jpVaL-BlP7FdbWwQkWgpxZZbUlpTVoFw3SRBUqA=.2419c090-2d90-4ff1-8be9-d9167b0dc1ef@github.com>
<aotwvmbaohOOvZhWXENl-PEcZWxAZYn9rHbqEg1f778=.4975299f-1344-47e9-b786-e8ba4775e6f2@github.com>
Message-ID: <VAtEfuy_gZfx_lcCe2cAlqRwTjJhV03qyaH-4jrFBS8=.49c2ce2e-ac38-4521-b5db-b4ebe6bf43f4@github.com>
On Wed, 4 Nov 2020 19:01:30 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:
>> Tobias Hartmann has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Pass PhaseValues to Node::destruct
>
> Good.
Thanks Vladimir!
-------------
PR: https://git.openjdk.java.net/jdk/pull/994
From chagedorn at openjdk.java.net Thu Nov 5 07:58:57 2020
From: chagedorn at openjdk.java.net (Christian Hagedorn)
Date: Thu, 5 Nov 2020 07:58:57 GMT
Subject: RFR: 8255665: C2 should aggressively remove temporary hook nodes
[v3]
In-Reply-To: <5wM4jpVaL-BlP7FdbWwQkWgpxZZbUlpTVoFw3SRBUqA=.2419c090-2d90-4ff1-8be9-d9167b0dc1ef@github.com>
References: <H9yHPRN01lNmqpbvEVHH61K_QslMdNJC1ruWFTbfyg8=.b66f1e8e-ae2a-4787-b409-074c96d6dfc9@github.com>
<5wM4jpVaL-BlP7FdbWwQkWgpxZZbUlpTVoFw3SRBUqA=.2419c090-2d90-4ff1-8be9-d9167b0dc1ef@github.com>
Message-ID: <I3nmbAMueXj2gp_vBA9Scd4YdEBJup77h9T8e0G5I3I=.af1ee73f-5f74-414a-a33a-5352c53d6093@github.com>
On Wed, 4 Nov 2020 07:08:06 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:
>> C2 often creates temporary "hook" nodes to keep other nodes alive. Although dead, these are sometimes not removed because they don't end up on the IGVN worklist. JDK-8040213 added detection of modified nodes that are not re-processed by IGVN but currently ignores dead nodes.
>>
>> This patch includes the following changes:
>> - Adjust detection of modified nodes such that dead nodes are includes as well. This revealed several locations were dead nodes are not eagerly destructed (or not even added to the worklist for later removal). I've fixed all of these.
>> - No need to yank node inputs before calling `destruct`.
>> - `kill_dead_code` accidentally re-adds dead nodes to the `_modified_nodes` list. `Compile::remove_modified_node` should be called at the end to avoid this.
>> - Some removal of dead code.
>>
>> Tested with tier1-3, higher tiers are running.
>>
>> JDK-8255670 will further improve detection.
>>
>> Thanks,
>> Tobias
>
> Tobias Hartmann has updated the pull request incrementally with one additional commit since the last revision:
>
> Pass PhaseValues to Node::destruct
Marked as reviewed by chagedorn (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/994
From thartmann at openjdk.java.net Thu Nov 5 08:05:56 2020
From: thartmann at openjdk.java.net (Tobias Hartmann)
Date: Thu, 5 Nov 2020 08:05:56 GMT
Subject: Integrated: 8255665: C2 should aggressively remove temporary hook
nodes
In-Reply-To: <H9yHPRN01lNmqpbvEVHH61K_QslMdNJC1ruWFTbfyg8=.b66f1e8e-ae2a-4787-b409-074c96d6dfc9@github.com>
References: <H9yHPRN01lNmqpbvEVHH61K_QslMdNJC1ruWFTbfyg8=.b66f1e8e-ae2a-4787-b409-074c96d6dfc9@github.com>
Message-ID: <FF2sdWJtwDruSihygytP6CdyIHjqpwhGn2MVUSn-hR8=.d003e533-3571-4457-ae7d-b1bd87eb604d@github.com>
On Mon, 2 Nov 2020 08:14:28 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:
> C2 often creates temporary "hook" nodes to keep other nodes alive. Although dead, these are sometimes not removed because they don't end up on the IGVN worklist. JDK-8040213 added detection of modified nodes that are not re-processed by IGVN but currently ignores dead nodes.
>
> This patch includes the following changes:
> - Adjust detection of modified nodes such that dead nodes are includes as well. This revealed several locations were dead nodes are not eagerly destructed (or not even added to the worklist for later removal). I've fixed all of these.
> - No need to yank node inputs before calling `destruct`.
> - `kill_dead_code` accidentally re-adds dead nodes to the `_modified_nodes` list. `Compile::remove_modified_node` should be called at the end to avoid this.
> - Some removal of dead code.
>
> Tested with tier1-3, higher tiers are running.
>
> JDK-8255670 will further improve detection.
>
> Thanks,
> Tobias
This pull request has now been integrated.
Changeset: eb85b8da
Author: Tobias Hartmann <thartmann at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/eb85b8da
Stats: 83 lines in 16 files changed: 4 ins; 42 del; 37 mod
8255665: C2 should aggressively remove temporary hook nodes
Reviewed-by: chagedorn, kvn
-------------
PR: https://git.openjdk.java.net/jdk/pull/994
From roland at openjdk.java.net Thu Nov 5 08:50:00 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Thu, 5 Nov 2020 08:50:00 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some" assertion
failure with Shenandoah
Message-ID: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
This is a Shenandoah bug but the proposed fix is in shared code.
In an infinite loop, a barrier is located right after the loop head
and above the never branch. When the barrier is expanded, control flow
is added between the loop and the never branch. During loop
verification the assert fires because it doesn't expect any control
flow between the never branch and the loop head.
While it would have been nice to fix this Shenandoah issue in
Shenandoah code, I think the cleaner fix is to preserve the invariant
that the never branch is always right after the loop head in an
infinite loop. In the proposed patch, this is achieved by moving all
uses of the loop head to the never branch when it's constructed.
-------------
Commit messages:
- fix & test
Changes: https://git.openjdk.java.net/jdk/pull/1073/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1073&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255936
Stats: 13 lines in 2 files changed: 6 ins; 5 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1073.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1073/head:pull/1073
PR: https://git.openjdk.java.net/jdk/pull/1073
From coleenp at openjdk.java.net Thu Nov 5 12:30:01 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 5 Nov 2020 12:30:01 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<jKyDBy89u-3EdYr0CCKsfGI-SM7FgGWCTFtG9YWk6yw=.196f1a2d-ac5d-4a6d-99cc-48854e747736@github.com>
<172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com>
Message-ID: <nw7ZiLLz8c9mc75ndZXLRTaVq43GM6lDRELxYsRw_J8=.4574de5e-b30e-41f8-952a-76d98c13065b@github.com>
On Wed, 4 Nov 2020 10:05:29 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> The comment is trying to describe the situation like:
>> 1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked)
>> 2. safepoint for heap walk
>> 2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A.
>> 3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened
>>
>> The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable.
>>
>> Both operations require the table to be locked.
>>
>> The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints.
>
> For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing,
and so still effectively be a serial subtask after all the parallel subtasks are done or mostly done.
Yes, you are right that the processing will be done serially and not by a parallel worker thread. This is could spawn a new GC worker thread to process the posts, as you suggest. We could do that if we find a customer that has a complaint about the pause time of this processing.
>> The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table.
>
> I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event.
Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 5 12:30:00 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 5 Nov 2020 12:30:00 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5]
In-Reply-To: <PRK7_SkeC4IUY2_R8RUGx4LPDRNQnEN6iO788E1kfW4=.f6871609-5e81-478c-9269-50824b166c40@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
<PRK7_SkeC4IUY2_R8RUGx4LPDRNQnEN6iO788E1kfW4=.f6871609-5e81-478c-9269-50824b166c40@github.com>
Message-ID: <KLod8zevEiiS0mzEMat35zbAuustKA8NLjuQwMOA_-U=.bf27dc47-1bad-4990-b12e-1a1e312ec9f8@github.com>
On Wed, 4 Nov 2020 08:56:54 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Code review comments from Kim and Albert.
>
> src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36:
>
>> 34: class JvmtiTagMapEntryClosure;
>> 35:
>> 36: class JvmtiTagMapEntry : public HashtableEntry<WeakHandle, mtServiceability> {
>
> By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%).
>
> It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost.
Yes, there is 64/32 bits extra per hashtable entry with the standard hashtable implementation. It wouldn't have been hard to replace the oop object, but using shared code was a goal of this change.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 5 12:30:01 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 5 Nov 2020 12:30:01 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <w2jGKoh-kG4Cxy42LGV4RImf9dUqZNX1mjK1ZWemPMU=.906a6ba1-2e5d-4836-b255-4e77ae41aa4b@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<Spqq9dORyH2M6ZrkCpGKDFA1jWfFTbiqxl3gNcQEPJk=.c964ac35-062a-4471-829a-856f5ec1c04e@github.com>
<t4O-bCM6Vmtj_7ho9Lp385qpLAzD7-2vMAWPqlsdHIE=.111c897f-96f9-4281-b28e-88e1d1345057@github.com>
<w2jGKoh-kG4Cxy42LGV4RImf9dUqZNX1mjK1ZWemPMU=.906a6ba1-2e5d-4836-b255-4e77ae41aa4b@github.com>
Message-ID: <e43o_EHW4_ci3iMq5OGvNsC7CqBYBGLnvrJ9wt5prHI=.c250ff2a-a53b-4315-985e-63eb6a4437b2@github.com>
On Wed, 4 Nov 2020 09:27:39 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.)
>
> It used to be that jvmti tagmap processing was all-in-one (in GC weak reference processing, with the weak clearing, dead table entry removal, and rehashing all done in one pass. This change has split that up, with the weak clearing happening in a different place (still as part of the GC's weak reference processing) than the others (which I claim can now be part of the mutator, whether further separated or not).
>
> "Concurrent GC" has nothing to do with whether tagmaps need rehashing. Any copying collector needs to do so. A non-copying collector (whether concurrent or not) would not. (We don't have any purely non-copying collectors, but G1 concurrent oldgen collection is non-copying.) And weak reference clearing (whether concurrent or not) has nothing to do with whether objects have been moved and so the hashing has been invalidated.
>
> There's also a "well known" issue with address-based hashing and generational or similar collectors, where a simple boolean "objects have moved" flag can be problematic, and which tagmaps seem likely to be prone to. The old do_weak_oops tries to mitigate it by recognizing when the object didn't move.
The new rehash function also doesn't move the objects either. It essentially does the same as the old weak_oops_do function.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 5 12:30:02 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 5 Nov 2020 12:30:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3]
In-Reply-To: <w_eD2tSaxZ05w2Yo3T2fjV00vT_DdUTnVzhrcLdsjfk=.0d35ce07-ca69-4dc9-8ecf-2fb705ebfb9f@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com>
<Z9BX2fia9O7E2P1XSb8qGDxqHd9mDGhyRmLFfOZcX-g=.febdd967-0516-4444-8d48-4db6e477da03@github.com>
<0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com>
<1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com>
<gxedU6v8A_Mkqc_lHxakcxetjdDJaM-6M2OZPGy8rlg=.59b52b6b-2c61-4ff9-b830-174bd14dd6d1@github.com>
<w_eD2tSaxZ05w2Yo3T2fjV00vT_DdUTnVzhrcLdsjfk=.0d35ce07-ca69-4dc9-8ecf-2fb705ebfb9f@github.com>
Message-ID: <z-eEVFhf2X4ElyR5lq3r5CmQSlOz-x00GZld_N1WrbQ=.7644405f-7f35-42a2-bf83-bbc040e7b17d@github.com>
On Wed, 4 Nov 2020 07:42:36 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>>> Ok, so I'm not sure what to do with this:
>>>
>>> enum Phase {
>>> // Serial phase.
>>> JVMTI_ONLY(jvmti)
>>> // Additional implicit phase values follow for oopstorages.
>>> `};`
>>>
>>> I've removed the only thing in this enum.
>>
>> Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums.
>
>> > Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet.
>>
>> This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting.
>
> I think the deletion stopped at the wrong place; it either went too far, or not far enough.
I restored the empty enum for Phase and can open a new RFE if there is more code to remove.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 5 12:45:01 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 5 Nov 2020 12:45:01 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6]
In-Reply-To: <TgRmyCL8pDACuMxxHfdsqN9mOVxyZaKNRpfXbssEl94=.69114d11-53a5-4220-b5ba-e9ecdfe96f26@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<rrqQkn7YRMjol0rpPIDSBsbXZXqFULDSRxMlMPJ5cjs=.f10aea2a-10db-4555-ba82-8664a472e393@github.com>
<TgRmyCL8pDACuMxxHfdsqN9mOVxyZaKNRpfXbssEl94=.69114d11-53a5-4220-b5ba-e9ecdfe96f26@github.com>
Message-ID: <q0OJ0RDIEfVUvYfaNAHV0oKWVO4RGch_9y1Qd6vEErI=.d54b2cee-4333-47e5-9469-7539b8bc1e43@github.com>
On Wed, 4 Nov 2020 22:09:21 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:
>> Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Add back WeakProcessorPhases::Phase enum.
>> - Serguei 1.
>
> Thank you for the update, Coleen!
> I leave it for you to decide to refactor the gc_notification or not.
> Thanks,
> Serguei
Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From eosterlund at openjdk.java.net Thu Nov 5 14:22:01 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Thu, 5 Nov 2020 14:22:01 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <D0tXRBpNv7FmHN9iMZdiHmDpVXcXD42Rhf04Mo7XSs8=.7a75f15b-27b5-4c08-8251-c0d4cb1ba282@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<D0tXRBpNv7FmHN9iMZdiHmDpVXcXD42Rhf04Mo7XSs8=.7a75f15b-27b5-4c08-8251-c0d4cb1ba282@github.com>
Message-ID: <F3TB_wJoGBgQxOnfztjq5C7tjtReiCINyguQZWsBqXo=.4cee88b8-e6fd-406e-b308-0d9910d4d715@github.com>
On Tue, 3 Nov 2020 21:14:04 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> src/hotspot/share/prims/jvmtiTagMap.cpp line 3018:
>>
>>> 3016: }
>>> 3017: // Later GC code will relocate the oops, so defer rehashing until then.
>>> 3018: tag_map->_needs_rehashing = true;
>>
>> This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.)
>>
>> For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing.
>>
>> (I think CMS oldgen remark may have been similar, for what that's worth.)
>
> Ok, so I'm going to need help to know where in all the different GCs to make this call. This seemed simpler at the expense of maybe causing a rehash at some points when it might not be necessary.
For what GC is this wrong? I can see that it might yield more work than required, when performing a full GC, but not that it would do too little work. In other words, I can't see how it is wrong, as opposed to inaccurate. Littering GCs with JVMTI hooks so that we can optimize away an operation we do every young GC, from a full GC, does not really seem worth it IMO.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From eosterlund at openjdk.java.net Thu Nov 5 14:40:02 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Thu, 5 Nov 2020 14:40:02 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <nw7ZiLLz8c9mc75ndZXLRTaVq43GM6lDRELxYsRw_J8=.4574de5e-b30e-41f8-952a-76d98c13065b@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<jKyDBy89u-3EdYr0CCKsfGI-SM7FgGWCTFtG9YWk6yw=.196f1a2d-ac5d-4a6d-99cc-48854e747736@github.com>
<172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com>
<nw7ZiLLz8c9mc75ndZXLRTaVq43GM6lDRELxYsRw_J8=.4574de5e-b30e-41f8-952a-76d98c13065b@github.com>
Message-ID: <OpyVqAff9GnzQVUeTMlTOXNPbHmBbAz1O4VKNg6neBY=.4bf447ac-3f45-468b-9f73-edd557846652@github.com>
On Wed, 4 Nov 2020 13:32:07 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event.
>
> Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times.
The point of this change was not necessarily to be lazy about updating the tagmap, until someone uses it. The point was to get rid of the last annoying serial GC phase. Doing it all lazily would certainly also achieve that. But it would also lead to situations where no event is ever posted from GC to GC. So you would get the event 20 GCs later, which might come as a surprise. It did come as a surprise to some tests, so it is reasonable to assume it would come as a surprise to users too. And I don't think we want such surprises unless we couldn't deal with them. And we can.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From eosterlund at openjdk.java.net Thu Nov 5 14:53:01 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Thu, 5 Nov 2020 14:53:01 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <nw7ZiLLz8c9mc75ndZXLRTaVq43GM6lDRELxYsRw_J8=.4574de5e-b30e-41f8-952a-76d98c13065b@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<jKyDBy89u-3EdYr0CCKsfGI-SM7FgGWCTFtG9YWk6yw=.196f1a2d-ac5d-4a6d-99cc-48854e747736@github.com>
<172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com>
<nw7ZiLLz8c9mc75ndZXLRTaVq43GM6lDRELxYsRw_J8=.4574de5e-b30e-41f8-952a-76d98c13065b@github.com>
Message-ID: <JKjD0OBw8P3VTZ29sWsyX2eT6u6rFYgtumRKtwpkR-w=.6a347da9-8220-4eeb-8ca8-c757dc3c00ac@github.com>
On Wed, 4 Nov 2020 13:22:57 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing,
and so still effectively be a serial subtask after all the parallel subtasks are done or mostly done.
>
> Yes, you are right that the processing will be done serially and not by a parallel worker thread. This is could spawn a new GC worker thread to process the posts, as you suggest. We could do that if we find a customer that has a complaint about the pause time of this processing.
So both before and now, this task is a single threaded task. The difference is that before that single threaded task could be performed in parallel to other tasks. So if the table is small, you probably won't be able to notice any difference as small table implies not much to do. And if the table is large, you still probably won't be able to notice any difference as a large table implies it will dominate the pause with both the old and new approach. Any difference at all is bounded at 2x processing time, as it was serial both before and after. But now if we have a perfectly medium balanced table, we can at the very worst observe a theoretical 2x worse processing of this JVMTI table. I think that if we truly did care about this difference, and that it is important to keep this code as well performed as possible, then we would not have a serial phase for this at all. The fact that this has been serial suggests to me that it is not a path that is critical, and therefore I don't think op
timizing the theoretical max 2x worse processing times for perfectly medium sized JVMTI tag map tables, is worth the hassle. At least I can't see why this would be of any importance.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 5 15:07:01 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 5 Nov 2020 15:07:01 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5]
In-Reply-To: <KLod8zevEiiS0mzEMat35zbAuustKA8NLjuQwMOA_-U=.bf27dc47-1bad-4990-b12e-1a1e312ec9f8@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<FdaHMHmb6co63PfIZwGt3QTiGVrbpJ4gYyDgaViy9oM=.524d21e9-fb9a-4f56-8ef6-18710372d89b@github.com>
<PRK7_SkeC4IUY2_R8RUGx4LPDRNQnEN6iO788E1kfW4=.f6871609-5e81-478c-9269-50824b166c40@github.com>
<KLod8zevEiiS0mzEMat35zbAuustKA8NLjuQwMOA_-U=.bf27dc47-1bad-4990-b12e-1a1e312ec9f8@github.com>
Message-ID: <_w3Wb6lCIkBeH6UxIGCZ-HoXgwx-qSYF6KpLY1txy6Y=.eeeb53b9-6b1f-41cb-b59b-109b714e8eed@github.com>
On Wed, 4 Nov 2020 13:19:21 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36:
>>
>>> 34: class JvmtiTagMapEntryClosure;
>>> 35:
>>> 36: class JvmtiTagMapEntry : public HashtableEntry<WeakHandle, mtServiceability> {
>>
>> By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%).
>>
>> It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost.
>
> Yes, there is 64/32 bits extra per hashtable entry with the standard hashtable implementation. It wouldn't have been hard to replace the oop object, but using shared code was a goal of this change.
So looking at the concurrent hashtable, the entries can be created without saving the hashcode. I was going to use that at first but didn't want to cut/paste the boilerplate to do so and the jvmti tag map hashtable is always accessed with a lock. This could be a future RFE if necessary and would also serve to eliminate another ad-hoc hashtable.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From zgu at openjdk.java.net Thu Nov 5 15:32:58 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 5 Nov 2020 15:32:58 GMT
Subject: RFR: 8255847: Shenandoah: Shenandoah should not mark through weak
roots [v2]
In-Reply-To: <94-imIwm3FHENBFcksLSvUK9ndfKUYfssdzeHf-yq0I=.ff46f281-6321-4a05-8ced-381c997b7d46@github.com>
References: <dJvQ4JLRc1zmIW86lSXrk7HU5mjVb5n_1hKm_G8WJQs=.5d0d7443-9391-4e17-80ef-91a988b3f501@github.com>
<94-imIwm3FHENBFcksLSvUK9ndfKUYfssdzeHf-yq0I=.ff46f281-6321-4a05-8ced-381c997b7d46@github.com>
Message-ID: <LOTdOwyESjKqVDSecxWW2EDFB9AZRUIeNr-wYWvK0N0=.76e8a184-ce71-456b-bd35-166e18dc6f85@github.com>
On Wed, 4 Nov 2020 21:38:10 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Looks good. Make sure to merge latest tip to get full pre-submit testing in GitHub actions.
Reran pre-submit tests, all clean.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1050
From zgu at openjdk.java.net Thu Nov 5 15:32:59 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 5 Nov 2020 15:32:59 GMT
Subject: Integrated: 8255847: Shenandoah: Shenandoah should not mark through
weak roots
In-Reply-To: <dJvQ4JLRc1zmIW86lSXrk7HU5mjVb5n_1hKm_G8WJQs=.5d0d7443-9391-4e17-80ef-91a988b3f501@github.com>
References: <dJvQ4JLRc1zmIW86lSXrk7HU5mjVb5n_1hKm_G8WJQs=.5d0d7443-9391-4e17-80ef-91a988b3f501@github.com>
Message-ID: <KDnsP0z-VCWq3LUnacRMneyfbmP8CQ31gBJ9pQxRuwg=.2ff89207-01c3-46d5-a398-8b00f643006d@github.com>
On Wed, 4 Nov 2020 00:09:13 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> After moving weak root processing into concurrent phase, Shenandoah should no longer marks through weak roots, even when class unloading is disabled, given weak root processing no longer contributes to latency.
>
> There are a couple of bugs:
> 1) ShenandoahRootVerifier was not updated to reflect the change.
> The problem did not show up due to SH::parallel_cleaning() uses wrong flag to determine if it should cleanup weak roots, and it will be addressed in separate CR.
>
> 2) Concurrent roots scanner should not mark through string dedup roots.
>
> Test:
> - [x] hotspot_gc_shenandoah
> - [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify -XX:-ClassUnloading
> - [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify -XX:+UseStringDeduplication
This pull request has now been integrated.
Changeset: 31918c55
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/31918c55
Stats: 11 lines in 3 files changed: 0 ins; 11 del; 0 mod
8255847: Shenandoah: Shenandoah should not mark through weak roots
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1050
From zgu at openjdk.java.net Thu Nov 5 18:23:02 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 5 Nov 2020 18:23:02 GMT
Subject: RFR: 8255955: Shenandoah: Only STW GC should process concurrent roots
at pauses
Message-ID: <wRaRYiMHz1JzQLXIj6zWsdtE8oFHBmYu_-SWtsIW0tg=.8b11da0d-6b7a-4ec9-a5d6-fc4f27627cc1@github.com>
This is a followup of JDK-8255847.
In early versions of Shenandoah, it marked through weak roots when class unloading is disabled. This is no longer the case.
After moving weak roots (except that JVMTI weak root) processing to concurrent phase for concurrent GC, only STW GCs (degenerated GC and full GC) need to process concurrent weak roots at pause.
-------------
Commit messages:
- Merge branch 'master' into JDK-8255955-stw-conc-roots
- 8255955: Shenandoah: Only STW GC should process concurrent roots at pauses
- Merge branch 'master' into JDK-8255847-root-verifier
- Remove StrDedup root from concurrent root scanner
- 8255847: Shenandoah: Shenandoah root verifier's roots_do() should not include weak roots
Changes: https://git.openjdk.java.net/jdk/pull/1081/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1081&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255955
Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
Patch: https://git.openjdk.java.net/jdk/pull/1081.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1081/head:pull/1081
PR: https://git.openjdk.java.net/jdk/pull/1081
From rkennke at openjdk.java.net Thu Nov 5 18:32:57 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 5 Nov 2020 18:32:57 GMT
Subject: RFR: 8255955: Shenandoah: Only STW GC should process concurrent
roots at pauses
In-Reply-To: <wRaRYiMHz1JzQLXIj6zWsdtE8oFHBmYu_-SWtsIW0tg=.8b11da0d-6b7a-4ec9-a5d6-fc4f27627cc1@github.com>
References: <wRaRYiMHz1JzQLXIj6zWsdtE8oFHBmYu_-SWtsIW0tg=.8b11da0d-6b7a-4ec9-a5d6-fc4f27627cc1@github.com>
Message-ID: <xkZvyTJY9pXxG_jMbOiBuBaWkG-tfzjG5ouY5zezGak=.e28a8925-8657-485a-8394-60503b5a9c80@github.com>
On Thu, 5 Nov 2020 18:18:09 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> This is a followup of JDK-8255847.
>
> In early versions of Shenandoah, it marked through weak roots when class unloading is disabled. This is no longer the case.
>
> After moving weak roots (except that JVMTI weak root) processing to concurrent phase for concurrent GC, only STW GCs (degenerated GC and full GC) need to process concurrent weak roots at pause.
Looks good to me, thank you!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1081
From zgu at openjdk.java.net Thu Nov 5 19:02:57 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 5 Nov 2020 19:02:57 GMT
Subject: Integrated: 8255955: Shenandoah: Only STW GC should process
concurrent roots at pauses
In-Reply-To: <wRaRYiMHz1JzQLXIj6zWsdtE8oFHBmYu_-SWtsIW0tg=.8b11da0d-6b7a-4ec9-a5d6-fc4f27627cc1@github.com>
References: <wRaRYiMHz1JzQLXIj6zWsdtE8oFHBmYu_-SWtsIW0tg=.8b11da0d-6b7a-4ec9-a5d6-fc4f27627cc1@github.com>
Message-ID: <vIgMuSfaN-yipccR-EQBwMwcImVHN4F_mWYmbKx8F3c=.32ab7284-3be2-40a1-932a-2be4e6c3bbf7@github.com>
On Thu, 5 Nov 2020 18:18:09 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> This is a followup of JDK-8255847.
>
> In early versions of Shenandoah, it marked through weak roots when class unloading is disabled. This is no longer the case.
>
> After moving weak roots (except that JVMTI weak root) processing to concurrent phase for concurrent GC, only STW GCs (degenerated GC and full GC) need to process concurrent weak roots at pause.
This pull request has now been integrated.
Changeset: fc894ab1
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/fc894ab1
Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
8255955: Shenandoah: Only STW GC should process concurrent roots at pauses
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1081
From zgu at openjdk.java.net Thu Nov 5 19:06:09 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 5 Nov 2020 19:06:09 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v5]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <d-mQuhllrwksSaXYV3WJLiYkvDh9IMkfKLG3Wvgv_9I=.1e002674-4221-45df-9a16-cf4ea6fa6a3f@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits:
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Move weak reference processing out of STWMark and fix its timings
- Merge branch 'JDK-8255019-sh-mark' of github.com:zhengyu123/jdk into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Merge branch 'master' into JDK-8255019-sh-mark
- Make ShenandoahMarkCompact stack allocated
- Split finish mark and pre-evaculation
- 8255019: Shenandoah: Split STW and concurrent mark into separate classes
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=04
Stats: 1969 lines in 21 files changed: 1072 ins; 737 del; 160 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From roland at openjdk.java.net Fri Nov 6 11:13:04 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Fri, 6 Nov 2020 11:13:04 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah [v2]
In-Reply-To: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
Message-ID: <dGIlql-cWnZXaROTZn-roVLs5gVcXmk29XyL9323UlI=.d62d69ca-d47f-44a7-962a-96e04dd59328@github.com>
> This is a Shenandoah bug but the proposed fix is in shared code.
>
> In an infinite loop, a barrier is located right after the loop head
> and above the never branch. When the barrier is expanded, control flow
> is added between the loop and the never branch. During loop
> verification the assert fires because it doesn't expect any control
> flow between the never branch and the loop head.
>
> While it would have been nice to fix this Shenandoah issue in
> Shenandoah code, I think the cleaner fix is to preserve the invariant
> that the never branch is always right after the loop head in an
> infinite loop. In the proposed patch, this is achieved by moving all
> uses of the loop head to the never branch when it's constructed.
Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains one commit:
fix & test
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1073/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1073&range=01
Stats: 13 lines in 2 files changed: 6 ins; 5 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1073.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1073/head:pull/1073
PR: https://git.openjdk.java.net/jdk/pull/1073
From roland at openjdk.java.net Fri Nov 6 11:17:55 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Fri, 6 Nov 2020 11:17:55 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah
In-Reply-To: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
Message-ID: <GdaiDzVNx_r596VloLQPzx9vtYCV0YGmlJxqoL758CM=.dcb2558f-23c4-4f80-8d17-7ac64a4d4824@github.com>
On Thu, 5 Nov 2020 08:44:03 GMT, Roland Westrelin <roland at openjdk.org> wrote:
> This is a Shenandoah bug but the proposed fix is in shared code.
>
> In an infinite loop, a barrier is located right after the loop head
> and above the never branch. When the barrier is expanded, control flow
> is added between the loop and the never branch. During loop
> verification the assert fires because it doesn't expect any control
> flow between the never branch and the loop head.
>
> While it would have been nice to fix this Shenandoah issue in
> Shenandoah code, I think the cleaner fix is to preserve the invariant
> that the never branch is always right after the loop head in an
> infinite loop. In the proposed patch, this is achieved by moving all
> uses of the loop head to the never branch when it's constructed.
I rebased the change
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From coleenp at openjdk.java.net Fri Nov 6 12:16:14 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Fri, 6 Nov 2020 12:16:14 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v7]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <l3OMmC1nHM6D6CciDKYX6G4bGVolf-dL9dz7SU-8jlA=.894ea4e7-2b53-40a2-97c7-693fe384ab86@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits:
- Merge branch 'master' into jvmti-table
- Add back WeakProcessorPhases::Phase enum.
- Serguei 1.
- Code review comments from Kim and Albert.
- Merge branch 'master' into jvmti-table
- Merge branch 'master' into jvmti-table
- More review comments from Stefan and ErikO
- Code review comments from StefanK.
- 8212879: Make JVMTI TagMap table not hash on oop address
-------------
Changes: https://git.openjdk.java.net/jdk/pull/967/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=06
Stats: 1748 lines in 41 files changed: 628 ins; 1018 del; 102 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at openjdk.java.net Fri Nov 6 17:51:01 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 6 Nov 2020 17:51:01 GMT
Subject: RFR: 8255991: Shenandoah: Apply 'weak' LRB on cmpxchg and xchg
Message-ID: <M_uTkjKmSGjUgDVGkrp_-cq0aYIp9YEi4vIvln4LsRg=.dee979b6-8a99-4a07-8c13-d1e451b2cf59@github.com>
It is possible to access a Reference's referent by using various cmpxchg and xchg intrinsics. When that happens, we need to apply the weak LRB to prevent resurrection.
Testing: hotspot_gc_shenandoah
-------------
Commit messages:
- 8255991: Shenandoah: Apply 'weak' LRB on cmpxchg and xchg
Changes: https://git.openjdk.java.net/jdk/pull/1098/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1098&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255991
Stats: 8 lines in 3 files changed: 3 ins; 0 del; 5 mod
Patch: https://git.openjdk.java.net/jdk/pull/1098.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1098/head:pull/1098
PR: https://git.openjdk.java.net/jdk/pull/1098
From rkennke at openjdk.java.net Fri Nov 6 21:16:03 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 6 Nov 2020 21:16:03 GMT
Subject: RFR: 8255999: Shenandoah: Invoke native-LRB only on non-strong refs,
again
Message-ID: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
In JDK-8255691, I changed the condition for invoking native-LRB only on non-strong-refs. Somehow, when merging JDK-8254315, I got the condition wrong again.
Testing: hotspot_gc_shenandoah
-------------
Commit messages:
- 8255999: Shenandoah: Invoke native-LRB only on non-strong refs, again
Changes: https://git.openjdk.java.net/jdk/pull/1101/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1101&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255999
Stats: 11 lines in 1 file changed: 6 ins; 4 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1101.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1101/head:pull/1101
PR: https://git.openjdk.java.net/jdk/pull/1101
From zgu at openjdk.java.net Fri Nov 6 22:57:55 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 6 Nov 2020 22:57:55 GMT
Subject: RFR: 8255999: Shenandoah: Invoke native-LRB only on non-strong
refs, again
In-Reply-To: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
References: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
Message-ID: <inQkm5u4eKJ4A3yfMBx2YWDhG28uY8QXNw3KQEQ26I4=.78153013-59b5-407a-ac34-8dbf6f77c9c1@github.com>
On Fri, 6 Nov 2020 21:11:08 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> In JDK-8255691, I changed the condition for invoking native-LRB only on non-strong-refs. Somehow, when merging JDK-8254315, I got the condition wrong again.
>
> Testing: hotspot_gc_shenandoah
Looks good.
-------------
Marked as reviewed by zgu (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1101
From rkennke at openjdk.java.net Sat Nov 7 20:15:01 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Sat, 7 Nov 2020 20:15:01 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects
Message-ID: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
Testing: hotspot_gc_shenandoah, tier1 & tier2 with -XX:+UseShenandoahGC
-------------
Commit messages:
- 8256011: Shenandoah: Don't resurrect finalizably reachable objects
Changes: https://git.openjdk.java.net/jdk/pull/1109/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256011
Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Sun Nov 8 10:03:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Sun, 8 Nov 2020 10:03:07 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v2]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <rzvXaG3sHtgRJh0Pg5n_XH-QqPQD6ypsFulC5b_obE4=.23228ca7-824a-41d0-96b8-00c541ccd961@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> Testing: hotspot_gc_shenandoah, tier1 & tier2 with -XX:+UseShenandoahGC
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Simplify condition, don't resurrect finalizably reachable objects even on phantom access
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/672333cc..189be782
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=00-01
Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From shade at openjdk.java.net Sun Nov 8 10:32:59 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Sun, 8 Nov 2020 10:32:59 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v2]
In-Reply-To: <rzvXaG3sHtgRJh0Pg5n_XH-QqPQD6ypsFulC5b_obE4=.23228ca7-824a-41d0-96b8-00c541ccd961@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<rzvXaG3sHtgRJh0Pg5n_XH-QqPQD6ypsFulC5b_obE4=.23228ca7-824a-41d0-96b8-00c541ccd961@github.com>
Message-ID: <MrkfFxL6H69z45ddz74fmosjWXdaADZLQ-4z3cnMyAI=.8b18199f-12e3-40ff-b4f4-956a37ddaad3@github.com>
On Sun, 8 Nov 2020 10:03:07 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> Testing: hotspot_gc_shenandoah, tier1 & tier2 with -XX:+UseShenandoahGC
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Simplify condition, don't resurrect finalizably reachable objects even on phantom access
This makes sense, thanks.
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 105:
> 103: inline oop ShenandoahBarrierSet::load_reference_barrier(oop obj, T* load_addr) {
> 104:
> 105: // Prevent resurrection of unreachable non-strorg references.
Have a chance to fix the typo, `non-strong` here?
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1109
From shade at openjdk.java.net Sun Nov 8 10:35:55 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Sun, 8 Nov 2020 10:35:55 GMT
Subject: RFR: 8255999: Shenandoah: Invoke native-LRB only on non-strong
refs, again
In-Reply-To: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
References: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
Message-ID: <AfncQFKDI86ajnBnGogqvMXnyHnqJqLs7jAI5zVPX98=.945473e1-b5fe-440a-80e8-7eac639024f6@github.com>
On Fri, 6 Nov 2020 21:11:08 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> In JDK-8255691, I changed the condition for invoking native-LRB only on non-strong-refs. Somehow, when merging JDK-8254315, I got the condition wrong again.
>
> Testing: hotspot_gc_shenandoah
This makes sense. Looks good.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1101
From shade at openjdk.java.net Sun Nov 8 14:56:59 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Sun, 8 Nov 2020 14:56:59 GMT
Subject: RFR: 8255991: Shenandoah: Apply 'weak' LRB on cmpxchg and xchg
In-Reply-To: <M_uTkjKmSGjUgDVGkrp_-cq0aYIp9YEi4vIvln4LsRg=.dee979b6-8a99-4a07-8c13-d1e451b2cf59@github.com>
References: <M_uTkjKmSGjUgDVGkrp_-cq0aYIp9YEi4vIvln4LsRg=.dee979b6-8a99-4a07-8c13-d1e451b2cf59@github.com>
Message-ID: <5d4PdExALw13yedls7jrZSAvOjd9hUKmw5nleWM_CMo=.117f449b-dadd-438d-a5b3-5d3fc4e74386@github.com>
On Fri, 6 Nov 2020 17:45:53 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> It is possible to access a Reference's referent by using various cmpxchg and xchg intrinsics. When that happens, we need to apply the weak LRB to prevent resurrection.
>
> Testing: hotspot_gc_shenandoah
Makes sense to me. Looks good.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1098
From rkennke at openjdk.java.net Mon Nov 9 09:21:58 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 9 Nov 2020 09:21:58 GMT
Subject: Integrated: 8255991: Shenandoah: Apply 'weak' LRB on cmpxchg and xchg
In-Reply-To: <M_uTkjKmSGjUgDVGkrp_-cq0aYIp9YEi4vIvln4LsRg=.dee979b6-8a99-4a07-8c13-d1e451b2cf59@github.com>
References: <M_uTkjKmSGjUgDVGkrp_-cq0aYIp9YEi4vIvln4LsRg=.dee979b6-8a99-4a07-8c13-d1e451b2cf59@github.com>
Message-ID: <JMrvv9TTPloMkosCKG5rlFD2UmltpJ-rJ0dvgGVg6g0=.8b2c51f8-7dea-4c32-b433-b6b19e43231a@github.com>
On Fri, 6 Nov 2020 17:45:53 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> It is possible to access a Reference's referent by using various cmpxchg and xchg intrinsics. When that happens, we need to apply the weak LRB to prevent resurrection.
>
> Testing: hotspot_gc_shenandoah
This pull request has now been integrated.
Changeset: d99e1f6c
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/d99e1f6c
Stats: 8 lines in 3 files changed: 3 ins; 0 del; 5 mod
8255991: Shenandoah: Apply 'weak' LRB on cmpxchg and xchg
Reviewed-by: shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1098
From rkennke at openjdk.java.net Mon Nov 9 09:31:00 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 9 Nov 2020 09:31:00 GMT
Subject: RFR: 8256040: Shenandoah: Allow NULL referent in
ShenandoahReferenceProcessor::should_discover()
Message-ID: <mU0wTUVDfkNDSA9dZ_9zj0BB53Ecph7fBhPdEndLlpM=.acf2050f-7b0d-477d-83f0-f8aeedb7ab3d@github.com>
Testing showed the following assert:
Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/oops/compressedOops.inline.hpp:54), pid=711923, tid=712264
assert(!is_null(v)) failed: narrow oop value can never be zero
V [libjvm.so+0x1629c84] bool ShenandoahReferenceProcessor::should_discover<narrowOop>(oop, ReferenceType) const+0x404
V [libjvm.so+0x162a01b] bool ShenandoahReferenceProcessor::discover<narrowOop>(oop, ReferenceType, unsigned int)+0x4b
V [libjvm.so+0x16239e8] ShenandoahReferenceProcessor::discover_reference(oop, ReferenceType)+0x138
V [libjvm.so+0x15b7a26] bool InstanceRefKlass::try_discover<narrowOop, ShenandoahMarkRefsMetadataClosure>(oop, ReferenceType, ShenandoahMarkRefsMetadataClosure*)+0x96
V [libjvm.so+0x15b7b3d] void InstanceRefKlass::oop_oop_iterate_discovery<narrowOop, ShenandoahMarkRefsMetadataClosure, AlwaysContains>(oop, ReferenceType, ShenandoahMarkRefsMetadataClosure*, AlwaysContains&) [clone .constprop.696]+0x3d
V [libjvm.so+0x15b7ce1] void InstanceRefKlass::oop_oop_iterate_ref_processing<narrowOop, ShenandoahMarkRefsMetadataClosure, AlwaysContains>(oop, ShenandoahMarkRefsMetadataClosure*, AlwaysContains&)+0x81
V [libjvm.so+0x15b835e] void OopOopIterateDispatch<ShenandoahMarkRefsMetadataClosure>::Table::oop_oop_iterate<InstanceRefKlass, narrowOop>(ShenandoahMarkRefsMetadataClosure*, oop, Klass*)+0x1ee
V [libjvm.so+0x15a905e] void ShenandoahConcurrentMark::do_task<ShenandoahMarkRefsMetadataClosure>(Padded<BufferedOverflowTaskQueue<ShenandoahMarkTask, (MEMFLAGS)5, 131072u>, 128ul>*, ShenandoahMarkRefsMetadataClosure*, unsigned short*, ShenandoahMarkTask*)+0x7be
V [libjvm.so+0x15ab81e] void ShenandoahConcurrentMark::mark_loop_work<ShenandoahMarkRefsMetadataClosure, true>(ShenandoahMarkRefsMetadataClosure*, unsigned short*, unsigned int, TaskTerminator*)+0x3ce
We have this code in ShRefProc::should_discover():
oop referent = CompressedOops::decode_not_null(heap_oop);
However, the referent can legally be NULL. We even treat NULL specially in is_inactive().
Testing:
- hotspot_gc_shenandoah
- tier1+UseShenandoahGC+ShenandoahVerify (which originally showed the problem)
-------------
Commit messages:
- 8256040: Shenandoah: Allow NULL referent in ShenandoahReferenceProcessor::should_discover()
Changes: https://git.openjdk.java.net/jdk/pull/1117/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1117&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256040
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1117.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1117/head:pull/1117
PR: https://git.openjdk.java.net/jdk/pull/1117
From shade at openjdk.java.net Mon Nov 9 09:40:56 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 9 Nov 2020 09:40:56 GMT
Subject: RFR: 8256040: Shenandoah: Allow NULL referent in
ShenandoahReferenceProcessor::should_discover()
In-Reply-To: <mU0wTUVDfkNDSA9dZ_9zj0BB53Ecph7fBhPdEndLlpM=.acf2050f-7b0d-477d-83f0-f8aeedb7ab3d@github.com>
References: <mU0wTUVDfkNDSA9dZ_9zj0BB53Ecph7fBhPdEndLlpM=.acf2050f-7b0d-477d-83f0-f8aeedb7ab3d@github.com>
Message-ID: <T3BUwx51uxLXCvZgaCTezhP8bIFg8T_RqIXmlx653ac=.6cd64299-f9be-4d37-aaf4-ac378a7f225d@github.com>
On Mon, 9 Nov 2020 09:25:41 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Testing showed the following assert:
>
> Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/oops/compressedOops.inline.hpp:54), pid=711923, tid=712264
> assert(!is_null(v)) failed: narrow oop value can never be zero
>
> V [libjvm.so+0x1629c84] bool ShenandoahReferenceProcessor::should_discover<narrowOop>(oop, ReferenceType) const+0x404
> V [libjvm.so+0x162a01b] bool ShenandoahReferenceProcessor::discover<narrowOop>(oop, ReferenceType, unsigned int)+0x4b
> V [libjvm.so+0x16239e8] ShenandoahReferenceProcessor::discover_reference(oop, ReferenceType)+0x138
> V [libjvm.so+0x15b7a26] bool InstanceRefKlass::try_discover<narrowOop, ShenandoahMarkRefsMetadataClosure>(oop, ReferenceType, ShenandoahMarkRefsMetadataClosure*)+0x96
> V [libjvm.so+0x15b7b3d] void InstanceRefKlass::oop_oop_iterate_discovery<narrowOop, ShenandoahMarkRefsMetadataClosure, AlwaysContains>(oop, ReferenceType, ShenandoahMarkRefsMetadataClosure*, AlwaysContains&) [clone .constprop.696]+0x3d
> V [libjvm.so+0x15b7ce1] void InstanceRefKlass::oop_oop_iterate_ref_processing<narrowOop, ShenandoahMarkRefsMetadataClosure, AlwaysContains>(oop, ShenandoahMarkRefsMetadataClosure*, AlwaysContains&)+0x81
> V [libjvm.so+0x15b835e] void OopOopIterateDispatch<ShenandoahMarkRefsMetadataClosure>::Table::oop_oop_iterate<InstanceRefKlass, narrowOop>(ShenandoahMarkRefsMetadataClosure*, oop, Klass*)+0x1ee
> V [libjvm.so+0x15a905e] void ShenandoahConcurrentMark::do_task<ShenandoahMarkRefsMetadataClosure>(Padded<BufferedOverflowTaskQueue<ShenandoahMarkTask, (MEMFLAGS)5, 131072u>, 128ul>*, ShenandoahMarkRefsMetadataClosure*, unsigned short*, ShenandoahMarkTask*)+0x7be
> V [libjvm.so+0x15ab81e] void ShenandoahConcurrentMark::mark_loop_work<ShenandoahMarkRefsMetadataClosure, true>(ShenandoahMarkRefsMetadataClosure*, unsigned short*, unsigned int, TaskTerminator*)+0x3ce
>
> We have this code in ShRefProc::should_discover():
> oop referent = CompressedOops::decode_not_null(heap_oop);
>
> However, the referent can legally be NULL. We even treat NULL specially in is_inactive().
>
> Testing:
> - hotspot_gc_shenandoah
> - tier1+UseShenandoahGC+ShenandoahVerify (which originally showed the problem)
Looks fine and trivial.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1117
From rkennke at openjdk.java.net Mon Nov 9 11:15:14 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 9 Nov 2020 11:15:14 GMT
Subject: RFR: 8255999: Shenandoah: Invoke native-LRB only on non-strong
refs, again [v2]
In-Reply-To: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
References: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
Message-ID: <pucC9Z_IXTzZXf7h9DGZhkygP_4n8nP9zQndC5eJiyg=.868606ec-d99a-40b6-b66e-fd386d78241b@github.com>
> In JDK-8255691, I changed the condition for invoking native-LRB only on non-strong-refs. Somehow, when merging JDK-8254315, I got the condition wrong again.
>
> Testing: hotspot_gc_shenandoah
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
- Merge branch 'master' into JDK-8255999
- 8255999: Shenandoah: Invoke native-LRB only on non-strong refs, again
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1101/files
- new: https://git.openjdk.java.net/jdk/pull/1101/files/b552dc70..20f503a8
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1101&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1101&range=00-01
Stats: 46846 lines in 372 files changed: 25991 ins; 13585 del; 7270 mod
Patch: https://git.openjdk.java.net/jdk/pull/1101.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1101/head:pull/1101
PR: https://git.openjdk.java.net/jdk/pull/1101
From rkennke at openjdk.java.net Mon Nov 9 11:21:04 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 9 Nov 2020 11:21:04 GMT
Subject: RFR: 8256046: Shenandoah: fatal error: DEBUG MESSAGE: unexpected null
in intrinsic
Message-ID: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
Testing found this problem (very rare/hard to reproduce):
# Internal Error (d:/a/jdk/jdk/jdk/src/hotspot/cpu/x86/macroAssembler_x86.cpp:880), pid=6160, tid=5828
# fatal error: DEBUG MESSAGE: unexpected null in intrinsic
Testing:
- [ ] hotspot_gc_shenandoah
- [ ] tier1+Shenandoah
- [ ] tier2+Shenandoah
-------------
Commit messages:
- 8256046: Shenandoah: fatal error: DEBUG MESSAGE: unexpected null in intrinsic
Changes: https://git.openjdk.java.net/jdk/pull/1120/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1120&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256046
Stats: 8 lines in 1 file changed: 7 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1120.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1120/head:pull/1120
PR: https://git.openjdk.java.net/jdk/pull/1120
From roland at openjdk.java.net Mon Nov 9 11:21:04 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Mon, 9 Nov 2020 11:21:04 GMT
Subject: RFR: 8256046: Shenandoah: fatal error: DEBUG MESSAGE: unexpected
null in intrinsic
In-Reply-To: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
References: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
Message-ID: <nAjd-Fbf9zH_6te2aUlc2qKw2W8x50CwojirLc-N_3Q=.a1d7de27-f23f-4181-bddd-4dd24e30bbab@github.com>
On Mon, 9 Nov 2020 11:10:27 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Testing found this problem (very rare/hard to reproduce):
>
> # Internal Error (d:/a/jdk/jdk/jdk/src/hotspot/cpu/x86/macroAssembler_x86.cpp:880), pid=6160, tid=5828
> # fatal error: DEBUG MESSAGE: unexpected null in intrinsic
>
> Testing:
> - [ ] hotspot_gc_shenandoah
> - [ ] tier1+Shenandoah
> - [ ] tier2+Shenandoah
Marked as reviewed by roland (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/1120
From rkennke at openjdk.java.net Mon Nov 9 11:29:10 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 9 Nov 2020 11:29:10 GMT
Subject: RFR: 8256046: Shenandoah: Mix-in NULL_PTR in non-strong
ShLRBNode's type [v2]
In-Reply-To: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
References: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
Message-ID: <tFosupMFk2JXNNrgO2H4o6v69Zj3O0kaF8r3ZfZQmpw=.61935429-0e59-4d9c-8279-ec8fe1bf4a4a@github.com>
> Testing found this problem (very rare/hard to reproduce):
>
> # Internal Error (d:/a/jdk/jdk/jdk/src/hotspot/cpu/x86/macroAssembler_x86.cpp:880), pid=6160, tid=5828
> # fatal error: DEBUG MESSAGE: unexpected null in intrinsic
>
> I believe this may be caused by non-strong LRB reporting a non-NULL type by passing-through its input's type, even though it may actually return NULL on non-NULL inputs. If this serves as input to an intrinsic, it may lead to elimination of surrounding null-check, and thus end up passing a NULL to the intrinsic even thought it should not.
>
> Testing:
> - [ ] hotspot_gc_shenandoah
> - [ ] tier1+Shenandoah
> - [ ] tier2+Shenandoah
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Omit useless cast to oopptr
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1120/files
- new: https://git.openjdk.java.net/jdk/pull/1120/files/6fbeabb3..d98ea79b
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1120&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1120&range=00-01
Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1120.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1120/head:pull/1120
PR: https://git.openjdk.java.net/jdk/pull/1120
From shade at openjdk.java.net Mon Nov 9 13:29:00 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 9 Nov 2020 13:29:00 GMT
Subject: RFR: 8256046: Shenandoah: Mix-in NULL_PTR in non-strong
ShLRBNode's type [v2]
In-Reply-To: <tFosupMFk2JXNNrgO2H4o6v69Zj3O0kaF8r3ZfZQmpw=.61935429-0e59-4d9c-8279-ec8fe1bf4a4a@github.com>
References: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
<tFosupMFk2JXNNrgO2H4o6v69Zj3O0kaF8r3ZfZQmpw=.61935429-0e59-4d9c-8279-ec8fe1bf4a4a@github.com>
Message-ID: <lYZNwsOF8yxRzjRnOfXlI1EGbU9Mr7lJrRJCjhnRRAY=.d470ae3a-1507-4196-bef1-d162c0dac4ea@github.com>
On Mon, 9 Nov 2020 11:29:10 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Testing found this problem (very rare/hard to reproduce):
>>
>> # Internal Error (d:/a/jdk/jdk/jdk/src/hotspot/cpu/x86/macroAssembler_x86.cpp:880), pid=6160, tid=5828
>> # fatal error: DEBUG MESSAGE: unexpected null in intrinsic
>>
>> I believe this may be caused by non-strong LRB reporting a non-NULL type by passing-through its input's type, even though it may actually return NULL on non-NULL inputs. If this serves as input to an intrinsic, it may lead to elimination of surrounding null-check, and thus end up passing a NULL to the intrinsic even thought it should not.
>>
>> Testing:
>> - [x] hotspot_gc_shenandoah
>> - [ ] tier1+Shenandoah
>> - [ ] tier2+Shenandoah
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Omit useless cast to oopptr
src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp line 2968:
> 2966: type = type->meet(TypePtr::NULL_PTR);
> 2967: }
> 2968: return type;
Can probably be written as:
if (kind() == ShenandoahBarrierSet::AccessKind::NORMAL) {
return t2;
}
return t2->meet(TypePtr::NULL_PTR);
...to follow the style. Your call.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1120
From zgu at openjdk.java.net Mon Nov 9 13:39:59 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 9 Nov 2020 13:39:59 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v2]
In-Reply-To: <rzvXaG3sHtgRJh0Pg5n_XH-QqPQD6ypsFulC5b_obE4=.23228ca7-824a-41d0-96b8-00c541ccd961@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<rzvXaG3sHtgRJh0Pg5n_XH-QqPQD6ypsFulC5b_obE4=.23228ca7-824a-41d0-96b8-00c541ccd961@github.com>
Message-ID: <brwoeSMVem0GU7WrSf1LnUW3ruM7VFbb-4ebahlmRjk=.ed525ec0-6912-4881-b47c-a9b29c0de970@github.com>
On Sun, 8 Nov 2020 10:03:07 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> Testing: hotspot_gc_shenandoah, tier1 & tier2 with -XX:+UseShenandoahGC
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Simplify condition, don't resurrect finalizably reachable objects even on phantom access
Looks good.
-------------
Marked as reviewed by zgu (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1109
From zgu at openjdk.java.net Mon Nov 9 17:04:02 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 9 Nov 2020 17:04:02 GMT
Subject: RFR: 8256039: Shenandoah:
runtime/stringtable/StringTableCleaningTest.java fails
Message-ID: <GNhlgVnHeSpxIAr5_aTRBT45rRD8b2U77Q3w209_3Fs=.0f9354e0-00c3-49e1-8369-41cf255f1d1d@github.com>
If parallel cleaning does not process concurrent weak roots, it should not notify GC on dead entries (which always 0).
-------------
Commit messages:
- Parallel cleaning should not report dead if it does not process concurrent weak roots
Changes: https://git.openjdk.java.net/jdk/pull/1133/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1133&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256039
Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1133.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1133/head:pull/1133
PR: https://git.openjdk.java.net/jdk/pull/1133
From shade at openjdk.java.net Mon Nov 9 17:20:56 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 9 Nov 2020 17:20:56 GMT
Subject: RFR: 8256039: Shenandoah:
runtime/stringtable/StringTableCleaningTest.java fails
In-Reply-To: <GNhlgVnHeSpxIAr5_aTRBT45rRD8b2U77Q3w209_3Fs=.0f9354e0-00c3-49e1-8369-41cf255f1d1d@github.com>
References: <GNhlgVnHeSpxIAr5_aTRBT45rRD8b2U77Q3w209_3Fs=.0f9354e0-00c3-49e1-8369-41cf255f1d1d@github.com>
Message-ID: <GYaNB4dwMEHoT84C0l49lZmziiCiQd0kGfCIeN2VGNo=.014e321f-dd70-4649-88b4-86b5cb9a2640@github.com>
On Mon, 9 Nov 2020 16:58:58 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> If parallel cleaning does not process concurrent weak roots, it should not notify GC on dead entries (which always 0).
Looks fine. This fixes the test for me.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1133
From rkennke at openjdk.java.net Mon Nov 9 17:34:58 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 9 Nov 2020 17:34:58 GMT
Subject: RFR: 8256039: Shenandoah:
runtime/stringtable/StringTableCleaningTest.java fails
In-Reply-To: <GNhlgVnHeSpxIAr5_aTRBT45rRD8b2U77Q3w209_3Fs=.0f9354e0-00c3-49e1-8369-41cf255f1d1d@github.com>
References: <GNhlgVnHeSpxIAr5_aTRBT45rRD8b2U77Q3w209_3Fs=.0f9354e0-00c3-49e1-8369-41cf255f1d1d@github.com>
Message-ID: <Xir8nCXgHlXyQeatRoHEjTWiS794q8_D0AuaxCSMuNs=.2ecbb3e2-99da-4ce4-a170-02cd8a5aff3c@github.com>
On Mon, 9 Nov 2020 16:58:58 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> If parallel cleaning does not process concurrent weak roots, it should not notify GC on dead entries (which always 0).
Looks good to me.
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1133
From coleenp at openjdk.java.net Mon Nov 9 20:43:06 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 9 Nov 2020 20:43:06 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6]
In-Reply-To: <q0OJ0RDIEfVUvYfaNAHV0oKWVO4RGch_9y1Qd6vEErI=.d54b2cee-4333-47e5-9469-7539b8bc1e43@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<rrqQkn7YRMjol0rpPIDSBsbXZXqFULDSRxMlMPJ5cjs=.f10aea2a-10db-4555-ba82-8664a472e393@github.com>
<TgRmyCL8pDACuMxxHfdsqN9mOVxyZaKNRpfXbssEl94=.69114d11-53a5-4220-b5ba-e9ecdfe96f26@github.com>
<q0OJ0RDIEfVUvYfaNAHV0oKWVO4RGch_9y1Qd6vEErI=.d54b2cee-4333-47e5-9469-7539b8bc1e43@github.com>
Message-ID: <Q2SPsAnxbJIW1H8zUkVtpvTpI2WrnbDN04LHXQqfSFU=.1d917f30-af8b-4323-9f52-d9786052912e@github.com>
On Thu, 5 Nov 2020 12:42:40 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> Thank you for the update, Coleen!
>> I leave it for you to decide to refactor the gc_notification or not.
>> Thanks,
>> Serguei
>
> Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change.
This change also passes tier 7,8 testing.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From zgu at openjdk.java.net Tue Nov 10 00:08:56 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 10 Nov 2020 00:08:56 GMT
Subject: Integrated: 8256039: Shenandoah:
runtime/stringtable/StringTableCleaningTest.java fails
In-Reply-To: <GNhlgVnHeSpxIAr5_aTRBT45rRD8b2U77Q3w209_3Fs=.0f9354e0-00c3-49e1-8369-41cf255f1d1d@github.com>
References: <GNhlgVnHeSpxIAr5_aTRBT45rRD8b2U77Q3w209_3Fs=.0f9354e0-00c3-49e1-8369-41cf255f1d1d@github.com>
Message-ID: <aWAXU8e4BzNzaqA8V4qzJoA3gLcAb11NJXRHnTKODzY=.6e7b136a-a6ce-46ad-9ccc-053eb18f97ce@github.com>
On Mon, 9 Nov 2020 16:58:58 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> If parallel cleaning does not process concurrent weak roots, it should not notify GC on dead entries (which always 0).
This pull request has now been integrated.
Changeset: 1332ba3c
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/1332ba3c
Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod
8256039: Shenandoah: runtime/stringtable/StringTableCleaningTest.java fails
Reviewed-by: shade, rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1133
From rkennke at openjdk.java.net Tue Nov 10 06:59:56 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 06:59:56 GMT
Subject: Integrated: 8256040: Shenandoah: Allow NULL referent in
ShenandoahReferenceProcessor::should_discover()
In-Reply-To: <mU0wTUVDfkNDSA9dZ_9zj0BB53Ecph7fBhPdEndLlpM=.acf2050f-7b0d-477d-83f0-f8aeedb7ab3d@github.com>
References: <mU0wTUVDfkNDSA9dZ_9zj0BB53Ecph7fBhPdEndLlpM=.acf2050f-7b0d-477d-83f0-f8aeedb7ab3d@github.com>
Message-ID: <r3_Die9NbwyLdcb2isbQ43_hG6nble2l8hJlbU9laPo=.9698502e-5b25-499a-9a03-75879189b5d2@github.com>
On Mon, 9 Nov 2020 09:25:41 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Testing showed the following assert:
>
> # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/oops/compressedOops.inline.hpp:54), pid=711923, tid=712264
> # assert(!is_null(v)) failed: narrow oop value can never be zero
>
> V [libjvm.so+0x1629c84] bool ShenandoahReferenceProcessor::should_discover<narrowOop>(oop, ReferenceType) const+0x404
> V [libjvm.so+0x162a01b] bool ShenandoahReferenceProcessor::discover<narrowOop>(oop, ReferenceType, unsigned int)+0x4b
> V [libjvm.so+0x16239e8] ShenandoahReferenceProcessor::discover_reference(oop, ReferenceType)+0x138
> V [libjvm.so+0x15b7a26] bool InstanceRefKlass::try_discover<narrowOop, ShenandoahMarkRefsMetadataClosure>(oop, ReferenceType, ShenandoahMarkRefsMetadataClosure*)+0x96
> V [libjvm.so+0x15b7b3d] void InstanceRefKlass::oop_oop_iterate_discovery<narrowOop, ShenandoahMarkRefsMetadataClosure, AlwaysContains>(oop, ReferenceType, ShenandoahMarkRefsMetadataClosure*, AlwaysContains&) [clone .constprop.696]+0x3d
> V [libjvm.so+0x15b7ce1] void InstanceRefKlass::oop_oop_iterate_ref_processing<narrowOop, ShenandoahMarkRefsMetadataClosure, AlwaysContains>(oop, ShenandoahMarkRefsMetadataClosure*, AlwaysContains&)+0x81
> V [libjvm.so+0x15b835e] void OopOopIterateDispatch<ShenandoahMarkRefsMetadataClosure>::Table::oop_oop_iterate<InstanceRefKlass, narrowOop>(ShenandoahMarkRefsMetadataClosure*, oop, Klass*)+0x1ee
> V [libjvm.so+0x15a905e] void ShenandoahConcurrentMark::do_task<ShenandoahMarkRefsMetadataClosure>(Padded<BufferedOverflowTaskQueue<ShenandoahMarkTask, (MEMFLAGS)5, 131072u>, 128ul>*, ShenandoahMarkRefsMetadataClosure*, unsigned short*, ShenandoahMarkTask*)+0x7be
> V [libjvm.so+0x15ab81e] void ShenandoahConcurrentMark::mark_loop_work<ShenandoahMarkRefsMetadataClosure, true>(ShenandoahMarkRefsMetadataClosure*, unsigned short*, unsigned int, TaskTerminator*)+0x3ce
>
> We have this code in ShRefProc::should_discover():
> oop referent = CompressedOops::decode_not_null(heap_oop);
>
> However, the referent can legally be NULL. We even treat NULL specially in is_inactive().
>
> Testing:
> - [x] hotspot_gc_shenandoah
> - [x] tier1+UseShenandoahGC+ShenandoahVerify (which originally showed the problem)
This pull request has now been integrated.
Changeset: a38dd534
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/a38dd534
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
8256040: Shenandoah: Allow NULL referent in ShenandoahReferenceProcessor::should_discover()
Reviewed-by: shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1117
From rkennke at openjdk.java.net Tue Nov 10 07:46:57 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 07:46:57 GMT
Subject: Withdrawn: 8255999: Shenandoah: Invoke native-LRB only on non-strong
refs, again
In-Reply-To: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
References: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
Message-ID: <MbP34Oyd30JnPMV1V3W-fzbmBIvZFlA16QaMJdu5nSg=.95e2de21-b73c-4e52-93f2-74e9d75f3fcf@github.com>
On Fri, 6 Nov 2020 21:11:08 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> In JDK-8255691, I changed the condition for invoking native-LRB only on non-strong-refs. Somehow, when merging JDK-8254315, I got the condition wrong again.
>
> Testing: hotspot_gc_shenandoah
This pull request has been closed without being integrated.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1101
From rkennke at openjdk.java.net Tue Nov 10 07:46:56 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 07:46:56 GMT
Subject: RFR: 8255999: Shenandoah: Invoke native-LRB only on non-strong
refs, again [v2]
In-Reply-To: <AfncQFKDI86ajnBnGogqvMXnyHnqJqLs7jAI5zVPX98=.945473e1-b5fe-440a-80e8-7eac639024f6@github.com>
References: <beUB0WYWGnGF6SpBX_d36fj8uaHZ2jIuz7oRJljMgD8=.674e206b-ae97-4c7e-99a8-b64998ec21e7@github.com>
<AfncQFKDI86ajnBnGogqvMXnyHnqJqLs7jAI5zVPX98=.945473e1-b5fe-440a-80e8-7eac639024f6@github.com>
Message-ID: <53UNhLAsUH6lUbDw2yR3b0O58bfjh-H7ZKmkRksmHqY=.1e252093-0e09-480f-a371-3bc273cd26ed@github.com>
On Sun, 8 Nov 2020 10:33:07 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
>>
>> - Merge branch 'master' into JDK-8255999
>> - 8255999: Shenandoah: Invoke native-LRB only on non-strong refs, again
>
> This makes sense. Looks good.
I'm closing this PR. It will be subsumed by #1109.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1101
From rkennke at openjdk.java.net Tue Nov 10 09:32:09 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 09:32:09 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v3]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <gn_1J7PW_yxSqpHStv42SwLdG6zIGUadsRm6bcwMGT4=.b0552770-9856-47ba-a388-0e8ac14f354c@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> Testing: hotspot_gc_shenandoah, tier1 & tier2 with -XX:+UseShenandoahGC
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
- Ensure correct strength and width of LRB runtime calls
- Merge branch 'master' into JDK-8256011
- Merge branch 'master' into JDK-8256011
- Simplify condition, don't resurrect finalizably reachable objects even on phantom access
- 8256011: Shenandoah: Don't resurrect finalizably reachable objects
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/189be782..8d979eea
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=01-02
Stats: 47771 lines in 313 files changed: 26190 ins; 14261 del; 7320 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From shade at openjdk.java.net Tue Nov 10 09:53:05 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 10 Nov 2020 09:53:05 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v3]
In-Reply-To: <gn_1J7PW_yxSqpHStv42SwLdG6zIGUadsRm6bcwMGT4=.b0552770-9856-47ba-a388-0e8ac14f354c@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<gn_1J7PW_yxSqpHStv42SwLdG6zIGUadsRm6bcwMGT4=.b0552770-9856-47ba-a388-0e8ac14f354c@github.com>
Message-ID: <VM2g-ehND8tZbAmv0rwsUO6--VzE6cGOpXx-Dj-Lrqw=.5eb44d4a-1e60-480f-ac3f-dff410f1955e@github.com>
On Tue, 10 Nov 2020 09:32:09 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> Testing: hotspot_gc_shenandoah, tier1 & tier2 with -XX:+UseShenandoahGC
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>
> - Ensure correct strength and width of LRB runtime calls
> - Merge branch 'master' into JDK-8256011
> - Merge branch 'master' into JDK-8256011
> - Simplify condition, don't resurrect finalizably reachable objects even on phantom access
> - 8256011: Shenandoah: Don't resurrect finalizably reachable objects
I have many questions :)
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 863:
> 861: __ call(RuntimeAddress(bs->load_reference_barrier_weak_rt_code_blob()->code_begin()));
> 862: } else {
> 863: assert(is_phantom, "only remaining strenght");
Typo: "strenght"
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 944:
> 942: if (UseCompressedOops) {
> 943: __ call_VM_leaf(CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_strong_narrow), c_rarg0,
> 944: c_rarg1);
Stick with newline style? Is `c_rarg1` on new line or not in this method?
src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 1063:
> 1061: Node* in2 = n->in(2);
> 1062:
> 1063: // If one input is NULL, then step over the barriers normal LRB barriers on the other input
"strong LRBs barriers" now... :)
src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp line 984:
> 982: calladdr = CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_strong);
> 983: }
> 984: name = "load_reference_barrier_strong";
Why not separate "load_reference_barrier_strong" and "load_reference_barrier_strong_narrow"?
src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp line 2910:
> 2908:
> 2909: ShenandoahLoadReferenceBarrierNode::ShenandoahLoadReferenceBarrierNode(Node* ctrl, Node* obj, DecoratorSet decorators)
> 2910: : Node(ctrl, obj), _decorators(decorators & (ON_STRONG_OOP_REF | ON_WEAK_OOP_REF | ON_PHANTOM_OOP_REF | ON_UNKNOWN_OOP_REF | IN_NATIVE)) {
Do we really want to strip bits from decorator here? Aren't we testing specific bits anyway downstream? I can imagine some downstream code in the future would be surprised not to see some bits that are actually set, but stripped here?
Is this for `LRBNode::hash/cmp` stability? It should be stripped in `hash/cmp` then.
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.hpp line 58:
> 56:
> 57: static bool is_strong_access(DecoratorSet decorators) {
> 58: return (decorators & (ON_WEAK_OOP_REF | ON_PHANTOM_OOP_REF | ON_UNKNOWN_OOP_REF)) == 0;
Is this the same as `decorators & ON_STRONG_OOP_REF`?
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 105:
> 103: inline oop ShenandoahBarrierSet::load_reference_barrier(oop obj, T* load_addr) {
> 104:
> 105: // Prevent resurrection of unreachable non-strorg references.
A chance to fix typo "non-strorg" here.
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.hpp line 62:
> 60:
> 61: static bool is_weak_access(DecoratorSet decorators) {
> 62: return (decorators & (ON_WEAK_OOP_REF | ON_UNKNOWN_OOP_REF)) != 0;
Um. Shouldn't `ON_UNKNOWN_OOP_REF` default to strong? I.e. the access via Unsafe should probably resurrect the object, otherwise it corrupts the heap?
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 108:
> 106: if (!HasDecorator<decorators, ON_STRONG_OOP_REF>::value && obj != NULL &&
> 107: _heap->is_concurrent_weak_root_in_progress() &&
> 108: !(HasDecorator<decorators, ON_PHANTOM_OOP_REF>::value ? _heap->marking_context()->is_marked(obj)
Maybe split out boolean locals for decorator tests? A matter of style, your call.
src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 1066:
> 1064: if (in1->bottom_type() == TypePtr::NULL_PTR &&
> 1065: !((in2->Opcode() == Op_ShenandoahLoadReferenceBarrier) &&
> 1066: (((ShenandoahLoadReferenceBarrierNode*)in2)->decorators() & ON_STRONG_OOP_REF) == 0)) {
Is this `ShenandoahBarrierSet::is_strong_access`?
-------------
Changes requested by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 10:23:03 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 10:23:03 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v3]
In-Reply-To: <VM2g-ehND8tZbAmv0rwsUO6--VzE6cGOpXx-Dj-Lrqw=.5eb44d4a-1e60-480f-ac3f-dff410f1955e@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<gn_1J7PW_yxSqpHStv42SwLdG6zIGUadsRm6bcwMGT4=.b0552770-9856-47ba-a388-0e8ac14f354c@github.com>
<VM2g-ehND8tZbAmv0rwsUO6--VzE6cGOpXx-Dj-Lrqw=.5eb44d4a-1e60-480f-ac3f-dff410f1955e@github.com>
Message-ID: <1MKAddYmmZwDvkKF8UxfITgxVfm0o2Tcbg4ZotOHXZk=.fc4a354d-668f-4d8d-8da3-1277a08bca4e@github.com>
On Tue, 10 Nov 2020 09:41:52 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>>
>> - Ensure correct strength and width of LRB runtime calls
>> - Merge branch 'master' into JDK-8256011
>> - Merge branch 'master' into JDK-8256011
>> - Simplify condition, don't resurrect finalizably reachable objects even on phantom access
>> - 8256011: Shenandoah: Don't resurrect finalizably reachable objects
>
> src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp line 984:
>
>> 982: calladdr = CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_strong);
>> 983: }
>> 984: name = "load_reference_barrier_strong";
>
> Why not separate "load_reference_barrier_strong" and "load_reference_barrier_strong_narrow"?
Dunno. Could do that. Within one instance of JVM, we would never see both narrow/non-narrow versions, though.
> src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp line 2910:
>
>> 2908:
>> 2909: ShenandoahLoadReferenceBarrierNode::ShenandoahLoadReferenceBarrierNode(Node* ctrl, Node* obj, DecoratorSet decorators)
>> 2910: : Node(ctrl, obj), _decorators(decorators & (ON_STRONG_OOP_REF | ON_WEAK_OOP_REF | ON_PHANTOM_OOP_REF | ON_UNKNOWN_OOP_REF | IN_NATIVE)) {
>
> Do we really want to strip bits from decorator here? Aren't we testing specific bits anyway downstream? I can imagine some downstream code in the future would be surprised not to see some bits that are actually set, but stripped here?
>
> Is this for `LRBNode::hash/cmp` stability? It should be stripped in `hash/cmp` then.
Yes, this is for hash/cmp. In particular, I suspect we might miss optimizations that don't find two LRBs equal only because some non-relevant access-bit is set. Will move it to hash/cmp then.
> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.hpp line 58:
>
>> 56:
>> 57: static bool is_strong_access(DecoratorSet decorators) {
>> 58: return (decorators & (ON_WEAK_OOP_REF | ON_PHANTOM_OOP_REF | ON_UNKNOWN_OOP_REF)) == 0;
>
> Is this the same as `decorators & ON_STRONG_OOP_REF`?
It is the same, but in my testing I found that ON_STRONG_OOP_REF is not always set (it is the default when no other ON_XYZ_OOP_REF is specified).
> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.hpp line 62:
>
>> 60:
>> 61: static bool is_weak_access(DecoratorSet decorators) {
>> 62: return (decorators & (ON_WEAK_OOP_REF | ON_UNKNOWN_OOP_REF)) != 0;
>
> Um. Shouldn't `ON_UNKNOWN_OOP_REF` default to strong? I.e. the access via Unsafe should probably resurrect the object, otherwise it corrupts the heap?
No. ON_UNKNOWN_OOP_REF *might* access the referent field by reflection. We must do the correct thing in this situation. Normally, ON_UNKNOWN_OOP_REF is indeed called on normal/strong references, though, however we do the correct thing in this case too, because the referent would be marked and thus go through regular LRB. It doesn't hurt to call into weak for unknown oop refs.
> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 108:
>
>> 106: if (!HasDecorator<decorators, ON_STRONG_OOP_REF>::value && obj != NULL &&
>> 107: _heap->is_concurrent_weak_root_in_progress() &&
>> 108: !(HasDecorator<decorators, ON_PHANTOM_OOP_REF>::value ? _heap->marking_context()->is_marked(obj)
>
> Maybe split out boolean locals for decorator tests? A matter of style, your call.
Hmm yeah. Actually I am thinking how to do the HasDecorator tests as outermost tests, because those are the compile-time-tests. Would need to duplicate the whole test once for WEAK/UNKNOWN and once for PHANTOM I guess.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From shade at openjdk.java.net Tue Nov 10 11:06:02 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 10 Nov 2020 11:06:02 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v3]
In-Reply-To: <1MKAddYmmZwDvkKF8UxfITgxVfm0o2Tcbg4ZotOHXZk=.fc4a354d-668f-4d8d-8da3-1277a08bca4e@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<gn_1J7PW_yxSqpHStv42SwLdG6zIGUadsRm6bcwMGT4=.b0552770-9856-47ba-a388-0e8ac14f354c@github.com>
<VM2g-ehND8tZbAmv0rwsUO6--VzE6cGOpXx-Dj-Lrqw=.5eb44d4a-1e60-480f-ac3f-dff410f1955e@github.com>
<1MKAddYmmZwDvkKF8UxfITgxVfm0o2Tcbg4ZotOHXZk=.fc4a354d-668f-4d8d-8da3-1277a08bca4e@github.com>
Message-ID: <5MmpX2SvrnBppVn3qGhkVt0v3Hrlqv57Y_3X3QTlvso=.471dec62-f0ac-4743-a86a-d911ea448e31@github.com>
On Tue, 10 Nov 2020 10:15:22 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp line 984:
>>
>>> 982: calladdr = CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_strong);
>>> 983: }
>>> 984: name = "load_reference_barrier_strong";
>>
>> Why not separate "load_reference_barrier_strong" and "load_reference_barrier_strong_narrow"?
>
> Dunno. Could do that. Within one instance of JVM, we would never see both narrow/non-narrow versions, though.
I would be more straight-forward looking, AFAICS. Is there any code that actually matches those strings?
>> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.hpp line 58:
>>
>>> 56:
>>> 57: static bool is_strong_access(DecoratorSet decorators) {
>>> 58: return (decorators & (ON_WEAK_OOP_REF | ON_PHANTOM_OOP_REF | ON_UNKNOWN_OOP_REF)) == 0;
>>
>> Is this the same as `decorators & ON_STRONG_OOP_REF`?
>
> It is the same, but in my testing I found that ON_STRONG_OOP_REF is not always set (it is the default when no other ON_XYZ_OOP_REF is specified).
Ouch. That means we should never trust `ON_STRONG_OOP_REF`, and instead go for `SBS::is_strong_access`?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From shade at openjdk.java.net Tue Nov 10 11:11:59 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 10 Nov 2020 11:11:59 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v3]
In-Reply-To: <1MKAddYmmZwDvkKF8UxfITgxVfm0o2Tcbg4ZotOHXZk=.fc4a354d-668f-4d8d-8da3-1277a08bca4e@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<gn_1J7PW_yxSqpHStv42SwLdG6zIGUadsRm6bcwMGT4=.b0552770-9856-47ba-a388-0e8ac14f354c@github.com>
<VM2g-ehND8tZbAmv0rwsUO6--VzE6cGOpXx-Dj-Lrqw=.5eb44d4a-1e60-480f-ac3f-dff410f1955e@github.com>
<1MKAddYmmZwDvkKF8UxfITgxVfm0o2Tcbg4ZotOHXZk=.fc4a354d-668f-4d8d-8da3-1277a08bca4e@github.com>
Message-ID: <I3lQeM7qmNkAZGCK4EaVNcF-GCidrKGdTFxJS7_UId8=.0067bf9c-fc09-4371-8ae0-4ed1f67d4dff@github.com>
On Tue, 10 Nov 2020 10:18:57 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.hpp line 62:
>>
>>> 60:
>>> 61: static bool is_weak_access(DecoratorSet decorators) {
>>> 62: return (decorators & (ON_WEAK_OOP_REF | ON_UNKNOWN_OOP_REF)) != 0;
>>
>> Um. Shouldn't `ON_UNKNOWN_OOP_REF` default to strong? I.e. the access via Unsafe should probably resurrect the object, otherwise it corrupts the heap?
>
> No. ON_UNKNOWN_OOP_REF *might* access the referent field by reflection. We must do the correct thing in this situation. Normally, ON_UNKNOWN_OOP_REF is indeed called on normal/strong references, though, however we do the correct thing in this case too, because the referent would be marked and thus go through regular LRB. It doesn't hurt to call into weak for unknown oop refs.
> No. ON_UNKNOWN_OOP_REF _might_ access the referent field by reflection.
Ew. This just goes against my intuition about the default strongness. For example, `Unsafe_GetReference` calls with `ON_UNKNOWN_OOP_REF` -- does that mean we would treat that access as weak? I don't think we should.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 11:19:55 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 11:19:55 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v3]
In-Reply-To: <5MmpX2SvrnBppVn3qGhkVt0v3Hrlqv57Y_3X3QTlvso=.471dec62-f0ac-4743-a86a-d911ea448e31@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<gn_1J7PW_yxSqpHStv42SwLdG6zIGUadsRm6bcwMGT4=.b0552770-9856-47ba-a388-0e8ac14f354c@github.com>
<VM2g-ehND8tZbAmv0rwsUO6--VzE6cGOpXx-Dj-Lrqw=.5eb44d4a-1e60-480f-ac3f-dff410f1955e@github.com>
<1MKAddYmmZwDvkKF8UxfITgxVfm0o2Tcbg4ZotOHXZk=.fc4a354d-668f-4d8d-8da3-1277a08bca4e@github.com>
<5MmpX2SvrnBppVn3qGhkVt0v3Hrlqv57Y_3X3QTlvso=.471dec62-f0ac-4743-a86a-d911ea448e31@github.com>
Message-ID: <Tv32cQln38p4BEPTEJThV_CB8gMYPyxgbmzAc5D5Cok=.a157b941-c1d6-4904-aac6-bc9bb072cc55@github.com>
On Tue, 10 Nov 2020 11:02:11 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Dunno. Could do that. Within one instance of JVM, we would never see both narrow/non-narrow versions, though.
>
> I would be more straight-forward looking, AFAICS. Is there any code that actually matches those strings?
I don't think any code matches those strings. I will change them to be more specific.
>> It is the same, but in my testing I found that ON_STRONG_OOP_REF is not always set (it is the default when no other ON_XYZ_OOP_REF is specified).
>
> Ouch. That means we should never trust `ON_STRONG_OOP_REF`, and instead go for `SBS::is_strong_access`?
Presumably, the Access machinery is supposed to fill it in when none of the others is set. In practice, it did not work for me. We could fix the access machinery too, but life's too short?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 11:25:57 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 11:25:57 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v3]
In-Reply-To: <I3lQeM7qmNkAZGCK4EaVNcF-GCidrKGdTFxJS7_UId8=.0067bf9c-fc09-4371-8ae0-4ed1f67d4dff@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<gn_1J7PW_yxSqpHStv42SwLdG6zIGUadsRm6bcwMGT4=.b0552770-9856-47ba-a388-0e8ac14f354c@github.com>
<VM2g-ehND8tZbAmv0rwsUO6--VzE6cGOpXx-Dj-Lrqw=.5eb44d4a-1e60-480f-ac3f-dff410f1955e@github.com>
<1MKAddYmmZwDvkKF8UxfITgxVfm0o2Tcbg4ZotOHXZk=.fc4a354d-668f-4d8d-8da3-1277a08bca4e@github.com>
<I3lQeM7qmNkAZGCK4EaVNcF-GCidrKGdTFxJS7_UId8=.0067bf9c-fc09-4371-8ae0-4ed1f67d4dff@github.com>
Message-ID: <9gTaeVuNDSAXmEIuSFNkoybzNLospwZlGqNIRhHe1uA=.6e6d4011-e375-4415-bd0f-5d416543573f@github.com>
On Tue, 10 Nov 2020 11:08:52 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> No. ON_UNKNOWN_OOP_REF *might* access the referent field by reflection. We must do the correct thing in this situation. Normally, ON_UNKNOWN_OOP_REF is indeed called on normal/strong references, though, however we do the correct thing in this case too, because the referent would be marked and thus go through regular LRB. It doesn't hurt to call into weak for unknown oop refs.
>
>> No. ON_UNKNOWN_OOP_REF _might_ access the referent field by reflection.
>
> Ew. This just goes against my intuition about the default strongness. For example, `Unsafe_GetReference` calls with `ON_UNKNOWN_OOP_REF` -- does that mean we would treat that access as weak? I don't think we should.
Yes that is the idea. Note, as explained above, it does not hurt. The alternative would be to generate code that figures out the strength *at runtime* and calls into the correct LRB. This would be complex, though. We already do it for SATB-barriers, but there it is necessary, while here it is harmless to call the weak-LRB for non-weak references.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 12:20:14 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 12:20:14 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v4]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <wXOB_KvM_qQPqGE74fklhE2uZ3pT7yOqWgp3Ls-2bSc=.1b15e918-d222-43bf-9e92-f60d3a3ba7aa@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [ ] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with five additional commits since the last revision:
- Whitespace changes
- Mask decorators in hash/cmp, not in ctor
- Use ShBarrierSet::is_strong_access() in CmpP optimization
- separate code-paths for phantom- and weak-access in runtime-LRB
- Give different name to different LRB runtime calls
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/8d979eea..ac2674f5
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=02-03
Stats: 33 lines in 4 files changed: 16 ins; 5 del; 12 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From shade at openjdk.java.net Tue Nov 10 12:33:56 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 10 Nov 2020 12:33:56 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v4]
In-Reply-To: <wXOB_KvM_qQPqGE74fklhE2uZ3pT7yOqWgp3Ls-2bSc=.1b15e918-d222-43bf-9e92-f60d3a3ba7aa@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<wXOB_KvM_qQPqGE74fklhE2uZ3pT7yOqWgp3Ls-2bSc=.1b15e918-d222-43bf-9e92-f60d3a3ba7aa@github.com>
Message-ID: <dGhtaCC1KV8WCA6zCFr1QlbFqXZijdXp-gwZ71Mtyzg=.df0f5070-1154-499f-b1a4-e3a093228a0d@github.com>
On Tue, 10 Nov 2020 12:20:14 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
>> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
>> - We can have strong and phantom native referents, and strong and weak in-heap referents
>> - Native referents are never compressed
>>
>> Note that this depends on PR#1140.
>>
>> Testing:
>> - [x] hotspot_gc_shenandoah
>> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
>> - [ ] tier2 +UseShenandoahGC +ShenandoahVerify
>
> Roman Kennke has updated the pull request incrementally with five additional commits since the last revision:
>
> - Whitespace changes
> - Mask decorators in hash/cmp, not in ctor
> - Use ShBarrierSet::is_strong_access() in CmpP optimization
> - separate code-paths for phantom- and weak-access in runtime-LRB
> - Give different name to different LRB runtime calls
This looks fine to me. Let Zhengyu look through this thoroughly.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 13:18:16 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 13:18:16 GMT
Subject: RFR: 8256046: Shenandoah: Mix-in NULL_PTR in non-strong
ShLRBNode's type [v3]
In-Reply-To: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
References: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
Message-ID: <tTKjp-NM_AX5kEfzCAc1B2VypkU6AEsMcUuHoM3NHls=.cb736285-a528-4951-aa7f-61a25ef3e9f2@github.com>
> Testing found this problem (very rare/hard to reproduce):
>
> # Internal Error (d:/a/jdk/jdk/jdk/src/hotspot/cpu/x86/macroAssembler_x86.cpp:880), pid=6160, tid=5828
> # fatal error: DEBUG MESSAGE: unexpected null in intrinsic
>
> I believe this may be caused by non-strong LRB reporting a non-NULL type by passing-through its input's type, even though it may actually return NULL on non-NULL inputs. If this serves as input to an intrinsic, it may lead to elimination of surrounding null-check, and thus end up passing a NULL to the intrinsic even thought it should not.
>
> Testing:
> - [x] hotspot_gc_shenandoah
> - [ ] tier1+Shenandoah
> - [ ] tier2+Shenandoah
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Shuffle new code-paths to match rest of methods
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1120/files
- new: https://git.openjdk.java.net/jdk/pull/1120/files/d98ea79b..8350d33e
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1120&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1120&range=01-02
Stats: 10 lines in 1 file changed: 2 ins; 1 del; 7 mod
Patch: https://git.openjdk.java.net/jdk/pull/1120.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1120/head:pull/1120
PR: https://git.openjdk.java.net/jdk/pull/1120
From shade at openjdk.java.net Tue Nov 10 13:18:16 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 10 Nov 2020 13:18:16 GMT
Subject: RFR: 8256046: Shenandoah: Mix-in NULL_PTR in non-strong
ShLRBNode's type [v3]
In-Reply-To: <tTKjp-NM_AX5kEfzCAc1B2VypkU6AEsMcUuHoM3NHls=.cb736285-a528-4951-aa7f-61a25ef3e9f2@github.com>
References: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
<tTKjp-NM_AX5kEfzCAc1B2VypkU6AEsMcUuHoM3NHls=.cb736285-a528-4951-aa7f-61a25ef3e9f2@github.com>
Message-ID: <aG6M9pqgOwihgbPsb9AJ5R22O6qmHqSHPOmvi_nJHgg=.d9ea7d88-072d-4020-8b54-9f0ca6f9596c@github.com>
On Tue, 10 Nov 2020 13:15:17 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Testing found this problem (very rare/hard to reproduce):
>>
>> # Internal Error (d:/a/jdk/jdk/jdk/src/hotspot/cpu/x86/macroAssembler_x86.cpp:880), pid=6160, tid=5828
>> # fatal error: DEBUG MESSAGE: unexpected null in intrinsic
>>
>> I believe this may be caused by non-strong LRB reporting a non-NULL type by passing-through its input's type, even though it may actually return NULL on non-NULL inputs. If this serves as input to an intrinsic, it may lead to elimination of surrounding null-check, and thus end up passing a NULL to the intrinsic even thought it should not.
>>
>> Testing:
>> - [x] hotspot_gc_shenandoah
>> - [ ] tier1+Shenandoah
>> - [ ] tier2+Shenandoah
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Shuffle new code-paths to match rest of methods
Thanks. Looks fine to me.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1120
From rkennke at openjdk.java.net Tue Nov 10 13:28:13 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 13:28:13 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v5]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <zo_ZoW442lhrRP0_vFShaeMI4DYHwSx2Jt0pACXm9AE=.912836ea-63da-4248-948c-8b92e0529484@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [ ] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Aarch64 parts
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/ac2674f5..d361e60b
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=04
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=03-04
Stats: 67 lines in 3 files changed: 9 ins; 11 del; 47 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From zgu at openjdk.java.net Tue Nov 10 14:17:01 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 10 Nov 2020 14:17:01 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v5]
In-Reply-To: <zo_ZoW442lhrRP0_vFShaeMI4DYHwSx2Jt0pACXm9AE=.912836ea-63da-4248-948c-8b92e0529484@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<zo_ZoW442lhrRP0_vFShaeMI4DYHwSx2Jt0pACXm9AE=.912836ea-63da-4248-948c-8b92e0529484@github.com>
Message-ID: <1EJmFGKXgKlDUsfnNa9F3sVc_CIt1byutdNXLeZWCJM=.f4357e91-301c-47c4-9777-77f6f8e62948@github.com>
On Tue, 10 Nov 2020 13:28:13 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
>> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
>> - We can have strong and phantom native referents, and strong and weak in-heap referents
>> - Native referents are never compressed
>>
>> Note that this depends on PR#1140.
>>
>> Testing:
>> - [x] hotspot_gc_shenandoah
>> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
>> - [ ] tier2 +UseShenandoahGC +ShenandoahVerify
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Aarch64 parts
Changes requested by zgu (Reviewer).
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 958:
> 956: }
> 957: #else
> 958: __ load_parameter(0, rax);
Dose not seem that x86_32 made adjustment as x86_64. Dose it even build on x86_32?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From zgu at openjdk.java.net Tue Nov 10 14:37:55 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 10 Nov 2020 14:37:55 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v5]
In-Reply-To: <zo_ZoW442lhrRP0_vFShaeMI4DYHwSx2Jt0pACXm9AE=.912836ea-63da-4248-948c-8b92e0529484@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<zo_ZoW442lhrRP0_vFShaeMI4DYHwSx2Jt0pACXm9AE=.912836ea-63da-4248-948c-8b92e0529484@github.com>
Message-ID: <A5ELqTYStVI7TFnhs7mCWvP4uUVd9vPBLpzL6JKcml8=.c6f6a9da-dc3e-4e01-b046-0696d0bc9ca5@github.com>
On Tue, 10 Nov 2020 13:28:13 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
>> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
>> - We can have strong and phantom native referents, and strong and weak in-heap referents
>> - Native referents are never compressed
>>
>> Note that this depends on PR#1140.
>>
>> Testing:
>> - [x] hotspot_gc_shenandoah
>> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
>> - [ ] tier2 +UseShenandoahGC +ShenandoahVerify
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Aarch64 parts
Changes requested by zgu (Reviewer).
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 278:
> 276: bool is_phantom = ShenandoahBarrierSet::is_phantom_access(decorators);
> 277: bool is_native = ShenandoahBarrierSet::is_native_access(decorators);
> 278: bool is_narrow = LP64_ONLY(UseCompressedOops &&) !is_native;
This seems wrong for 32-bits. should be:
is_narrow = LP64_ONLY(UseCompressedOops &&) NOT_LP64(false &&) !is_native;
src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 237:
> 235: bool is_phantom = ShenandoahBarrierSet::is_phantom_access(decorators);
> 236: bool is_native = ShenandoahBarrierSet::is_native_access(decorators);
> 237: bool is_narrow = LP64_ONLY(UseCompressedOops &&) !is_native;
Don't need LP64_ONLY, aarch64 always 64-bits.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 15:26:14 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 15:26:14 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v6]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <-X5aHA0aJFAUOjdrVoKttDntOBth2QQv7vD2cBVcAZo=.70f3d572-66dc-4ca6-b512-0e9ce0577131@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [ ] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
- Remove superfluous LP64 in aarch64 part
- Fixes/missing parts for x86_64
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/d361e60b..37c97786
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=05
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=04-05
Stats: 20 lines in 3 files changed: 4 ins; 6 del; 10 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 15:28:58 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 15:28:58 GMT
Subject: Integrated: 8256046: Shenandoah: Mix-in NULL_PTR in non-strong
ShLRBNode's type
In-Reply-To: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
References: <gFX4qffeagLCCwS6tfPrYygX_-GVfUcqYB5UdAlABBI=.eea784b1-a944-4478-8e01-4b4fcb614384@github.com>
Message-ID: <bkutmPb8w1Zg7eHpZ1krpP9bHrinfn24tkWjrODIGNQ=.4d4d149d-6a55-4f5a-a939-54ecb1d37e9d@github.com>
On Mon, 9 Nov 2020 11:10:27 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Testing found this problem (very rare/hard to reproduce):
>
> # Internal Error (d:/a/jdk/jdk/jdk/src/hotspot/cpu/x86/macroAssembler_x86.cpp:880), pid=6160, tid=5828
> # fatal error: DEBUG MESSAGE: unexpected null in intrinsic
>
> I believe this may be caused by non-strong LRB reporting a non-NULL type by passing-through its input's type, even though it may actually return NULL on non-NULL inputs. If this serves as input to an intrinsic, it may lead to elimination of surrounding null-check, and thus end up passing a NULL to the intrinsic even thought it should not.
>
> Testing:
> - [x] hotspot_gc_shenandoah
> - [ ] tier1+Shenandoah
> - [ ] tier2+Shenandoah
This pull request has now been integrated.
Changeset: 97d6e4ae
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/97d6e4ae
Stats: 11 lines in 1 file changed: 8 ins; 0 del; 3 mod
8256046: Shenandoah: Mix-in NULL_PTR in non-strong ShLRBNode's type
Reviewed-by: roland, shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1120
From zgu at openjdk.java.net Tue Nov 10 15:58:57 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 10 Nov 2020 15:58:57 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v6]
In-Reply-To: <-X5aHA0aJFAUOjdrVoKttDntOBth2QQv7vD2cBVcAZo=.70f3d572-66dc-4ca6-b512-0e9ce0577131@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<-X5aHA0aJFAUOjdrVoKttDntOBth2QQv7vD2cBVcAZo=.70f3d572-66dc-4ca6-b512-0e9ce0577131@github.com>
Message-ID: <hVlztsfrxFvvxtrWlj3fu8cfWoNqOtXCoXqujxjcE6E=.ff31831a-c6ff-442d-8d37-34da38708930@github.com>
On Tue, 10 Nov 2020 15:26:14 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
>> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
>> - We can have strong and phantom native referents, and strong and weak in-heap referents
>> - Native referents are never compressed
>>
>> Note that this depends on PR#1140.
>>
>> Testing:
>> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
>> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
>> - [ ] tier2 +UseShenandoahGC +ShenandoahVerify
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>
> - Remove superfluous LP64 in aarch64 part
> - Fixes/missing parts for x86_64
Marked as reviewed by zgu (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 17:08:10 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 17:08:10 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v7]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <i4tbe7CgmCfOUcfBY17ebIv1OvkWGhGsHT_ASkM78YU=.b6018204-ec93-41df-b91e-5285a0dc67f5@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [ ] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Don't make phantom-access narrow (mistake when doing 32bit parts
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/37c97786..92a92fcd
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=06
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=05-06
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Tue Nov 10 20:28:17 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 10 Nov 2020 20:28:17 GMT
Subject: RFR: 8256020: Don't resurrect objects on argument-dependency
access [v2]
In-Reply-To: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
References: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
Message-ID: <QIue_LeWRh820lcE2-Dxq41wYREEGg-V_lv9cShS5Js=.6d63704f-5649-4ded-a450-16a64476c671@github.com>
> In Shenandoah-testing, we noticed that compiler/jsr292/CallSiteDepContextTest.java fails with the following error:
>
> CONF=linux-x86_64-server-fastdebug make run-test TEST=compiler/jsr292/CallSiteDepContextTest.java TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+ShenandoahVerify"
>
> # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp:92), pid=2318905, tid=2318938
> # Error: Before Updating References, Marked; Must be marked in complete bitmap
>
> Referenced from:
> interior location: 0x00000000fff8514c
> 0x00000000fff85140 - klass 0x000000010004ecd8 java.lang.invoke.MutableCallSite
> allocated after mark start
> not after update watermark
> marked strong
> marked weak
> not in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 2565|R |BTE fff80000, fffc0000, fffc0000|TAMS fff80000|UWM fffc0000|U 256K|T 0B|G 256K|S 0B|L 0B|CP 0
>
> Object:
> 0x00000000d80a9210 - klass 0x000000010004cf58 java.lang.invoke.DirectMethodHandle
> not allocated after mark start
> not after update watermark
> not marked strong
> not marked weak
> in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 9|CS |BTE d8080000, d80c0000, d80c0000|TAMS d80c0000|UWM d80c0000|U 256K|T 256K|G 0B|S 0B|L 22464B|CP 0
>
> Forwardee:
> (the object itself)
>
> In other words, a reachable (marked) MutableCallSite references an unreachable DirectMethodHandle. That reference would subsequently become dangling and lead to crashes if accessed.
>
> I narrowed it down to the access in Dependencies::DepStream::recorded_oop_at(int i) which is done as 'strong', which means that it would return the reference even if it is unreachable, e.g. during concurrent class-unloading. This resurrection of the unreachable DMH is potentially fatal: eventually the reference will become dangling (after GC) and lead to crashes when accessed. I believe that access should be 'phantom' instead which causes GCs like Shenandoah and ZGC to return NULL when encountering unreachable objects.
>
> (Notice that the bug only manifested after JDK-8255691, we accidentally applied the resurrection-preventing weak-LRB on strong access too)
>
> Testing:
> - [x] the offending CallSiteDepContextTest.java +UseShenandoah
> - [ ] hotspot_gc_shenandoah
> - [ ] tier1+UseShenandoahGC+ShenandoahVerify
> - [ ] tier2+UseShenandoahGC+ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Revert original fix; Prevent resurrection of unreachable objects during evacuation on AS_NO_KEEPALIVE
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1113/files
- new: https://git.openjdk.java.net/jdk/pull/1113/files/a478871f..80380b80
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1113&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1113&range=00-01
Stats: 10 lines in 2 files changed: 1 ins; 0 del; 9 mod
Patch: https://git.openjdk.java.net/jdk/pull/1113.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1113/head:pull/1113
PR: https://git.openjdk.java.net/jdk/pull/1113
From rkennke at openjdk.java.net Wed Nov 11 07:44:13 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 11 Nov 2020 07:44:13 GMT
Subject: RFR: 8256020: Shenandoah: Don't resurrect objects during
evacuation on AS_NO_KEEPALIVE [v3]
In-Reply-To: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
References: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
Message-ID: <ycUKfm-WN-wncIfU6eRu19hRUJ66ldK_TM6ze_7w_zs=.057b6122-c82a-4922-bf90-8d22b901856a@github.com>
> In Shenandoah-testing, we noticed that compiler/jsr292/CallSiteDepContextTest.java fails with the following error:
>
> CONF=linux-x86_64-server-fastdebug make run-test TEST=compiler/jsr292/CallSiteDepContextTest.java TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+ShenandoahVerify"
>
> # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp:92), pid=2318905, tid=2318938
> # Error: Before Updating References, Marked; Must be marked in complete bitmap
>
> Referenced from:
> interior location: 0x00000000fff8514c
> 0x00000000fff85140 - klass 0x000000010004ecd8 java.lang.invoke.MutableCallSite
> allocated after mark start
> not after update watermark
> marked strong
> marked weak
> not in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 2565|R |BTE fff80000, fffc0000, fffc0000|TAMS fff80000|UWM fffc0000|U 256K|T 0B|G 256K|S 0B|L 0B|CP 0
>
> Object:
> 0x00000000d80a9210 - klass 0x000000010004cf58 java.lang.invoke.DirectMethodHandle
> not allocated after mark start
> not after update watermark
> not marked strong
> not marked weak
> in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 9|CS |BTE d8080000, d80c0000, d80c0000|TAMS d80c0000|UWM d80c0000|U 256K|T 256K|G 0B|S 0B|L 22464B|CP 0
>
> Forwardee:
> (the object itself)
>
> In other words, a reachable (marked) MutableCallSite references an unreachable DirectMethodHandle. That reference would subsequently become dangling and lead to crashes if accessed.
>
> I narrowed it down to the access in Dependencies::DepStream::recorded_oop_at(int i) which is done as 'strong', which means that it would return the reference even if it is unreachable, e.g. during concurrent class-unloading. This resurrection of the unreachable DMH is potentially fatal: eventually the reference will become dangling (after GC) and lead to crashes when accessed. I believe that access should be 'phantom' instead which causes GCs like Shenandoah and ZGC to return NULL when encountering unreachable objects.
>
> (Notice that the bug only manifested after JDK-8255691, we accidentally applied the resurrection-preventing weak-LRB on strong access too)
>
> Testing:
> - [x] the offending CallSiteDepContextTest.java +UseShenandoah
> - [x] hotspot_gc_shenandoah
> - [x] tier1+UseShenandoahGC+ShenandoahVerify
> - [x] tier2+UseShenandoahGC+ShenandoahVerify
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
- Merge branch 'master' into JDK-8256020
- Revert original fix; Prevent resurrection of unreachable objects during evacuation on AS_NO_KEEPALIVE
- 8256020: Don't resurrect objects on argument-dependency access
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1113/files
- new: https://git.openjdk.java.net/jdk/pull/1113/files/80380b80..07260346
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1113&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1113&range=01-02
Stats: 53129 lines in 468 files changed: 29895 ins; 15100 del; 8134 mod
Patch: https://git.openjdk.java.net/jdk/pull/1113.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1113/head:pull/1113
PR: https://git.openjdk.java.net/jdk/pull/1113
From rkennke at openjdk.java.net Wed Nov 11 15:17:15 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 11 Nov 2020 15:17:15 GMT
Subject: RFR: 8256020: Shenandoah: Don't resurrect objects during
evacuation on AS_NO_KEEPALIVE [v4]
In-Reply-To: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
References: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
Message-ID: <vpQmggzbOvcVS-kVM5dxd-YT1VyN7gtkIfMBSUSkUPo=.73157ed1-8d0b-4096-ace0-fee1b1da0213@github.com>
> In Shenandoah-testing, we noticed that compiler/jsr292/CallSiteDepContextTest.java fails with the following error:
>
> CONF=linux-x86_64-server-fastdebug make run-test TEST=compiler/jsr292/CallSiteDepContextTest.java TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+ShenandoahVerify"
>
> # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp:92), pid=2318905, tid=2318938
> # Error: Before Updating References, Marked; Must be marked in complete bitmap
>
> Referenced from:
> interior location: 0x00000000fff8514c
> 0x00000000fff85140 - klass 0x000000010004ecd8 java.lang.invoke.MutableCallSite
> allocated after mark start
> not after update watermark
> marked strong
> marked weak
> not in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 2565|R |BTE fff80000, fffc0000, fffc0000|TAMS fff80000|UWM fffc0000|U 256K|T 0B|G 256K|S 0B|L 0B|CP 0
>
> Object:
> 0x00000000d80a9210 - klass 0x000000010004cf58 java.lang.invoke.DirectMethodHandle
> not allocated after mark start
> not after update watermark
> not marked strong
> not marked weak
> in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 9|CS |BTE d8080000, d80c0000, d80c0000|TAMS d80c0000|UWM d80c0000|U 256K|T 256K|G 0B|S 0B|L 22464B|CP 0
>
> Forwardee:
> (the object itself)
>
> In other words, a reachable (marked) MutableCallSite references an unreachable DirectMethodHandle. That reference would subsequently become dangling and lead to crashes if accessed.
>
> I narrowed it down to the access in Dependencies::DepStream::recorded_oop_at(int i) which is done as 'strong', which means that it would return the reference even if it is unreachable, e.g. during concurrent class-unloading. This resurrection of the unreachable DMH is potentially fatal: eventually the reference will become dangling (after GC) and lead to crashes when accessed. I believe that access should be 'phantom' instead which causes GCs like Shenandoah and ZGC to return NULL when encountering unreachable objects.
>
> (Notice that the bug only manifested after JDK-8255691, we accidentally applied the resurrection-preventing weak-LRB on strong access too)
>
> Testing:
> - [x] the offending CallSiteDepContextTest.java +UseShenandoah
> - [x] hotspot_gc_shenandoah
> - [x] tier1+UseShenandoahGC+ShenandoahVerify
> - [x] tier2+UseShenandoahGC+ShenandoahVerify
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
- Merge branch 'master' into JDK-8256020
- Merge branch 'master' into JDK-8256020
- Merge branch 'master' into JDK-8256020
- Revert original fix; Prevent resurrection of unreachable objects during evacuation on AS_NO_KEEPALIVE
- 8256020: Don't resurrect objects on argument-dependency access
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1113/files
- new: https://git.openjdk.java.net/jdk/pull/1113/files/07260346..492255ed
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1113&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1113&range=02-03
Stats: 687 lines in 13 files changed: 579 ins; 40 del; 68 mod
Patch: https://git.openjdk.java.net/jdk/pull/1113.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1113/head:pull/1113
PR: https://git.openjdk.java.net/jdk/pull/1113
From rkennke at openjdk.java.net Wed Nov 11 15:55:19 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 11 Nov 2020 15:55:19 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v8]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <62FWNENDnaI16zsUXVc9v0RuANXHS9FQRzNYajRXls0=.d2ad75ec-30b3-40b4-a66c-1d50c534dfae@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 18 additional commits since the last revision:
- Fix/invert condition in CmpP optimization
- Fix after merge
- Merge branch 'master' into JDK-8256011
- Merge branch 'master' into JDK-8256011
- Don't make phantom-access narrow (mistake when doing 32bit parts
- Remove superfluous LP64 in aarch64 part
- Fixes/missing parts for x86_64
- Aarch64 parts
- Whitespace changes
- Mask decorators in hash/cmp, not in ctor
- ... and 8 more: https://git.openjdk.java.net/jdk/compare/8d27361d...c0ee9346
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/92a92fcd..c0ee9346
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=07
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=06-07
Stats: 6278 lines in 184 files changed: 4337 ins; 942 del; 999 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From zgu at openjdk.java.net Wed Nov 11 16:58:10 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 11 Nov 2020 16:58:10 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v6]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <pZ5NCr9KF4hnTOJ00UfS3765rP33diJfMnXfxHlH8yU=.894d97ad-8669-426d-a83d-df16689b9eef@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits:
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Move weak reference processing out of STWMark and fix its timings
- Merge branch 'JDK-8255019-sh-mark' of github.com:zhengyu123/jdk into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Merge branch 'master' into JDK-8255019-sh-mark
- Make ShenandoahMarkCompact stack allocated
- ... and 2 more: https://git.openjdk.java.net/jdk/compare/2e19026d...6896e06a
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=05
Stats: 1969 lines in 21 files changed: 1072 ins; 737 del; 160 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From shade at openjdk.java.net Wed Nov 11 16:59:04 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 11 Nov 2020 16:59:04 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v8]
In-Reply-To: <62FWNENDnaI16zsUXVc9v0RuANXHS9FQRzNYajRXls0=.d2ad75ec-30b3-40b4-a66c-1d50c534dfae@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<62FWNENDnaI16zsUXVc9v0RuANXHS9FQRzNYajRXls0=.d2ad75ec-30b3-40b4-a66c-1d50c534dfae@github.com>
Message-ID: <DV9Ec-VRpHxFFoeeR-GYErViL6B08zh-siALtDwfxsE=.7c33767c-9500-4c1f-af15-f06cb21ccf3e@github.com>
On Wed, 11 Nov 2020 15:55:19 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
>> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
>> - We can have strong and phantom native referents, and strong and weak in-heap referents
>> - Native referents are never compressed
>>
>> Note that this depends on PR#1140.
>>
>> Testing:
>> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
>> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
>> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 18 additional commits since the last revision:
>
> - Fix/invert condition in CmpP optimization
> - Fix after merge
> - Merge branch 'master' into JDK-8256011
> - Merge branch 'master' into JDK-8256011
> - Don't make phantom-access narrow (mistake when doing 32bit parts
> - Remove superfluous LP64 in aarch64 part
> - Fixes/missing parts for x86_64
> - Aarch64 parts
> - Whitespace changes
> - Mask decorators in hash/cmp, not in ctor
> - ... and 8 more: https://git.openjdk.java.net/jdk/compare/682e0e24...c0ee9346
This looks fine to me. Good to go, assuming the tests pass.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1109
From shade at openjdk.java.net Wed Nov 11 17:05:00 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 11 Nov 2020 17:05:00 GMT
Subject: RFR: 8256020: Shenandoah: Don't resurrect objects during
evacuation on AS_NO_KEEPALIVE [v4]
In-Reply-To: <vpQmggzbOvcVS-kVM5dxd-YT1VyN7gtkIfMBSUSkUPo=.73157ed1-8d0b-4096-ace0-fee1b1da0213@github.com>
References: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
<vpQmggzbOvcVS-kVM5dxd-YT1VyN7gtkIfMBSUSkUPo=.73157ed1-8d0b-4096-ace0-fee1b1da0213@github.com>
Message-ID: <F3_BBrlGMXFgsmKJc2jyjGEUNa8Li4xYZ1cKLBsvNu0=.08105fa3-0a46-4c76-b546-d8e98ea812a1@github.com>
On Wed, 11 Nov 2020 15:17:15 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In Shenandoah-testing, we noticed that compiler/jsr292/CallSiteDepContextTest.java fails with the following error:
>>
>> CONF=linux-x86_64-server-fastdebug make run-test TEST=compiler/jsr292/CallSiteDepContextTest.java TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+ShenandoahVerify"
>>
>> # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp:92), pid=2318905, tid=2318938
>> # Error: Before Updating References, Marked; Must be marked in complete bitmap
>>
>> Referenced from:
>> interior location: 0x00000000fff8514c
>> 0x00000000fff85140 - klass 0x000000010004ecd8 java.lang.invoke.MutableCallSite
>> allocated after mark start
>> not after update watermark
>> marked strong
>> marked weak
>> not in collection set
>> mark: mark(is_neutral no_hash age=0)
>> region: | 2565|R |BTE fff80000, fffc0000, fffc0000|TAMS fff80000|UWM fffc0000|U 256K|T 0B|G 256K|S 0B|L 0B|CP 0
>>
>> Object:
>> 0x00000000d80a9210 - klass 0x000000010004cf58 java.lang.invoke.DirectMethodHandle
>> not allocated after mark start
>> not after update watermark
>> not marked strong
>> not marked weak
>> in collection set
>> mark: mark(is_neutral no_hash age=0)
>> region: | 9|CS |BTE d8080000, d80c0000, d80c0000|TAMS d80c0000|UWM d80c0000|U 256K|T 256K|G 0B|S 0B|L 22464B|CP 0
>>
>> Forwardee:
>> (the object itself)
>>
>> In other words, a reachable (marked) MutableCallSite references an unreachable DirectMethodHandle. That reference would subsequently become dangling and lead to crashes if accessed.
>>
>> I narrowed it down to the access in Dependencies::DepStream::recorded_oop_at(int i) which is done as 'strong', which means that it would return the reference even if it is unreachable, e.g. during concurrent class-unloading. This resurrection of the unreachable DMH is potentially fatal: eventually the reference will become dangling (after GC) and lead to crashes when accessed. I believe that access should be 'phantom' instead which causes GCs like Shenandoah and ZGC to return NULL when encountering unreachable objects.
>>
>> (Notice that the bug only manifested after JDK-8255691, we accidentally applied the resurrection-preventing weak-LRB on strong access too)
>>
>> Testing:
>> - [x] the offending CallSiteDepContextTest.java +UseShenandoah
>> - [x] hotspot_gc_shenandoah
>> - [x] tier1+UseShenandoahGC+ShenandoahVerify
>> - [x] tier2+UseShenandoahGC+ShenandoahVerify
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>
> - Merge branch 'master' into JDK-8256020
> - Merge branch 'master' into JDK-8256020
> - Merge branch 'master' into JDK-8256020
> - Revert original fix; Prevent resurrection of unreachable objects during evacuation on AS_NO_KEEPALIVE
> - 8256020: Don't resurrect objects on argument-dependency access
src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 115:
> 113: // concurrent class-unloading.
> 114: if (HasDecorator<decorators, AS_NO_KEEPALIVE>::value && obj != NULL &&
> 115: _heap->has_forwarded_objects() &&
Why `has_forwarded_objects()` check here? It seems to me that without forwarded objects, the `load_reference_barrier` below would return the same `obj` anyway?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1113
From shade at openjdk.java.net Wed Nov 11 17:41:57 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 11 Nov 2020 17:41:57 GMT
Subject: RFR: 8256051: nmethod_entry_barrier stub miscalculates xmm spill
size on x86_32
In-Reply-To: <5rwWQ1KNBVs3x8STyNXBWYLrP43atygm6TwaF-4D5wM=.4f9b85ec-b5af-4515-90b0-885ec817b7a6@github.com>
References: <5rwWQ1KNBVs3x8STyNXBWYLrP43atygm6TwaF-4D5wM=.4f9b85ec-b5af-4515-90b0-885ec817b7a6@github.com>
Message-ID: <rZKf5XqQYLsWvD1JMRDjKEUFpXZAgq8lIhAqdgDOfJo=.a2a4273f-3984-400a-844d-f1da718ddd31@github.com>
On Wed, 11 Nov 2020 16:46:43 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> nmethod_entry_barrier stub miscalculates xmm spill size on x86_32, instead of 4 * wordSize, it uses 2 * wordSize.
>
> This bug only affects Shenandoah, as it is the only GC that supports concurrent class unloading on x86_32.
D'oh. Seems like a simple error while copy-pasting from `stubGenerator_x86_64.cpp`. Looks fine and trivial to me. I ran `x86_32` + `tier1` + `Shenandoah` tests on my own with this change, and it improves the test results a lot.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1170
From rkennke at openjdk.java.net Wed Nov 11 17:49:57 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 11 Nov 2020 17:49:57 GMT
Subject: RFR: 8256020: Shenandoah: Don't resurrect objects during
evacuation on AS_NO_KEEPALIVE [v4]
In-Reply-To: <F3_BBrlGMXFgsmKJc2jyjGEUNa8Li4xYZ1cKLBsvNu0=.08105fa3-0a46-4c76-b546-d8e98ea812a1@github.com>
References: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
<vpQmggzbOvcVS-kVM5dxd-YT1VyN7gtkIfMBSUSkUPo=.73157ed1-8d0b-4096-ace0-fee1b1da0213@github.com>
<F3_BBrlGMXFgsmKJc2jyjGEUNa8Li4xYZ1cKLBsvNu0=.08105fa3-0a46-4c76-b546-d8e98ea812a1@github.com>
Message-ID: <GtTdGAchilnA5xWPTn8rVTAA_yNSB4vLIXbsX4OIJXw=.35409826-865a-41e1-8138-ec21749240b6@github.com>
On Wed, 11 Nov 2020 17:02:13 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>>
>> - Merge branch 'master' into JDK-8256020
>> - Merge branch 'master' into JDK-8256020
>> - Merge branch 'master' into JDK-8256020
>> - Revert original fix; Prevent resurrection of unreachable objects during evacuation on AS_NO_KEEPALIVE
>> - 8256020: Don't resurrect objects on argument-dependency access
>
> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 115:
>
>> 113: // concurrent class-unloading.
>> 114: if (HasDecorator<decorators, AS_NO_KEEPALIVE>::value && obj != NULL &&
>> 115: _heap->has_forwarded_objects() &&
>
> Why `has_forwarded_objects()` check here? It seems to me that without forwarded objects, the `load_reference_barrier` below would return the same `obj` anyway?
Mostly because we check if the object is (un-)marked and that would be unreliable outside of has_forwarded. Also, secondarily, because this block is intended to prevent resurrection-by-evacuation, which would not happen outside of has_forwarded. We could, infact, narrow it to is_evacuation_in_progress() I guess.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1113
From rkennke at openjdk.java.net Wed Nov 11 17:56:12 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 11 Nov 2020 17:56:12 GMT
Subject: RFR: 8256020: Shenandoah: Don't resurrect objects during
evacuation on AS_NO_KEEPALIVE [v5]
In-Reply-To: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
References: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
Message-ID: <YpNBze78xGXVgyIvaVLAJZ1skxeanYo6aROPhiGtJmg=.3e5be7da-84a0-43f1-bf68-da2caaa9c667@github.com>
> In Shenandoah-testing, we noticed that compiler/jsr292/CallSiteDepContextTest.java fails with the following error:
>
> CONF=linux-x86_64-server-fastdebug make run-test TEST=compiler/jsr292/CallSiteDepContextTest.java TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+ShenandoahVerify"
>
> # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp:92), pid=2318905, tid=2318938
> # Error: Before Updating References, Marked; Must be marked in complete bitmap
>
> Referenced from:
> interior location: 0x00000000fff8514c
> 0x00000000fff85140 - klass 0x000000010004ecd8 java.lang.invoke.MutableCallSite
> allocated after mark start
> not after update watermark
> marked strong
> marked weak
> not in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 2565|R |BTE fff80000, fffc0000, fffc0000|TAMS fff80000|UWM fffc0000|U 256K|T 0B|G 256K|S 0B|L 0B|CP 0
>
> Object:
> 0x00000000d80a9210 - klass 0x000000010004cf58 java.lang.invoke.DirectMethodHandle
> not allocated after mark start
> not after update watermark
> not marked strong
> not marked weak
> in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 9|CS |BTE d8080000, d80c0000, d80c0000|TAMS d80c0000|UWM d80c0000|U 256K|T 256K|G 0B|S 0B|L 22464B|CP 0
>
> Forwardee:
> (the object itself)
>
> In other words, a reachable (marked) MutableCallSite references an unreachable DirectMethodHandle. That reference would subsequently become dangling and lead to crashes if accessed.
>
> I narrowed it down to the access in Dependencies::DepStream::recorded_oop_at(int i) which is done as 'strong', which means that it would return the reference even if it is unreachable, e.g. during concurrent class-unloading. This resurrection of the unreachable DMH is potentially fatal: eventually the reference will become dangling (after GC) and lead to crashes when accessed. I believe that access should be 'phantom' instead which causes GCs like Shenandoah and ZGC to return NULL when encountering unreachable objects.
>
> (Notice that the bug only manifested after JDK-8255691, we accidentally applied the resurrection-preventing weak-LRB on strong access too)
>
> Testing:
> - [x] the offending CallSiteDepContextTest.java +UseShenandoah
> - [x] hotspot_gc_shenandoah
> - [x] tier1+UseShenandoahGC+ShenandoahVerify
> - [x] tier2+UseShenandoahGC+ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Narrow phase in which to prevent resurrection-by-evacuation
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1113/files
- new: https://git.openjdk.java.net/jdk/pull/1113/files/492255ed..65f902cd
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1113&range=04
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1113&range=03-04
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1113.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1113/head:pull/1113
PR: https://git.openjdk.java.net/jdk/pull/1113
From shade at openjdk.java.net Wed Nov 11 18:03:01 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 11 Nov 2020 18:03:01 GMT
Subject: RFR: 8256020: Shenandoah: Don't resurrect objects during
evacuation on AS_NO_KEEPALIVE [v5]
In-Reply-To: <YpNBze78xGXVgyIvaVLAJZ1skxeanYo6aROPhiGtJmg=.3e5be7da-84a0-43f1-bf68-da2caaa9c667@github.com>
References: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
<YpNBze78xGXVgyIvaVLAJZ1skxeanYo6aROPhiGtJmg=.3e5be7da-84a0-43f1-bf68-da2caaa9c667@github.com>
Message-ID: <aGoF6XRVmoH5jgujfUO53UmL2CzaOXYc022_pvdlwZ4=.f6048e3d-25c5-4ae5-a443-0d34885d2a4f@github.com>
On Wed, 11 Nov 2020 17:56:12 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In Shenandoah-testing, we noticed that compiler/jsr292/CallSiteDepContextTest.java fails with the following error:
>>
>> CONF=linux-x86_64-server-fastdebug make run-test TEST=compiler/jsr292/CallSiteDepContextTest.java TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+ShenandoahVerify"
>>
>> # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp:92), pid=2318905, tid=2318938
>> # Error: Before Updating References, Marked; Must be marked in complete bitmap
>>
>> Referenced from:
>> interior location: 0x00000000fff8514c
>> 0x00000000fff85140 - klass 0x000000010004ecd8 java.lang.invoke.MutableCallSite
>> allocated after mark start
>> not after update watermark
>> marked strong
>> marked weak
>> not in collection set
>> mark: mark(is_neutral no_hash age=0)
>> region: | 2565|R |BTE fff80000, fffc0000, fffc0000|TAMS fff80000|UWM fffc0000|U 256K|T 0B|G 256K|S 0B|L 0B|CP 0
>>
>> Object:
>> 0x00000000d80a9210 - klass 0x000000010004cf58 java.lang.invoke.DirectMethodHandle
>> not allocated after mark start
>> not after update watermark
>> not marked strong
>> not marked weak
>> in collection set
>> mark: mark(is_neutral no_hash age=0)
>> region: | 9|CS |BTE d8080000, d80c0000, d80c0000|TAMS d80c0000|UWM d80c0000|U 256K|T 256K|G 0B|S 0B|L 22464B|CP 0
>>
>> Forwardee:
>> (the object itself)
>>
>> In other words, a reachable (marked) MutableCallSite references an unreachable DirectMethodHandle. That reference would subsequently become dangling and lead to crashes if accessed.
>>
>> I narrowed it down to the access in Dependencies::DepStream::recorded_oop_at(int i) which is done as 'strong', which means that it would return the reference even if it is unreachable, e.g. during concurrent class-unloading. This resurrection of the unreachable DMH is potentially fatal: eventually the reference will become dangling (after GC) and lead to crashes when accessed. I believe that access should be 'phantom' instead which causes GCs like Shenandoah and ZGC to return NULL when encountering unreachable objects.
>>
>> (Notice that the bug only manifested after JDK-8255691, we accidentally applied the resurrection-preventing weak-LRB on strong access too)
>>
>> Testing:
>> - [x] the offending CallSiteDepContextTest.java +UseShenandoah
>> - [x] hotspot_gc_shenandoah
>> - [x] tier1+UseShenandoahGC+ShenandoahVerify
>> - [x] tier2+UseShenandoahGC+ShenandoahVerify
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Narrow phase in which to prevent resurrection-by-evacuation
Yes, that matches the synopsis better too.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1113
From rkennke at openjdk.java.net Wed Nov 11 18:07:02 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 11 Nov 2020 18:07:02 GMT
Subject: Integrated: 8256020: Shenandoah: Don't resurrect objects during
evacuation on AS_NO_KEEPALIVE
In-Reply-To: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
References: <JlzuXp3fQMcvn0wBp9toVjFFD7iX5ErkYusesMvUb8U=.5ce051f0-7343-4e27-9130-d0b3edea67c3@github.com>
Message-ID: <t_BdSHIXkSDVN7sIYeANxCYhRBjW09dPs9vgVfeb2BA=.b2ce6bae-b944-4fea-a0bc-afb0431a696b@github.com>
On Sun, 8 Nov 2020 21:35:29 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> In Shenandoah-testing, we noticed that compiler/jsr292/CallSiteDepContextTest.java fails with the following error:
>
> CONF=linux-x86_64-server-fastdebug make run-test TEST=compiler/jsr292/CallSiteDepContextTest.java TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+ShenandoahVerify"
>
> # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp:92), pid=2318905, tid=2318938
> # Error: Before Updating References, Marked; Must be marked in complete bitmap
>
> Referenced from:
> interior location: 0x00000000fff8514c
> 0x00000000fff85140 - klass 0x000000010004ecd8 java.lang.invoke.MutableCallSite
> allocated after mark start
> not after update watermark
> marked strong
> marked weak
> not in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 2565|R |BTE fff80000, fffc0000, fffc0000|TAMS fff80000|UWM fffc0000|U 256K|T 0B|G 256K|S 0B|L 0B|CP 0
>
> Object:
> 0x00000000d80a9210 - klass 0x000000010004cf58 java.lang.invoke.DirectMethodHandle
> not allocated after mark start
> not after update watermark
> not marked strong
> not marked weak
> in collection set
> mark: mark(is_neutral no_hash age=0)
> region: | 9|CS |BTE d8080000, d80c0000, d80c0000|TAMS d80c0000|UWM d80c0000|U 256K|T 256K|G 0B|S 0B|L 22464B|CP 0
>
> Forwardee:
> (the object itself)
>
> In other words, a reachable (marked) MutableCallSite references an unreachable DirectMethodHandle. That reference would subsequently become dangling and lead to crashes if accessed.
>
> I narrowed it down to the access in Dependencies::DepStream::recorded_oop_at(int i) which is done as 'strong', which means that it would return the reference even if it is unreachable, e.g. during concurrent class-unloading. This resurrection of the unreachable DMH is potentially fatal: eventually the reference will become dangling (after GC) and lead to crashes when accessed. I believe that access should be 'phantom' instead which causes GCs like Shenandoah and ZGC to return NULL when encountering unreachable objects.
>
> (Notice that the bug only manifested after JDK-8255691, we accidentally applied the resurrection-preventing weak-LRB on strong access too)
>
> Testing:
> - [x] the offending CallSiteDepContextTest.java +UseShenandoah
> - [x] hotspot_gc_shenandoah
> - [x] tier1+UseShenandoahGC+ShenandoahVerify
> - [x] tier2+UseShenandoahGC+ShenandoahVerify
This pull request has now been integrated.
Changeset: 3c3469b9
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/3c3469b9
Stats: 9 lines in 1 file changed: 1 ins; 0 del; 8 mod
8256020: Shenandoah: Don't resurrect objects during evacuation on AS_NO_KEEPALIVE
Reviewed-by: shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1113
From rkennke at openjdk.java.net Wed Nov 11 18:13:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 11 Nov 2020 18:13:07 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v9]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <tpd3eIJcwwFrmFElkB1egpojb9JE4Drx4C56ahatUq4=.5840e7d6-55fe-4432-84b7-ec5bac1757c1@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits:
- Move resurrection-barrier from JDK-8256020 into right place after merge
- Merge branch 'master' into JDK-8256011
- Fix/invert condition in CmpP optimization
- Fix after merge
- Merge branch 'master' into JDK-8256011
- Merge branch 'master' into JDK-8256011
- Don't make phantom-access narrow (mistake when doing 32bit parts
- Remove superfluous LP64 in aarch64 part
- Fixes/missing parts for x86_64
- Aarch64 parts
- ... and 10 more: https://git.openjdk.java.net/jdk/compare/96e02610...492cea4d
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1109/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=08
Stats: 337 lines in 16 files changed: 82 ins; 85 del; 170 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From zgu at openjdk.java.net Wed Nov 11 19:15:57 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 11 Nov 2020 19:15:57 GMT
Subject: Integrated: 8256051: nmethod_entry_barrier stub miscalculates xmm
spill size on x86_32
In-Reply-To: <5rwWQ1KNBVs3x8STyNXBWYLrP43atygm6TwaF-4D5wM=.4f9b85ec-b5af-4515-90b0-885ec817b7a6@github.com>
References: <5rwWQ1KNBVs3x8STyNXBWYLrP43atygm6TwaF-4D5wM=.4f9b85ec-b5af-4515-90b0-885ec817b7a6@github.com>
Message-ID: <0qb1bbBFIQOhMJ27jemslBFuCplazCNVRm9kdNU3eTs=.097ce556-ba33-4b2d-a94e-084d99953b41@github.com>
On Wed, 11 Nov 2020 16:46:43 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> nmethod_entry_barrier stub miscalculates xmm spill size on x86_32, instead of 4 * wordSize, it uses 2 * wordSize.
>
> This bug only affects Shenandoah, as it is the only GC that supports concurrent class unloading on x86_32.
This pull request has now been integrated.
Changeset: bfa060f0
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/bfa060f0
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
8256051: nmethod_entry_barrier stub miscalculates xmm spill size on x86_32
Reviewed-by: shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1170
From zgu at openjdk.java.net Wed Nov 11 19:15:55 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 11 Nov 2020 19:15:55 GMT
Subject: RFR: 8256051: nmethod_entry_barrier stub miscalculates xmm spill
size on x86_32
In-Reply-To: <rZKf5XqQYLsWvD1JMRDjKEUFpXZAgq8lIhAqdgDOfJo=.a2a4273f-3984-400a-844d-f1da718ddd31@github.com>
References: <5rwWQ1KNBVs3x8STyNXBWYLrP43atygm6TwaF-4D5wM=.4f9b85ec-b5af-4515-90b0-885ec817b7a6@github.com>
<rZKf5XqQYLsWvD1JMRDjKEUFpXZAgq8lIhAqdgDOfJo=.a2a4273f-3984-400a-844d-f1da718ddd31@github.com>
Message-ID: <njFAVcS4EZTBUwHYvSB-zKos5baN0QAtk-TgkRRhrzg=.de8641aa-55df-45ad-8dac-d8223fabe2df@github.com>
On Wed, 11 Nov 2020 17:39:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> D'oh. Seems like a simple error while copy-pasting from `stubGenerator_x86_64.cpp`. Looks fine and trivial to me. I ran `x86_32` + `tier1` + `Shenandoah` tests on my own with this change, and it improves the test results a lot.
Thanks, Aleksey.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1170
From rkennke at openjdk.java.net Wed Nov 11 20:34:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 11 Nov 2020 20:34:07 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v10]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <1a7sW4Qf-6GcT82aBF5kbIW47ulaLfGO-nob6jl5uFI=.235b42d8-b4fc-4737-b1ee-8ee9ce478d56@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 21 commits:
- Merge branch 'master' into JDK-8256011
- Move resurrection-barrier from JDK-8256020 into right place after merge
- Merge branch 'master' into JDK-8256011
- Fix/invert condition in CmpP optimization
- Fix after merge
- Merge branch 'master' into JDK-8256011
- Merge branch 'master' into JDK-8256011
- Don't make phantom-access narrow (mistake when doing 32bit parts
- Remove superfluous LP64 in aarch64 part
- Fixes/missing parts for x86_64
- ... and 11 more: https://git.openjdk.java.net/jdk/compare/59965c17...3eeee3c3
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1109/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=09
Stats: 337 lines in 16 files changed: 82 ins; 85 del; 170 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From zgu at openjdk.java.net Wed Nov 11 21:28:01 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 11 Nov 2020 21:28:01 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v7]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <bIxuLWKINEQfJB09LQ3oC5tpYqelQ7452agJxYaNsXU=.a76c47b3-86ea-4cac-a031-c8140e39d23b@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits:
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Move weak reference processing out of STWMark and fix its timings
- Merge branch 'JDK-8255019-sh-mark' of github.com:zhengyu123/jdk into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Merge branch 'master' into JDK-8255019-sh-mark
- ... and 3 more: https://git.openjdk.java.net/jdk/compare/59965c17...71b38b4d
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=06
Stats: 1969 lines in 21 files changed: 1072 ins; 737 del; 160 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From rkennke at openjdk.java.net Thu Nov 12 09:54:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 12 Nov 2020 09:54:07 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v11]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <S-O2sk-ZyIvSTHwBYaYMzAFDAFMw03y4uVbqM6H_ZXc=.03b831a5-5fd0-424a-8485-abd3be8544f6@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Call phantom LRB when ON_PHANTOM_OOP_REF is requested in C1 LRB stub
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/3eeee3c3..29347682
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=10
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=09-10
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Thu Nov 12 12:41:12 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 12 Nov 2020 12:41:12 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v12]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <_cPA4cY7jgq7ZOiXVcPTTZ6SdoBiAIIH_OGrsswU2tg=.2788ba6a-985c-46b0-b73d-c9d9a198bdfd@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
- Avoid null-check and skip cset-check on non-strong refs in C1 LRB stub
- Call weak-LRB with ON_WEAK_OOP_REF, not ON_UNKNOWN_OOP_REF (cosmetic)
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/29347682..83c07cb8
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=11
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=10-11
Stats: 26 lines in 2 files changed: 6 ins; 7 del; 13 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From zgu at openjdk.java.net Thu Nov 12 13:08:08 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 12 Nov 2020 13:08:08 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v8]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <y7ODsDVpjrxfAtpCODoYuJ09E63LJ-digUUXOdb5A-s=.b1e72e7a-04bf-425b-96da-309be6235f11@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Removed obsoleted class
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1009/files
- new: https://git.openjdk.java.net/jdk/pull/1009/files/71b38b4d..91886ed4
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=07
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=06-07
Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From thartmann at openjdk.java.net Thu Nov 12 15:00:01 2020
From: thartmann at openjdk.java.net (Tobias Hartmann)
Date: Thu, 12 Nov 2020 15:00:01 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah [v2]
In-Reply-To: <dGIlql-cWnZXaROTZn-roVLs5gVcXmk29XyL9323UlI=.d62d69ca-d47f-44a7-962a-96e04dd59328@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
<dGIlql-cWnZXaROTZn-roVLs5gVcXmk29XyL9323UlI=.d62d69ca-d47f-44a7-962a-96e04dd59328@github.com>
Message-ID: <oHDtwNYKsbX6FqRvWsmfd7UCDeP7LIwJVmilNzlFCjE=.072f83ba-f47f-4d10-b50d-8e765e1a811d@github.com>
On Fri, 6 Nov 2020 11:13:04 GMT, Roland Westrelin <roland at openjdk.org> wrote:
>> This is a Shenandoah bug but the proposed fix is in shared code.
>>
>> In an infinite loop, a barrier is located right after the loop head
>> and above the never branch. When the barrier is expanded, control flow
>> is added between the loop and the never branch. During loop
>> verification the assert fires because it doesn't expect any control
>> flow between the never branch and the loop head.
>>
>> While it would have been nice to fix this Shenandoah issue in
>> Shenandoah code, I think the cleaner fix is to preserve the invariant
>> that the never branch is always right after the loop head in an
>> infinite loop. In the proposed patch, this is achieved by moving all
>> uses of the loop head to the never branch when it's constructed.
>
> Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains one commit:
>
> fix & test
Otherwise looks good to me.
src/hotspot/share/opto/loopnode.cpp line 4569:
> 4567: set_loop(if_t, l);
> 4568:
> 4569: Node* cfg = NULL; // Find the One True Control User of m
Should be removed.
src/hotspot/share/opto/loopnode.cpp line 4576:
> 4574: int nb = x->replace_edge(m, if_t);
> 4575: assert(nb > 0, "use should have been updated");
> 4576: --j, jmax -= nb;
Indentation is wrong.
-------------
Marked as reviewed by thartmann (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1073
From rkennke at openjdk.java.net Thu Nov 12 15:18:10 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 12 Nov 2020 15:18:10 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v13]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <0ZJsK6RHI2m-zr7RoheY-uQCsO-AxfxNxMyVjERjD18=.f22c8dff-d391-48bd-8aca-a65592a0ae30@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Don't call compressed-oops LRB on IN_NATIVE from C1
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/83c07cb8..51811e31
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=12
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=11-12
Stats: 29 lines in 3 files changed: 24 ins; 2 del; 3 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From roland at openjdk.java.net Thu Nov 12 15:38:07 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Thu, 12 Nov 2020 15:38:07 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah [v2]
In-Reply-To: <oHDtwNYKsbX6FqRvWsmfd7UCDeP7LIwJVmilNzlFCjE=.072f83ba-f47f-4d10-b50d-8e765e1a811d@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
<dGIlql-cWnZXaROTZn-roVLs5gVcXmk29XyL9323UlI=.d62d69ca-d47f-44a7-962a-96e04dd59328@github.com>
<oHDtwNYKsbX6FqRvWsmfd7UCDeP7LIwJVmilNzlFCjE=.072f83ba-f47f-4d10-b50d-8e765e1a811d@github.com>
Message-ID: <LRF2Rso25yIeEDdTpyKq0rkuPuNMNvs8P9pMDzN_u2Q=.353dc8a4-8f58-42cd-b428-30dc9f3edcdf@github.com>
On Thu, 12 Nov 2020 14:55:43 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:
>> Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase.
>
> src/hotspot/share/opto/loopnode.cpp line 4575:
>
>> 4573: _igvn.rehash_node_delayed(x);
>> 4574: int nb = x->replace_edge(m, if_t);
>> 4575: assert(nb > 0, "use should have been updated");
>
> Indentation is wrong.
Updated. Thanks for the review.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From roland at openjdk.java.net Thu Nov 12 15:38:06 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Thu, 12 Nov 2020 15:38:06 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah [v3]
In-Reply-To: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
Message-ID: <jG1N-2cWJDmoApu-Frm4zsPgBDpmhyyS4cH_FPPsmnk=.4dba7ebe-b038-47a3-b906-252a5f23f084@github.com>
> This is a Shenandoah bug but the proposed fix is in shared code.
>
> In an infinite loop, a barrier is located right after the loop head
> and above the never branch. When the barrier is expanded, control flow
> is added between the loop and the never branch. During loop
> verification the assert fires because it doesn't expect any control
> flow between the never branch and the loop head.
>
> While it would have been nice to fix this Shenandoah issue in
> Shenandoah code, I think the cleaner fix is to preserve the invariant
> that the never branch is always right after the loop head in an
> infinite loop. In the proposed patch, this is achieved by moving all
> uses of the loop head to the never branch when it's constructed.
Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits:
- review
- fix & test
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1073/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1073&range=02
Stats: 14 lines in 2 files changed: 6 ins; 6 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1073.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1073/head:pull/1073
PR: https://git.openjdk.java.net/jdk/pull/1073
From zgu at openjdk.java.net Thu Nov 12 16:00:04 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 12 Nov 2020 16:00:04 GMT
Subject: RFR: 8256278: Shenandoah: Avoid num of dead callback from weak
processor in Shenandoah root verifier
Message-ID: <wx12HCljhSrkQIvIrS-R7K_2Ewb7MEdM48s2RvZvuD8=.9e96d209-fe3e-4df5-a258-fb7684d56845@github.com>
Shenandoah root verifier currently uses WeakProcessor to verify weak oops, but it always generates report_num_dead callback if registered. This callback fails runtime/stringtable/StringTableCleaningTest.java test with -XX:+ShenandoahVerify.
Test:
- [x] hotspot_gc_shenandoah
- [x] Passed failed test
TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:-ShenandoahVerify" make
CONF=linux-x86_64-server-release run-test TEST=runtime/stringtable/StringTableCleaningTest.java
-------------
Commit messages:
- Avoid num of dead callback from weak processor in Shenandoah root verifier
Changes: https://git.openjdk.java.net/jdk/pull/1189/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1189&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256278
Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1189.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1189/head:pull/1189
PR: https://git.openjdk.java.net/jdk/pull/1189
From coleenp at openjdk.java.net Thu Nov 12 16:23:23 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 12 Nov 2020 16:23:23 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v8]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <pRGCVDv6C52tCs27ouLhVH4UR7u0MnAslZOHZcargZ0=.8f0ae077-f0de-4705-b005-9abda7ed2787@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
Add logging to event posting in case of pauses.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/6dec83d8..0487b84c
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=07
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=06-07
Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From zgu at openjdk.java.net Thu Nov 12 16:45:16 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 12 Nov 2020 16:45:16 GMT
Subject: RFR: 8256278: Shenandoah: Avoid num of dead callback from weak
processor in Shenandoah root verifier [v2]
In-Reply-To: <wx12HCljhSrkQIvIrS-R7K_2Ewb7MEdM48s2RvZvuD8=.9e96d209-fe3e-4df5-a258-fb7684d56845@github.com>
References: <wx12HCljhSrkQIvIrS-R7K_2Ewb7MEdM48s2RvZvuD8=.9e96d209-fe3e-4df5-a258-fb7684d56845@github.com>
Message-ID: <SZtkBHMYIZ7hW-31akc6wWHHVkj0MlJPszO-g3kMDuA=.757610e4-27ac-41d1-989e-3813dd6b3207@github.com>
> Shenandoah root verifier currently uses WeakProcessor to verify weak oops, but it always generates report_num_dead callback if registered. This callback fails runtime/stringtable/StringTableCleaningTest.java test with -XX:+ShenandoahVerify.
>
> Test:
>
> - [x] hotspot_gc_shenandoah
> - [x] Passed failed test
> TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:-ShenandoahVerify" make
> CONF=linux-x86_64-server-release run-test TEST=runtime/stringtable/StringTableCleaningTest.java
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Removed unused header
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1189/files
- new: https://git.openjdk.java.net/jdk/pull/1189/files/e9821d97..229cbaf7
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1189&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1189&range=00-01
Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
Patch: https://git.openjdk.java.net/jdk/pull/1189.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1189/head:pull/1189
PR: https://git.openjdk.java.net/jdk/pull/1189
From rkennke at openjdk.java.net Thu Nov 12 17:38:58 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 12 Nov 2020 17:38:58 GMT
Subject: RFR: 8256278: Shenandoah: Avoid num of dead callback from weak
processor in Shenandoah root verifier [v2]
In-Reply-To: <SZtkBHMYIZ7hW-31akc6wWHHVkj0MlJPszO-g3kMDuA=.757610e4-27ac-41d1-989e-3813dd6b3207@github.com>
References: <wx12HCljhSrkQIvIrS-R7K_2Ewb7MEdM48s2RvZvuD8=.9e96d209-fe3e-4df5-a258-fb7684d56845@github.com>
<SZtkBHMYIZ7hW-31akc6wWHHVkj0MlJPszO-g3kMDuA=.757610e4-27ac-41d1-989e-3813dd6b3207@github.com>
Message-ID: <aPRtIdRWylDxD8NydEt4GoTnGl6mzv8RjpQRtv-Hxc8=.62e13f54-2d0d-4c09-a3f6-fbe3d4f3e8ab@github.com>
On Thu, 12 Nov 2020 16:45:16 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> Shenandoah root verifier currently uses WeakProcessor to verify weak oops, but it always generates report_num_dead callback if registered. This callback fails runtime/stringtable/StringTableCleaningTest.java test with -XX:+ShenandoahVerify.
>>
>> Test:
>>
>> - [x] hotspot_gc_shenandoah
>> - [x] Passed failed test
>> TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:-ShenandoahVerify" make
>> CONF=linux-x86_64-server-release run-test TEST=runtime/stringtable/StringTableCleaningTest.java
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>
> Removed unused header
Looks good to me. Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1189
From shade at openjdk.java.net Thu Nov 12 17:42:58 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 12 Nov 2020 17:42:58 GMT
Subject: RFR: 8256278: Shenandoah: Avoid num of dead callback from weak
processor in Shenandoah root verifier [v2]
In-Reply-To: <SZtkBHMYIZ7hW-31akc6wWHHVkj0MlJPszO-g3kMDuA=.757610e4-27ac-41d1-989e-3813dd6b3207@github.com>
References: <wx12HCljhSrkQIvIrS-R7K_2Ewb7MEdM48s2RvZvuD8=.9e96d209-fe3e-4df5-a258-fb7684d56845@github.com>
<SZtkBHMYIZ7hW-31akc6wWHHVkj0MlJPszO-g3kMDuA=.757610e4-27ac-41d1-989e-3813dd6b3207@github.com>
Message-ID: <z4MCXg2wfKc8rUreg9PBvgsWKS4DlsLM3zX6igvBmSQ=.390a8bf5-22dc-40f8-a2e6-0f245cc7d07d@github.com>
On Thu, 12 Nov 2020 16:45:16 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> Shenandoah root verifier currently uses WeakProcessor to verify weak oops, but it always generates report_num_dead callback if registered. This callback fails runtime/stringtable/StringTableCleaningTest.java test with -XX:+ShenandoahVerify.
>>
>> Test:
>>
>> - [x] hotspot_gc_shenandoah
>> - [x] Passed failed test
>> TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:-ShenandoahVerify" make
>> CONF=linux-x86_64-server-release run-test TEST=runtime/stringtable/StringTableCleaningTest.java
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>
> Removed unused header
Marked as reviewed by shade (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/1189
From rkennke at openjdk.java.net Thu Nov 12 19:35:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 12 Nov 2020 19:35:07 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v14]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <1xZs_DTTaoyM6EJz6wxsGbZ8TQIL7Az0CWcjI2Lfe_M=.df15f501-6ba8-4f2d-89ec-86ec15c5364f@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Remaining aarch64 changes
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/51811e31..eef816c8
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=13
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=12-13
Stats: 48 lines in 2 files changed: 18 ins; 20 del; 10 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Thu Nov 12 19:44:14 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 12 Nov 2020 19:44:14 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v15]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <eZ_BlLRqLAYNJ7K3HY6olG0a6KuyNynHUItq10ZTJ7I=.00740fd4-edc1-463f-96eb-eafcfd70be3c@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Whitespace fixes
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/eef816c8..03633594
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=14
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=13-14
Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Thu Nov 12 20:14:12 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 12 Nov 2020 20:14:12 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v16]
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <GeOqNKOu1Tbrlhc54Jdh7n41B1sXLKmt0c-0ntaa1nk=.92cc65c3-e125-40d1-90aa-c2a24a91fc06@github.com>
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Asserts against impossible combinations of weak/phantom vs in-native/in-heap
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1109/files
- new: https://git.openjdk.java.net/jdk/pull/1109/files/03633594..c3b1b19a
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=15
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1109&range=14-15
Stats: 5 lines in 3 files changed: 4 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1109.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1109/head:pull/1109
PR: https://git.openjdk.java.net/jdk/pull/1109
From zgu at openjdk.java.net Thu Nov 12 21:41:55 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 12 Nov 2020 21:41:55 GMT
Subject: Integrated: 8256278: Shenandoah: Avoid num of dead callback from weak
processor in Shenandoah root verifier
In-Reply-To: <wx12HCljhSrkQIvIrS-R7K_2Ewb7MEdM48s2RvZvuD8=.9e96d209-fe3e-4df5-a258-fb7684d56845@github.com>
References: <wx12HCljhSrkQIvIrS-R7K_2Ewb7MEdM48s2RvZvuD8=.9e96d209-fe3e-4df5-a258-fb7684d56845@github.com>
Message-ID: <_ECZ9UIGRT2yxohp7KEpgxgeGpEbVyKMjmLZTjNKQC8=.d382b88f-765b-4960-ac86-87d1fb9b0041@github.com>
On Thu, 12 Nov 2020 15:38:10 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> Shenandoah root verifier currently uses WeakProcessor to verify weak oops, but it always generates report_num_dead callback if registered. This callback fails runtime/stringtable/StringTableCleaningTest.java test with -XX:+ShenandoahVerify.
>
> Test:
>
> - [x] hotspot_gc_shenandoah
> - [x] Passed failed test
> TEST_VM_OPTS="-XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:-ShenandoahVerify" make
> CONF=linux-x86_64-server-release run-test TEST=runtime/stringtable/StringTableCleaningTest.java
This pull request has now been integrated.
Changeset: 531c56ea
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/531c56ea
Stats: 3 lines in 1 file changed: 0 ins; 1 del; 2 mod
8256278: Shenandoah: Avoid num of dead callback from weak processor in Shenandoah root verifier
Reviewed-by: rkennke, shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1189
From kemperw at amazon.com Thu Nov 12 21:53:10 2020
From: kemperw at amazon.com (Kemper, William)
Date: Thu, 12 Nov 2020 21:53:10 +0000
Subject: Can the shenandoah visualizer project be moved to GitHub?
Message-ID: <1605217990818.92512@amazon.com>
The project is hosted here: http://icedtea.classpath.org/hg/shenandoah-visualizer/?. Having it on GitHub would make it easier to propose contributions. The project is also missing a LICENSE file, though there is a reference to GPLv2 in a header comment.
Thanks,
William?
From rkennke at redhat.com Thu Nov 12 21:57:54 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 12 Nov 2020 22:57:54 +0100
Subject: Can the shenandoah visualizer project be moved to GitHub?
In-Reply-To: <1605217990818.92512@amazon.com>
References: <1605217990818.92512@amazon.com>
Message-ID: <bdbf27df-96c9-2c4b-49e8-31bdab28cba1@redhat.com>
Hi William,
This is a good idea. I am not even sure if it still works, it's probably
a little bit-rotten by now. I will look into moving it to Github soon.
Thanks,
Roman
> The project is hosted here: http://icedtea.classpath.org/hg/shenandoah-visualizer/?. Having it on GitHub would make it easier to propose contributions. The project is also missing a LICENSE file, though there is a reference to GPLv2 in a header comment.
>
>
> Thanks,
>
> William?
>
From shade at openjdk.java.net Thu Nov 12 22:22:09 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 12 Nov 2020 22:22:09 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
On Fri, 6 Nov 2020 20:00:25 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
While waiting for merge to "adaptive", here is a round of minor things from the first read.
Thanks for updates. I would like to dive in with some testing. Second read nits below, meanwhile.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 135:
> 133: if (available < min_threshold) {
> 134: if (log_is_enabled(Info, gc)) {
> 135: log_info(gc)("Trigger: Free (" SIZE_FORMAT "%s) is below minimum threshold (" SIZE_FORMAT "%s)",
Why do we need these additional checks? Is this to avoid computing the arguments? I was under impression `log_info` and friends do this right already. Are they?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 167:
> 165: // 1. Some space to absorb allocation spikes
> 166: // 2. Accumulated penalties from Degenerated and Full GC
> 167: ShenandoahHeap* heap = ShenandoahHeap::heap();
Does not seem to be used.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 78:
> 76:
> 77: // Track allocation rate even if we decide to start a cycle for other reasons.
> 78: ShenandoahReactiveHeuristics *heuristic = const_cast<ShenandoahReactiveHeuristics *>(this);
Why this cast is needed? Aren't local fields accessible directly, even without `heuristics->`, or `this->`, for that matter?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 219:
> 217: }
> 218:
> 219: static double safe_adjust(double value, double adjustment, double min, double max) {
Seems like this is just:
static double saturate(double value, double min, double max) {
return MAX2(MIN2(value, max), min);
}
...in fact, we probably use this pattern without the explicit method everywhere else.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 236:
> 234: } else if (_last_trigger == SPIKE) {
> 235: adjust_spike_threshold(amount);
> 236: }
Please use this style:
switch(_last_trigger) {
case RATE:
adjust_margin_of_error(amount);
break;
case SPIKE:
adjust_spike_threshold(amount);
break;
case OTHER:
// Do nothing
break;
default:
ShouldNotReachHere();
}
src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 158:
> 156: range(0,1.0) \
> 157: \
> 158: \
Double new-line.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 96:
> 94: // Don't call into our immediate parent class, we've already done
> 95: // everything it would do (and more).
> 96: return ShenandoahHeuristics::should_start_gc(); // NOLINT(bugprone-parent-virtual-call)
We don't do lint (disable) warnings in the source code. I understand it might be useful for bugprone, but we don't do that anyway. This should go away once changes are merged with "adaptive", AFAICS.
src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 130:
> 128: range(0,100) \
> 129: \
> 130: product(uintx, ShenandoahReactiveSampleFrequencyHz, 10, EXPERIMENTAL, \
Minor nit: these are now `ShenandoahAdaptive*`?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 212:
> 210: ShenandoahAdaptiveHeuristics *heuristic = const_cast<ShenandoahAdaptiveHeuristics *>(this);
> 211: heuristic->_allocation_rate.sample(bytes_allocated_since_gc_start);
> 212: heuristic->_last_trigger = OTHER;
Shouldn't `_last_trigger` updates be near `return true;` in this method? I.e. we should not probably overwrite `_last_trigger` if nothing was actually triggered? This probably affects the super-call to `ShenandoahHeuristics::should_start_gc()` as well?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 331:
> 329: }
> 330:
> 331:
Double newline.
-------------
Changes requested by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 12 22:22:06 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 12 Nov 2020 22:22:06 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to missing
load spikes
Message-ID: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
This change adds a "reactive" heuristic for triggering concurrent GC cycles.
The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
The "adaptive" heuristic remains the default.
Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
-------------
Commit messages:
- Merge 'reactive' heuristic into 'adaptive' heuristic
- Incorporate review feed back
- Add a "reactive" heuristic for triggering concurrent GC cycles
Changes: https://git.openjdk.java.net/jdk/pull/1099/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8255984
Stats: 378 lines in 4 files changed: 357 ins; 12 del; 9 mod
Patch: https://git.openjdk.java.net/jdk/pull/1099.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1099/head:pull/1099
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Thu Nov 12 22:22:09 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 12 Nov 2020 22:22:09 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <AqCDASd-Rjud7VX8ITAzgESzuY9kON1uIBnEAgAFbAM=.2bdf3615-3e27-4043-a609-e2b2facdcecc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<AqCDASd-Rjud7VX8ITAzgESzuY9kON1uIBnEAgAFbAM=.2bdf3615-3e27-4043-a609-e2b2facdcecc@github.com>
Message-ID: <2BbeV--DMmaNRrnth78sUbfpAUPUy7kPdZiWQtsN4lc=.28557486-d087-4ce3-ba8d-897305eebd92@github.com>
On Fri, 6 Nov 2020 21:44:10 GMT, Dalibor Topic <robilad at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> Hi,
>
> Please contact me at Dalibor.topic at oracle.com so that I can verify your GitHub account.
>
> Thanks!
This looks interesting. I have to look closer at this, but maybe we should instead fold this to `adaptive`. This would also resolve the testing question: Shenandoah tests usually run with most (all?) heuristics, and so new heuristics needs to be followed with lots of test changes. Folding this to `adaptive` implicitly fixes that.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From robilad at openjdk.java.net Thu Nov 12 22:22:09 2020
From: robilad at openjdk.java.net (Dalibor Topic)
Date: Thu, 12 Nov 2020 22:22:09 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <AqCDASd-Rjud7VX8ITAzgESzuY9kON1uIBnEAgAFbAM=.2bdf3615-3e27-4043-a609-e2b2facdcecc@github.com>
On Fri, 6 Nov 2020 20:00:25 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
Hi,
Please contact me at Dalibor.topic at oracle.com so that I can verify your GitHub account.
Thanks!
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Thu Nov 12 22:22:10 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 12 Nov 2020 22:22:10 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
Message-ID: <wiUn_ibMnBMC6dkx62Qvo2LfvOzwctoa7evm98AFQsg=.e0f54e89-af81-45bb-9dfc-e16654dad433@github.com>
On Wed, 11 Nov 2020 08:14:40 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> Thanks for updates. I would like to dive in with some testing. Second read nits below, meanwhile.
Oh, and change PR title to "8255984: Shenandoah: "adaptive" heuristic is prone to missing load spikes", so that bots know what issue it refers to.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 12 22:22:09 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 12 Nov 2020 22:22:09 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <2BbeV--DMmaNRrnth78sUbfpAUPUy7kPdZiWQtsN4lc=.28557486-d087-4ce3-ba8d-897305eebd92@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<AqCDASd-Rjud7VX8ITAzgESzuY9kON1uIBnEAgAFbAM=.2bdf3615-3e27-4043-a609-e2b2facdcecc@github.com>
<2BbeV--DMmaNRrnth78sUbfpAUPUy7kPdZiWQtsN4lc=.28557486-d087-4ce3-ba8d-897305eebd92@github.com>
Message-ID: <6D5_wivsBMlCvOstXFgwhxIMgP6IJDKQSP9khDcG4xw=.30514170-9ac9-4520-b1f7-b3efcc18a5e6@github.com>
On Sun, 8 Nov 2020 10:37:38 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Hi,
>>
>> Please contact me at Dalibor.topic at oracle.com so that I can verify your GitHub account.
>>
>> Thanks!
>
> This looks interesting. I have to look closer at this, but maybe we should instead fold this to `adaptive`. This would also resolve the testing question: Shenandoah tests usually run with most (all?) heuristics, and so new heuristics needs to be followed with lots of test changes. Folding this to `adaptive` implicitly fixes that.
I think folding it in to `adaptive` makes sense. I went the conservative approach originally just to keep the changes isolated. I'll prepare a commit with these changes built directly in the `adaptive` heuristic.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 12 22:22:13 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 12 Nov 2020 22:22:13 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
Message-ID: <XoLx7ruw3OSAgcx-glyd0B9PUkMJ_0_U-o_sVKgZ5dc=.08a75392-a48e-43e2-a661-5c10c808f341@github.com>
On Tue, 10 Nov 2020 17:30:40 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 135:
>
>> 133: if (available < min_threshold) {
>> 134: if (log_is_enabled(Info, gc)) {
>> 135: log_info(gc)("Trigger: Free (" SIZE_FORMAT "%s) is below minimum threshold (" SIZE_FORMAT "%s)",
>
> Why do we need these additional checks? Is this to avoid computing the arguments? I was under impression `log_info` and friends do this right already. Are they?
You're right, the `log_info` macro performs this check before evaluating the arguments. I'll fix this.
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 167:
>
>> 165: // 1. Some space to absorb allocation spikes
>> 166: // 2. Accumulated penalties from Degenerated and Full GC
>> 167: ShenandoahHeap* heap = ShenandoahHeap::heap();
>
> Does not seem to be used.
Used on line 180 - I will inline the variable where it's used.
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 78:
>
>> 76:
>> 77: // Track allocation rate even if we decide to start a cycle for other reasons.
>> 78: ShenandoahReactiveHeuristics *heuristic = const_cast<ShenandoahReactiveHeuristics *>(this);
>
> Why this cast is needed? Aren't local fields accessible directly, even without `heuristics->`, or `this->`, for that matter?
`should_start_gc` is declared `const`, so it's cast away here. I didn't want to change the signature of `should_start_gc`, but if it were not `const` we could get rid of the cast here.
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 219:
>
>> 217: }
>> 218:
>> 219: static double safe_adjust(double value, double adjustment, double min, double max) {
>
> Seems like this is just:
>
> static double saturate(double value, double min, double max) {
> return MAX2(MIN2(value, max), min);
> }
>
> ...in fact, we probably use this pattern without the explicit method everywhere else.
Refactored this to the pattern you suggested.
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 236:
>
>> 234: } else if (_last_trigger == SPIKE) {
>> 235: adjust_spike_threshold(amount);
>> 236: }
>
> Please use this style:
>
> switch(_last_trigger) {
> case RATE:
> adjust_margin_of_error(amount);
> break;
> case SPIKE:
> adjust_spike_threshold(amount);
> break;
> case OTHER:
> // Do nothing
> break;
> default:
> ShouldNotReachHere();
> }
Done.
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 96:
>
>> 94: // Don't call into our immediate parent class, we've already done
>> 95: // everything it would do (and more).
>> 96: return ShenandoahHeuristics::should_start_gc(); // NOLINT(bugprone-parent-virtual-call)
>
> We don't do lint (disable) warnings in the source code. I understand it might be useful for bugprone, but we don't do that anyway. This should go away once changes are merged with "adaptive", AFAICS.
Yes, this is gone now.
> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 130:
>
>> 128: range(0,100) \
>> 129: \
>> 130: product(uintx, ShenandoahReactiveSampleFrequencyHz, 10, EXPERIMENTAL, \
>
> Minor nit: these are now `ShenandoahAdaptive*`?
Yes, will fix this.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Thu Nov 12 22:22:14 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 12 Nov 2020 22:22:14 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <XoLx7ruw3OSAgcx-glyd0B9PUkMJ_0_U-o_sVKgZ5dc=.08a75392-a48e-43e2-a661-5c10c808f341@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
<XoLx7ruw3OSAgcx-glyd0B9PUkMJ_0_U-o_sVKgZ5dc=.08a75392-a48e-43e2-a661-5c10c808f341@github.com>
Message-ID: <GMs06O2QAhMvknNzk4dX43xGdtEADW2YeaCCI9aAFxM=.58101b4f-2a51-4795-a240-2a745ee5d6b9@github.com>
On Tue, 10 Nov 2020 20:10:29 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahReactiveHeuristics.cpp line 78:
>>
>>> 76:
>>> 77: // Track allocation rate even if we decide to start a cycle for other reasons.
>>> 78: ShenandoahReactiveHeuristics *heuristic = const_cast<ShenandoahReactiveHeuristics *>(this);
>>
>> Why this cast is needed? Aren't local fields accessible directly, even without `heuristics->`, or `this->`, for that matter?
>
> `should_start_gc` is declared `const`, so it's cast away here. I didn't want to change the signature of `should_start_gc`, but if it were not `const` we could get rid of the cast here.
Right. Let's drop the `const` from declarations then!
>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 167:
>>
>>> 165: // 1. Some space to absorb allocation spikes
>>> 166: // 2. Accumulated penalties from Degenerated and Full GC
>>> 167: ShenandoahHeap* heap = ShenandoahHeap::heap();
>>
>> Does not seem to be used.
>
> Used on line 180 - I will inline the variable where it's used.
Right. Ignore this comment then.
>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 135:
>>
>>> 133: if (available < min_threshold) {
>>> 134: if (log_is_enabled(Info, gc)) {
>>> 135: log_info(gc)("Trigger: Free (" SIZE_FORMAT "%s) is below minimum threshold (" SIZE_FORMAT "%s)",
>>
>> Why do we need these additional checks? Is this to avoid computing the arguments? I was under impression `log_info` and friends do this right already. Are they?
>
> You're right, the `log_info` macro performs this check before evaluating the arguments. I'll fix this.
Good! (please click "Resolve conversation")
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 12 22:25:57 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 12 Nov 2020 22:25:57 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
Message-ID: <MbNZpDJESiGogMy13ATN-HbhaF-DpzhzOpYj1Oi-YJg=.fa744a6d-6ffa-44da-9dc4-de9945a80208@github.com>
On Wed, 11 Nov 2020 08:12:50 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 212:
>
>> 210: ShenandoahAdaptiveHeuristics *heuristic = const_cast<ShenandoahAdaptiveHeuristics *>(this);
>> 211: heuristic->_allocation_rate.sample(bytes_allocated_since_gc_start);
>> 212: heuristic->_last_trigger = OTHER;
>
> Shouldn't `_last_trigger` updates be near `return true;` in this method? I.e. we should not probably overwrite `_last_trigger` if nothing was actually triggered? This probably affects the super-call to `ShenandoahHeuristics::should_start_gc()` as well?
Would it make more sense if I `OTHER` were `NONE`? The intention here is to only adjust the allocation rate or spike detection parameters if they triggered the cycle. Here we reset the `_last_trigger` in one place without having to touch all the actual decision points. As it stands, once a gc cycle is started, the control thread won't call back into this method until the cycle is complete.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 12 22:57:12 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 12 Nov 2020 22:57:12 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
- Remove const qualifier from should_start_gc
This lets the heuristics update state without const_casts.
- Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1099/files
- new: https://git.openjdk.java.net/jdk/pull/1099/files/7f1de19b..275e80a2
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=00-01
Stats: 31 lines in 13 files changed: 0 ins; 2 del; 29 mod
Patch: https://git.openjdk.java.net/jdk/pull/1099.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1099/head:pull/1099
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 00:24:58 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 00:24:58 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <wiUn_ibMnBMC6dkx62Qvo2LfvOzwctoa7evm98AFQsg=.e0f54e89-af81-45bb-9dfc-e16654dad433@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
<wiUn_ibMnBMC6dkx62Qvo2LfvOzwctoa7evm98AFQsg=.e0f54e89-af81-45bb-9dfc-e16654dad433@github.com>
Message-ID: <2xZoPzvwpiECqeQ68zs7qbESTBmXOOeJjJi2vfLUm_w=.89ec8176-ad52-4c38-a7c9-319b81a55be7@github.com>
On Wed, 11 Nov 2020 08:15:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Thanks for updates. I would like to dive in with some testing. Second read nits below, meanwhile.
>
> Oh, and change PR title to "8255984: Shenandoah: "adaptive" heuristic is prone to missing load spikes", so that bots know what issue it refers to.
Though we tested this with several benchmarks and internal services, these changes were primarily developed using the [HyperAlloc](https://github.com/corretto/heapothesys/tree/master/HyperAlloc) benchmark. The attached chart shows stacked histograms for the heuristics that triggered a collection cycle during a run of HyperAlloc.
Each test run was given a 3g heap. HyperAlloc targeted a 1g/s allocation rate (-a 1000) with a max object size of 1000 bytes (-x 1000). It was configured to maintain 1gb of live objects on the heap (-s 1024) and it ran for two minutes (-d 120). Each row in the chart is for a different _allocation smoothness factor_. The top row has an allocation smoothness of zero (i.e., most spiky), the bottom row has an allocation smoothness of 0.4, still spiky, but manageable. Each column in the chart is a different heuristic. On the left is the original `adaptive` heuristic (with default parameters). On the far right is the `compact` heuristic. In the middle are runs of the original `reactive` heuristic with different sensitivities to allocations spikes. The purple bars are allocation failures. The light blue bars are instances where an allocation spike was detected and triggered a cycle.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From thartmann at openjdk.java.net Fri Nov 13 06:28:56 2020
From: thartmann at openjdk.java.net (Tobias Hartmann)
Date: Fri, 13 Nov 2020 06:28:56 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah [v3]
In-Reply-To: <jG1N-2cWJDmoApu-Frm4zsPgBDpmhyyS4cH_FPPsmnk=.4dba7ebe-b038-47a3-b906-252a5f23f084@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
<jG1N-2cWJDmoApu-Frm4zsPgBDpmhyyS4cH_FPPsmnk=.4dba7ebe-b038-47a3-b906-252a5f23f084@github.com>
Message-ID: <NtrgJb8ccO1xTY7arW5IG5IkemX5vm5_U06bwl5ZwW0=.2219e475-3977-41dc-a891-eb2945a66ca2@github.com>
On Thu, 12 Nov 2020 15:38:06 GMT, Roland Westrelin <roland at openjdk.org> wrote:
>> This is a Shenandoah bug but the proposed fix is in shared code.
>>
>> In an infinite loop, a barrier is located right after the loop head
>> and above the never branch. When the barrier is expanded, control flow
>> is added between the loop and the never branch. During loop
>> verification the assert fires because it doesn't expect any control
>> flow between the never branch and the loop head.
>>
>> While it would have been nice to fix this Shenandoah issue in
>> Shenandoah code, I think the cleaner fix is to preserve the invariant
>> that the never branch is always right after the loop head in an
>> infinite loop. In the proposed patch, this is achieved by moving all
>> uses of the loop head to the never branch when it's constructed.
>
> Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits:
>
> - review
> - fix & test
Marked as reviewed by thartmann (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From thartmann at openjdk.java.net Fri Nov 13 06:28:57 2020
From: thartmann at openjdk.java.net (Tobias Hartmann)
Date: Fri, 13 Nov 2020 06:28:57 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah
In-Reply-To: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
Message-ID: <UkJ0_VHY7PRx0k4hRhtunAWTP-QegGPIFhs1pdmvJDs=.f76ea562-4173-429e-b2fd-16dd7ebc7715@github.com>
On Thu, 5 Nov 2020 08:44:03 GMT, Roland Westrelin <roland at openjdk.org> wrote:
> This is a Shenandoah bug but the proposed fix is in shared code.
>
> In an infinite loop, a barrier is located right after the loop head
> and above the never branch. When the barrier is expanded, control flow
> is added between the loop and the never branch. During loop
> verification the assert fires because it doesn't expect any control
> flow between the never branch and the loop head.
>
> While it would have been nice to fix this Shenandoah issue in
> Shenandoah code, I think the cleaner fix is to preserve the invariant
> that the never branch is always right after the loop head in an
> infinite loop. In the proposed patch, this is achieved by moving all
> uses of the loop head to the never branch when it's constructed.
@rwestrel please don't use force-push because it prevents getting incremental changes in the PR ("New changes since you last viewed").
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From xliu at openjdk.java.net Fri Nov 13 06:56:55 2020
From: xliu at openjdk.java.net (Xin Liu)
Date: Fri, 13 Nov 2020 06:56:55 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah
In-Reply-To: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
Message-ID: <5gcJ-y5dzLhf1AAx2Hl4I5Lo3EBbu6zvdia5d_qzuAU=.c0fcfef9-6429-4481-bb7d-9a43227476ef@github.com>
On Thu, 5 Nov 2020 08:44:03 GMT, Roland Westrelin <roland at openjdk.org> wrote:
> This is a Shenandoah bug but the proposed fix is in shared code.
>
> In an infinite loop, a barrier is located right after the loop head
> and above the never branch. When the barrier is expanded, control flow
> is added between the loop and the never branch. During loop
> verification the assert fires because it doesn't expect any control
> flow between the never branch and the loop head.
>
> While it would have been nice to fix this Shenandoah issue in
> Shenandoah code, I think the cleaner fix is to preserve the invariant
> that the never branch is always right after the loop head in an
> infinite loop. In the proposed patch, this is achieved by moving all
> uses of the loop head to the never branch when it's constructed.
hi, @rwestrel
Can we have an assertion to make this loop invariance more prominent?
`I think the cleaner fix is to preserve the invariant that the never branch is always right after the loop head in an infinite loop. `
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From shade at openjdk.java.net Fri Nov 13 07:59:56 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 13 Nov 2020 07:59:56 GMT
Subject: RFR: 8256011: Shenandoah: Don't resurrect finalizably reachable
objects [v16]
In-Reply-To: <GeOqNKOu1Tbrlhc54Jdh7n41B1sXLKmt0c-0ntaa1nk=.92cc65c3-e125-40d1-90aa-c2a24a91fc06@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
<GeOqNKOu1Tbrlhc54Jdh7n41B1sXLKmt0c-0ntaa1nk=.92cc65c3-e125-40d1-90aa-c2a24a91fc06@github.com>
Message-ID: <BlNdzGvEHn-GPAMIkV2z9EL0ksI24f_dPw6n6WcQn44=.e7fb76b0-dc68-4082-924d-7c9c0d74f81f@github.com>
On Thu, 12 Nov 2020 20:14:12 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>>
>> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
>> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
>> - We can have strong and phantom native referents, and strong and weak in-heap referents
>> - Native referents are never compressed
>>
>> Note that this depends on PR#1140.
>>
>> Testing:
>> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
>> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
>> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Asserts against impossible combinations of weak/phantom vs in-native/in-heap
Okay, looks good.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1109
From rkennke at openjdk.java.net Fri Nov 13 09:49:56 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 13 Nov 2020 09:49:56 GMT
Subject: Integrated: 8256011: Shenandoah: Don't resurrect finalizably
reachable objects
In-Reply-To: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
References: <WxTxIJIJpFG-7qaoVeRhsoQaCDTZHHlyo97xFZOf9ms=.279a5ff8-257f-4c1d-b72a-18c92aa21c19@github.com>
Message-ID: <SpM-toeEEW4bXPmJV6boLk9fs7I0aTwOcoD5qfhthNU=.87247beb-b4c0-4212-9a8d-cdf387a7d330@github.com>
On Sat, 7 Nov 2020 20:10:14 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> In the weak-LRB we currently return referents when it is 'marked', that is when it's either reachable strongly or through a finalizable object. This means a finalizable object can be resurrected by Reference.get(), which is wrong. Only truly strongly reachable objects should be returned by Reference.get() during weak-reference-processing.
>
> I had to reconsider the way we call into runtime-LRBs from generated code for these reasons:
> - We need to distinguish phantom, weak and strong reference strength, and native vs in-heap access. Those are two orthogonal dimensions
> - We can have strong and phantom native referents, and strong and weak in-heap referents
> - Native referents are never compressed
>
> Note that this depends on PR#1140.
>
> Testing:
> - [x] hotspot_gc_shenandoah (x86_64, x64_32, aarch64)
> - [x] tier1 +UseShenandoahGC +ShenandoahVerify
> - [x] tier2 +UseShenandoahGC +ShenandoahVerify
This pull request has now been integrated.
Changeset: b0c28fad
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/b0c28fad
Stats: 390 lines in 16 files changed: 107 ins; 87 del; 196 mod
8256011: Shenandoah: Don't resurrect finalizably reachable objects
Reviewed-by: shade, zgu
-------------
PR: https://git.openjdk.java.net/jdk/pull/1109
From mathiske at amazon.com Fri Nov 13 13:01:52 2020
From: mathiske at amazon.com (Mathiske, Bernd)
Date: Fri, 13 Nov 2020 13:01:52 +0000
Subject: Can the shenandoah visualizer project be moved to GitHub?
In-Reply-To: <bdbf27df-96c9-2c4b-49e8-31bdab28cba1@redhat.com>
References: <1605217990818.92512@amazon.com>
<bdbf27df-96c9-2c4b-49e8-31bdab28cba1@redhat.com>
Message-ID: <8C4597FC-EE02-4B27-A77E-6EFB7C3BF0E8@amazon.com>
It works and we like it very much. Soon, we will have enhancements that show how regions age and transition between generations if running with "-XX::ShenandoahGCMode=generational" in a GenShen prototype...
?On 11/12/20, 10:58 PM, "shenandoah-dev on behalf of Roman Kennke" <shenandoah-dev-retn at openjdk.java.net on behalf of rkennke at redhat.com> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hi William,
This is a good idea. I am not even sure if it still works, it's probably
a little bit-rotten by now. I will look into moving it to Github soon.
Thanks,
Roman
> The project is hosted here: http://icedtea.classpath.org/hg/shenandoah-visualizer/?. Having it on GitHub would make it easier to propose contributions. The project is also missing a LICENSE file, though there is a reference to GPLv2 in a header comment.
>
>
> Thanks,
>
> William?
>
From rkennke at redhat.com Fri Nov 13 14:43:28 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 13 Nov 2020 15:43:28 +0100
Subject: Can the shenandoah visualizer project be moved to GitHub?
In-Reply-To: <8C4597FC-EE02-4B27-A77E-6EFB7C3BF0E8@amazon.com>
References: <1605217990818.92512@amazon.com>
<bdbf27df-96c9-2c4b-49e8-31bdab28cba1@redhat.com>
<8C4597FC-EE02-4B27-A77E-6EFB7C3BF0E8@amazon.com>
Message-ID: <d0861044-b721-3361-212d-96c8ce4461dd@redhat.com>
Hi Bernd,
This is great news!
I have asked ops@ to create a new repository under github.com/openjdk, I
expect that it shows up within the next few days, then I need to migrate
the code over to there, and we should be ready to go (unless we get some
unnecessary red tape in our way, in which case I'd put it someplace else)
Very much looking forward to see your prototype in action!
How is progress with GenShen prototype? I haven't seen/heard of you
lately... ;-)
Cheers,
Roman
> It works and we like it very much. Soon, we will have enhancements that show how regions age and transition between generations if running with "-XX::ShenandoahGCMode=generational" in a GenShen prototype...
>
> ?On 11/12/20, 10:58 PM, "shenandoah-dev on behalf of Roman Kennke" <shenandoah-dev-retn at openjdk.java.net on behalf of rkennke at redhat.com> wrote:
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> Hi William,
>
> This is a good idea. I am not even sure if it still works, it's probably
> a little bit-rotten by now. I will look into moving it to Github soon.
>
> Thanks,
> Roman
>
>
> > The project is hosted here: http://icedtea.classpath.org/hg/shenandoah-visualizer/?. Having it on GitHub would make it easier to propose contributions. The project is also missing a LICENSE file, though there is a reference to GPLv2 in a header comment.
> >
> >
> > Thanks,
> >
> > William?
> >
>
>
From zgu at openjdk.java.net Fri Nov 13 15:20:08 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 13 Nov 2020 15:20:08 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v9]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <ZrfnDs0f5fXrVIjt5a3bIEbCNS9N83kQjF0bmAAmGhk=.8a01d20b-0595-47f2-9599-7a2c14ef5fef@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits:
- Merge branch 'master' into JDK-8255019-sh-mark
- Removed obsoleted class
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Move weak reference processing out of STWMark and fix its timings
- Merge branch 'JDK-8255019-sh-mark' of github.com:zhengyu123/jdk into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- ... and 5 more: https://git.openjdk.java.net/jdk/compare/1c47244b...43224908
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=08
Stats: 1969 lines in 21 files changed: 1072 ins; 738 del; 159 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From mathiske at amazon.com Fri Nov 13 15:57:59 2020
From: mathiske at amazon.com (Mathiske, Bernd)
Date: Fri, 13 Nov 2020 15:57:59 +0000
Subject: Can the shenandoah visualizer project be moved to GitHub?
In-Reply-To: <d0861044-b721-3361-212d-96c8ce4461dd@redhat.com>
References: <1605217990818.92512@amazon.com>
<bdbf27df-96c9-2c4b-49e8-31bdab28cba1@redhat.com>
<8C4597FC-EE02-4B27-A77E-6EFB7C3BF0E8@amazon.com>
<d0861044-b721-3361-212d-96c8ce4461dd@redhat.com>
Message-ID: <5B75F37A-C649-4374-813B-EF66BE1AF590@amazon.com>
Hi Roman,
I figured you were more than busy given the emails I see flying by. (
We plan to set up a little demo script so you can reproduce the visualization.
With milestone 3 (tenuring and promotion), we ran into some bugs, still spending a lot of time on debugging what happens to some promoted objects. See the code in question in my fork, in branch "Milestone-3" (https://github.com/bernd-aws/shenandoah/tree/Milestone-3).
BTW, would you have time to accept another PR (https://github.com/openjdk/shenandoah/pull/5) into the genshen branch of your repo? I would keep them coming... (
Best,
Bernd
?On 11/13/20, 3:43 PM, "Roman Kennke" <rkennke at redhat.com> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hi Bernd,
This is great news!
I have asked ops@ to create a new repository under github.com/openjdk, I
expect that it shows up within the next few days, then I need to migrate
the code over to there, and we should be ready to go (unless we get some
unnecessary red tape in our way, in which case I'd put it someplace else)
Very much looking forward to see your prototype in action!
How is progress with GenShen prototype? I haven't seen/heard of you
lately... ;-)
Cheers,
Roman
> It works and we like it very much. Soon, we will have enhancements that show how regions age and transition between generations if running with "-XX::ShenandoahGCMode=generational" in a GenShen prototype...
>
> On 11/12/20, 10:58 PM, "shenandoah-dev on behalf of Roman Kennke" <shenandoah-dev-retn at openjdk.java.net on behalf of rkennke at redhat.com> wrote:
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> Hi William,
>
> This is a good idea. I am not even sure if it still works, it's probably
> a little bit-rotten by now. I will look into moving it to Github soon.
>
> Thanks,
> Roman
>
>
> > The project is hosted here: http://icedtea.classpath.org/hg/shenandoah-visualizer/?. Having it on GitHub would make it easier to propose contributions. The project is also missing a LICENSE file, though there is a reference to GPLv2 in a header comment.
> >
> >
> > Thanks,
> >
> > William?
> >
>
>
From shade at openjdk.java.net Fri Nov 13 17:49:15 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 13 Nov 2020 17:49:15 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
Message-ID: <wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
On Thu, 12 Nov 2020 22:57:12 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>
> - Remove const qualifier from should_start_gc
>
> This lets the heuristics update state without const_casts.
> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
The more I study this patch, the more I like it, good job. Another round of changes below. I fixed a few mentioned issues in my private working copy while doing initial performance testing. Would be good to have an updated PR for testing too.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 367:
> 365: _last_sample_time = os::javaTimeNanos();
> 366: _last_sample_value = 0;
> 367: }
This method seems unused.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 383:
> 381:
> 382: double ShenandoahAllocationRate::instantaneous_rate(size_t bytes_allocated_since_gc_start) const {
> 383: size_t allocation_delta = bytes_allocated_since_gc_start - _last_sample_value;
`allocation_delta` underflows here in my benchmarks, causing back-to-back "Instantaneous rate" triggers. Note that `sample()` method seems to check for it.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 293:
> 291: double instantaneous_rate = _allocation_rate.instantaneous_rate(bytes_allocated_since_gc_start);
> 292: if (_allocation_rate.is_spiking(instantaneous_rate) && average_cycle_seconds > allocation_headroom / instantaneous_rate) {
> 293: log_info(gc)("Trigger: Instantaneous allocation rate (%.0f %sB/s) will deplete free headroom (" SIZE_FORMAT "%s) before average time (%.2f ms) to complete GC cycle.",
If you compare the message with previous trigger, you'll probably notice has the different order of printed values: gc time, allocation rate, head room. This one should have the same order. You can change the previous trigger to read:
log_info(gc)("Trigger: Average GC time (%.2f ms) is above the time for average allocation rate (%.0f %sB/s) to deplete free headroom (" SIZE_FORMAT "%s) (margin of error = %.2f)"
...and this one to read:
log_info(gc)("Trigger: Average GC time (%.2f ms) is above the time for instantaneous allocation rate (%.0f %sB/s) to deplete free headroom (" SIZE_FORMAT "%s) (margin of error = %.2f)",
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 332:
> 330: ShenandoahAllocationRate::ShenandoahAllocationRate(ShenandoahAdaptiveHeuristics *heuristics) :
> 331: _heuristics(heuristics),
> 332: _last_sample_time(os::javaTimeNanos()),
My IDE complains that `size_t` and `jlong` are not compatible here. That might be the problem for x86_32. Note that existing heuristics code uses `os::elapsedTime()`, which returns `double`. Is there a reason not to use it?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 223:
> 221: if (is_allocation_rate_too_high(capacity, available, bytes_allocated_since_gc_start)) {
> 222: return true;
> 223: }
Does this split really win us anything readability-wise? I inlined the methods back, and the method is still quite readable, and does not require tracking `return`-s and pushing local variables there. I wonder if that is just a left-over from adaptive/reactive split.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 266:
> 264: size_t allocation_headroom = available;
> 265:
> 266: size_t spike_headroom = capacity / 100 * ShenandoahAllocSpikeFactor;
I suspect `ShenandoahAllocSpikeFactor` handling is not needed anymore, as this new code is supposed to provide the similar cushion of few sigmas off the average value. That was my poor-mans solution to the same problem this patch resolves. You can remove the flag and update the comments in this block to mention only accumulated penalties.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 130:
> 128: ShenandoahHeuristics::record_success_concurrent();
> 129:
> 130: double available = ShenandoahHeap::heap()->free_set()->available();
This `ShenandoahFreeSet::available()` returns `size_t`. If you keep it `size_t` as local variable, then you would not need to convert it back to `size_t` below. Plus, implicit conversions to `double` would happen in the computation in the next block.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 345:
> 343: size_t allocation_delta = bytes_allocated_since_gc_start - _last_sample_value;
> 344: size_t time_delta_ns = now - _last_sample_time;
> 345: double alloc_bytes_per_second = ((double) allocation_delta * NANOUNITS) / time_delta_ns;
I wonder if you can just call `instantaneous_rate()` here. Note the comment about that method too.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 207:
> 205: available = (available > soft_tail) ? (available - soft_tail) : 0;
> 206:
> 207: size_t bytes_allocated_since_gc_start = heap->bytes_allocated_since_gc_start();
Name this local variable `allocated`, it unclutters some code.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 272:
> 270: allocation_headroom -= MIN2(allocation_headroom, penalties);
> 271:
> 272: double average_cycle_seconds = _gc_time_history->davg() + (_margin_of_error_sd * _gc_time_history->dsd());
Name this local variable `avg_cycle_time`, makes it easier to read.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 273:
> 271:
> 272: double average_cycle_seconds = _gc_time_history->davg() + (_margin_of_error_sd * _gc_time_history->dsd());
> 273: double bytes_allocated_per_second = _allocation_rate.upper_bound(_margin_of_error_sd);
Name this local variable `avg_alloc_rate`.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 291:
> 289: }
> 290:
> 291: double instantaneous_rate = _allocation_rate.instantaneous_rate(bytes_allocated_since_gc_start);
Name this variable `instant_alloc_rate`.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 270:
> 268:
> 269: allocation_headroom -= MIN2(allocation_headroom, spike_headroom);
> 270: allocation_headroom -= MIN2(allocation_headroom, penalties);
...in fact, maybe even penalties are not needed since we are widening the SD "windows" on degenerated/full GC? We can clean this up later if we are not sure.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 369:
> 367: }
> 368:
> 369: bool ShenandoahAllocationRate::is_spiking(double instantaneous_rate) const {
Name the argument just `rate`?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 370:
> 368:
> 369: bool ShenandoahAllocationRate::is_spiking(double instantaneous_rate) const {
> 370: double standard_deviation = _rate.sd();
Name it `sd`, that is pretty conventional.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 386:
> 384: size_t time_delta_ns = os::javaTimeNanos() - _last_sample_time;
> 385: double alloc_bytes_per_second = ((double) allocation_delta * NANOUNITS) / time_delta_ns;
> 386: return alloc_bytes_per_second;
Inline local variable here.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 382:
> 380: }
> 381:
> 382: double ShenandoahAllocationRate::instantaneous_rate(size_t bytes_allocated_since_gc_start) const {
Name the argument just `allocated`?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 339:
> 337: }
> 338:
> 339: void ShenandoahAllocationRate::sample(size_t bytes_allocated_since_gc_start) {
Name the argument just `allocated`?
-------------
Changes requested by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Fri Nov 13 18:02:57 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 13 Nov 2020 18:02:57 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
<wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
Message-ID: <O90h88zBB_L5N7OHa41WKEHBKZZRH03ePSaLcJox6Ts=.3d12b818-3a88-4f90-94eb-d56b28a0ffc0@github.com>
On Fri, 13 Nov 2020 17:46:09 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Remove const qualifier from should_start_gc
>>
>> This lets the heuristics update state without const_casts.
>> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
>
> The more I study this patch, the more I like it, good job. Another round of changes below. I fixed a few mentioned issues in my private working copy while doing initial performance testing. Would be good to have an updated PR for testing too.
> 
Yes, it mostly matches what I would expect from this heuristic improvement. As it stands right now, "adaptive" balances way too closely to the cliff, and so it is expected that improvement would make GC cycles more frequent to move GC triggers away from it. On the chart, "Headroom" probably means "Average allocation rate" trigger? It still a bit sad we cannot handle `smoothness=0.0`, but that probably requires future-predicting capabilities not yet available at current technology level.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From rkennke at redhat.com Fri Nov 13 18:36:13 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 13 Nov 2020 19:36:13 +0100
Subject: Can the shenandoah visualizer project be moved to GitHub?
In-Reply-To: <5B75F37A-C649-4374-813B-EF66BE1AF590@amazon.com>
References: <1605217990818.92512@amazon.com>
<bdbf27df-96c9-2c4b-49e8-31bdab28cba1@redhat.com>
<8C4597FC-EE02-4B27-A77E-6EFB7C3BF0E8@amazon.com>
<d0861044-b721-3361-212d-96c8ce4461dd@redhat.com>
<5B75F37A-C649-4374-813B-EF66BE1AF590@amazon.com>
Message-ID: <9aa98a33-941b-533d-5baf-dcf187c43721@redhat.com>
Hi Bernd,
> Hi Roman,
>
> I figured you were more than busy given the emails I see flying by. (
>
> We plan to set up a little demo script so you can reproduce the visualization.
Very nice!
> With milestone 3 (tenuring and promotion), we ran into some bugs, still spending a lot of time on debugging what happens to some promoted objects. See the code in question in my fork, in branch "Milestone-3" (https://github.com/bernd-aws/shenandoah/tree/Milestone-3).
I will check it out!
> BTW, would you have time to accept another PR (https://github.com/openjdk/shenandoah/pull/5) into the genshen branch of your repo? I would keep them coming... (
Aww, why haven't I seen this? You should better ping me when you file a
PR and I don't respond within 24hours. I'll have a look now!
Thanks,
Roman
> Best,
> Bernd
>
> ?On 11/13/20, 3:43 PM, "Roman Kennke" <rkennke at redhat.com> wrote:
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> Hi Bernd,
>
> This is great news!
>
> I have asked ops@ to create a new repository under github.com/openjdk, I
> expect that it shows up within the next few days, then I need to migrate
> the code over to there, and we should be ready to go (unless we get some
> unnecessary red tape in our way, in which case I'd put it someplace else)
>
> Very much looking forward to see your prototype in action!
>
> How is progress with GenShen prototype? I haven't seen/heard of you
> lately... ;-)
>
> Cheers,
> Roman
>
> > It works and we like it very much. Soon, we will have enhancements that show how regions age and transition between generations if running with "-XX::ShenandoahGCMode=generational" in a GenShen prototype...
> >
> > On 11/12/20, 10:58 PM, "shenandoah-dev on behalf of Roman Kennke" <shenandoah-dev-retn at openjdk.java.net on behalf of rkennke at redhat.com> wrote:
> >
> > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> >
> >
> >
> > Hi William,
> >
> > This is a good idea. I am not even sure if it still works, it's probably
> > a little bit-rotten by now. I will look into moving it to Github soon.
> >
> > Thanks,
> > Roman
> >
> >
> > > The project is hosted here: http://icedtea.classpath.org/hg/shenandoah-visualizer/?. Having it on GitHub would make it easier to propose contributions. The project is also missing a LICENSE file, though there is a reference to GPLv2 in a header comment.
> > >
> > >
> > > Thanks,
> > >
> > > William?
> > >
> >
> >
>
>
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 18:49:58 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 18:49:58 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
<wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
Message-ID: <-HxUTCCA29AvKGiJBERmvvYJehetuC6Ouoe4qvVuGkA=.9205cd31-0fd1-415d-8d32-d32457629f58@github.com>
On Fri, 13 Nov 2020 17:05:46 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Remove const qualifier from should_start_gc
>>
>> This lets the heuristics update state without const_casts.
>> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 367:
>
>> 365: _last_sample_time = os::javaTimeNanos();
>> 366: _last_sample_value = 0;
>> 367: }
>
> This method seems unused.
Oops, good catch. This was meant to be called from `record_cycle_start`, lost it when I merged reactive into adaptive. Will push the fix soon.
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 383:
>
>> 381:
>> 382: double ShenandoahAllocationRate::instantaneous_rate(size_t bytes_allocated_since_gc_start) const {
>> 383: size_t allocation_delta = bytes_allocated_since_gc_start - _last_sample_value;
>
> `allocation_delta` underflows here in my benchmarks, causing back-to-back "Instantaneous rate" triggers. Note that `sample()` method seems to check for it.
Yep, that's because I fumbled the `allocation_counter_reset` call.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 19:01:58 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 19:01:58 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
<wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
Message-ID: <7SWlZHdENzqYnxVCySAOms98pZgQ_Ks_TxGCqNqMSYQ=.f6b02dd3-4558-4752-8b26-d1e2a749578e@github.com>
On Fri, 13 Nov 2020 17:11:56 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Remove const qualifier from should_start_gc
>>
>> This lets the heuristics update state without const_casts.
>> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 293:
>
>> 291: double instantaneous_rate = _allocation_rate.instantaneous_rate(bytes_allocated_since_gc_start);
>> 292: if (_allocation_rate.is_spiking(instantaneous_rate) && average_cycle_seconds > allocation_headroom / instantaneous_rate) {
>> 293: log_info(gc)("Trigger: Instantaneous allocation rate (%.0f %sB/s) will deplete free headroom (" SIZE_FORMAT "%s) before average time (%.2f ms) to complete GC cycle.",
>
> If you compare the message with previous trigger, you'll probably notice has the different order of printed values: gc time, allocation rate, head room. This one should have the same order. You can change the previous trigger to read:
>
> log_info(gc)("Trigger: Average GC time (%.2f ms) is above the time for average allocation rate (%.0f %sB/s) to deplete free headroom (" SIZE_FORMAT "%s) (margin of error = %.2f)"
>
> ...and this one to read:
>
> log_info(gc)("Trigger: Average GC time (%.2f ms) is above the time for instantaneous allocation rate (%.0f %sB/s) to deplete free headroom (" SIZE_FORMAT "%s) (margin of error = %.2f)",
Done.
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 223:
>
>> 221: if (is_allocation_rate_too_high(capacity, available, bytes_allocated_since_gc_start)) {
>> 222: return true;
>> 223: }
>
> Does this split really win us anything readability-wise? I inlined the methods back, and the method is still quite readable, and does not require tracking `return`-s and pushing local variables there. I wonder if that is just a left-over from adaptive/reactive split.
The change was certainly originally motivated by having these changes in a separate heuristic. I don't think it hurts readability, but I'll inline them back in for the sake of performance.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From rkennke at openjdk.java.net Fri Nov 13 20:38:10 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 13 Nov 2020 20:38:10 GMT
Subject: RFR: Card marking write barrier for generational mode.
In-Reply-To: <UXC_VSsfsTWf0X1qPkKI-ywQJ2zLyrBYgxePAZ9-beY=.ff77d174-a867-4c29-a161-1dfb7aa59ac2@github.com>
References: <UXC_VSsfsTWf0X1qPkKI-ywQJ2zLyrBYgxePAZ9-beY=.ff77d174-a867-4c29-a161-1dfb7aa59ac2@github.com>
Message-ID: <HsAeQzM4uLl3F8F8OQQE5pGP5C7isvReugE6oW8nIAM=.eca8cba4-bc07-420d-a161-ae338fefc0f8@github.com>
On Tue, 27 Oct 2020 12:00:28 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:
> Adding a typical generational card marking write barrier, which is only active in generational mode and has no other effect yet than writing into a card table that is allocated at startup on the side and covers the whole heap as specified by -Xmx.
>
> The code for card marking is copied verbatim from existing shared barrier code. A previous version inherited this code, by making ShenandoahBarrierSet a subclass of CardTableBarrierSet. Now the code pertinent to post write barriers is just copied down instead. This allows adding the new code without any changes to shared code (code that is not specific to Shenandoah).
>
> Initializing the card table in proper sequence comes with slightly rearranging heap initialization. This can be seen near the bottom of the listing here.
>
> Testing: ran the same SPECjvm2008 benchmark programs as before. Performance seems to be 1-1.5% less with card marking active, in average for this kind of benchmark. In non-generational mode, no change observed.
>
> For now we only prepare x86 code. Support for the other architectures will provided later with additional commits.
It looks good to me, and surprisingly straightforward! Nice!
Marked as reviewed by rkennke (Reviewer).
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/shenandoah/pull/5
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 21:02:05 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 21:02:05 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
<wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
Message-ID: <eEMq83Bf39QlSn6nEG4F18OJXN0SQsByYC9EVu7QPnk=.f797c4c2-7f40-4f82-a32f-fe57cde6b7ec@github.com>
On Fri, 13 Nov 2020 17:24:01 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Remove const qualifier from should_start_gc
>>
>> This lets the heuristics update state without const_casts.
>> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 266:
>
>> 264: size_t allocation_headroom = available;
>> 265:
>> 266: size_t spike_headroom = capacity / 100 * ShenandoahAllocSpikeFactor;
>
> I suspect `ShenandoahAllocSpikeFactor` handling is not needed anymore, as this new code is supposed to provide the similar cushion of few sigmas off the average value. That was my poor-mans solution to the same problem this patch resolves. You can remove the flag and update the comments in this block to mention only accumulated penalties.
I'll make some test runs without this flag.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 21:17:04 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 21:17:04 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
<wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
Message-ID: <GjPjcsSW-lh518LSM8d6H6_XG6SE5-h3RF044TKhKR0=.bf63be98-5b61-4804-9ddc-fb6945f314c1@github.com>
On Fri, 13 Nov 2020 17:40:07 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Remove const qualifier from should_start_gc
>>
>> This lets the heuristics update state without const_casts.
>> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 270:
>
>> 268:
>> 269: allocation_headroom -= MIN2(allocation_headroom, spike_headroom);
>> 270: allocation_headroom -= MIN2(allocation_headroom, penalties);
>
> ...in fact, maybe even penalties are not needed since we are widening the SD "windows" on degenerated/full GC? We can clean this up later if we are not sure.
I'll try testing without these adjustments as well.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 21:25:02 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 21:25:02 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <O90h88zBB_L5N7OHa41WKEHBKZZRH03ePSaLcJox6Ts=.3d12b818-3a88-4f90-94eb-d56b28a0ffc0@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
<wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
<O90h88zBB_L5N7OHa41WKEHBKZZRH03ePSaLcJox6Ts=.3d12b818-3a88-4f90-94eb-d56b28a0ffc0@github.com>
Message-ID: <zgVABvn4qeCsO9DDfky_FCZamRPiMJyNszvNMc6tD4s=.ce5e6924-1c65-43d1-ba86-9f17c54cd83c@github.com>
On Fri, 13 Nov 2020 18:00:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> The more I study this patch, the more I like it, good job. Another round of changes below. I fixed a few mentioned issues in my private working copy while doing initial performance testing. Would be good to have an updated PR for testing too.
>
>> 
>
> Yes, it mostly matches what I would expect from this heuristic improvement. As it stands right now, "adaptive" balances way too closely to the cliff, and so it is expected that improvement would make GC cycles more frequent to move GC triggers away from it. On the chart, "Headroom" probably means "Average allocation rate" trigger? It still a bit sad we cannot handle `smoothness=0.0`, but that probably requires future-predicting capabilities not yet available at current technology level.
I'll do more testing. I was hunting for a workload that was manageable by the `compact` heuristic, but caused degenerated cycles with `adaptive`. In theory, the trigger sensitivity and margin of error ought to be able to adapt to the worst case scenario by running cycles with the same frequency as the `compact` heuristic. I also want to find the breaking point of `compact` in this test harness.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 22:22:57 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 22:22:57 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
<wEjzjFCQRXjw4Sf4NEMVjysvbIy7Rv4bDDcky8K0MCw=.78bd6e56-519d-4797-826b-feb705c6133b@github.com>
Message-ID: <Bk2wVP2zxn_uhVfbwhi67jyKab1Yxf5ViUCWlF0g_dg=.3037e77c-0d26-4d61-a7b4-ee9c596dad96@github.com>
On Fri, 13 Nov 2020 17:14:45 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Remove const qualifier from should_start_gc
>>
>> This lets the heuristics update state without const_casts.
>> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 332:
>
>> 330: ShenandoahAllocationRate::ShenandoahAllocationRate(ShenandoahAdaptiveHeuristics *heuristics) :
>> 331: _heuristics(heuristics),
>> 332: _last_sample_time(os::javaTimeNanos()),
>
> My IDE complains that `size_t` and `jlong` are not compatible here. That might be the problem for x86_32. Note that existing heuristics code uses `os::elapsedTime()`, which returns `double`. Is there a reason not to use it?
Changed to use `os::elapsedTime` .
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 22:26:58 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 22:26:58 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
Message-ID: <CvDA6qNJ2sCuuSrmoeCu1CE7QwuScztgyrbmvkaob5U=.41f97706-66f9-4330-9876-c84fe1cd96e1@github.com>
On Thu, 12 Nov 2020 22:57:12 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>
> - Remove const qualifier from should_start_gc
>
> This lets the heuristics update state without const_casts.
> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 335:
> 333: _last_sample_value(0),
> 334: _interval_ns(NANOUNITS / ShenandoahAdaptiveSampleFrequencyHz),
> 335: _rate(ShenandoahAdaptiveSampleSizeSeconds * ShenandoahAdaptiveSampleFrequencyHz, ShenandoahAdaptiveDecayFactor),
I get a warning here because `TruncatedSeq` constructor takes a signed `int` for length (SampleSize and Frequency are unsigned). Could we change that constructor?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 13 22:37:12 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 13 Nov 2020 22:37:12 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
earthling-amzn has updated the pull request incrementally with six additional commits since the last revision:
- Inline calls to gc decision methods (vestige of an earlier design)
- Use os::elapsedTime to avoid type issues and to be consistent with other heuristics code
- Reuse instantaneous_rate method instead of duplicating code
- Rename variables to improve readability
- Make logging messages more consistent
- Restore call to reset allocation counter at cycle start
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1099/files
- new: https://git.openjdk.java.net/jdk/pull/1099/files/275e80a2..6ef8dc68
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=01-02
Stats: 92 lines in 2 files changed: 13 ins; 44 del; 35 mod
Patch: https://git.openjdk.java.net/jdk/pull/1099.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1099/head:pull/1099
PR: https://git.openjdk.java.net/jdk/pull/1099
From zgu at openjdk.java.net Sat Nov 14 12:33:02 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Sat, 14 Nov 2020 12:33:02 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v10]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <be7Z1mhBaVJ27nT1wJSjSHRffzAV4Xjy3l1Bss19j4k=.2f119432-25f0-4699-87da-dad7c18a2064@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits:
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Removed obsoleted class
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Move weak reference processing out of STWMark and fix its timings
- Merge branch 'JDK-8255019-sh-mark' of github.com:zhengyu123/jdk into JDK-8255019-sh-mark
- ... and 6 more: https://git.openjdk.java.net/jdk/compare/1e9a432d...7524feb1
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=09
Stats: 1969 lines in 21 files changed: 1072 ins; 738 del; 159 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From gnu.andrew at redhat.com Mon Nov 16 06:17:28 2020
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Mon, 16 Nov 2020 06:17:28 +0000
Subject: [RFR] [8u] 8u275-b01 Upstream Sync
Message-ID: <20201116061728.GA226173@rincewind>
Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/
Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/root/merge.changeset
Changes in aarch64-shenandoah-jdk8u275-b01:
- JDK-8214440: ldap over a TLS connection negotiate failed with "javax.net.ssl.SSLPeerUnverifiedException: hostname of the server '' does not match the hostname in the server's certificate"
- JDK-8223940: Private key not supported by chosen signature algorithm
- JDK-8236512: PKCS11 Connection closed after Cipher.doFinal and NoPadding
- JDK-8250861: Crash in MinINode::Ideal(PhaseGVN*, bool)
Main issues of note:
None, clean merge (only HotSpot change raises no conflicts).
diffstat for root
b/.hgtags | 4 ++++
1 file changed, 4 insertions(+)
diffstat for corba
b/.hgtags | 4 ++++
1 file changed, 4 insertions(+)
diffstat for jaxp
b/.hgtags | 4 ++++
1 file changed, 4 insertions(+)
diffstat for jaxws
b/.hgtags | 4 ++++
1 file changed, 4 insertions(+)
diffstat for langtools
b/.hgtags | 4 ++++
1 file changed, 4 insertions(+)
diffstat for nashorn
b/.hgtags | 4 ++++
1 file changed, 4 insertions(+)
diffstat for jdk
b/.hgtags | 4
b/src/share/classes/com/sun/jndi/ldap/ext/StartTlsResponseImpl.java | 3
b/src/share/classes/sun/security/pkcs11/P11AEADCipher.java | 38 ++--
b/src/share/classes/sun/security/pkcs11/P11Cipher.java | 31 +--
b/src/share/classes/sun/security/pkcs11/P11Mac.java | 20 +-
b/src/share/classes/sun/security/pkcs11/P11PSSSignature.java | 30 +--
b/src/share/classes/sun/security/pkcs11/P11RSACipher.java | 60 +++----
b/src/share/classes/sun/security/pkcs11/P11Signature.java | 82 ++++------
b/src/share/classes/sun/security/ssl/CertificateVerify.java | 54 +++---
b/src/share/classes/sun/security/ssl/DHServerKeyExchange.java | 30 +--
b/src/share/classes/sun/security/ssl/ECDHServerKeyExchange.java | 28 +--
b/src/share/classes/sun/security/ssl/SignatureScheme.java | 64 ++++++-
12 files changed, 235 insertions(+), 209 deletions(-)
diffstat for hotspot
b/.hgtags | 4 ++++
b/src/share/vm/opto/addnode.cpp | 4 ++--
2 files changed, 6 insertions(+), 2 deletions(-)
Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le & aarch64.
Ok to push?
Thanks,
--
Andrew :)
Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)
PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222
From shade at openjdk.java.net Mon Nov 16 08:23:00 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 16 Nov 2020 08:23:00 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v2]
In-Reply-To: <CvDA6qNJ2sCuuSrmoeCu1CE7QwuScztgyrbmvkaob5U=.41f97706-66f9-4330-9876-c84fe1cd96e1@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<8T74ez_nGcoGmhPjg1X3xG-EHUNCMH-MH401ZS0o4qM=.df3da6e4-cfd2-4645-9986-c9d9bb93763a@github.com>
<CvDA6qNJ2sCuuSrmoeCu1CE7QwuScztgyrbmvkaob5U=.41f97706-66f9-4330-9876-c84fe1cd96e1@github.com>
Message-ID: <zjKi_rqLt4SSOzZUUMxc8kfLae8-neep4DRCBq1FtSc=.83654813-da38-45b9-87b0-840068574844@github.com>
On Fri, 13 Nov 2020 22:24:22 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Remove const qualifier from should_start_gc
>>
>> This lets the heuristics update state without const_casts.
>> - Rename ShenandoahReactive flags to ShenandoahAdaptive flags.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 335:
>
>> 333: _last_sample_value(0),
>> 334: _interval_ns(NANOUNITS / ShenandoahAdaptiveSampleFrequencyHz),
>> 335: _rate(ShenandoahAdaptiveSampleSizeSeconds * ShenandoahAdaptiveSampleFrequencyHz, ShenandoahAdaptiveDecayFactor),
>
> I get a warning here because `TruncatedSeq` constructor takes a signed `int` for length (SampleSize and Frequency are unsigned). Could we change that constructor?
We better not. I think it is fine to have explicit cast to `int` to avoid this warning.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Mon Nov 16 08:41:59 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 16 Nov 2020 08:41:59 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
Message-ID: <xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
On Fri, 13 Nov 2020 22:37:12 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> earthling-amzn has updated the pull request incrementally with six additional commits since the last revision:
>
> - Inline calls to gc decision methods (vestige of an earlier design)
> - Use os::elapsedTime to avoid type issues and to be consistent with other heuristics code
> - Reuse instantaneous_rate method instead of duplicating code
> - Rename variables to improve readability
> - Make logging messages more consistent
> - Restore call to reset allocation counter at cycle start
I did some performance tests on the variant of this patch, and while it regresses throughput in some scenarios, that's the price we pay for additional spike safety. I think we can drop current alloc-spike and penalty tracking (even from `shenandoahHeuristics`?).
Another round of tune-ups follows.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 360:
> 358: double time_delta_sec = time - _last_sample_time;
> 359: return allocation_delta / time_delta_sec;
> 360: }
GitHub review says there is no new-line at the end of file. Please add.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 357:
> 355:
> 356: double ShenandoahAllocationRate::instantaneous_rate(double time, size_t allocated) const {
> 357: size_t allocation_delta = allocated - _last_sample_value;
Please protect from accidental underflow and division by zero:
size_t last_value = _last_sample_value;
size_t last_time = _last_sample_time;
size_t allocation_delta = (allocated > last_value) ? (allocated - last_value) : 0;
double time_delta = (time - last_time);
return (time_delta > 0) ? (allocation_delta / time_delta) : 0;
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 305:
> 303:
> 304: ShenandoahAllocationRate::ShenandoahAllocationRate(ShenandoahAdaptiveHeuristics *heuristics) :
> 305: _heuristics(heuristics),
Are we carrying `heuristics` here just to get access to `spike_threshold_sd`? I wonder if it is cleaner to just pass spike threshold to `is_spiking` then?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.hpp line 51:
> 49:
> 50: double instantaneous_rate(double time, size_t allocated) const;
> 51:
Group the declarations like this:
public:
explicit ShenandoahAllocationRate(ShenandoahAdaptiveHeuristics* heuristics);
void allocation_counter_reset();
void sample(size_t allocated);
double instantaneous_rate(size_t allocated) const;
double upper_bound(double standard_deviations) const;
bool is_spiking(double rate) const;
private:
double instantaneous_rate(double time, size_t allocated) const;
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 326:
> 324: }
> 325:
> 326: double ShenandoahAllocationRate::upper_bound(double standard_deviations) const {
Name the parameter `sds`?
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.hpp line 94:
> 92: const static double HIGHEST_EXPECTED_AVAILABLE_AT_END;
> 93:
> 94: friend class ShenandoahAllocationRate;
Does it still need to be friends? I.e. are there non-public members in `ShenandoahAllocationRate` this class needs to access?
-------------
Changes requested by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at redhat.com Mon Nov 16 08:43:05 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 16 Nov 2020 09:43:05 +0100
Subject: [RFR] [8u] 8u275-b01 Upstream Sync
In-Reply-To: <20201116061728.GA226173@rincewind>
References: <20201116061728.GA226173@rincewind>
Message-ID: <4edf1f3f-5b4f-944f-3db4-3acd04aa1dcf@redhat.com>
On 11/16/20 7:17 AM, Andrew Hughes wrote:
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jaxws/merge.changeset
Look trivially good.
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jdk/merge.changeset
Looks good.
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/hotspot/merge.changeset
Looks good.
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/langtools/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/nashorn/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/root/merge.changeset
Look trivially good.
> Ok to push?
Yes.
--
Thanks,
-Aleksey
From bmathiske at openjdk.java.net Mon Nov 16 13:37:34 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Mon, 16 Nov 2020 13:37:34 GMT
Subject: Integrated: Card marking write barrier for generational mode.
In-Reply-To: <UXC_VSsfsTWf0X1qPkKI-ywQJ2zLyrBYgxePAZ9-beY=.ff77d174-a867-4c29-a161-1dfb7aa59ac2@github.com>
References: <UXC_VSsfsTWf0X1qPkKI-ywQJ2zLyrBYgxePAZ9-beY=.ff77d174-a867-4c29-a161-1dfb7aa59ac2@github.com>
Message-ID: <Udzcc-yGfBMA_aUa3q3FlVg8wYqWtaQDdlvlJ_7owB4=.94dca36f-ecb0-4eeb-aa69-a4b1cc1e96dd@github.com>
On Tue, 27 Oct 2020 12:00:28 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:
> Adding a typical generational card marking write barrier, which is only active in generational mode and has no other effect yet than writing into a card table that is allocated at startup on the side and covers the whole heap as specified by -Xmx.
>
> The code for card marking is copied verbatim from existing shared barrier code. A previous version inherited this code, by making ShenandoahBarrierSet a subclass of CardTableBarrierSet. Now the code pertinent to post write barriers is just copied down instead. This allows adding the new code without any changes to shared code (code that is not specific to Shenandoah).
>
> Initializing the card table in proper sequence comes with slightly rearranging heap initialization. This can be seen near the bottom of the listing here.
>
> Testing: ran the same SPECjvm2008 benchmark programs as before. Performance seems to be 1-1.5% less with card marking active, in average for this kind of benchmark. In non-generational mode, no change observed.
>
> For now we only prepare x86 code. Support for the other architectures will provided later with additional commits.
This pull request has now been integrated.
Changeset: 73a63881
Author: Bernd Mathiske <bmathiske at openjdk.org>
Committer: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/shenandoah/commit/73a63881
Stats: 541 lines in 13 files changed: 499 ins; 22 del; 20 mod
Card marking write barrier for generational mode.
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/5
From gnu.andrew at redhat.com Mon Nov 16 13:45:20 2020
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Mon, 16 Nov 2020 13:45:20 +0000
Subject: [RFR] [8u] 8u275-b01 Upstream Sync
In-Reply-To: <4edf1f3f-5b4f-944f-3db4-3acd04aa1dcf@redhat.com>
References: <20201116061728.GA226173@rincewind>
<4edf1f3f-5b4f-944f-3db4-3acd04aa1dcf@redhat.com>
Message-ID: <20201116134520.GA227942@rincewind>
On 09:43 Mon 16 Nov , Aleksey Shipilev wrote:
> On 11/16/20 7:17 AM, Andrew Hughes wrote:
> > Merge changesets:
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/corba/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jaxp/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jaxws/merge.changeset
>
> Look trivially good.
>
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/jdk/merge.changeset
>
> Looks good.
>
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/hotspot/merge.changeset
>
> Looks good.
>
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/langtools/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/nashorn/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u275-b01/root/merge.changeset
>
> Look trivially good.
>
> > Ok to push?
>
> Yes.
>
> --
> Thanks,
> -Aleksey
>
Thanks. Pushed.
--
Andrew :)
Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)
PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222
From bmathiske at openjdk.java.net Mon Nov 16 13:55:29 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Mon, 16 Nov 2020 13:55:29 GMT
Subject: RFR: Generation labeling for regions and allocation requests.
Message-ID: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
The original generational Shenandoah patch authored by Roman Kennke.
The rest of the project will build upon this base.
-------------
Commit messages:
- Generation labeling for regions and allocation requests.
Changes: https://git.openjdk.java.net/shenandoah/pull/6/files
Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=6&range=00
Stats: 71 lines in 5 files changed: 61 ins; 0 del; 10 mod
Patch: https://git.openjdk.java.net/shenandoah/pull/6.diff
Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/6/head:pull/6
PR: https://git.openjdk.java.net/shenandoah/pull/6
From rkennke at openjdk.java.net Mon Nov 16 14:18:23 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 16 Nov 2020 14:18:23 GMT
Subject: RFR: Generation labeling for regions and allocation requests.
In-Reply-To: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
References: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
Message-ID: <9MQ7EUbRdyNcaFqY7r_IFUCes6FDpy5oiVjvb5nRgH4=.383a0b5f-333f-4a4a-a1f5-60333efad361@github.com>
On Mon, 16 Nov 2020 13:51:12 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:
> The original generational Shenandoah patch authored by Roman Kennke.
> The rest of the project will build upon this base.
One small thing (and I leave it to you if you want to change it), otherwise looks good.
src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 267:
> 265: if (target_gen == YOUNG_GEN) {
> 266: markWord mark = p->mark();
> 267: if (mark.is_marked()) {
This could probably use ShenandoahForwarding::is_marked() instead.
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/shenandoah/pull/6
From duke at openjdk.java.net Mon Nov 16 15:19:22 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 15:19:22 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Add jcheck config
Message-ID: <0a2a035a-1c0b-4a0c-aaf4-03b67e3c4283@openjdk.java.net>
Changeset: 300b6a12
Author: Roman Kennke <rkennke at redhat.com>
Date: 2020-11-16 16:18:50 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/300b6a12
Add jcheck config
+ .jcheck/conf
From bmathiske at openjdk.java.net Mon Nov 16 16:14:38 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Mon, 16 Nov 2020 16:14:38 GMT
Subject: RFR: Generation labeling for regions and allocation requests. [v2]
In-Reply-To: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
References: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
Message-ID: <_yY73naCUYn8w5YvHPWJxMCV1-iM6kkrwyeal--JfGE=.b36d70bc-e79f-4e65-abfe-a240d99c35a0@github.com>
> The original generational Shenandoah patch authored by Roman Kennke.
> The rest of the project will build upon this base.
Bernd Mathiske has updated the pull request incrementally with one additional commit since the last revision:
Simplify forwarding conditional in evacuation.
-------------
Changes:
- all: https://git.openjdk.java.net/shenandoah/pull/6/files
- new: https://git.openjdk.java.net/shenandoah/pull/6/files/b90f4dc9..26ae8b70
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=6&range=01
- incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=6&range=00-01
Stats: 10 lines in 1 file changed: 1 ins; 5 del; 4 mod
Patch: https://git.openjdk.java.net/shenandoah/pull/6.diff
Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/6/head:pull/6
PR: https://git.openjdk.java.net/shenandoah/pull/6
From bmathiske at openjdk.java.net Mon Nov 16 16:14:39 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Mon, 16 Nov 2020 16:14:39 GMT
Subject: RFR: Generation labeling for regions and allocation requests. [v2]
In-Reply-To: <9MQ7EUbRdyNcaFqY7r_IFUCes6FDpy5oiVjvb5nRgH4=.383a0b5f-333f-4a4a-a1f5-60333efad361@github.com>
References: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
<9MQ7EUbRdyNcaFqY7r_IFUCes6FDpy5oiVjvb5nRgH4=.383a0b5f-333f-4a4a-a1f5-60333efad361@github.com>
Message-ID: <rC03vXwkwSm_1XuqIuTOC5F_b_jqZoXbWkUZGlGsvQQ=.cd824ae9-c2c7-4878-b483-9f37ff16069e@github.com>
On Mon, 16 Nov 2020 14:14:43 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Bernd Mathiske has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Simplify forwarding conditional in evacuation.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 267:
>
>> 265: if (target_gen == YOUNG_GEN) {
>> 266: markWord mark = p->mark();
>> 267: if (mark.is_marked()) {
>
> This could probably use ShenandoahForwarding::is_marked() instead.
Fixed.
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/6
From rkennke at openjdk.java.net Mon Nov 16 16:18:18 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 16 Nov 2020 16:18:18 GMT
Subject: RFR: Generation labeling for regions and allocation requests. [v2]
In-Reply-To: <9MQ7EUbRdyNcaFqY7r_IFUCes6FDpy5oiVjvb5nRgH4=.383a0b5f-333f-4a4a-a1f5-60333efad361@github.com>
References: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
<9MQ7EUbRdyNcaFqY7r_IFUCes6FDpy5oiVjvb5nRgH4=.383a0b5f-333f-4a4a-a1f5-60333efad361@github.com>
Message-ID: <Bx5ZgxHp4UaIIWU1-hhEER-OECBLQcYvsV3ZJPLtSl0=.42079067-6aa5-4929-9d45-45af7301e8ec@github.com>
On Mon, 16 Nov 2020 14:15:50 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Bernd Mathiske has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Simplify forwarding conditional in evacuation.
>
> One small thing (and I leave it to you if you want to change it), otherwise looks good.
Thanks! I already approved it, you can /integrate it now :-)
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/6
From github.com+71722661+earthling-amzn at openjdk.java.net Mon Nov 16 17:03:04 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 16 Nov 2020 17:03:04 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
Message-ID: <RfwDFc1DGscKYfe2_cGRTlsFThw-rmGobU5Q5t2EqCU=.47864d9d-b846-4595-903c-d916643e7e02@github.com>
On Mon, 16 Nov 2020 08:27:59 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with six additional commits since the last revision:
>>
>> - Inline calls to gc decision methods (vestige of an earlier design)
>> - Use os::elapsedTime to avoid type issues and to be consistent with other heuristics code
>> - Reuse instantaneous_rate method instead of duplicating code
>> - Rename variables to improve readability
>> - Make logging messages more consistent
>> - Restore call to reset allocation counter at cycle start
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 305:
>
>> 303:
>> 304: ShenandoahAllocationRate::ShenandoahAllocationRate(ShenandoahAdaptiveHeuristics *heuristics) :
>> 305: _heuristics(heuristics),
>
> Are we carrying `heuristics` here just to get access to `spike_threshold_sd`? I wonder if it is cleaner to just pass spike threshold to `is_spiking` then?
I'll re-work this dependency and also get rid of the `friends` declaration.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From coleenp at openjdk.java.net Mon Nov 16 17:12:32 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 16 Nov 2020 17:12:32 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v9]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <Zw6TjZaVyVEGnBTeraBNjMAsJ4uAbhyqQeoHKCYVQXw=.34b82cff-c1bf-4c52-af0e-af50e4ac5354@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with three additional commits since the last revision:
- Add shenandoah set_needs_cleaning but this doesn't work.
- fix vmTestbase/nsk/jvmti tests
- improve tagmap cleanup and objectfree event posting
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/0487b84c..283696f6
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=08
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=07-08
Stats: 223 lines in 16 files changed: 174 ins; 12 del; 37 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From bmathiske at openjdk.java.net Mon Nov 16 17:38:23 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Mon, 16 Nov 2020 17:38:23 GMT
Subject: Integrated: Generation labeling for regions and allocation requests.
In-Reply-To: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
References: <CWBHYedXVwog1BscOE9oH9ax3vNsRjZagiRx_i6qOHs=.f6481330-a388-43b4-a09f-5013b641f917@github.com>
Message-ID: <hSc3qAbLvzjh58bit_baDS6zzLGqjK18ZqlP6rbxkjY=.f36b3305-d809-4e06-a1c3-de30374f0977@github.com>
On Mon, 16 Nov 2020 13:51:12 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:
> The original generational Shenandoah patch authored by Roman Kennke.
> The rest of the project will build upon this base.
This pull request has now been integrated.
Changeset: 7e4a14c0
Author: Bernd Mathiske <bmathiske at openjdk.org>
Committer: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/shenandoah/commit/7e4a14c0
Stats: 68 lines in 5 files changed: 57 ins; 0 del; 11 mod
Generation labeling for regions and allocation requests.
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/6
From duke at openjdk.java.net Mon Nov 16 18:32:16 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 18:32:16 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Turn README into proper
Markdown
Message-ID: <e6160f10-1ff9-4cec-ae16-0ed4710dc8b0@openjdk.java.net>
Changeset: 5e395602
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 19:24:07 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/5e395602
Turn README into proper Markdown
- README
+ README.md
From duke at openjdk.java.net Mon Nov 16 18:32:49 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 18:32:49 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Update README again
Message-ID: <a7be3690-a005-4994-ad60-3d42be6f43d4@openjdk.java.net>
Changeset: 9cf52799
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 19:32:15 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/9cf52799
Update README again
! README.md
From duke at openjdk.java.net Mon Nov 16 18:35:09 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 18:35:09 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Update README.md
Message-ID: <a87f7879-228e-410b-b385-38f005888c6c@openjdk.java.net>
Changeset: ffb924a6
Author: Aleksey Shipil?v <shade at redhat.com>
Committer: GitHub <noreply at github.com>
Date: 2020-11-16 19:34:23 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/ffb924a6
Update README.md
! README.md
From duke at openjdk.java.net Mon Nov 16 18:36:11 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 18:36:11 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Rename .hgignore ->
.gitignore
Message-ID: <7569bb62-e271-4ff7-b832-5f8ac5662dfd@openjdk.java.net>
Changeset: 25d5c06d
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 19:35:30 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/25d5c06d
Rename .hgignore -> .gitignore
= .gitignore
From duke at openjdk.java.net Mon Nov 16 18:38:36 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 18:38:36 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Basic pre-integration
tests
Message-ID: <ca417098-c1b6-4a0b-9bc8-24aa9a22a283@openjdk.java.net>
Changeset: 24509d1d
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 19:37:20 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/24509d1d
Basic pre-integration tests
+ .github/workflows/pre-integration.yml
From duke at openjdk.java.net Mon Nov 16 18:39:30 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 18:39:30 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Pre-integration tests:
JDK 7 is actually unsupported
Message-ID: <6bafed2c-6883-4860-bb2a-3ae301e640c2@openjdk.java.net>
Changeset: 1b8285ef
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 19:38:58 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/1b8285ef
Pre-integration tests: JDK 7 is actually unsupported
! .github/workflows/pre-integration.yml
From duke at openjdk.java.net Mon Nov 16 19:02:33 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 19:02:33 GMT
Subject: git: openjdk/shenandoah-visualizer: master: 4 new changesets
Message-ID: <913d3e73-1922-4ccb-84bd-8dc625b6acae@openjdk.java.net>
Changeset: 2d585846
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 19:42:53 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/2d585846
Updated dangling sentence
! README.md
Changeset: e4724083
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 19:55:56 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/e4724083
Add missing license texts and headers
+ LICENSE
! pom.xml
+ src/license/gpl_cpe/header.txt
! src/main/java/org/openjdk/shenandoah/Colors.java
! src/main/java/org/openjdk/shenandoah/DataProvider.java
! src/main/java/org/openjdk/shenandoah/Phase.java
! src/main/java/org/openjdk/shenandoah/RegionStat.java
! src/main/java/org/openjdk/shenandoah/RegionState.java
! src/main/java/org/openjdk/shenandoah/Snapshot.java
! src/main/java/org/openjdk/shenandoah/SnapshotView.java
! src/test/java/org/openjdk/shenandoah/RenderLegendTest.java
Changeset: c88f04ce
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 19:57:49 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/c88f04ce
Make sure consistent CRLF for all files: dos2unix all
! src/main/java/org/openjdk/shenandoah/ShenandoahVisualizer.java
Changeset: c0805617
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 20:01:35 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/c0805617
Fix copyright dates, based on repo history
! src/main/java/org/openjdk/shenandoah/Phase.java
! src/main/java/org/openjdk/shenandoah/ShenandoahVisualizer.java
From github.com+71722661+earthling-amzn at openjdk.java.net Mon Nov 16 19:03:19 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 16 Nov 2020 19:03:19 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v4]
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <qiIZUVXXKNNGagxtP44pQ66w9kcyLyxoIshfoC70Dng=.1bae3672-0bb6-4fa1-91e0-e6bf0b81c81f@github.com>
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
- Remove dependency from allocation rate to adaptive heuristic
- Defend against underflow and division by zero
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1099/files
- new: https://git.openjdk.java.net/jdk/pull/1099/files/6ef8dc68..3ec8f3c5
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=02-03
Stats: 29 lines in 2 files changed: 5 ins; 10 del; 14 mod
Patch: https://git.openjdk.java.net/jdk/pull/1099.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1099/head:pull/1099
PR: https://git.openjdk.java.net/jdk/pull/1099
From duke at openjdk.java.net Mon Nov 16 19:18:10 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 19:18:10 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Sample screenshot
Message-ID: <8d6392bd-afd0-42a0-8a34-a7f485dd6a73@openjdk.java.net>
Changeset: c097e6a5
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 20:17:47 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/c097e6a5
Sample screenshot
+ sample-screenshot.png
From duke at openjdk.java.net Mon Nov 16 19:19:56 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 19:19:56 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Move the screenshot to
images
Message-ID: <4bb737d9-5b26-42a4-90e0-ab7947795337@openjdk.java.net>
Changeset: fed9b79f
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 20:19:16 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/fed9b79f
Move the screenshot to images
= images/sample-screenshot.png
From duke at openjdk.java.net Mon Nov 16 19:21:01 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 19:21:01 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Add screenshot to README
Message-ID: <e7cbc0df-60f3-4f94-b750-ea895b8bc9fe@openjdk.java.net>
Changeset: 72961647
Author: Aleksey Shipil?v <shade at redhat.com>
Committer: GitHub <noreply at github.com>
Date: 2020-11-16 20:20:21 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/72961647
Add screenshot to README
! README.md
From duke at openjdk.java.net Mon Nov 16 19:26:06 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 19:26:06 GMT
Subject: git: openjdk/shenandoah-visualizer: master: 2 new changesets
Message-ID: <29362082-a82c-4e3d-916c-abb2ec849f0b@openjdk.java.net>
Changeset: 6f8078bf
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 20:22:47 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/6f8078bf
Optimize imports
! src/main/java/org/openjdk/shenandoah/RegionStat.java
! src/main/java/org/openjdk/shenandoah/ShenandoahVisualizer.java
Changeset: bcf8d85b
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-16 20:24:06 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/bcf8d85b
Traversal does not longer exist
! src/main/java/org/openjdk/shenandoah/Colors.java
! src/main/java/org/openjdk/shenandoah/Phase.java
! src/main/java/org/openjdk/shenandoah/ShenandoahVisualizer.java
! src/main/java/org/openjdk/shenandoah/Snapshot.java
From shade at openjdk.java.net Mon Nov 16 19:41:08 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 16 Nov 2020 19:41:08 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v4]
In-Reply-To: <MbNZpDJESiGogMy13ATN-HbhaF-DpzhzOpYj1Oi-YJg=.fa744a6d-6ffa-44da-9dc4-de9945a80208@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<rR0gwszlXlQLC4mUiK2KWlRWC7hcyyJHClYzuMv9lEM=.d1066e7b-367b-401c-8831-ef4056f967ee@github.com>
<MbNZpDJESiGogMy13ATN-HbhaF-DpzhzOpYj1Oi-YJg=.fa744a6d-6ffa-44da-9dc4-de9945a80208@github.com>
Message-ID: <1BXaXma8kFkbiWaDzqWz_AmDbp7yGLKlT-dGGSS_p6s=.2c12591a-0a5d-4501-be79-e4e6c4e09f7c@github.com>
On Thu, 12 Nov 2020 22:23:27 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 212:
>>
>>> 210: ShenandoahAdaptiveHeuristics *heuristic = const_cast<ShenandoahAdaptiveHeuristics *>(this);
>>> 211: heuristic->_allocation_rate.sample(bytes_allocated_since_gc_start);
>>> 212: heuristic->_last_trigger = OTHER;
>>
>> Shouldn't `_last_trigger` updates be near `return true;` in this method? I.e. we should not probably overwrite `_last_trigger` if nothing was actually triggered? This probably affects the super-call to `ShenandoahHeuristics::should_start_gc()` as well?
>
> Would it make more sense if I `OTHER` were `NONE`? The intention here is to only adjust the allocation rate or spike detection parameters if they triggered the cycle. Here we reset the `_last_trigger` in one place without having to touch all the actual decision points. As it stands, once a gc cycle is started, the control thread won't call back into this method until the cycle is complete.
(realized I have not replied here) `OTHER` is fine. `NONE` implies no trigger at all, which is a lie when some trigger actually acts :)
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From duke at openjdk.java.net Mon Nov 16 19:51:13 2020
From: duke at openjdk.java.net (duke)
Date: Mon, 16 Nov 2020 19:51:13 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Fix mistake in README
Message-ID: <8989065b-8b4e-40a2-b544-84c67015c43a@openjdk.java.net>
Changeset: aa990e89
Author: Roman Kennke <rkennke at redhat.com>
Date: 2020-11-16 20:50:40 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/aa990e89
Fix mistake in README
! README.md
From rkennke at redhat.com Mon Nov 16 19:54:27 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 16 Nov 2020 20:54:27 +0100
Subject: Can the shenandoah visualizer project be moved to GitHub?
In-Reply-To: <5B75F37A-C649-4374-813B-EF66BE1AF590@amazon.com>
References: <1605217990818.92512@amazon.com>
<bdbf27df-96c9-2c4b-49e8-31bdab28cba1@redhat.com>
<8C4597FC-EE02-4B27-A77E-6EFB7C3BF0E8@amazon.com>
<d0861044-b721-3361-212d-96c8ce4461dd@redhat.com>
<5B75F37A-C649-4374-813B-EF66BE1AF590@amazon.com>
Message-ID: <c32f3165-8151-4322-6754-a3fa6798284b@redhat.com>
Hi Bernd & William,
We migrated the visualizer to GitHub:
https://github.com/openjdk/shenandoah-visualizer
and Aleksey updated the docs, copyrights, license and added a test pipeline.
Best regards,
Roman
On 13/11/2020 16:57, Mathiske, Bernd wrote:
> Hi Roman,
>
> I figured you were more than busy given the emails I see flying by. (
>
> We plan to set up a little demo script so you can reproduce the visualization.
>
> With milestone 3 (tenuring and promotion), we ran into some bugs, still spending a lot of time on debugging what happens to some promoted objects. See the code in question in my fork, in branch "Milestone-3" (https://github.com/bernd-aws/shenandoah/tree/Milestone-3).
>
> BTW, would you have time to accept another PR (https://github.com/openjdk/shenandoah/pull/5) into the genshen branch of your repo? I would keep them coming... (
>
> Best,
> Bernd
>
> ?On 11/13/20, 3:43 PM, "Roman Kennke" <rkennke at redhat.com> wrote:
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> Hi Bernd,
>
> This is great news!
>
> I have asked ops@ to create a new repository under github.com/openjdk, I
> expect that it shows up within the next few days, then I need to migrate
> the code over to there, and we should be ready to go (unless we get some
> unnecessary red tape in our way, in which case I'd put it someplace else)
>
> Very much looking forward to see your prototype in action!
>
> How is progress with GenShen prototype? I haven't seen/heard of you
> lately... ;-)
>
> Cheers,
> Roman
>
> > It works and we like it very much. Soon, we will have enhancements that show how regions age and transition between generations if running with "-XX::ShenandoahGCMode=generational" in a GenShen prototype...
> >
> > On 11/12/20, 10:58 PM, "shenandoah-dev on behalf of Roman Kennke" <shenandoah-dev-retn at openjdk.java.net on behalf of rkennke at redhat.com> wrote:
> >
> > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> >
> >
> >
> > Hi William,
> >
> > This is a good idea. I am not even sure if it still works, it's probably
> > a little bit-rotten by now. I will look into moving it to Github soon.
> >
> > Thanks,
> > Roman
> >
> >
> > > The project is hosted here: http://icedtea.classpath.org/hg/shenandoah-visualizer/?. Having it on GitHub would make it easier to propose contributions. The project is also missing a LICENSE file, though there is a reference to GPLv2 in a header comment.
> > >
> > >
> > > Thanks,
> > >
> > > William?
> > >
> >
> >
>
>
From kemperw at amazon.com Mon Nov 16 21:16:54 2020
From: kemperw at amazon.com (Kemper, William)
Date: Mon, 16 Nov 2020 21:16:54 +0000
Subject: Can the shenandoah visualizer project be moved to GitHub?
In-Reply-To: <c32f3165-8151-4322-6754-a3fa6798284b@redhat.com>
References: <1605217990818.92512@amazon.com>
<bdbf27df-96c9-2c4b-49e8-31bdab28cba1@redhat.com>
<8C4597FC-EE02-4B27-A77E-6EFB7C3BF0E8@amazon.com>
<d0861044-b721-3361-212d-96c8ce4461dd@redhat.com>
<5B75F37A-C649-4374-813B-EF66BE1AF590@amazon.com>,
<c32f3165-8151-4322-6754-a3fa6798284b@redhat.com>
Message-ID: <1605561414568.9304@amazon.com>
Excellent! Thank you so much!
________________________________________
From: Roman Kennke <rkennke at redhat.com>
Sent: Monday, November 16, 2020 11:54 AM
To: Mathiske, Bernd; Kemper, William; shenandoah-dev at openjdk.java.net
Subject: RE: [EXTERNAL] Can the shenandoah visualizer project be moved to GitHub?
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hi Bernd & William,
We migrated the visualizer to GitHub:
https://github.com/openjdk/shenandoah-visualizer
and Aleksey updated the docs, copyrights, license and added a test pipeline.
Best regards,
Roman
On 13/11/2020 16:57, Mathiske, Bernd wrote:
> Hi Roman,
>
> I figured you were more than busy given the emails I see flying by. (
>
> We plan to set up a little demo script so you can reproduce the visualization.
>
> With milestone 3 (tenuring and promotion), we ran into some bugs, still spending a lot of time on debugging what happens to some promoted objects. See the code in question in my fork, in branch "Milestone-3" (https://github.com/bernd-aws/shenandoah/tree/Milestone-3).
>
> BTW, would you have time to accept another PR (https://github.com/openjdk/shenandoah/pull/5) into the genshen branch of your repo? I would keep them coming... (
>
> Best,
> Bernd
>
> ?On 11/13/20, 3:43 PM, "Roman Kennke" <rkennke at redhat.com> wrote:
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> Hi Bernd,
>
> This is great news!
>
> I have asked ops@ to create a new repository under github.com/openjdk, I
> expect that it shows up within the next few days, then I need to migrate
> the code over to there, and we should be ready to go (unless we get some
> unnecessary red tape in our way, in which case I'd put it someplace else)
>
> Very much looking forward to see your prototype in action!
>
> How is progress with GenShen prototype? I haven't seen/heard of you
> lately... ;-)
>
> Cheers,
> Roman
>
> > It works and we like it very much. Soon, we will have enhancements that show how regions age and transition between generations if running with "-XX::ShenandoahGCMode=generational" in a GenShen prototype...
> >
> > On 11/12/20, 10:58 PM, "shenandoah-dev on behalf of Roman Kennke" <shenandoah-dev-retn at openjdk.java.net on behalf of rkennke at redhat.com> wrote:
> >
> > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> >
> >
> >
> > Hi William,
> >
> > This is a good idea. I am not even sure if it still works, it's probably
> > a little bit-rotten by now. I will look into moving it to Github soon.
> >
> > Thanks,
> > Roman
> >
> >
> > > The project is hosted here: http://icedtea.classpath.org/hg/shenandoah-visualizer/?. Having it on GitHub would make it easier to propose contributions. The project is also missing a LICENSE file, though there is a reference to GPLv2 in a header comment.
> > >
> > >
> > > Thanks,
> > >
> > > William?
> > >
> >
> >
>
>
From coleenp at openjdk.java.net Mon Nov 16 23:10:21 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 16 Nov 2020 23:10:21 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v10]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits:
- Reverse remove_dead_entries_locked function names.
- Merge branch 'master' into jvmti-table
- Add shenandoah set_needs_cleaning but this doesn't work.
- fix vmTestbase/nsk/jvmti tests
- improve tagmap cleanup and objectfree event posting
- Add logging to event posting in case of pauses.
- Merge branch 'master' into jvmti-table
- Add back WeakProcessorPhases::Phase enum.
- Serguei 1.
- Code review comments from Kim and Albert.
- ... and 5 more: https://git.openjdk.java.net/jdk/compare/0357db35...daaa13fe
-------------
Changes: https://git.openjdk.java.net/jdk/pull/967/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=09
Stats: 1884 lines in 49 files changed: 768 ins; 993 del; 123 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Mon Nov 16 23:19:14 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 16 Nov 2020 23:19:14 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4]
In-Reply-To: <OpyVqAff9GnzQVUeTMlTOXNPbHmBbAz1O4VKNg6neBY=.4bf447ac-3f45-468b-9f73-edd557846652@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<xYdvaPQJbpAUjaws1GqL6lgrxi01wKMv9bv7-msAS1Q=.1f9aa991-a73f-4ad0-b770-2ef3870027a4@github.com>
<ZTks_geJldq1VIq96rDUvhFz86dwEee4urYeLZZUcds=.96a1f000-fb32-479d-8a37-411266151f2e@github.com>
<jKyDBy89u-3EdYr0CCKsfGI-SM7FgGWCTFtG9YWk6yw=.196f1a2d-ac5d-4a6d-99cc-48854e747736@github.com>
<172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com>
<nw7ZiLLz8c9mc75ndZXLRTaVq43GM6lDRELxYsRw_J8=.4574de5e-b30e-41f8-952a-76d98c13065b@github.com>
<OpyVqAff9GnzQVUeTMlTOXNPbHmBbAz1O4VKNg6neBY=.4bf447ac-3f45-468b-9f73-edd557846652@github.com>
Message-ID: <Ihi-8Ma86r3YD_eRHUeNoD_DFkX7Gdv1wp7iYyosOOk=.6afbbd95-8ee2-4251-9064-345f25ae3cb9@github.com>
On Thu, 5 Nov 2020 14:36:44 GMT, Erik ?sterlund <eosterlund at openjdk.org> wrote:
>> Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times.
>
> The point of this change was not necessarily to be lazy about updating the tagmap, until someone uses it. The point was to get rid of the last annoying serial GC phase. Doing it all lazily would certainly also achieve that. But it would also lead to situations where no event is ever posted from GC to GC. So you would get the event 20 GCs later, which might come as a surprise. It did come as a surprise to some tests, so it is reasonable to assume it would come as a surprise to users too. And I don't think we want such surprises unless we couldn't deal with them. And we can.
Kim's change to post the events from the service thread or before other JVMTI operations removes posting events from the gc_notification, which was the objection.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Mon Nov 16 23:19:13 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 16 Nov 2020 23:19:13 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6]
In-Reply-To: <Q2SPsAnxbJIW1H8zUkVtpvTpI2WrnbDN04LHXQqfSFU=.1d917f30-af8b-4323-9f52-d9786052912e@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<rrqQkn7YRMjol0rpPIDSBsbXZXqFULDSRxMlMPJ5cjs=.f10aea2a-10db-4555-ba82-8664a472e393@github.com>
<TgRmyCL8pDACuMxxHfdsqN9mOVxyZaKNRpfXbssEl94=.69114d11-53a5-4220-b5ba-e9ecdfe96f26@github.com>
<q0OJ0RDIEfVUvYfaNAHV0oKWVO4RGch_9y1Qd6vEErI=.d54b2cee-4333-47e5-9469-7539b8bc1e43@github.com>
<Q2SPsAnxbJIW1H8zUkVtpvTpI2WrnbDN04LHXQqfSFU=.1d917f30-af8b-4323-9f52-d9786052912e@github.com>
Message-ID: <DA7oOizeDFJQjUEuY6gDSJjIRQjeDFs1lucOjbhcJZM=.c9b29751-9746-4aa4-b958-a6dd9e46b052@github.com>
On Mon, 9 Nov 2020 20:39:40 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change.
>
> This change also passes tier 7,8 testing.
does this work?
I've added two commits from @kimbarrett that defer the ObjectFree posting to the service thread or to a place where it could be removed before posting.
I also remerged and added the call JvmtiTagMap::set_needs_cleaning() to shenandoah which works after merging the latest code from shenandoah.
Testing tiers 1-6 currently. jvmti/jdi tests pass with G1 and ZGC stress options, and JVMTI tests pass with shenandoah.
Thanks! Coleen
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Mon Nov 16 23:30:25 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Mon, 16 Nov 2020 23:30:25 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
Fix minimal build.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/daaa13fe..1940eaf1
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=10
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=09-10
Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From kbarrett at openjdk.java.net Tue Nov 17 03:52:14 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Tue, 17 Nov 2020 03:52:14 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
Message-ID: <KyTV20qnxYd1LsYOKYBhfS1Sip_SZH7CnXPdweuWzm8=.56fa0b5a-14fd-4e80-9d1c-8aeacb3fa37e@github.com>
On Mon, 16 Nov 2020 23:30:25 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> Fix minimal build.
Marked as reviewed by kbarrett (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From bmathiske at openjdk.java.net Tue Nov 17 10:35:35 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Tue, 17 Nov 2020 10:35:35 GMT
Subject: RFR: Generation affiliation transitions for heap regions.
Message-ID: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
-------------
Commit messages:
- Generation affiliation transitions for heap regions.
Changes: https://git.openjdk.java.net/shenandoah/pull/7/files
Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=7&range=00
Stats: 178 lines in 11 files changed: 138 ins; 5 del; 35 mod
Patch: https://git.openjdk.java.net/shenandoah/pull/7.diff
Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/7/head:pull/7
PR: https://git.openjdk.java.net/shenandoah/pull/7
From rkennke at openjdk.java.net Tue Nov 17 11:18:18 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 17 Nov 2020 11:18:18 GMT
Subject: RFR: Generation affiliation transitions for heap regions.
In-Reply-To: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
Message-ID: <HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
On Tue, 17 Nov 2020 10:31:20 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:
> Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
>
> The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
>
> The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
Nice changes. A few questions too...
BTW, I'd like to bring in latest upstream changes so that we don't diverge too much. In particular, this will bring in concurrent weak reference processing. I suggest that I do this after this PR is integrated, ok?
src/hotspot/share/gc/shenandoah/shenandoahCardTable.cpp line 26:
> 24:
> 25: #include "precompiled.hpp"
> 26: #include "gc/shared/memset_with_concurrent_readers.hpp"
What's this?
src/hotspot/share/gc/shenandoah/shenandoahCardTable.hpp line 41:
> 39: virtual void initialize();
> 40:
> 41: virtual inline bool is_in_young(oop obj) const {
Why do we need virtual here? Is it overridden anywhere?
src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 550:
> 548: bool is_in(const void* p) const;
> 549:
> 550: bool is_in_young(const void* p) const;
This overrides CollectedHeap method, right? So either put 'virtual' here to indicate this, or even better 'override'. I believe we can use that new C++11 specifier now...
-------------
Changes requested by rkennke (Reviewer).
PR: https://git.openjdk.java.net/shenandoah/pull/7
From roland at openjdk.java.net Tue Nov 17 12:38:23 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Tue, 17 Nov 2020 12:38:23 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah [v4]
In-Reply-To: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
Message-ID: <jGIIRVQHL2oyAk7SOQY5021eCxfpvTnrh628pMdj8o4=.76ff5729-705b-4a70-b1bf-2143608eec86@github.com>
> This is a Shenandoah bug but the proposed fix is in shared code.
>
> In an infinite loop, a barrier is located right after the loop head
> and above the never branch. When the barrier is expanded, control flow
> is added between the loop and the never branch. During loop
> verification the assert fires because it doesn't expect any control
> flow between the never branch and the loop head.
>
> While it would have been nice to fix this Shenandoah issue in
> Shenandoah code, I think the cleaner fix is to preserve the invariant
> that the never branch is always right after the loop head in an
> infinite loop. In the proposed patch, this is achieved by moving all
> uses of the loop head to the never branch when it's constructed.
Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
- fix
- test
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1073/files
- new: https://git.openjdk.java.net/jdk/pull/1073/files/bd85cdfc..15fbccd8
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1073&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1073&range=02-03
Stats: 13157 lines in 238 files changed: 8705 ins; 2419 del; 2033 mod
Patch: https://git.openjdk.java.net/jdk/pull/1073.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1073/head:pull/1073
PR: https://git.openjdk.java.net/jdk/pull/1073
From bmathiske at openjdk.java.net Tue Nov 17 13:25:16 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Tue, 17 Nov 2020 13:25:16 GMT
Subject: RFR: Generation affiliation transitions for heap regions.
In-Reply-To: <HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
<HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
Message-ID: <-1BWw_x4UpeEBz7TgtJuZQ7gmwkjwuAs77K8W7oQbRI=.6922b461-0464-4743-b3cd-55f91f6a36ca@github.com>
On Tue, 17 Nov 2020 11:08:18 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
>>
>> The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
>>
>> The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
>
> src/hotspot/share/gc/shenandoah/shenandoahCardTable.cpp line 26:
>
>> 24:
>> 25: #include "precompiled.hpp"
>> 26: #include "gc/shared/memset_with_concurrent_readers.hpp"
>
> What's this?
Not needed. Removing it.
(This is a remnant from a former version of mine where more functionality was hosted in this class. Tsktsk.)
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/7
From bmathiske at openjdk.java.net Tue Nov 17 13:28:24 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Tue, 17 Nov 2020 13:28:24 GMT
Subject: RFR: Generation affiliation transitions for heap regions.
In-Reply-To: <HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
<HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
Message-ID: <4qazVu-qLhNCvYal6c28833xW5_-tTNF68ET4nnI30M=.9793152c-ad49-4431-85ea-517bbe70f561@github.com>
On Tue, 17 Nov 2020 11:09:07 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
>>
>> The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
>>
>> The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
>
> src/hotspot/share/gc/shenandoah/shenandoahCardTable.hpp line 41:
>
>> 39: virtual void initialize();
>> 40:
>> 41: virtual inline bool is_in_young(oop obj) const {
>
> Why do we need virtual here? Is it overridden anywhere?
Removing "virtual".
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/7
From bmathiske at openjdk.java.net Tue Nov 17 13:32:21 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Tue, 17 Nov 2020 13:32:21 GMT
Subject: RFR: Generation affiliation transitions for heap regions.
In-Reply-To: <HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
<HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
Message-ID: <WVVoDZnY6ICrQgS4km8fHFglC3svbt2rlVve7S8pZlg=.532dc0a4-c3fa-488e-be8d-096b6de76cc5@github.com>
On Tue, 17 Nov 2020 11:12:50 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
>>
>> The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
>>
>> The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 550:
>
>> 548: bool is_in(const void* p) const;
>> 549:
>> 550: bool is_in_young(const void* p) const;
>
> This overrides CollectedHeap method, right? So either put 'virtual' here to indicate this, or even better 'override'. I believe we can use that new C++11 specifier now...
This method does not exist in CollectedHeap. It does exist in GenCollectedHeap, but ShenandoahHeap is not a subclass of the latter. (GenCollectedHeap is rather specialized, not suitable for us. It assumes contiguous generation spaces. G1 does not use it either.)
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/7
From bmathiske at openjdk.java.net Tue Nov 17 13:35:38 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Tue, 17 Nov 2020 13:35:38 GMT
Subject: RFR: Generation affiliation transitions for heap regions. [v2]
In-Reply-To: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
Message-ID: <WP956VryoxRu_XicLp7mh0YARMLCnPZTnlggLfYIe6I=.3bb277e2-6522-45fa-94b8-b37dbb766d7e@github.com>
> Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
>
> The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
>
> The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
Bernd Mathiske has updated the pull request incrementally with one additional commit since the last revision:
Removetwo uneeded declarations.
-------------
Changes:
- all: https://git.openjdk.java.net/shenandoah/pull/7/files
- new: https://git.openjdk.java.net/shenandoah/pull/7/files/e5539836..dadb608e
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=7&range=01
- incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=7&range=00-01
Stats: 2 lines in 2 files changed: 0 ins; 1 del; 1 mod
Patch: https://git.openjdk.java.net/shenandoah/pull/7.diff
Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/7/head:pull/7
PR: https://git.openjdk.java.net/shenandoah/pull/7
From bmathiske at openjdk.java.net Tue Nov 17 13:35:39 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Tue, 17 Nov 2020 13:35:39 GMT
Subject: RFR: Generation affiliation transitions for heap regions. [v2]
In-Reply-To: <HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
<HqyS7yVV_cr-gQbi3_UciP1VsB4GkXef62VDbY5blCA=.09ee2e11-e70c-477b-84fd-f1b87a376e4b@github.com>
Message-ID: <gCtJ_n45xUCxlDuZ0AXP1rTNL5CcGnK3JIWExtsINoM=.f928e285-654d-48a5-9996-a3c4477b0c9c@github.com>
On Tue, 17 Nov 2020 11:15:15 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Bernd Mathiske has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Removetwo uneeded declarations.
>
> Nice changes. A few questions too...
>
> BTW, I'd like to bring in latest upstream changes so that we don't diverge too much. In particular, this will bring in concurrent weak reference processing. I suggest that I do this after this PR is integrated, ok?
Please feel free to bring in upstream changes any time. Meanwhile, I'll prepare the next PR, but I will be happy to rebase it before landing.
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/7
From bmathiske at openjdk.java.net Tue Nov 17 13:40:39 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Tue, 17 Nov 2020 13:40:39 GMT
Subject: RFR: Generation affiliation transitions for heap regions. [v3]
In-Reply-To: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
Message-ID: <tmhOOxtnCkPLJhcstRbpyu9RETRNvQahw0D1kaMJjaM=.b2142a15-5886-495c-ae0c-ac77b30351b0@github.com>
> Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
>
> The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
>
> The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
Bernd Mathiske has updated the pull request incrementally with one additional commit since the last revision:
Add missing precomiled header include for Windows builds.
-------------
Changes:
- all: https://git.openjdk.java.net/shenandoah/pull/7/files
- new: https://git.openjdk.java.net/shenandoah/pull/7/files/dadb608e..131a8b3b
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=7&range=02
- incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=7&range=01-02
Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
Patch: https://git.openjdk.java.net/shenandoah/pull/7.diff
Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/7/head:pull/7
PR: https://git.openjdk.java.net/shenandoah/pull/7
From roland at openjdk.java.net Tue Nov 17 13:52:08 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Tue, 17 Nov 2020 13:52:08 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah
In-Reply-To: <wKaJDOduE_A6NtWVMR7SElWF0QMC0K9u60XMzkP4z84=.68cd805e-f5b2-4188-8636-288404cc6529@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
<5gcJ-y5dzLhf1AAx2Hl4I5Lo3EBbu6zvdia5d_qzuAU=.c0fcfef9-6429-4481-bb7d-9a43227476ef@github.com>
<wKaJDOduE_A6NtWVMR7SElWF0QMC0K9u60XMzkP4z84=.68cd805e-f5b2-4188-8636-288404cc6529@github.com>
Message-ID: <7xRg4xNeDVIQI8KOYK_-7nnwBGBZSKaxu3KL-rpECHs=.bce2af90-9de9-4908-821d-30691b610695@github.com>
On Tue, 17 Nov 2020 13:47:25 GMT, Roland Westrelin <roland at openjdk.org> wrote:
>> hi, @rwestrel
>>
>> Can we have an assertion to make this loop invariant more prominent?
>> `I think the cleaner fix is to preserve the invariant that the never branch is always right after the loop head in an infinite loop. `
>
>> Can we have an assertion to make this loop invariant more prominent?
>> `I think the cleaner fix is to preserve the invariant that the never branch is always right after the loop head in an infinite loop. `
>
> Thanks for the suggestion. Actually I reworked the fix so it has nothing to do with the previous one.
I pushed a new fix that's different from the previous one. I realized that NeverBranchNode::Ideal() should remove the NeverBranch once the barrier is expanded but we don't run igvn after barrier expansion which is a mistake.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From roland at openjdk.java.net Tue Nov 17 13:52:08 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Tue, 17 Nov 2020 13:52:08 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah
In-Reply-To: <5gcJ-y5dzLhf1AAx2Hl4I5Lo3EBbu6zvdia5d_qzuAU=.c0fcfef9-6429-4481-bb7d-9a43227476ef@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
<5gcJ-y5dzLhf1AAx2Hl4I5Lo3EBbu6zvdia5d_qzuAU=.c0fcfef9-6429-4481-bb7d-9a43227476ef@github.com>
Message-ID: <wKaJDOduE_A6NtWVMR7SElWF0QMC0K9u60XMzkP4z84=.68cd805e-f5b2-4188-8636-288404cc6529@github.com>
On Fri, 13 Nov 2020 06:54:18 GMT, Xin Liu <xliu at openjdk.org> wrote:
> Can we have an assertion to make this loop invariant more prominent?
> `I think the cleaner fix is to preserve the invariant that the never branch is always right after the loop head in an infinite loop. `
Thanks for the suggestion. Actually I reworked the fix so it has nothing to do with the previous one.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From rkennke at openjdk.java.net Tue Nov 17 15:30:17 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 17 Nov 2020 15:30:17 GMT
Subject: RFR: 8256426: Shenandoah: Remove superfluous assert is
ShBS::load_reference_barrier()
Message-ID: <URHj--NZpQ2lEWdNOt2Z_THTMUqC9KumRtBPD8AMcaI=.7ab79b3f-b4ed-4c6f-9ec3-3b69539ce209@github.com>
Some of heap walk related tests in nsk/jdi suite failed after concurrent weak reference processing.
Can be easily reproduced:
TEST_VM_OPTS="-XX:+UseShenandoahGC" make CONF=linux-x86_64-server-fastdebug run-test TEST=vmTestbase/nsk/jdi/ObjectReference/referringObjects/referringObjects001/referringObjects001.java
It asserts that no non-Java thread accesses references with weak/unknown strength. However, JVMTI will do just that when heap-walking and I can't see how that would be problematic, as long as we don't resurrect objects, and we don't.
Testing:
- [x] the failing test
- [x] vmTestbase_nsk_jdi +UseShenandoahGC
- [ ] hotspot_gc_shenandoah
- [ ] tier1 +UseShenandoahGC
- [ ] tier2 +UseShenandoahGC
-------------
Commit messages:
- 8256426: Shenandoah: Remove superfluous assert is ShBS::load_reference_barrier()
Changes: https://git.openjdk.java.net/jdk/pull/1261/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1261&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256426
Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
Patch: https://git.openjdk.java.net/jdk/pull/1261.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1261/head:pull/1261
PR: https://git.openjdk.java.net/jdk/pull/1261
From github.com+71722661+earthling-amzn at openjdk.java.net Tue Nov 17 16:58:16 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Tue, 17 Nov 2020 16:58:16 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v5]
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <1ObVZ6s6Ot_2pxEf1axEwRoexTWLAmpO_F1Q-S9CB4k=.0c716b79-dfdd-41cd-8b17-e12eecf47826@github.com>
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
earthling-amzn has updated the pull request incrementally with one additional commit since the last revision:
Fix wrong type for os::elapsedTime
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1099/files
- new: https://git.openjdk.java.net/jdk/pull/1099/files/3ec8f3c5..f6bb0391
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=04
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=03-04
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1099.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1099/head:pull/1099
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Tue Nov 17 17:03:02 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 17 Nov 2020 17:03:02 GMT
Subject: RFR: 8256426: Shenandoah: Remove superfluous assert is
ShBS::load_reference_barrier()
In-Reply-To: <URHj--NZpQ2lEWdNOt2Z_THTMUqC9KumRtBPD8AMcaI=.7ab79b3f-b4ed-4c6f-9ec3-3b69539ce209@github.com>
References: <URHj--NZpQ2lEWdNOt2Z_THTMUqC9KumRtBPD8AMcaI=.7ab79b3f-b4ed-4c6f-9ec3-3b69539ce209@github.com>
Message-ID: <ojA96V2LFwrlH46DuEXZ05KWw3821F2nAwFLI2t5Ncc=.8cf679c0-a711-42d3-94f4-fe72690e53c7@github.com>
On Tue, 17 Nov 2020 15:25:37 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Some of heap walk related tests in nsk/jdi suite failed after concurrent weak reference processing.
>
> Can be easily reproduced:
> TEST_VM_OPTS="-XX:+UseShenandoahGC" make CONF=linux-x86_64-server-fastdebug run-test TEST=vmTestbase/nsk/jdi/ObjectReference/referringObjects/referringObjects001/referringObjects001.java
>
> It asserts that no non-Java thread accesses references with weak/unknown strength. However, JVMTI will do just that when heap-walking and I can't see how that would be problematic, as long as we don't resurrect objects, and we don't.
>
> Testing:
> - [x] the failing test
> - [x] vmTestbase_nsk_jdi +UseShenandoahGC
> - [ ] hotspot_gc_shenandoah
> - [ ] tier1 +UseShenandoahGC
> - [ ] tier2 +UseShenandoahGC
Ok, fine.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1261
From github.com+71722661+earthling-amzn at openjdk.java.net Tue Nov 17 17:07:10 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Tue, 17 Nov 2020 17:07:10 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
Message-ID: <aKb1Q8HifNM2W_UVJRzXIFTUPqJt7mrww7b0JFXug3w=.1177ee33-e553-4b33-a00a-3280dc1a307d@github.com>
On Mon, 16 Nov 2020 08:39:26 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with six additional commits since the last revision:
>>
>> - Inline calls to gc decision methods (vestige of an earlier design)
>> - Use os::elapsedTime to avoid type issues and to be consistent with other heuristics code
>> - Reuse instantaneous_rate method instead of duplicating code
>> - Rename variables to improve readability
>> - Make logging messages more consistent
>> - Restore call to reset allocation counter at cycle start
>
> I did some performance tests on the variant of this patch, and while it regresses throughput in some scenarios, that's the price we pay for additional spike safety. I think we can drop current alloc-spike and penalty tracking (even from `shenandoahHeuristics`?).
>
> Another round of tune-ups follows.
@shipilev , what workload are you running? I'd like to run it as well.
I'm reluctant to remove the headroom and penalty adjustments. They're an additional layer of safety and let the user provide information about the application that we wouldn't otherwise know. Perhaps we could externalize the penalty adjustments for headroom as command line arguments? If somebody wants to run closer to the edge, they could set the headroom and penalties to zero.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From rkennke at openjdk.java.net Tue Nov 17 17:13:06 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 17 Nov 2020 17:13:06 GMT
Subject: Integrated: 8256426: Shenandoah: Remove superfluous assert is
ShBS::load_reference_barrier()
In-Reply-To: <URHj--NZpQ2lEWdNOt2Z_THTMUqC9KumRtBPD8AMcaI=.7ab79b3f-b4ed-4c6f-9ec3-3b69539ce209@github.com>
References: <URHj--NZpQ2lEWdNOt2Z_THTMUqC9KumRtBPD8AMcaI=.7ab79b3f-b4ed-4c6f-9ec3-3b69539ce209@github.com>
Message-ID: <L5h7a95QbnW0Hbvo7rmE1y1qtaYdwe-KzsAYlD8iwEo=.24b271b9-aec7-47a6-84b9-a0dc9be7315b@github.com>
On Tue, 17 Nov 2020 15:25:37 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Some of heap walk related tests in nsk/jdi suite failed after concurrent weak reference processing.
>
> Can be easily reproduced:
> TEST_VM_OPTS="-XX:+UseShenandoahGC" make CONF=linux-x86_64-server-fastdebug run-test TEST=vmTestbase/nsk/jdi/ObjectReference/referringObjects/referringObjects001/referringObjects001.java
>
> It asserts that no non-Java thread accesses references with weak/unknown strength. However, JVMTI will do just that when heap-walking and I can't see how that would be problematic, as long as we don't resurrect objects, and we don't.
>
> Testing:
> - [x] the failing test
> - [x] vmTestbase_nsk_jdi +UseShenandoahGC
> - [x] hotspot_gc_shenandoah
> - [x] tier1 +UseShenandoahGC
> - [x] tier2 +UseShenandoahGC
This pull request has now been integrated.
Changeset: 9efbb463
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/9efbb463
Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
8256426: Shenandoah: Remove superfluous assert is ShBS::load_reference_barrier()
Reviewed-by: shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1261
From shade at openjdk.java.net Tue Nov 17 17:15:05 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 17 Nov 2020 17:15:05 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
Message-ID: <t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
On Mon, 16 Nov 2020 08:39:26 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with six additional commits since the last revision:
>>
>> - Inline calls to gc decision methods (vestige of an earlier design)
>> - Use os::elapsedTime to avoid type issues and to be consistent with other heuristics code
>> - Reuse instantaneous_rate method instead of duplicating code
>> - Rename variables to improve readability
>> - Make logging messages more consistent
>> - Restore call to reset allocation counter at cycle start
>
> I did some performance tests on the variant of this patch, and while it regresses throughput in some scenarios, that's the price we pay for additional spike safety. I think we can drop current alloc-spike and penalty tracking (even from `shenandoahHeuristics`?).
>
> Another round of tune-ups follows.
> @shipilev , what workload are you running? I'd like to run it as well.
SPECjbb2015 and SPECjvm2008, hacked^W adapted for newer JDKs.
> I'm reluctant to remove the headroom and penalty adjustments. They're an additional layer of safety and let the user provide information about the application that we wouldn't otherwise know. Perhaps we could externalize the penalty adjustments for headroom as command line arguments? If somebody wants to run closer to the edge, they could set the headroom and penalties to zero.
I think spike threshold already covers both parts. Does HyperAlloc perform worse if you remove both `AllocSpikeFactor` and GC/Degen penalties? If you are unsure, we can leave these as is. But the flipside is that we make heuristics hoard more free memory than it already does.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Tue Nov 17 18:17:06 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 17 Nov 2020 18:17:06 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
<t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
Message-ID: <1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
On Tue, 17 Nov 2020 17:12:01 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> I did some performance tests on the variant of this patch, and while it regresses throughput in some scenarios, that's the price we pay for additional spike safety. I think we can drop current alloc-spike and penalty tracking (even from `shenandoahHeuristics`?).
>>
>> Another round of tune-ups follows.
>
>> @shipilev , what workload are you running? I'd like to run it as well.
>
> SPECjbb2015 and SPECjvm2008, hacked^W adapted for newer JDKs.
>
>> I'm reluctant to remove the headroom and penalty adjustments. They're an additional layer of safety and let the user provide information about the application that we wouldn't otherwise know. Perhaps we could externalize the penalty adjustments for headroom as command line arguments? If somebody wants to run closer to the edge, they could set the headroom and penalties to zero.
>
> I think spike threshold already covers both parts. Does HyperAlloc perform worse if you remove both `AllocSpikeFactor` and GC/Degen penalties? If you are unsure, we can leave these as is. But the flipside is that we make heuristics hoard more free memory than it already does.
I think we are nearing the integration. Please merge (don't rebase) from master to get GH actions updates. (In fact, I cannot see "Testing" section in your first comment -- that one is supposed to be added by bots. Do you have GH actions disabled?)
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From zgu at openjdk.java.net Tue Nov 17 19:35:10 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 17 Nov 2020 19:35:10 GMT
Subject: RFR: 8256415: Shenandoah: Should evacuate/update codecache
concurrently when class unloading is off
Message-ID: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
During isolating GC work (JDK-8255765), I found that Shenandoah still evacuate/update codecache roots at final mark safepoint when class unloading is off.
Actually, it can be done concurrently by utilizing nmethod_entry_barrier.
Shenandoah should always enable nmethod_entry_barrier for evacuation, regardless class unloading status, and perform code cache evacuation/updating during strong root processing.
Key points:
1) Always enable nmethod entry barrier
2) Always attach ShenandoahNMethod to nmethod, regardless weather class unloading is enabled
3) Only evacuate/update thread and serial weak roots at final mark pause.
4) Code roots are evacuated/updated concurrently during weak roots (when class unloading is on) or strong roots (when class unloading is off)
- [x] hotspot_gc_shenandoah
- [x] tier1 with ShenandoahGC and +/-ClassUnloading
-------------
Commit messages:
- Always disarm on-stack nmethods during init evaculation
- Merge branch 'master' into JDK-8256415-conc-code-no-classunloading
- Evacuate/update code roots concurrently when class unloading is off
Changes: https://git.openjdk.java.net/jdk/pull/1271/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1271&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256415
Stats: 170 lines in 10 files changed: 56 ins; 90 del; 24 mod
Patch: https://git.openjdk.java.net/jdk/pull/1271.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1271/head:pull/1271
PR: https://git.openjdk.java.net/jdk/pull/1271
From github.com+71722661+earthling-amzn at openjdk.java.net Tue Nov 17 19:36:06 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Tue, 17 Nov 2020 19:36:06 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
<t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
<1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
Message-ID: <3_BiEmzQbP-BcNrPIHmDZfm8ooysWFoGN5Lyo90Dw88=.e071f740-c83c-4e06-9784-b42bb60cc922@github.com>
On Tue, 17 Nov 2020 18:13:47 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>>> @shipilev , what workload are you running? I'd like to run it as well.
>>
>> SPECjbb2015 and SPECjvm2008, hacked^W adapted for newer JDKs.
>>
>>> I'm reluctant to remove the headroom and penalty adjustments. They're an additional layer of safety and let the user provide information about the application that we wouldn't otherwise know. Perhaps we could externalize the penalty adjustments for headroom as command line arguments? If somebody wants to run closer to the edge, they could set the headroom and penalties to zero.
>>
>> I think spike threshold already covers both parts. Does HyperAlloc perform worse if you remove both `AllocSpikeFactor` and GC/Degen penalties? If you are unsure, we can leave these as is. But the flipside is that we make heuristics hoard more free memory than it already does.
>
> I think we are nearing the integration. Please merge (don't rebase) from master to get GH actions updates. (In fact, I cannot see "Testing" section in your first comment -- that one is supposed to be added by bots. Do you have GH actions disabled?)
I don't have GH actions disabled, but I'll merge from master. I'm running more tests after all these changes (there was a regression when we changed from `os::javaTimeNanos` to `os::elapsedTime`).
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From zgu at openjdk.java.net Tue Nov 17 19:57:16 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 17 Nov 2020 19:57:16 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v11]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <spM5sJrFi9n9DWk7RgHdks0I6hSwD99kf7hnjXBlrmg=.cfb5d86b-c78b-4925-807f-6e5d5b312cf7@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits:
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Removed obsoleted class
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Move weak reference processing out of STWMark and fix its timings
- ... and 7 more: https://git.openjdk.java.net/jdk/compare/9efbb463...701880d5
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=10
Stats: 1969 lines in 21 files changed: 1072 ins; 738 del; 159 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From rkennke at openjdk.java.net Tue Nov 17 21:27:22 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 17 Nov 2020 21:27:22 GMT
Subject: RFR: Generation affiliation transitions for heap regions. [v3]
In-Reply-To: <tmhOOxtnCkPLJhcstRbpyu9RETRNvQahw0D1kaMJjaM=.b2142a15-5886-495c-ae0c-ac77b30351b0@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
<tmhOOxtnCkPLJhcstRbpyu9RETRNvQahw0D1kaMJjaM=.b2142a15-5886-495c-ae0c-ac77b30351b0@github.com>
Message-ID: <bSVkMNzo_yPQ_hjB2XbR2Jhy9GKPSkf78hRcQBLyXWc=.df4090f1-cae3-421f-a519-199d36fa1cd0@github.com>
On Tue, 17 Nov 2020 13:40:39 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:
>> Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
>>
>> The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
>>
>> The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
>
> Bernd Mathiske has updated the pull request incrementally with one additional commit since the last revision:
>
> Add missing precomiled header include for Windows builds.
Looks good to me! Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/shenandoah/pull/7
From rkennke at openjdk.java.net Tue Nov 17 21:37:06 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 17 Nov 2020 21:37:06 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v11]
In-Reply-To: <spM5sJrFi9n9DWk7RgHdks0I6hSwD99kf7hnjXBlrmg=.cfb5d86b-c78b-4925-807f-6e5d5b312cf7@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
<spM5sJrFi9n9DWk7RgHdks0I6hSwD99kf7hnjXBlrmg=.cfb5d86b-c78b-4925-807f-6e5d5b312cf7@github.com>
Message-ID: <VvWAZgTG68CQ0PNPQWtIa2Un02Mfoy_tfksKPTf5KMg=.abfd67e8-f314-41c0-b934-181b05770eac@github.com>
On Tue, 17 Nov 2020 19:57:16 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>>
>> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>>
>> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>>
>> First step, I would like to split STW and concurrent mark, so that:
>> 1) Code has to special case for STW and concurrent mark.
>> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
>> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
>> 4) STW mark does not need to remark some of roots.
>>
>> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>>
>> A few changes:
>> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
>> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
>> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
>> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
>
> Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits:
>
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Removed obsoleted class
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge
> - Move weak reference processing out of STWMark and fix its timings
> - ... and 7 more: https://git.openjdk.java.net/jdk/compare/9efbb463...701880d5
Overall a great refactoring!
I wonder if we should keep marking state in a single class e.g. ShenandoahMarkingState or even just add it to ShenandoahMarkingContext, instead of scattering it over ShenandoahHeap?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1009
From rkennke at openjdk.java.net Tue Nov 17 21:51:11 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 17 Nov 2020 21:51:11 GMT
Subject: RFR: 8256415: Shenandoah: Should evacuate/update codecache
concurrently when class unloading is off
In-Reply-To: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
References: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
Message-ID: <8yUfr1RPiAGSH5dHf_YIQ1z_60HJtSXYgCaX2aTJUIs=.aad2f58a-e18c-4bb2-8e17-3810ea969c4c@github.com>
On Tue, 17 Nov 2020 19:28:18 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> During isolating GC work (JDK-8255765), I found that Shenandoah still evacuate/update codecache roots at final mark safepoint when class unloading is off.
>
> Actually, it can be done concurrently by utilizing nmethod_entry_barrier.
>
> Shenandoah should always enable nmethod_entry_barrier for evacuation, regardless class unloading status, and perform code cache evacuation/updating during strong root processing.
>
> Key points:
> 1) Always enable nmethod entry barrier
> 2) Always attach ShenandoahNMethod to nmethod, regardless weather class unloading is enabled
> 3) Only evacuate/update thread and serial weak roots at final mark pause.
> 4) Code roots are evacuated/updated concurrently during weak roots (when class unloading is on) or strong roots (when class unloading is off)
>
>
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with ShenandoahGC and +/-ClassUnloading
Looks good, except change one comment a little.
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1746:
> 1744: if (!is_degenerated_gc_in_progress()) {
> 1745: // We need to arm nmethods for concurrent codecache process, regardless concurrent
> 1746: // class unloading
Comment should probably not refer to previous implementations. Future readers will not see the connection there.
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1271
From zgu at openjdk.java.net Tue Nov 17 22:37:18 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 17 Nov 2020 22:37:18 GMT
Subject: RFR: 8256415: Shenandoah: Should evacuate/update codecache
concurrently when class unloading is off [v2]
In-Reply-To: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
References: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
Message-ID: <w4kpnKlOA79H8sfvS4O1m4cC0ypN2iWTiSYImS4GevA=.4c8fc760-cd7e-44da-b859-9b5f7e3fddad@github.com>
> During isolating GC work (JDK-8255765), I found that Shenandoah still evacuate/update codecache roots at final mark safepoint when class unloading is off.
>
> Actually, it can be done concurrently by utilizing nmethod_entry_barrier.
>
> Shenandoah should always enable nmethod_entry_barrier for evacuation, regardless class unloading status, and perform code cache evacuation/updating during strong root processing.
>
> Key points:
> 1) Always enable nmethod entry barrier
> 2) Always attach ShenandoahNMethod to nmethod, regardless weather class unloading is enabled
> 3) Only evacuate/update thread and serial weak roots at final mark pause.
> 4) Code roots are evacuated/updated concurrently during weak roots (when class unloading is on) or strong roots (when class unloading is off)
>
>
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with ShenandoahGC and +/-ClassUnloading
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Update comment
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1271/files
- new: https://git.openjdk.java.net/jdk/pull/1271/files/5e96a2c5..76ba8735
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1271&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1271&range=00-01
Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1271.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1271/head:pull/1271
PR: https://git.openjdk.java.net/jdk/pull/1271
From zgu at openjdk.java.net Wed Nov 18 00:32:25 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 18 Nov 2020 00:32:25 GMT
Subject: RFR: 8256415: Shenandoah: Should evacuate/update codecache
concurrently when class unloading is off [v3]
In-Reply-To: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
References: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
Message-ID: <IwpZSb8U0LJ58YRdOQhtaLlf2-w2zS7gcGqIRCjeGXk=.7b1e7d7a-bea4-4f5d-9450-e435fc918d0d@github.com>
> During isolating GC work (JDK-8255765), I found that Shenandoah still evacuate/update codecache roots at final mark safepoint when class unloading is off.
>
> Actually, it can be done concurrently by utilizing nmethod_entry_barrier.
>
> Shenandoah should always enable nmethod_entry_barrier for evacuation, regardless class unloading status, and perform code cache evacuation/updating during strong root processing.
>
> Key points:
> 1) Always enable nmethod entry barrier
> 2) Always attach ShenandoahNMethod to nmethod, regardless weather class unloading is enabled
> 3) Only evacuate/update thread and serial weak roots at final mark pause.
> 4) Code roots are evacuated/updated concurrently during weak roots (when class unloading is on) or strong roots (when class unloading is off)
>
>
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with ShenandoahGC and +/-ClassUnloading
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Update comment
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1271/files
- new: https://git.openjdk.java.net/jdk/pull/1271/files/76ba8735..9884043f
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1271&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1271&range=01-02
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.java.net/jdk/pull/1271.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1271/head:pull/1271
PR: https://git.openjdk.java.net/jdk/pull/1271
From zgu at openjdk.java.net Wed Nov 18 00:32:26 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 18 Nov 2020 00:32:26 GMT
Subject: RFR: 8256415: Shenandoah: Should evacuate/update codecache
concurrently when class unloading is off [v3]
In-Reply-To: <8yUfr1RPiAGSH5dHf_YIQ1z_60HJtSXYgCaX2aTJUIs=.aad2f58a-e18c-4bb2-8e17-3810ea969c4c@github.com>
References: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
<8yUfr1RPiAGSH5dHf_YIQ1z_60HJtSXYgCaX2aTJUIs=.aad2f58a-e18c-4bb2-8e17-3810ea969c4c@github.com>
Message-ID: <87V__XnKoEND6jQenESFiyvJQaLHMlT8gOVh2N_ImDI=.3aa824e8-15f3-4665-ad34-188b58203803@github.com>
On Tue, 17 Nov 2020 21:46:05 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Update comment
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1746:
>
>> 1744: if (!is_degenerated_gc_in_progress()) {
>> 1745: // We need to arm nmethods for concurrent codecache process, regardless concurrent
>> 1746: // class unloading
>
> Comment should probably not refer to previous implementations. Future readers will not see the connection there.
Thanks for reviewing.
Comment updated to "Arm nmethods for concurrent codecache processing."
-------------
PR: https://git.openjdk.java.net/jdk/pull/1271
From xliu at openjdk.java.net Wed Nov 18 02:11:07 2020
From: xliu at openjdk.java.net (Xin Liu)
Date: Wed, 18 Nov 2020 02:11:07 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah
In-Reply-To: <7xRg4xNeDVIQI8KOYK_-7nnwBGBZSKaxu3KL-rpECHs=.bce2af90-9de9-4908-821d-30691b610695@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
<5gcJ-y5dzLhf1AAx2Hl4I5Lo3EBbu6zvdia5d_qzuAU=.c0fcfef9-6429-4481-bb7d-9a43227476ef@github.com>
<wKaJDOduE_A6NtWVMR7SElWF0QMC0K9u60XMzkP4z84=.68cd805e-f5b2-4188-8636-288404cc6529@github.com>
<7xRg4xNeDVIQI8KOYK_-7nnwBGBZSKaxu3KL-rpECHs=.bce2af90-9de9-4908-821d-30691b610695@github.com>
Message-ID: <Qw6aQdB49o2Z2d3XKgo7JPi2jQVGiXv-9bvGPgVIE5c=.ba32a829-3d1e-43e0-ad75-db52b29bdac6@github.com>
On Tue, 17 Nov 2020 13:49:34 GMT, Roland Westrelin <roland at openjdk.org> wrote:
>>> Can we have an assertion to make this loop invariant more prominent?
>>> `I think the cleaner fix is to preserve the invariant that the never branch is always right after the loop head in an infinite loop. `
>>
>> Thanks for the suggestion. Actually I reworked the fix so it has nothing to do with the previous one.
>
> I pushed a new fix that's different from the previous one. I realized that NeverBranchNode::Ideal() should remove the NeverBranch once the barrier is expanded but we don't run igvn after barrier expansion which is a mistake.
yes, looks good to me.
Happy to see a more elegant way to solve this problem.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From thartmann at openjdk.java.net Wed Nov 18 08:05:08 2020
From: thartmann at openjdk.java.net (Tobias Hartmann)
Date: Wed, 18 Nov 2020 08:05:08 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah [v4]
In-Reply-To: <jGIIRVQHL2oyAk7SOQY5021eCxfpvTnrh628pMdj8o4=.76ff5729-705b-4a70-b1bf-2143608eec86@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
<jGIIRVQHL2oyAk7SOQY5021eCxfpvTnrh628pMdj8o4=.76ff5729-705b-4a70-b1bf-2143608eec86@github.com>
Message-ID: <W9a5Vnqls4O1l2d8kU-UXySFqU8num4Msb0-roy9iE4=.772a893f-2b45-4a8a-8141-8e0d14848466@github.com>
On Tue, 17 Nov 2020 12:38:23 GMT, Roland Westrelin <roland at openjdk.org> wrote:
>> This is a Shenandoah bug but the proposed fix is in shared code.
>>
>> In an infinite loop, a barrier is located right after the loop head
>> and above the never branch. When the barrier is expanded, control flow
>> is added between the loop and the never branch. During loop
>> verification the assert fires because it doesn't expect any control
>> flow between the never branch and the loop head.
>>
>> While it would have been nice to fix this Shenandoah issue in
>> Shenandoah code, I think the cleaner fix is to preserve the invariant
>> that the never branch is always right after the loop head in an
>> infinite loop. In the proposed patch, this is achieved by moving all
>> uses of the loop head to the never branch when it's constructed.
>
> Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
>
> - fix
> - test
The reworked fix looks good to me too.
-------------
Marked as reviewed by thartmann (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1073
From rkennke at openjdk.java.net Wed Nov 18 08:59:09 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 18 Nov 2020 08:59:09 GMT
Subject: RFR: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah [v4]
In-Reply-To: <jGIIRVQHL2oyAk7SOQY5021eCxfpvTnrh628pMdj8o4=.76ff5729-705b-4a70-b1bf-2143608eec86@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
<jGIIRVQHL2oyAk7SOQY5021eCxfpvTnrh628pMdj8o4=.76ff5729-705b-4a70-b1bf-2143608eec86@github.com>
Message-ID: <0Z4Kw5n0Db87jhFM59XmeOMEOcIY6PO2WzqI6Vp-Tlg=.4651e1c6-5b2f-4135-9250-f12c6e65f7bf@github.com>
On Tue, 17 Nov 2020 12:38:23 GMT, Roland Westrelin <roland at openjdk.org> wrote:
>> This is a Shenandoah bug but the proposed fix is in shared code.
>>
>> In an infinite loop, a barrier is located right after the loop head
>> and above the never branch. When the barrier is expanded, control flow
>> is added between the loop and the never branch. During loop
>> verification the assert fires because it doesn't expect any control
>> flow between the never branch and the loop head.
>>
>> While it would have been nice to fix this Shenandoah issue in
>> Shenandoah code, I think the cleaner fix is to preserve the invariant
>> that the never branch is always right after the loop head in an
>> infinite loop. In the proposed patch, this is achieved by moving all
>> uses of the loop head to the never branch when it's constructed.
>
> Roland Westrelin has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits:
>
> - fix
> - test
Looks good to me! Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1073
From roland at openjdk.java.net Wed Nov 18 09:28:04 2020
From: roland at openjdk.java.net (Roland Westrelin)
Date: Wed, 18 Nov 2020 09:28:04 GMT
Subject: Integrated: 8255936: "parsing found no loops but there are some"
assertion failure with Shenandoah
In-Reply-To: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
References: <m3lhlnDDwxKNdEY_rKtszeAASuvrqGNcnqXMCkpOUH4=.9c3acbf9-b6fe-4db8-ab1b-ba46b569902e@github.com>
Message-ID: <oyJqhb4TJOHk_BthaZS2emfJu053r3e86ocUAgsC7kw=.a7dfbc63-b79c-430e-b7ae-e1f3e5ee250c@github.com>
On Thu, 5 Nov 2020 08:44:03 GMT, Roland Westrelin <roland at openjdk.org> wrote:
> This is a Shenandoah bug but the proposed fix is in shared code.
>
> In an infinite loop, a barrier is located right after the loop head
> and above the never branch. When the barrier is expanded, control flow
> is added between the loop and the never branch. During loop
> verification the assert fires because it doesn't expect any control
> flow between the never branch and the loop head.
>
> While it would have been nice to fix this Shenandoah issue in
> Shenandoah code, I think the cleaner fix is to preserve the invariant
> that the never branch is always right after the loop head in an
> infinite loop. In the proposed patch, this is achieved by moving all
> uses of the loop head to the never branch when it's constructed.
This pull request has now been integrated.
Changeset: 655bb619
Author: Roland Westrelin <roland at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/655bb619
Stats: 3 lines in 2 files changed: 2 ins; 0 del; 1 mod
8255936: "parsing found no loops but there are some" assertion failure with Shenandoah
Reviewed-by: thartmann, rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1073
From bmathiske at openjdk.java.net Wed Nov 18 10:06:25 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Wed, 18 Nov 2020 10:06:25 GMT
Subject: Integrated: Generation affiliation transitions for heap regions.
In-Reply-To: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
References: <0ejD0m7JFotdWrFr6xVngqrlVbGjYnm4CPOitwq82wI=.7a0232b2-ac80-41e4-abb9-8f3df2360aed@github.com>
Message-ID: <xqO1dW0fnKopmA28sREyGmMHCnfcB4QkqdwIfJ6XPGk=.6af1533c-b907-46a7-9ca0-a60dc8301578@github.com>
On Tue, 17 Nov 2020 10:31:20 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:
> Preparatory renaming generation->affiliation since we will want to have a class called ShenandoahGeneration that is not an enum. The next PR will build on this. It will be a big refactoring that pulls code out of the ShenandoahHeap class and puts it into generation-specific classes that are subclasses of ShenandoahGeneration.
>
> The Affiliation declaration still sits in AllocRequest and not in ShenandoahHeapRegion or ShenandoahGeneration for the same reason as before: it is hard to get anything to compile in other combinations of what goes where. I tried.
>
> The card table update code in the set_affiliation() will be revised and augmented later, but seems OK as a first step.
This pull request has now been integrated.
Changeset: 6620f0eb
Author: Bernd Mathiske <bmathiske at openjdk.org>
Committer: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.java.net/shenandoah/commit/6620f0eb
Stats: 177 lines in 9 files changed: 138 ins; 5 del; 34 mod
Generation affiliation transitions for heap regions.
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/shenandoah/pull/7
From duke at openjdk.java.net Wed Nov 18 10:19:22 2020
From: duke at openjdk.java.net (duke)
Date: Wed, 18 Nov 2020 10:19:22 GMT
Subject: git: openjdk/shenandoah-visualizer: master: Minor cleanups after code
inspection
Message-ID: <cb208305-997a-4da3-9018-aa59ca6c196c@openjdk.java.net>
Changeset: bc55c9c0
Author: Aleksey Shipilev <shade at redhat.com>
Date: 2020-11-18 11:18:14 +0000
URL: https://github.com/openjdk/shenandoah-visualizer/commit/bc55c9c0
Minor cleanups after code inspection
! src/main/java/org/openjdk/shenandoah/DataProvider.java
! src/main/java/org/openjdk/shenandoah/RegionStat.java
! src/main/java/org/openjdk/shenandoah/ShenandoahVisualizer.java
From shade at redhat.com Wed Nov 18 12:09:55 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 18 Nov 2020 13:09:55 +0100
Subject: [11u] RFR (S) 8255457: Shenandoah: cleanup ShenandoahMarkTask
Message-ID: <a88e7043-30cb-158d-b21d-e714cc492c5d@redhat.com>
Original RFE:
https://bugs.openjdk.java.net/browse/JDK-8255457
https://git.openjdk.java.net/jdk/commit/1215b1a8
This patch does not apply cleanly to 11u, because of the contextual differences. Most significant
difference is that in HEAD JDK the affected class is trivially copyable, and the shared code already
adjusted to accept that. 11u, however, would fail to compile due to odd cv-stripping casts.
Therefore, I had to keep copying constructors and assignment operators.
11u webrev:
https://cr.openjdk.java.net/~shade/8255457/webrev.11u.01/
Testing: Linux {x86_64, x86_32} hotspot_gc_shenandoah
--
Thanks,
-Aleksey
From bmathiske at openjdk.java.net Wed Nov 18 12:36:55 2020
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Wed, 18 Nov 2020 12:36:55 GMT
Subject: RFR: Refactor to differentiate young generation marking.
Message-ID: <4Qfenq7nBOTZi4269M6vny9pxb13_z-d7OHdkw7HH0I=.35f9a949-66f9-4603-9862-cef0121de112@github.com>
This change is large to achieve uninterrupted functionality for single-generational Shenandoah while planting code that differentiates two distinct generations and concurrent marking for the young generation.
One main part of the change is redistributing existing code from the heap class to generation-specific classes for further modifications.
The other is a pervasive case distinction throughout marking code that differentiates between young and global collections. Where necessary we use templates to not impose runtime overhead. In code sections that are not runtime-critical we use conditions and switches over an enum that indicates which generational collection mode we are in (young or global).
We temporarily make all collections in generational mode young collections. Global and old collections will come later.
-------------
Commit messages:
- Refactor to differentiate young generation marking.
Changes: https://git.openjdk.java.net/shenandoah/pull/8/files
Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=8&range=00
Stats: 1400 lines in 21 files changed: 935 ins; 329 del; 136 mod
Patch: https://git.openjdk.java.net/shenandoah/pull/8.diff
Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/8/head:pull/8
PR: https://git.openjdk.java.net/shenandoah/pull/8
From zgu at openjdk.java.net Wed Nov 18 13:03:05 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 18 Nov 2020 13:03:05 GMT
Subject: Integrated: 8256415: Shenandoah: Should evacuate/update codecache
concurrently when class unloading is off
In-Reply-To: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
References: <OG0i59F05POao53hcf7bwM33DOvDtbtHXDIO-yWC6O4=.98648a4a-4a96-447a-a49e-81ab603e1759@github.com>
Message-ID: <VjUgzcecz3rTFOAxgZXfF5yDRuBbt4r10jTPJGRyOZk=.051fc88f-fc2d-4503-ab3d-f17fdcf8644d@github.com>
On Tue, 17 Nov 2020 19:28:18 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> During isolating GC work (JDK-8255765), I found that Shenandoah still evacuate/update codecache roots at final mark safepoint when class unloading is off.
>
> Actually, it can be done concurrently by utilizing nmethod_entry_barrier.
>
> Shenandoah should always enable nmethod_entry_barrier for evacuation, regardless class unloading status, and perform code cache evacuation/updating during strong root processing.
>
> Key points:
> 1) Always enable nmethod entry barrier
> 2) Always attach ShenandoahNMethod to nmethod, regardless weather class unloading is enabled
> 3) Only evacuate/update thread and serial weak roots at final mark pause.
> 4) Code roots are evacuated/updated concurrently during weak roots (when class unloading is on) or strong roots (when class unloading is off)
>
>
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with ShenandoahGC and +/-ClassUnloading
This pull request has now been integrated.
Changeset: eab170c0
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/eab170c0
Stats: 170 lines in 10 files changed: 56 ins; 91 del; 23 mod
8256415: Shenandoah: Should evacuate/update codecache concurrently when class unloading is off
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1271
From rkennke at openjdk.java.net Wed Nov 18 13:47:19 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 18 Nov 2020 13:47:19 GMT
Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs
In-Reply-To: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
References: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
Message-ID: <aBrQ4zNNNOUToycDbVEKsfOt5yfXVUcVlAhbgUmN-JU=.73b858f8-ba80-4499-bdb0-10b058cf8caa@github.com>
On Tue, 17 Nov 2020 19:02:03 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling.
>
> Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah.
>
> Additional testing:
> - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops)
> - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC`
Looks good to me! Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1268
From shade at openjdk.java.net Wed Nov 18 13:47:19 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 18 Nov 2020 13:47:19 GMT
Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs
Message-ID: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling.
Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah.
Additional testing:
- [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops)
- [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC`
-------------
Commit messages:
- Remove TODO
- 8256497: Zero: enable G1 and Shenandoah GCs
Changes: https://git.openjdk.java.net/jdk/pull/1268/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1268&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256497
Stats: 72 lines in 5 files changed: 54 ins; 16 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1268.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1268/head:pull/1268
PR: https://git.openjdk.java.net/jdk/pull/1268
From sspitsyn at openjdk.java.net Wed Nov 18 19:34:12 2020
From: sspitsyn at openjdk.java.net (Serguei Spitsyn)
Date: Wed, 18 Nov 2020 19:34:12 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
Message-ID: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
On Mon, 16 Nov 2020 23:30:25 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>
> Fix minimal build.
Hi Coleen,
It looks good to me.
Just a couple of nits below.
src/hotspot/share/prims/jvmtiTagMap.cpp:
There is a double-check for _needs_cleaning, so the one at line 136 can be removed:
136 if (_needs_cleaning &&
137 post_events &&
138 env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) {
139 remove_dead_entries(true /* post_object_free */);
1158 void JvmtiTagMap::remove_dead_entries(bool post_object_free) {
1159 assert(is_locked(), "precondition");
1160 if (_needs_cleaning) {
1161 log_info(jvmti, table)("TagMap table needs cleaning%s",
1162 (post_object_free ? " and posting" : ""));
1163 hashmap()->remove_dead_entries(env(), post_object_free);
1164 _needs_cleaning = false;
1165 }
1166 }
test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp:
The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same:
- nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni);
+
+ /* Flush any pending ObjectFree events, which may set success to 1 */
+ if (jvmti->SetEventNotificationMode(JVMTI_DISABLE,
+ JVMTI_EVENT_OBJECT_FREE,
+ NULL) != JVMTI_ERROR_NONE) {
+ success = 0;
+ }
+
+ nsk_aod_agentFinished(jni, agentName, success);
}
-------------
Marked as reviewed by sspitsyn (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/967
From zgu at openjdk.java.net Wed Nov 18 19:49:14 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 18 Nov 2020 19:49:14 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v12]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <r66sGM9ajzIWKJHFdTi1xk5gEUbcKLw4hG5Bnpz7pqk=.513e4fe9-bd50-4a58-927b-06f3d9933424@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 18 commits:
- Merge
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Removed obsoleted class
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- ... and 8 more: https://git.openjdk.java.net/jdk/compare/50a2c22f...f65e3da0
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=11
Stats: 1953 lines in 21 files changed: 1064 ins; 730 del; 159 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From zgu at openjdk.java.net Wed Nov 18 22:03:19 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 18 Nov 2020 22:03:19 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v13]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <GmPLECvNFCbROoFaibaR5DuY0_QUOu36iJ5MKpCjZPY=.1f66b9ec-6ce7-48dc-9535-6fd445a77632@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Moved task queues to marking context
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1009/files
- new: https://git.openjdk.java.net/jdk/pull/1009/files/f65e3da0..403ec58d
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=12
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=11-12
Stats: 43 lines in 6 files changed: 16 ins; 19 del; 8 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From erikj at openjdk.java.net Wed Nov 18 22:32:03 2020
From: erikj at openjdk.java.net (Erik Joelsson)
Date: Wed, 18 Nov 2020 22:32:03 GMT
Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs
In-Reply-To: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
References: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
Message-ID: <eCFR6sKwiQ8ysx1dNvUH-HMefNyn82bs1AVwQgEc4KQ=.ccd40b4b-8f88-469c-8628-cac73fe0f20b@github.com>
On Tue, 17 Nov 2020 19:02:03 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling.
>
> Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah.
>
> Additional testing:
> - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops)
> - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC`
Build change looks ok.
-------------
Marked as reviewed by erikj (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1268
From coleenp at openjdk.java.net Wed Nov 18 23:24:12 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 18 Nov 2020 23:24:12 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v10]
In-Reply-To: <4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com>
Message-ID: <6MQhIGtA1LWvXvSFSZWRkyU6d9AXf_LVdkE0zAq0ekc=.151e621a-eec4-4366-8ffa-729eb1d5bb63@github.com>
On Mon, 16 Nov 2020 23:10:21 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>>
>> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>>
>> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>>
>> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
>> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>>
>> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>>
>> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>>
>> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>>
>> It has also been tested with tier1-6.
>>
>> Thank you to Stefan, Erik and Kim for their help with this change.
>
> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits:
>
> - Reverse remove_dead_entries_locked function names.
> - Merge branch 'master' into jvmti-table
> - Add shenandoah set_needs_cleaning but this doesn't work.
> - fix vmTestbase/nsk/jvmti tests
> - improve tagmap cleanup and objectfree event posting
> - Add logging to event posting in case of pauses.
> - Merge branch 'master' into jvmti-table
> - Add back WeakProcessorPhases::Phase enum.
> - Serguei 1.
> - Code review comments from Kim and Albert.
> - ... and 5 more: https://git.openjdk.java.net/jdk/compare/0357db35...daaa13fe
test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp line 72:
> 70: Java_nsk_jvmti_AttachOnDemand_attach021_attach021Target_shutdownAgent(JNIEnv * jni,
> 71: jclass klass) {
> 72:
@kimbarrett ? I'm not sure why you made this change. See Serguei's comment.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Wed Nov 18 23:29:10 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 18 Nov 2020 23:29:10 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
<8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
Message-ID: <F5tWyEnnYqZCtPjA_haRqbwUli59TwojI2jbIjAjjfY=.b947441d-ecac-4d31-8a95-0ba2ddb6bd49@github.com>
On Wed, 18 Nov 2020 19:31:44 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:
> There is a double-check for _needs_cleaning, so the one at line 136 can be removed:
I want to leave _needs_cleaning at 136 because even though the boolean is checked twice, it doesn't hurt performance and it has a nice symmetry in that function.
I asked Kim about the other change.
Thank you for reviewing, Serguei!
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 19 00:20:24 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 19 Nov 2020 00:20:24 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v12]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <c-KbT0JKQMejEPztZP7N2QYMwJiCvJuHjsupyZReVAo=.aa0cd4a8-539a-4749-8e37-dad8713252ff@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits:
- Merge branch 'master' into jvmti-table
- Fix minimal build.
- Reverse remove_dead_entries_locked function names.
- Merge branch 'master' into jvmti-table
- Add shenandoah set_needs_cleaning but this doesn't work.
- fix vmTestbase/nsk/jvmti tests
- improve tagmap cleanup and objectfree event posting
- Add logging to event posting in case of pauses.
- Merge branch 'master' into jvmti-table
- Add back WeakProcessorPhases::Phase enum.
- ... and 7 more: https://git.openjdk.java.net/jdk/compare/2b155713...9ef44f28
-------------
Changes: https://git.openjdk.java.net/jdk/pull/967/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=11
Stats: 1884 lines in 49 files changed: 768 ins; 992 del; 124 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From kbarrett at openjdk.java.net Thu Nov 19 00:43:09 2020
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Thu, 19 Nov 2020 00:43:09 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
<8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
Message-ID: <ywAQf_CJ9rxJlXivn5XU6vk1qlZwPTgGSIooNkkWGwU=.2aeed564-87d7-4bbc-807f-7193c60a9fda@github.com>
On Wed, 18 Nov 2020 19:31:44 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:
> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp:
>
> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same:
>
> ```
> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni);
> +
> + /* Flush any pending ObjectFree events, which may set success to 1 */
> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE,
> + JVMTI_EVENT_OBJECT_FREE,
> + NULL) != JVMTI_ERROR_NONE) {
> + success = 0;
> + }
> +
> + nsk_aod_agentFinished(jni, agentName, success);
> }
> ```
This change really is needed.
The success variable in the test is a global, initially 0, set to 1 by the
ObjectFree handler.
In the old code, if the ObjectFree event hasn't been posted yet, we pass the
initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where
it's a local variable (so unaffected by any changes to the variable in the
test), so stays 0 through to the call to nsk_aod_agentFinished. So the test
fails.
The split in the change causes the updated post-ObjectFree event success
value of 1 to be passed to agentFinished. So the test passes.
That required some head scratching to find at the time. That's the point of
the comment about flushing pending events. Maybe the comment should be
improved.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 19 01:26:35 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 19 Nov 2020 01:26:35 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v6]
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <DykCR83ieEXgOZdkVbeKOG-2YIz_x7GV5IFr6sJDeU0=.9244664f-362b-474e-9be4-e0d72425c032@github.com>
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
earthling-amzn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 17 additional commits since the last revision:
- Remove unused member
- Avoid recomputing instantaneous allocation rate
- Merge branch 'master' into shenandoah-reactive-heuristic
- Fix wrong type for os::elapsedTime
- Remove dependency from allocation rate to adaptive heuristic
- Defend against underflow and division by zero
- Inline calls to gc decision methods (vestige of an earlier design)
- Use os::elapsedTime to avoid type issues and to be consistent with other heuristics code
- Reuse instantaneous_rate method instead of duplicating code
- Rename variables to improve readability
- ... and 7 more: https://git.openjdk.java.net/jdk/compare/7bde62e5...68ba6285
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1099/files
- new: https://git.openjdk.java.net/jdk/pull/1099/files/f6bb0391..68ba6285
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=05
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=04-05
Stats: 77106 lines in 993 files changed: 43874 ins; 22060 del; 11172 mod
Patch: https://git.openjdk.java.net/jdk/pull/1099.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1099/head:pull/1099
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Thu Nov 19 07:21:05 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 19 Nov 2020 07:21:05 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <3_BiEmzQbP-BcNrPIHmDZfm8ooysWFoGN5Lyo90Dw88=.e071f740-c83c-4e06-9784-b42bb60cc922@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
<t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
<1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
<3_BiEmzQbP-BcNrPIHmDZfm8ooysWFoGN5Lyo90Dw88=.e071f740-c83c-4e06-9784-b42bb60cc922@github.com>
Message-ID: <vo5O28vV2lBh8QlKbnFafdHbFbq5PzPSuxHjXzqKRYc=.a446201a-adfc-457f-b67a-abc740547822@github.com>
On Tue, 17 Nov 2020 19:33:07 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> I think we are nearing the integration. Please merge (don't rebase) from master to get GH actions updates. (In fact, I cannot see "Testing" section in your first comment -- that one is supposed to be added by bots. Do you have GH actions disabled?)
>
> I don't have GH actions disabled, but I'll merge from master. I'm running more tests after all these changes (there was a regression when we changed from `os::javaTimeNanos` to `os::elapsedTime`).
I don't understand why GH actions do not test your patch :/ Can you go to https://github.com/earthling-amzn/jdk/settings/ and check that both "Actions" and "Secrets" do not have anything special defined? "Actions" should say "Allow all actions", "Secrets" should not have JDK_SUBMIT_PLATFORMS defined.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Thu Nov 19 08:32:14 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 19 Nov 2020 08:32:14 GMT
Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs [v2]
In-Reply-To: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
References: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
Message-ID: <Jqr8E1u-g9edoPzB9qtKQWfcRL35vrTMolruivwp088=.fe2e1e32-81aa-468b-a177-b86bc617e752@github.com>
> Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling.
>
> Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah.
>
> Additional testing:
> - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops)
> - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC`
Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
- Merge branch 'master' into JDK-8256497-zero-g1-shenandoah
- Remove TODO
- 8256497: Zero: enable G1 and Shenandoah GCs
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1268/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1268&range=01
Stats: 72 lines in 5 files changed: 54 ins; 16 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1268.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1268/head:pull/1268
PR: https://git.openjdk.java.net/jdk/pull/1268
From ihse at openjdk.java.net Thu Nov 19 09:58:05 2020
From: ihse at openjdk.java.net (Magnus Ihse Bursie)
Date: Thu, 19 Nov 2020 09:58:05 GMT
Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs [v2]
In-Reply-To: <Jqr8E1u-g9edoPzB9qtKQWfcRL35vrTMolruivwp088=.fe2e1e32-81aa-468b-a177-b86bc617e752@github.com>
References: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
<Jqr8E1u-g9edoPzB9qtKQWfcRL35vrTMolruivwp088=.fe2e1e32-81aa-468b-a177-b86bc617e752@github.com>
Message-ID: <uAFO6co1K5FQqf7AJYCXNkEEFxGSfZpOAdMpV3tlod4=.76da4f25-7763-4b86-a2d5-aadf68f584f6@github.com>
On Thu, 19 Nov 2020 08:32:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling.
>>
>> Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah.
>>
>> Additional testing:
>> - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops)
>> - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC`
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
>
> - Merge branch 'master' into JDK-8256497-zero-g1-shenandoah
> - Remove TODO
> - 8256497: Zero: enable G1 and Shenandoah GCs
Build changes look good.
-------------
Marked as reviewed by ihse (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1268
From sspitsyn at openjdk.java.net Thu Nov 19 10:13:13 2020
From: sspitsyn at openjdk.java.net (Serguei Spitsyn)
Date: Thu, 19 Nov 2020 10:13:13 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <ywAQf_CJ9rxJlXivn5XU6vk1qlZwPTgGSIooNkkWGwU=.2aeed564-87d7-4bbc-807f-7193c60a9fda@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
<8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
<ywAQf_CJ9rxJlXivn5XU6vk1qlZwPTgGSIooNkkWGwU=.2aeed564-87d7-4bbc-807f-7193c60a9fda@github.com>
Message-ID: <zHyAgV_fjFjDegqqFGUFD_VzVKd02L-qinlNuyqmqjU=.f9dafb62-45fe-4896-a238-9d67d304d749@github.com>
On Thu, 19 Nov 2020 00:39:52 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:
>> Hi Coleen,
>> It looks good to me.
>> Just a couple of nits below.
>>
>> src/hotspot/share/prims/jvmtiTagMap.cpp:
>>
>> There is a double-check for _needs_cleaning, so the one at line 136 can be removed:
>> 136 if (_needs_cleaning &&
>> 137 post_events &&
>> 138 env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) {
>> 139 remove_dead_entries(true /* post_object_free */);
>> 1158 void JvmtiTagMap::remove_dead_entries(bool post_object_free) {
>> 1159 assert(is_locked(), "precondition");
>> 1160 if (_needs_cleaning) {
>> 1161 log_info(jvmti, table)("TagMap table needs cleaning%s",
>> 1162 (post_object_free ? " and posting" : ""));
>> 1163 hashmap()->remove_dead_entries(env(), post_object_free);
>> 1164 _needs_cleaning = false;
>> 1165 }
>> 1166 }
>> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp:
>>
>> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same:
>> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni);
>> +
>> + /* Flush any pending ObjectFree events, which may set success to 1 */
>> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE,
>> + JVMTI_EVENT_OBJECT_FREE,
>> + NULL) != JVMTI_ERROR_NONE) {
>> + success = 0;
>> + }
>> +
>> + nsk_aod_agentFinished(jni, agentName, success);
>> }
>
>> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp:
>>
>> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same:
>>
>> ```
>> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni);
>> +
>> + /* Flush any pending ObjectFree events, which may set success to 1 */
>> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE,
>> + JVMTI_EVENT_OBJECT_FREE,
>> + NULL) != JVMTI_ERROR_NONE) {
>> + success = 0;
>> + }
>> +
>> + nsk_aod_agentFinished(jni, agentName, success);
>> }
>> ```
>
> This change really is needed.
>
> The success variable in the test is a global, initially 0, set to 1 by the
> ObjectFree handler.
>
> In the old code, if the ObjectFree event hasn't been posted yet, we pass the
> initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where
> it's a local variable (so unaffected by any changes to the variable in the
> test), so stays 0 through to the call to nsk_aod_agentFinished. So the test
> fails.
>
> The split in the change causes the updated post-ObjectFree event success
> value of 1 to be passed to agentFinished. So the test passes.
>
> That required some head scratching to find at the time. That's the point of
> the comment about flushing pending events. Maybe the comment should be
> improved.
@kimbarrett
Okay, thank you for explanation.
I agree, it'd make sense to improve the comment a little bit.
Thanks,
Serguei
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From rkennke at redhat.com Thu Nov 19 12:27:06 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 19 Nov 2020 13:27:06 +0100
Subject: [11u] RFR (S) 8255457: Shenandoah: cleanup ShenandoahMarkTask
In-Reply-To: <a88e7043-30cb-158d-b21d-e714cc492c5d@redhat.com>
References: <a88e7043-30cb-158d-b21d-e714cc492c5d@redhat.com>
Message-ID: <eb957ba1-d988-f286-0cb1-42f004b26bcd@redhat.com>
Looks good to me!
Thanks,
Roman
On 18/11/2020 13:09, Aleksey Shipilev wrote:
> Original RFE:
> ? https://bugs.openjdk.java.net/browse/JDK-8255457
> ? https://git.openjdk.java.net/jdk/commit/1215b1a8
>
> This patch does not apply cleanly to 11u, because of the contextual
> differences. Most significant difference is that in HEAD JDK the
> affected class is trivially copyable, and the shared code already
> adjusted to accept that. 11u, however, would fail to compile due to odd
> cv-stripping casts. Therefore, I had to keep copying constructors and
> assignment operators.
>
> 11u webrev:
> ? https://cr.openjdk.java.net/~shade/8255457/webrev.11u.01/
>
> Testing: Linux {x86_64, x86_32} hotspot_gc_shenandoah
>
From shade at redhat.com Thu Nov 19 12:34:32 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 19 Nov 2020 13:34:32 +0100
Subject: [11u] RFR (S) 8255457: Shenandoah: cleanup ShenandoahMarkTask
In-Reply-To: <eb957ba1-d988-f286-0cb1-42f004b26bcd@redhat.com>
References: <a88e7043-30cb-158d-b21d-e714cc492c5d@redhat.com>
<eb957ba1-d988-f286-0cb1-42f004b26bcd@redhat.com>
Message-ID: <8bac41c4-e3f0-3f9b-c71e-79cdd91f5123@redhat.com>
On 11/19/20 1:27 PM, Roman Kennke wrote:
> Looks good to me!
Thanks, tagged.
--
-Aleksey
From coleenp at openjdk.java.net Thu Nov 19 12:39:14 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 19 Nov 2020 12:39:14 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <zHyAgV_fjFjDegqqFGUFD_VzVKd02L-qinlNuyqmqjU=.f9dafb62-45fe-4896-a238-9d67d304d749@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
<8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
<ywAQf_CJ9rxJlXivn5XU6vk1qlZwPTgGSIooNkkWGwU=.2aeed564-87d7-4bbc-807f-7193c60a9fda@github.com>
<zHyAgV_fjFjDegqqFGUFD_VzVKd02L-qinlNuyqmqjU=.f9dafb62-45fe-4896-a238-9d67d304d749@github.com>
Message-ID: <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com>
On Thu, 19 Nov 2020 10:10:06 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:
>>> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp:
>>>
>>> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same:
>>>
>>> ```
>>> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni);
>>> +
>>> + /* Flush any pending ObjectFree events, which may set success to 1 */
>>> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE,
>>> + JVMTI_EVENT_OBJECT_FREE,
>>> + NULL) != JVMTI_ERROR_NONE) {
>>> + success = 0;
>>> + }
>>> +
>>> + nsk_aod_agentFinished(jni, agentName, success);
>>> }
>>> ```
>>
>> This change really is needed.
>>
>> The success variable in the test is a global, initially 0, set to 1 by the
>> ObjectFree handler.
>>
>> In the old code, if the ObjectFree event hasn't been posted yet, we pass the
>> initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where
>> it's a local variable (so unaffected by any changes to the variable in the
>> test), so stays 0 through to the call to nsk_aod_agentFinished. So the test
>> fails.
>>
>> The split in the change causes the updated post-ObjectFree event success
>> value of 1 to be passed to agentFinished. So the test passes.
>>
>> That required some head scratching to find at the time. That's the point of
>> the comment about flushing pending events. Maybe the comment should be
>> improved.
>
> @kimbarrett
> Okay, thank you for explanation.
> I agree, it'd make sense to improve the comment a little bit.
> Thanks,
> Serguei
So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change?
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 19 12:57:25 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 19 Nov 2020 12:57:25 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
<8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
<ywAQf_CJ9rxJlXivn5XU6vk1qlZwPTgGSIooNkkWGwU=.2aeed564-87d7-4bbc-807f-7193c60a9fda@github.com>
<zHyAgV_fjFjDegqqFGUFD_VzVKd02L-qinlNuyqmqjU=.f9dafb62-45fe-4896-a238-9d67d304d749@github.com>
<6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com>
Message-ID: <sDDWpJdJ4WMKIaiiRQhNQSh3EC3udb10uN5bTJofGkI=.4568f4f2-00f3-4d2e-b746-98a047f97c88@github.com>
On Thu, 19 Nov 2020 12:36:46 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> @kimbarrett
>> Okay, thank you for explanation.
>> I agree, it'd make sense to improve the comment a little bit.
>> Thanks,
>> Serguei
>
> So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change?
> Ok, with this change, it's not posted yet and the success variable for nsk_aod_agentFinished is the local variable. We should fix this in an RFE filed: https://bugs.openjdk.java.net/browse/JDK-8256651
/* Flush any pending ObjectFree events, which will set global success variable to 1
for any pending ObjectFree events. */
How about this? The word 'global' helps me.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 19 12:57:24 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 19 Nov 2020 12:57:24 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v13]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <j3vvsTphdI68E0yGSGtQtIJXPVnOlvfFMB8OxlitOmI=.c855101d-75f4-4239-827c-0e48996e6da5@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
Update comment in jvmti test.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/9ef44f28..40168e63
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=12
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=11-12
Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 19 12:57:25 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 19 Nov 2020 12:57:25 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <sDDWpJdJ4WMKIaiiRQhNQSh3EC3udb10uN5bTJofGkI=.4568f4f2-00f3-4d2e-b746-98a047f97c88@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
<8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
<ywAQf_CJ9rxJlXivn5XU6vk1qlZwPTgGSIooNkkWGwU=.2aeed564-87d7-4bbc-807f-7193c60a9fda@github.com>
<zHyAgV_fjFjDegqqFGUFD_VzVKd02L-qinlNuyqmqjU=.f9dafb62-45fe-4896-a238-9d67d304d749@github.com>
<6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com>
<sDDWpJdJ4WMKIaiiRQhNQSh3EC3udb10uN5bTJofGkI=.4568f4f2-00f3-4d2e-b746-98a047f97c88@github.com>
Message-ID: <lrZ_5WDSi_r7SQrIEfl57xHL1C7w6_K2lgw91e1wxwY=.76d7c889-c80f-489d-971b-c03febb720d6@github.com>
On Thu, 19 Nov 2020 12:51:11 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change?
>> Ok, with this change, it's not posted yet and the success variable for nsk_aod_agentFinished is the local variable. We should fix this in an RFE filed: https://bugs.openjdk.java.net/browse/JDK-8256651
>
> /* Flush any pending ObjectFree events, which will set global success variable to 1
> for any pending ObjectFree events. */
> How about this? The word 'global' helps me.
With remerging into shenandoah, all the jdi tests pass with shenandoah also.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 19 13:01:22 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 19 Nov 2020 13:01:22 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v14]
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <u1AB5qY6JoMPhopRT6nb_JS1t-2SIdXhUAIeyLU187Q=.ac921bdc-885e-4cf6-aaa6-6164c527e7df@github.com>
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
Fix copyrights in test changes.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/967/files
- new: https://git.openjdk.java.net/jdk/pull/967/files/40168e63..589e4c5b
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=13
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=12-13
Stats: 5 lines in 5 files changed: 0 ins; 0 del; 5 mod
Patch: https://git.openjdk.java.net/jdk/pull/967.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 19 14:33:16 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 19 Nov 2020 14:33:16 GMT
Subject: Integrated: 8212879: Make JVMTI TagMap table concurrent
In-Reply-To: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
Message-ID: <XJbTNVPJgFyO62O_jn1Mo0KOs52qXuqNJX8X2O5Qeew=.60971263-b83c-4962-949e-3e6d8c0e2430@github.com>
On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently.
>
> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use.
>
> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways:
>
> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post.
> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event.
>
> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads.
>
> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks.
>
> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code.
>
> It has also been tested with tier1-6.
>
> Thank you to Stefan, Erik and Kim for their help with this change.
This pull request has now been integrated.
Changeset: ba721f5f
Author: Coleen Phillimore <coleenp at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/ba721f5f
Stats: 1891 lines in 49 files changed: 769 ins; 992 del; 130 mod
8212879: Make JVMTI TagMap table concurrent
Co-authored-by: Kim Barrett <kbarrett at openjdk.org>
Co-authored-by: Coleen Phillimore <coleenp at openjdk.org>
Reviewed-by: stefank, ihse, zgu, eosterlund, sspitsyn, kbarrett
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From coleenp at openjdk.java.net Thu Nov 19 14:33:12 2020
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Thu, 19 Nov 2020 14:33:12 GMT
Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11]
In-Reply-To: <lrZ_5WDSi_r7SQrIEfl57xHL1C7w6_K2lgw91e1wxwY=.76d7c889-c80f-489d-971b-c03febb720d6@github.com>
References: <wRMAEoPWqRvAMnc3F8_t1E79O6UxCDl76pvDC3upSfg=.bba0d609-281c-4b1a-abbf-610726652f32@github.com>
<KREcwckFV7QnIq91b02VCVGdeOPOV7g5DFKyeGr1KX0=.346b9e80-fb16-40dd-bfbd-ca91dc81a6df@github.com>
<8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com>
<ywAQf_CJ9rxJlXivn5XU6vk1qlZwPTgGSIooNkkWGwU=.2aeed564-87d7-4bbc-807f-7193c60a9fda@github.com>
<zHyAgV_fjFjDegqqFGUFD_VzVKd02L-qinlNuyqmqjU=.f9dafb62-45fe-4896-a238-9d67d304d749@github.com>
<6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com>
<sDDWpJdJ4WMKIaiiRQhNQSh3EC3udb10uN5bTJofGkI=.4568f4f2-00f3-4d2e-b746-98a047f97c88@github.com>
<lrZ_5WDSi_r7SQrIEfl57xHL1C7w6_K2lgw91e1wxwY=.76d7c889-c80f-489d-971b-c03febb720d6@github.com>
Message-ID: <RU03_qPCx0L_7SJ8GbQjxMXcy3sjlO1vFBAyM0MdwnI=.1341fce3-aca5-4f42-8749-64999f6c8110@github.com>
On Thu, 19 Nov 2020 12:54:23 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> /* Flush any pending ObjectFree events, which will set global success variable to 1
>> for any pending ObjectFree events. */
>> How about this? The word 'global' helps me.
>
> With remerging into shenandoah, all the jdi tests pass with shenandoah also.
Thank you to all the reviewers.
-------------
PR: https://git.openjdk.java.net/jdk/pull/967
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 19 18:41:14 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 19 Nov 2020 18:41:14 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <vo5O28vV2lBh8QlKbnFafdHbFbq5PzPSuxHjXzqKRYc=.a446201a-adfc-457f-b67a-abc740547822@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
<t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
<1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
<3_BiEmzQbP-BcNrPIHmDZfm8ooysWFoGN5Lyo90Dw88=.e071f740-c83c-4e06-9784-b42bb60cc922@github.com>
<vo5O28vV2lBh8QlKbnFafdHbFbq5PzPSuxHjXzqKRYc=.a446201a-adfc-457f-b67a-abc740547822@github.com>
Message-ID: <dLPB1BOOaSMIafSlFjZqkmSdSptRoYwCG21rdxwEFLQ=.45da83bb-81d6-4e0f-a903-c8cb8eac8509@github.com>
On Thu, 19 Nov 2020 07:18:07 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> I don't have GH actions disabled, but I'll merge from master. I'm running more tests after all these changes (there was a regression when we changed from `os::javaTimeNanos` to `os::elapsedTime`).
>
> I don't understand why GH actions do not test your patch :/ Can you go to https://github.com/earthling-amzn/jdk/settings/ and check that both "Actions" and "Secrets" do not have anything special defined? "Actions" should say "Allow all actions", "Secrets" should not have JDK_SUBMIT_PLATFORMS defined.
"Allow all actions" is checked, nothing defined for secrets. Is it because the title didn't have the JBS issue number in it when I opened the PR?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Thu Nov 19 18:47:09 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 19 Nov 2020 18:47:09 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <dLPB1BOOaSMIafSlFjZqkmSdSptRoYwCG21rdxwEFLQ=.45da83bb-81d6-4e0f-a903-c8cb8eac8509@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
<t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
<1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
<3_BiEmzQbP-BcNrPIHmDZfm8ooysWFoGN5Lyo90Dw88=.e071f740-c83c-4e06-9784-b42bb60cc922@github.com>
<vo5O28vV2lBh8QlKbnFafdHbFbq5PzPSuxHjXzqKRYc=.a446201a-adfc-457f-b67a-abc740547822@github.com>
<dLPB1BOOaSMIafSlFjZqkmSdSptRoYwCG21rdxwEFLQ=.45da83bb-81d6-4e0f-a903-c8cb8eac8509@github.com>
Message-ID: <YkkGNmojndt4oWLzK_Ngfdr9DEYknOXEx-PqxAnMFvA=.2001042d-4746-4815-bb03-c3800c26931d@github.com>
On Thu, 19 Nov 2020 18:38:17 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> I don't understand why GH actions do not test your patch :/ Can you go to https://github.com/earthling-amzn/jdk/settings/ and check that both "Actions" and "Secrets" do not have anything special defined? "Actions" should say "Allow all actions", "Secrets" should not have JDK_SUBMIT_PLATFORMS defined.
>
> "Allow all actions" is checked, nothing defined for secrets. Is it because the title didn't have the JBS issue number in it when I opened the PR?
Unlikely. If you go to https://github.com/earthling-amzn/jdk/actions, can you trigger the workflow by hand?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 19 19:39:04 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 19 Nov 2020 19:39:04 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <YkkGNmojndt4oWLzK_Ngfdr9DEYknOXEx-PqxAnMFvA=.2001042d-4746-4815-bb03-c3800c26931d@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
<t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
<1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
<3_BiEmzQbP-BcNrPIHmDZfm8ooysWFoGN5Lyo90Dw88=.e071f740-c83c-4e06-9784-b42bb60cc922@github.com>
<vo5O28vV2lBh8QlKbnFafdHbFbq5PzPSuxHjXzqKRYc=.a446201a-adfc-457f-b67a-abc740547822@github.com>
<dLPB1BOOaSMIafSlFjZqkmSdSptRoYwCG21rdxwEFLQ=.45da83bb-81d6-4e0f-a903-c8cb8eac8509@github.com>
<YkkGNmojndt4oWLzK_Ngfdr9DEYknOXEx-PqxAnMFvA=.2001042d-4746-4815-bb03-c3800c26931d@github.com>
Message-ID: <oX8mxqJPO0m0SOAnKqNmVVHKAzTqlrfo0ZXhBcsOOIk=.3621fe09-bc0d-4dca-956d-464e8c9e2d40@github.com>
On Thu, 19 Nov 2020 18:44:36 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> "Allow all actions" is checked, nothing defined for secrets. Is it because the title didn't have the JBS issue number in it when I opened the PR?
>
> Unlikely. If you go to https://github.com/earthling-amzn/jdk/actions, can you trigger the workflow by hand?
Ah, it says this at the actions link:
>Workflows aren?t being run on this forked repository
Because this repository contained workflow files when it was forked, we have disabled them from running on this fork. Make sure you understand the configured workflows and their expected usage before enabling Actions on this repository.
I've re-enabled them and the pre-submit action is running now.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Thu Nov 19 19:39:05 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 19 Nov 2020 19:39:05 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <oX8mxqJPO0m0SOAnKqNmVVHKAzTqlrfo0ZXhBcsOOIk=.3621fe09-bc0d-4dca-956d-464e8c9e2d40@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
<t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
<1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
<3_BiEmzQbP-BcNrPIHmDZfm8ooysWFoGN5Lyo90Dw88=.e071f740-c83c-4e06-9784-b42bb60cc922@github.com>
<vo5O28vV2lBh8QlKbnFafdHbFbq5PzPSuxHjXzqKRYc=.a446201a-adfc-457f-b67a-abc740547822@github.com>
<dLPB1BOOaSMIafSlFjZqkmSdSptRoYwCG21rdxwEFLQ=.45da83bb-81d6-4e0f-a903-c8cb8eac8509@github.com>
<YkkGNmojndt4oWLzK_Ngfdr9DEYknOXEx-PqxAnMFvA=.2001042d-4746-4815-bb03-c3800c26931d@github.com>
<oX8mxqJPO0m0SOAnKqNmVVHKAzTqlrfo0ZXhBcsOOIk=.3621fe09-bc0d-4dca-956d-464e8c9e2d40@github.com>
Message-ID: <hI3wOjV3H-SojhbDdnjocMQAmbHF8MJa88EtAAWhUyY=.e1cc6137-efa8-427a-90d1-8bd3760994a2@github.com>
On Thu, 19 Nov 2020 19:33:56 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> Unlikely. If you go to https://github.com/earthling-amzn/jdk/actions, can you trigger the workflow by hand?
>
> Ah, it says this at the actions link:
>>Workflows aren?t being run on this forked repository
> Because this repository contained workflow files when it was forked, we have disabled them from running on this fork. Make sure you understand the configured workflows and their expected usage before enabling Actions on this repository.
>
> I've re-enabled them and the pre-submit action is running now.
I'm also running my own test suite (heapotheysis/HyperAlloc) with different permutations for the margin of error, spike threshold and sample frequency. I'll run the whole thing again without any headroom buffer. Will have the results by end of day.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From zgu at openjdk.java.net Fri Nov 20 01:07:10 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 01:07:10 GMT
Subject: RFR: 8256688: Shenandoah: Lock rank inversion after JDK-8212879
Message-ID: <mmuSTleUVUaLMGwy043pLzdBORzz7OnPOAjLzhk6spI=.94ba6d44-5cf4-49e0-83c2-425d233058c1@github.com>
After JDK-8212879, gc_notification() takes JvmtiTagMap_lock/24, while ShenandoahConcurrentWeakRootsEvacUpdateTask holds StringDedupQueue_lock/11, which results lock rank inversion during concurrent weak root processing.
This patch ensures that no lock is held when triggers gc notification.
Test:
- [x] hotspot_gc_shenandoah
-------------
Commit messages:
- Fixed ShenandoahHeapIterationRootScanner
Changes: https://git.openjdk.java.net/jdk/pull/1334/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1334&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256688
Stats: 17 lines in 3 files changed: 15 ins; 0 del; 2 mod
Patch: https://git.openjdk.java.net/jdk/pull/1334.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1334/head:pull/1334
PR: https://git.openjdk.java.net/jdk/pull/1334
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 20 01:41:05 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 20 Nov 2020 01:41:05 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v3]
In-Reply-To: <hI3wOjV3H-SojhbDdnjocMQAmbHF8MJa88EtAAWhUyY=.e1cc6137-efa8-427a-90d1-8bd3760994a2@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<oNrpUXvTgxtrYtGFHN89So9VnQ6gw0fQEwnGGclH1xc=.28987f0c-925c-4db1-accf-2fc219bf7f1d@github.com>
<xkpG_PFGF318OmdLxFm4GQvUp5OGsenCcCwSOXUiFHE=.26f3581f-28c4-4b86-90dd-83249ac0466f@github.com>
<t89VVxgYc_8qGDPphaQWOvE09L-iaHIB00KLqHLR_2A=.423cd65f-27a7-470a-aa3c-b01609dfc495@github.com>
<1yk2DZRdcYG16RhlVQJSJ4y9PCjNzRtpaPoZhYJt47I=.76b1b6bb-4d3c-4412-9120-f68cc839c420@github.com>
<3_BiEmzQbP-BcNrPIHmDZfm8ooysWFoGN5Lyo90Dw88=.e071f740-c83c-4e06-9784-b42bb60cc922@github.com>
<vo5O28vV2lBh8QlKbnFafdHbFbq5PzPSuxHjXzqKRYc=.a446201a-adfc-457f-b67a-abc740547822@github.com>
<dLPB1BOOaSMIafSlFjZqkmSdSptRoYwCG21rdxwEFLQ=.45da83bb-81d6-4e0f-a903-c8cb8eac8509@github.com>
<YkkGNmojndt4oWLzK_Ngfdr9DEYknOXEx-PqxAnMFvA=.2001042d-4746-4815-bb03-c3800c26931d@github.com>
<oX8mxqJPO0m0SOAnKqNmVVHKAzTqlrfo0ZXhBcsOOIk=.3621fe09-bc0d-4dca-956d-464e8c9e2d40@github.com>
<hI3wOjV3H-SojhbDdnjocMQAmbHF8MJa88E
tAAWhUyY=.e1cc6137-efa8-427a-90d1-8bd3760994a2@github.com>
Message-ID: <e6qE-ce6LMC_3Xdey75GmmCse6KdI_zWpt_c5nmsdLI=.58df23b2-2e19-4922-9b18-84c3a4e637da@github.com>
On Thu, 19 Nov 2020 19:36:12 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> Ah, it says this at the actions link:
>>>Workflows aren?t being run on this forked repository
>> Because this repository contained workflow files when it was forked, we have disabled them from running on this fork. Make sure you understand the configured workflows and their expected usage before enabling Actions on this repository.
>>
>> I've re-enabled them and the pre-submit action is running now.
>
> I'm also running my own test suite (heapotheysis/HyperAlloc) with different permutations for the margin of error, spike threshold and sample frequency. I'll run the whole thing again without any headroom buffer. Will have the results by end of day.
Finished running more tests. My preference is still to keep the headroom/penalties in place. It can run without these, but there is a modest increase in degenerated cycles. If we did remove these safeguards, I'd recommend increasing the default sample frequency to 100hz.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From zgu at openjdk.java.net Fri Nov 20 02:10:11 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 02:10:11 GMT
Subject: RFR: 8256664: Shenandoah: Cleanup after JDK-8212879
Message-ID: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
JDK-8212879 removed last serial weak roots, that means:
1) Concurrent GC no longer needs to cleanup any weak roots at pause
2) Root verifier no longer needs to distinguish concurrent/serial weak roots.
Test:
- [x] hotspot_gc_shenandoah
-------------
Commit messages:
- Update comment
- Cleanup after JDK-8212879
Changes: https://git.openjdk.java.net/jdk/pull/1336/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1336&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256664
Stats: 70 lines in 9 files changed: 9 ins; 37 del; 24 mod
Patch: https://git.openjdk.java.net/jdk/pull/1336.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1336/head:pull/1336
PR: https://git.openjdk.java.net/jdk/pull/1336
From shade at openjdk.java.net Fri Nov 20 07:06:17 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 20 Nov 2020 07:06:17 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v6]
In-Reply-To: <DykCR83ieEXgOZdkVbeKOG-2YIz_x7GV5IFr6sJDeU0=.9244664f-362b-474e-9be4-e0d72425c032@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<DykCR83ieEXgOZdkVbeKOG-2YIz_x7GV5IFr6sJDeU0=.9244664f-362b-474e-9be4-e0d72425c032@github.com>
Message-ID: <GrcfbPh9OB7shc2G7LcN4DdaOXZqHehL6DtcWMt2ctU=.75524b67-5c50-4ba8-b2e0-1adc53615e1e@github.com>
On Thu, 19 Nov 2020 01:26:35 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> earthling-amzn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 17 additional commits since the last revision:
>
> - Remove unused member
> - Avoid recomputing instantaneous allocation rate
> - Merge branch 'master' into shenandoah-reactive-heuristic
> - Fix wrong type for os::elapsedTime
> - Remove dependency from allocation rate to adaptive heuristic
> - Defend against underflow and division by zero
> - Inline calls to gc decision methods (vestige of an earlier design)
> - Use os::elapsedTime to avoid type issues and to be consistent with other heuristics code
> - Reuse instantaneous_rate method instead of duplicating code
> - Rename variables to improve readability
> - ... and 7 more: https://git.openjdk.java.net/jdk/compare/0e541a6f...68ba6285
Okay then! Time to integrate and do anything else in follow-ups.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1099
From rkennke at openjdk.java.net Fri Nov 20 11:28:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 20 Nov 2020 11:28:07 GMT
Subject: RFR: 8256688: Shenandoah: Lock rank inversion after JDK-8212879
In-Reply-To: <mmuSTleUVUaLMGwy043pLzdBORzz7OnPOAjLzhk6spI=.94ba6d44-5cf4-49e0-83c2-425d233058c1@github.com>
References: <mmuSTleUVUaLMGwy043pLzdBORzz7OnPOAjLzhk6spI=.94ba6d44-5cf4-49e0-83c2-425d233058c1@github.com>
Message-ID: <ALdbOz8sCzlZQHNmRWRz1kjBFMqerT2V9pHczZOXHRE=.e145fbd6-0664-4a9a-98d2-ce4ef8b18977@github.com>
On Fri, 20 Nov 2020 01:02:20 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> After JDK-8212879, gc_notification() takes JvmtiTagMap_lock/24, while ShenandoahConcurrentWeakRootsEvacUpdateTask holds StringDedupQueue_lock/11, which results lock rank inversion during concurrent weak root processing.
>
> This patch ensures that no lock is held when triggers gc notification.
>
> Test:
> - [x] hotspot_gc_shenandoah
Looks good, thank you!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1334
From rkennke at openjdk.java.net Fri Nov 20 11:49:06 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 20 Nov 2020 11:49:06 GMT
Subject: RFR: 8256664: Shenandoah: Cleanup after JDK-8212879
In-Reply-To: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
References: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
Message-ID: <qske28wuUwZmvj5gIHfHZsNTpcLCj3A7LB3TQBFPgDk=.4341bf14-6e1c-431e-bb8b-8b7a966b2c81@github.com>
On Fri, 20 Nov 2020 02:04:26 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> JDK-8212879 removed last serial weak roots, that means:
>
> 1) Concurrent GC no longer needs to cleanup any weak roots at pause
> 2) Root verifier no longer needs to distinguish concurrent/serial weak roots.
>
> Test:
> - [x] hotspot_gc_shenandoah
Looks good to me! Thank you!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1336
From rkennke at openjdk.java.net Fri Nov 20 11:49:07 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 20 Nov 2020 11:49:07 GMT
Subject: RFR: 8256664: Shenandoah: Cleanup after JDK-8212879
In-Reply-To: <qske28wuUwZmvj5gIHfHZsNTpcLCj3A7LB3TQBFPgDk=.4341bf14-6e1c-431e-bb8b-8b7a966b2c81@github.com>
References: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
<qske28wuUwZmvj5gIHfHZsNTpcLCj3A7LB3TQBFPgDk=.4341bf14-6e1c-431e-bb8b-8b7a966b2c81@github.com>
Message-ID: <XJE6BRlFtB4V09Gf1xYZ_BOQXYcYr4GFkyIzJszSR3k=.2dda7075-2244-407d-9ec9-e47edc77f86c@github.com>
On Fri, 20 Nov 2020 11:43:48 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> JDK-8212879 removed last serial weak roots, that means:
>>
>> 1) Concurrent GC no longer needs to cleanup any weak roots at pause
>> 2) Root verifier no longer needs to distinguish concurrent/serial weak roots.
>>
>> Test:
>> - [x] hotspot_gc_shenandoah
>
> Looks good to me! Thank you!
There is one failure in tier1/linux/x86_64, make sure this is not caused by this change!
-------------
PR: https://git.openjdk.java.net/jdk/pull/1336
From shade at openjdk.java.net Fri Nov 20 11:55:02 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 20 Nov 2020 11:55:02 GMT
Subject: RFR: 8256664: Shenandoah: Cleanup after JDK-8212879
In-Reply-To: <XJE6BRlFtB4V09Gf1xYZ_BOQXYcYr4GFkyIzJszSR3k=.2dda7075-2244-407d-9ec9-e47edc77f86c@github.com>
References: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
<qske28wuUwZmvj5gIHfHZsNTpcLCj3A7LB3TQBFPgDk=.4341bf14-6e1c-431e-bb8b-8b7a966b2c81@github.com>
<XJE6BRlFtB4V09Gf1xYZ_BOQXYcYr4GFkyIzJszSR3k=.2dda7075-2244-407d-9ec9-e47edc77f86c@github.com>
Message-ID: <lev0Hq7yrTU8hCPH8gMwWFIPIZdXs-db96Lb9G2TzDc=.5646c2b6-af28-4159-8761-8fc5e37f35c4@github.com>
On Fri, 20 Nov 2020 11:46:20 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> There is one failure in tier1/linux/x86_64, make sure this is not caused by this change!
It seems to be https://bugs.openjdk.java.net/browse/JDK-8256641, and it manifests across many runs today.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1336
From zgu at openjdk.java.net Fri Nov 20 12:54:05 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 12:54:05 GMT
Subject: Integrated: 8256688: Shenandoah: Lock rank inversion after JDK-8212879
In-Reply-To: <mmuSTleUVUaLMGwy043pLzdBORzz7OnPOAjLzhk6spI=.94ba6d44-5cf4-49e0-83c2-425d233058c1@github.com>
References: <mmuSTleUVUaLMGwy043pLzdBORzz7OnPOAjLzhk6spI=.94ba6d44-5cf4-49e0-83c2-425d233058c1@github.com>
Message-ID: <AIGob6CYKG3yOnLQUEspuq_rUYpn_I5uhqKLhv6kJCI=.1f91b34b-7308-4c0c-ba9f-e797faa4954a@github.com>
On Fri, 20 Nov 2020 01:02:20 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> After JDK-8212879, gc_notification() takes JvmtiTagMap_lock/24, while ShenandoahConcurrentWeakRootsEvacUpdateTask holds StringDedupQueue_lock/11, which results lock rank inversion during concurrent weak root processing.
>
> This patch ensures that no lock is held when triggers gc notification.
>
> Test:
> - [x] hotspot_gc_shenandoah
This pull request has now been integrated.
Changeset: 266dea06
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/266dea06
Stats: 17 lines in 3 files changed: 15 ins; 0 del; 2 mod
8256688: Shenandoah: Lock rank inversion after JDK-8212879
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1334
From zgu at openjdk.java.net Fri Nov 20 12:54:03 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 12:54:03 GMT
Subject: RFR: 8256688: Shenandoah: Lock rank inversion after JDK-8212879
In-Reply-To: <ALdbOz8sCzlZQHNmRWRz1kjBFMqerT2V9pHczZOXHRE=.e145fbd6-0664-4a9a-98d2-ce4ef8b18977@github.com>
References: <mmuSTleUVUaLMGwy043pLzdBORzz7OnPOAjLzhk6spI=.94ba6d44-5cf4-49e0-83c2-425d233058c1@github.com>
<ALdbOz8sCzlZQHNmRWRz1kjBFMqerT2V9pHczZOXHRE=.e145fbd6-0664-4a9a-98d2-ce4ef8b18977@github.com>
Message-ID: <FMxObjiCzEOT6rWmwKMcArQoDPfQKXAFI62PVThRmJ0=.bb85092d-3085-44c8-a3f4-fb880c2a27d2@github.com>
On Fri, 20 Nov 2020 11:25:47 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Looks good, thank you!
Thanks for reviewing. Integrated.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1334
From zgu at openjdk.java.net Fri Nov 20 13:00:03 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 13:00:03 GMT
Subject: RFR: 8256664: Shenandoah: Cleanup after JDK-8212879
In-Reply-To: <qske28wuUwZmvj5gIHfHZsNTpcLCj3A7LB3TQBFPgDk=.4341bf14-6e1c-431e-bb8b-8b7a966b2c81@github.com>
References: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
<qske28wuUwZmvj5gIHfHZsNTpcLCj3A7LB3TQBFPgDk=.4341bf14-6e1c-431e-bb8b-8b7a966b2c81@github.com>
Message-ID: <07o-on17LSDCKW3zzRH0wuHM_aPEUk0AFyZSmLEtNA4=.ae38e51e-7b4f-4bcb-96c0-d43298bbf345@github.com>
On Fri, 20 Nov 2020 11:43:48 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Looks good to me! Thank you!
Thanks.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1336
From zgu at openjdk.java.net Fri Nov 20 13:00:05 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 13:00:05 GMT
Subject: RFR: 8256664: Shenandoah: Cleanup after JDK-8212879
In-Reply-To: <lev0Hq7yrTU8hCPH8gMwWFIPIZdXs-db96Lb9G2TzDc=.5646c2b6-af28-4159-8761-8fc5e37f35c4@github.com>
References: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
<qske28wuUwZmvj5gIHfHZsNTpcLCj3A7LB3TQBFPgDk=.4341bf14-6e1c-431e-bb8b-8b7a966b2c81@github.com>
<XJE6BRlFtB4V09Gf1xYZ_BOQXYcYr4GFkyIzJszSR3k=.2dda7075-2244-407d-9ec9-e47edc77f86c@github.com>
<lev0Hq7yrTU8hCPH8gMwWFIPIZdXs-db96Lb9G2TzDc=.5646c2b6-af28-4159-8761-8fc5e37f35c4@github.com>
Message-ID: <DJL4Xrhb_uWd6CKC-1OoHzSF3LjcziSJwHDgPV_1VYg=.6eae70a8-cdae-4ce1-8a15-6667550d0cfd@github.com>
On Fri, 20 Nov 2020 11:51:50 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> > There is one failure in tier1/linux/x86_64, make sure this is not caused by this change!
>
> It seems to be https://bugs.openjdk.java.net/browse/JDK-8256641, and it manifests across many runs today.
Yes, it is not related to this change.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1336
From zgu at openjdk.java.net Fri Nov 20 13:00:06 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 13:00:06 GMT
Subject: Integrated: 8256664: Shenandoah: Cleanup after JDK-8212879
In-Reply-To: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
References: <x8aJ41x11kBFyk6ds6_i6IUcz3Wkm0dX4Z5REtTgSuc=.386a987f-df90-452b-a5c9-980bb2af0b3b@github.com>
Message-ID: <hsGESIXFdYIm1lGlKy27KwKxeCYUeiTzkH3StxuHiUk=.638a2537-cb34-4d6f-a906-dcb2b8742219@github.com>
On Fri, 20 Nov 2020 02:04:26 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> JDK-8212879 removed last serial weak roots, that means:
>
> 1) Concurrent GC no longer needs to cleanup any weak roots at pause
> 2) Root verifier no longer needs to distinguish concurrent/serial weak roots.
>
> Test:
> - [x] hotspot_gc_shenandoah
This pull request has now been integrated.
Changeset: 98a5d5a6
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/98a5d5a6
Stats: 70 lines in 9 files changed: 9 ins; 37 del; 24 mod
8256664: Shenandoah: Cleanup after JDK-8212879
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1336
From zgu at openjdk.java.net Fri Nov 20 14:42:11 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 14:42:11 GMT
Subject: RFR: 8256658: Shenandoah: Deadlock between nmethod_entry_barrier and
concurrent code root evacuator
Message-ID: <KWp_15pdSwZb7S9YU--zXQUj2pNTMTs9UOQPqgb0C_s=.0a90f7f5-0802-4a3f-b240-1ef829597d34@github.com>
Nightly tests revealed a deadlock after JDK-8256415.
The new concurrent code root evacuating/updating task sets up EvacOOM scope on the top level. If OOM occurs and nmethod_entry_barrier holds per-nmethod lock, while concurrent code root evacuating/updating task tries to acquire the exact per-nmethod lock, we deadlock.
The solution is to move EvacOOM scope under per-nmethod lock.
Test:
- [x] hotspot_gc_shenandoah
- [x] nightly tests
-------------
Commit messages:
- Merge branch 'master' into JDK-8256658-coderoot-deadlock
- Fix deadlock after JDK-8256415
Changes: https://git.openjdk.java.net/jdk/pull/1352/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1352&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256658
Stats: 17 lines in 1 file changed: 7 ins; 1 del; 9 mod
Patch: https://git.openjdk.java.net/jdk/pull/1352.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1352/head:pull/1352
PR: https://git.openjdk.java.net/jdk/pull/1352
From rkennke at openjdk.java.net Fri Nov 20 14:52:03 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 20 Nov 2020 14:52:03 GMT
Subject: RFR: 8256658: Shenandoah: Deadlock between nmethod_entry_barrier
and concurrent code root evacuator
In-Reply-To: <KWp_15pdSwZb7S9YU--zXQUj2pNTMTs9UOQPqgb0C_s=.0a90f7f5-0802-4a3f-b240-1ef829597d34@github.com>
References: <KWp_15pdSwZb7S9YU--zXQUj2pNTMTs9UOQPqgb0C_s=.0a90f7f5-0802-4a3f-b240-1ef829597d34@github.com>
Message-ID: <sEIixlP6R9gfNQuF7cn6AVRw4TeSExD5ox1p7XgF7v4=.3d2748f9-1acc-4ecf-b966-5dcd35f8c34f@github.com>
On Fri, 20 Nov 2020 14:36:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> Nightly tests revealed a deadlock after JDK-8256415.
>
> The new concurrent code root evacuating/updating task sets up EvacOOM scope on the top level. If OOM occurs and nmethod_entry_barrier holds per-nmethod lock, while concurrent code root evacuating/updating task tries to acquire the exact per-nmethod lock, we deadlock.
>
> The solution is to move EvacOOM scope under per-nmethod lock.
>
> Test:
> - [x] hotspot_gc_shenandoah
> - [x] nightly tests
Looks good to me! Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1352
From zgu at openjdk.java.net Fri Nov 20 17:47:17 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 17:47:17 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v14]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <nfffzhT__khJIRza_GCMoyTVS2iElHeHZRImvplBfDU=.6b782457-d36a-46fc-9df6-f0270207d9be@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits:
- Merge
- Moved task queues to marking context
- Merge
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Removed obsoleted class
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- ... and 10 more: https://git.openjdk.java.net/jdk/compare/98a5d5a6...8c58f6f4
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=13
Stats: 1978 lines in 23 files changed: 1071 ins; 740 del; 167 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From zgu at openjdk.java.net Fri Nov 20 18:39:05 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 20 Nov 2020 18:39:05 GMT
Subject: Integrated: 8256658: Shenandoah: Deadlock between
nmethod_entry_barrier and concurrent code root evacuator
In-Reply-To: <KWp_15pdSwZb7S9YU--zXQUj2pNTMTs9UOQPqgb0C_s=.0a90f7f5-0802-4a3f-b240-1ef829597d34@github.com>
References: <KWp_15pdSwZb7S9YU--zXQUj2pNTMTs9UOQPqgb0C_s=.0a90f7f5-0802-4a3f-b240-1ef829597d34@github.com>
Message-ID: <EIohgrfNJhlor9CcUB7rdmKDXjkSP9qIgLGi_HuiRbY=.214a26d9-bb53-4299-98b4-f1bc145783c0@github.com>
On Fri, 20 Nov 2020 14:36:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
> Nightly tests revealed a deadlock after JDK-8256415.
>
> The new concurrent code root evacuating/updating task sets up EvacOOM scope on the top level. If OOM occurs and nmethod_entry_barrier holds per-nmethod lock, while concurrent code root evacuating/updating task tries to acquire the exact per-nmethod lock, we deadlock.
>
> The solution is to move EvacOOM scope under per-nmethod lock.
>
> Test:
> - [x] hotspot_gc_shenandoah
> - [x] nightly tests
This pull request has now been integrated.
Changeset: 5ad1e228
Author: Zhengyu Gu <zgu at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/5ad1e228
Stats: 17 lines in 1 file changed: 7 ins; 1 del; 9 mod
8256658: Shenandoah: Deadlock between nmethod_entry_barrier and concurrent code root evacuator
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1352
From rkennke at redhat.com Fri Nov 20 19:17:11 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 20 Nov 2020 20:17:11 +0100
Subject: RFR: [backport] 8202976: Add C1 lea patching support for x86
Message-ID: <02bae8a4-04ff-f6f4-5cb0-170bd82d3511@redhat.com>
This backports this change:
https://bugs.openjdk.java.net/browse/JDK-8202976
to shenandoah/jdk8
This is prerequisite for SFX barriers which I will post next. It is not
Shenandoah-specific per-se, but only required for Shenandoah atm. I
don't think it makes sense to bother upstream with it.
The change itself is a straightforward backport from jdk11:
http://cr.openjdk.java.net/~rkennke/JDK-8202976-jdk8/webrev.00/
Testing: hotspot_gc_shenandoah (also with upcoming SFX change, which
actually uses it)
Ok?
Thanks,
Roman
From shade at openjdk.java.net Fri Nov 20 19:21:10 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 20 Nov 2020 19:21:10 GMT
Subject: RFR: 8256806: Shenandoah: optimize
shenandoah/jni/TestPinnedGarbage.java test
Message-ID: <kl_bnxl5SQxrqTCzpiNDvrPGLlJ9g76-ipa-lmyWAg8=.b8bd0af0-21c4-4d37-bed7-2e7e558d6ae7@github.com>
This test is way too slow for Zero configuration, taking about 2 hours. No reasonable timeout factor accounts for this. We can clean the test up and make it more lenient. It would also improve server testing time.
Current proposal cuts down the test heap 4x, and thus the number of objects it needs to handle 4x as well. Additionally, moving `Random` initialization out of tested method marginally improves the times. I also went back to remember why this test even exists, and it has to do with [Full GC handling pinned objects specially](http://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/bc0f0cfed315). I injected the bug back, and the test still caught it.
Additional testing:
- [x] Linux x86_64 Zero fastdebug, affected test
- [x] Linux x86_64 Server fastdebug, affected test
- [x] Linux x86_32 Server fastdebug, affected test
-------------
Commit messages:
- 8256806: Shenandoah: optimize shenandoah/jni/TestPinnedGarbage.java test
Changes: https://git.openjdk.java.net/jdk/pull/1358/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1358&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256806
Stats: 10 lines in 1 file changed: 1 ins; 1 del; 8 mod
Patch: https://git.openjdk.java.net/jdk/pull/1358.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1358/head:pull/1358
PR: https://git.openjdk.java.net/jdk/pull/1358
From rkennke at redhat.com Fri Nov 20 19:23:51 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 20 Nov 2020 20:23:51 +0100
Subject: RFR: [backport] 8231087: Shenandoah: Self-fixing load reference
barriers for C1/C2
Message-ID: <3d3c1f61-04f6-afac-1895-d4ad31827cab@redhat.com>
This backports:
https://bugs.openjdk.java.net/browse/JDK-8231087
to sh/jdk8. It accounts for lack of GC interfaces, but otherwise is
relatively straightforward:
http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.00/
Testing: hotspot_gc_shenandoah
Ok?
Roman
From rkennke at openjdk.java.net Fri Nov 20 19:52:03 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 20 Nov 2020 19:52:03 GMT
Subject: RFR: 8256806: Shenandoah: optimize
shenandoah/jni/TestPinnedGarbage.java test
In-Reply-To: <kl_bnxl5SQxrqTCzpiNDvrPGLlJ9g76-ipa-lmyWAg8=.b8bd0af0-21c4-4d37-bed7-2e7e558d6ae7@github.com>
References: <kl_bnxl5SQxrqTCzpiNDvrPGLlJ9g76-ipa-lmyWAg8=.b8bd0af0-21c4-4d37-bed7-2e7e558d6ae7@github.com>
Message-ID: <qStaC5T_qbCUPcQuVk516upTYtJYYK5Xadt9eLpPpJA=.6fc70d50-d205-4b99-9f8d-62d5f9ee6f10@github.com>
On Fri, 20 Nov 2020 19:15:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> This test is way too slow for Zero configuration, taking about 2 hours. No reasonable timeout factor accounts for this. We can clean the test up and make it more lenient. It would also improve server testing time.
>
> Current proposal cuts down the test heap 4x, and thus the number of objects it needs to handle 4x as well. Additionally, moving `Random` initialization out of tested method marginally improves the times. I also went back to remember why this test even exists, and it has to do with [Full GC handling pinned objects specially](http://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/bc0f0cfed315). I injected the bug back, and the test still caught it.
>
> Additional testing:
> - [x] Linux x86_64 Zero fastdebug, affected test
> - [x] Linux x86_64 Server fastdebug, affected test
> - [x] Linux x86_32 Server fastdebug, affected test
Looks good to me! Thank you!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1358
From rkennke at openjdk.java.net Fri Nov 20 20:31:08 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 20 Nov 2020 20:31:08 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v14]
In-Reply-To: <nfffzhT__khJIRza_GCMoyTVS2iElHeHZRImvplBfDU=.6b782457-d36a-46fc-9df6-f0270207d9be@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
<nfffzhT__khJIRza_GCMoyTVS2iElHeHZRImvplBfDU=.6b782457-d36a-46fc-9df6-f0270207d9be@github.com>
Message-ID: <WfC3iKU1scKESjXx5b7AjCVoAo5bqPestJBgWyTcncY=.b1c7cd8a-cfe6-4008-afed-778d16217a31@github.com>
On Fri, 20 Nov 2020 17:47:17 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>>
>> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>>
>> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>>
>> First step, I would like to split STW and concurrent mark, so that:
>> 1) Code has to special case for STW and concurrent mark.
>> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
>> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
>> 4) STW mark does not need to remark some of roots.
>>
>> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>>
>> A few changes:
>> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
>> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
>> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
>> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
>
> Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits:
>
> - Merge
> - Moved task queues to marking context
> - Merge
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Removed obsoleted class
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - Merge branch 'master' into JDK-8255019-sh-mark
> - ... and 10 more: https://git.openjdk.java.net/jdk/compare/98a5d5a6...8c58f6f4
Good work! I have a few questions...
src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 373:
> 371: // call the entry method below
> 372: void vmop_entry_init_mark(ShenandoahConcurrentMark* mark);
> 373: void vmop_entry_final_mark(ShenandoahConcurrentMark* mark);
I find it odd that those entries now take ShenandoahConcurrentMark instance as argument, while all others don't. In particular because it implies it is the concurrent version anyway. Why is this necessary?
src/hotspot/share/gc/shenandoah/shenandoahMarkCompact.cpp line 119:
> 117: // b. Cancel concurrent mark, if in progress
> 118: if (heap->is_concurrent_mark_in_progress()) {
> 119: ShenandoahConcurrentMark::cancel();
While cancel() doesn't exactly need to be an instance-method, it feels strange to do a static call here...
src/hotspot/share/gc/shenandoah/shenandoahMarkCompact.cpp line 126:
> 124: // c. Update roots if this full GC is due to evac-oom, which may carry from-space pointers in roots.
> 125: if (has_forwarded_objects) {
> 126: ShenandoahConcurrentMark::update_roots(ShenandoahPhaseTimings::full_gc_update_roots);
Same as with cancel() above.
-------------
Changes requested by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1009
From shade at openjdk.java.net Fri Nov 20 21:48:05 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 20 Nov 2020 21:48:05 GMT
Subject: Integrated: 8256806: Shenandoah: optimize
shenandoah/jni/TestPinnedGarbage.java test
In-Reply-To: <kl_bnxl5SQxrqTCzpiNDvrPGLlJ9g76-ipa-lmyWAg8=.b8bd0af0-21c4-4d37-bed7-2e7e558d6ae7@github.com>
References: <kl_bnxl5SQxrqTCzpiNDvrPGLlJ9g76-ipa-lmyWAg8=.b8bd0af0-21c4-4d37-bed7-2e7e558d6ae7@github.com>
Message-ID: <4JALutGVHAIg-0ApDUjHlvl0JwO82t5KzZzKPYKFX5k=.04b39cb7-1561-4396-8238-9ca287f2f375@github.com>
On Fri, 20 Nov 2020 19:15:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> This test is way too slow for Zero configuration, taking about 2 hours. No reasonable timeout factor accounts for this. We can clean the test up and make it more lenient. It would also improve server testing time.
>
> Current proposal cuts down the test heap 4x, and thus the number of objects it needs to handle 4x as well. Additionally, moving `Random` initialization out of tested method marginally improves the times. I also went back to remember why this test even exists, and it has to do with [Full GC handling pinned objects specially](http://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/bc0f0cfed315). I injected the bug back, and the test still caught it.
>
> Additional testing:
> - [x] Linux x86_64 Zero fastdebug, affected test
> - [x] Linux x86_64 Server fastdebug, affected test
> - [x] Linux x86_32 Server fastdebug, affected test
This pull request has now been integrated.
Changeset: 86f36027
Author: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/86f36027
Stats: 10 lines in 1 file changed: 1 ins; 1 del; 8 mod
8256806: Shenandoah: optimize shenandoah/jni/TestPinnedGarbage.java test
Reviewed-by: rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1358
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 20 23:30:23 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 20 Nov 2020 23:30:23 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v7]
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <0KF9L60_nXXLJPDF6jEGrVKVeWjt9GkodTGMLDfAJ7w=.243d156a-2d15-404c-ac8b-9ac4890bc6e1@github.com>
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
- Do not re-sample allocation rate with higher frequency
This causes too many false positives and triggers unnecessary cycles.
- Sample allocation rate even if cumulative bytes allocated hasn't changed
Without this, the average allocation rate will be much higher than it should be.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1099/files
- new: https://git.openjdk.java.net/jdk/pull/1099/files/68ba6285..8b58ec85
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=06
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=05-06
Stats: 12 lines in 1 file changed: 1 ins; 0 del; 11 mod
Patch: https://git.openjdk.java.net/jdk/pull/1099.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1099/head:pull/1099
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Fri Nov 20 23:41:08 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 20 Nov 2020 23:41:08 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v6]
In-Reply-To: <GrcfbPh9OB7shc2G7LcN4DdaOXZqHehL6DtcWMt2ctU=.75524b67-5c50-4ba8-b2e0-1adc53615e1e@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<DykCR83ieEXgOZdkVbeKOG-2YIz_x7GV5IFr6sJDeU0=.9244664f-362b-474e-9be4-e0d72425c032@github.com>
<GrcfbPh9OB7shc2G7LcN4DdaOXZqHehL6DtcWMt2ctU=.75524b67-5c50-4ba8-b2e0-1adc53615e1e@github.com>
Message-ID: <u_EpnIkTHlCLCXoXZb3zE-eundNOTgFuy4Ws5F6jhsc=.83299c64-71dd-4b13-a854-e46a538297e1@github.com>
On Fri, 20 Nov 2020 07:03:38 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 17 additional commits since the last revision:
>>
>> - Remove unused member
>> - Avoid recomputing instantaneous allocation rate
>> - Merge branch 'master' into shenandoah-reactive-heuristic
>> - Fix wrong type for os::elapsedTime
>> - Remove dependency from allocation rate to adaptive heuristic
>> - Defend against underflow and division by zero
>> - Inline calls to gc decision methods (vestige of an earlier design)
>> - Use os::elapsedTime to avoid type issues and to be consistent with other heuristics code
>> - Reuse instantaneous_rate method instead of duplicating code
>> - Rename variables to improve readability
>> - ... and 7 more: https://git.openjdk.java.net/jdk/compare/e152dd98...68ba6285
>
> Okay then! Time to integrate and do anything else in follow-ups.
Sorry, added one bug fix and another small change. The bug caused the average allocation rate to be much higher than it should be. The other change is to only consider the values which are actually sampled as a "spike" (as opposed to previously when it would recompute the instantaneous rate on every call to `should_start_gc`). For what it's worth, I made these changes while the headroom/penalty adjustments were disabled. I'm more comfortable with taking those adjustments out now.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Sun Nov 22 10:05:27 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Sun, 22 Nov 2020 10:05:27 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v7]
In-Reply-To: <0KF9L60_nXXLJPDF6jEGrVKVeWjt9GkodTGMLDfAJ7w=.243d156a-2d15-404c-ac8b-9ac4890bc6e1@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<0KF9L60_nXXLJPDF6jEGrVKVeWjt9GkodTGMLDfAJ7w=.243d156a-2d15-404c-ac8b-9ac4890bc6e1@github.com>
Message-ID: <TUHhUfdFSEVtWe9CbwDH4UbjJAXq6zeOK5y-anh9Flg=.7b1d84a9-73ae-4243-9054-b0f4f95a37a5@github.com>
On Fri, 20 Nov 2020 23:30:23 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
>> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>>
>> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>>
>> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>>
>> The "adaptive" heuristic remains the default.
>>
>> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
>
> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>
> - Do not re-sample allocation rate with higher frequency
>
> This causes too many false positives and triggers unnecessary cycles.
> - Sample allocation rate even if cumulative bytes allocated hasn't changed
>
> Without this, the average allocation rate will be much higher than it should be.
Okay, I have a minor suggestion that seems still preserve the semantics. It is generally looks good, the testing passes, and benchmarks look okay (some regressions, but expected). Feel free to integrate.
src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 261:
> 259: }
> 260:
> 261: if (rate > 0.0) {
I'd prefer to do this in `is_spiking` instead, because it seems to conceptually relate to the definition of a spike. E.g.:
bool ShenandoahAllocationRate::is_spiking(double rate, double threshold) const {
if (rate <= 0) {
return false;
}
...
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Sun Nov 22 18:12:41 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Sun, 22 Nov 2020 18:12:41 GMT
Subject: Integrated: 8256497: Zero: enable G1 and Shenandoah GCs
In-Reply-To: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
References: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
Message-ID: <CgHP2PL6sU_mSHriXokYa7uIr66V63xSSyzcnxAvkr8=.e419dd66-6da2-40cd-82e4-f0ea58a520e2@github.com>
On Tue, 17 Nov 2020 19:02:03 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling.
>
> Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah.
>
> Additional testing:
> - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops)
> - [x] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC`
This pull request has now been integrated.
Changeset: e06a6839
Author: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/e06a6839
Stats: 72 lines in 5 files changed: 54 ins; 16 del; 2 mod
8256497: Zero: enable G1 and Shenandoah GCs
Reviewed-by: rkennke, erikj, ihse
-------------
PR: https://git.openjdk.java.net/jdk/pull/1268
From shade at openjdk.java.net Sun Nov 22 18:12:39 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Sun, 22 Nov 2020 18:12:39 GMT
Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs [v2]
In-Reply-To: <uAFO6co1K5FQqf7AJYCXNkEEFxGSfZpOAdMpV3tlod4=.76da4f25-7763-4b86-a2d5-aadf68f584f6@github.com>
References: <bKt0UifZg3LZ7r_gcro9vA0BcUf1Rqgi26_TkTbalqI=.8d25411d-54fb-42f6-b5bc-84ae3e92e976@github.com>
<Jqr8E1u-g9edoPzB9qtKQWfcRL35vrTMolruivwp088=.fe2e1e32-81aa-468b-a177-b86bc617e752@github.com>
<uAFO6co1K5FQqf7AJYCXNkEEFxGSfZpOAdMpV3tlod4=.76da4f25-7763-4b86-a2d5-aadf68f584f6@github.com>
Message-ID: <FGWWypQzLJulYJj4ygpPPF4hBdbZe9OhBaG0SwDAKvo=.c6f06dd9-83c8-409b-b3a9-461ef534e106@github.com>
On Thu, 19 Nov 2020 09:55:04 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:
>> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
>>
>> - Merge branch 'master' into JDK-8256497-zero-g1-shenandoah
>> - Remove TODO
>> - 8256497: Zero: enable G1 and Shenandoah GCs
>
> Build changes look good.
Thanks! By now I completed more thorough tests with Shenandoah, and those look fine.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1268
From shade at redhat.com Mon Nov 23 12:11:17 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 23 Nov 2020 13:11:17 +0100
Subject: RFR: [backport] 8202976: Add C1 lea patching support for x86
In-Reply-To: <02bae8a4-04ff-f6f4-5cb0-170bd82d3511@redhat.com>
References: <02bae8a4-04ff-f6f4-5cb0-170bd82d3511@redhat.com>
Message-ID: <3f28dd8b-5e4c-c285-0a99-ba177389683a@redhat.com>
On 11/20/20 8:17 PM, Roman Kennke wrote:
> http://cr.openjdk.java.net/~rkennke/JDK-8202976-jdk8/webrev.00/
Looks fine.
--
Thanks,
-Aleksey
From shade at redhat.com Mon Nov 23 12:22:23 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 23 Nov 2020 13:22:23 +0100
Subject: RFR: [backport] 8231087: Shenandoah: Self-fixing load reference
barriers for C1/C2
In-Reply-To: <3d3c1f61-04f6-afac-1895-d4ad31827cab@redhat.com>
References: <3d3c1f61-04f6-afac-1895-d4ad31827cab@redhat.com>
Message-ID: <abab9862-2b4d-a4e2-b842-db498b5d3f36@redhat.com>
On 11/20/20 8:23 PM, Roman Kennke wrote:
> This backports:
>
> https://bugs.openjdk.java.net/browse/JDK-8231087
>
> to sh/jdk8. It accounts for lack of GC interfaces, but otherwise is
> relatively straightforward:
>
> http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.00/
I am quite a bit confused.
*) Where are aarch64 changes? It should match c1_LIRGenerator_x86.cpp change, no?
*) Where are shenandoahBarrierSetAssembler_{x86|aarch64} changes?
--
Thanks,
-Aleksey
From rkennke at redhat.com Mon Nov 23 13:01:36 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 23 Nov 2020 14:01:36 +0100
Subject: RFR: [backport] 8231087: Shenandoah: Self-fixing load reference
barriers for C1/C2
In-Reply-To: <abab9862-2b4d-a4e2-b842-db498b5d3f36@redhat.com>
References: <3d3c1f61-04f6-afac-1895-d4ad31827cab@redhat.com>
<abab9862-2b4d-a4e2-b842-db498b5d3f36@redhat.com>
Message-ID: <88f3d872-e0bb-38ed-d855-653ad3d92898@redhat.com>
I need to paddle-back a little. This change requires JDK-8228369 and
JDK-8229709 to be backported first. And perhaps also JDK-8229470 and
JDK-8229977. Let's withdraw this for now.
Roman
> On 11/20/20 8:23 PM, Roman Kennke wrote:
>> This backports:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8231087
>>
>> to sh/jdk8. It accounts for lack of GC interfaces, but otherwise is
>> relatively straightforward:
>>
>> http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.00/
>
> I am quite a bit confused.
>
> *) Where are aarch64 changes? It should match c1_LIRGenerator_x86.cpp
> change, no?
>
> *) Where are shenandoahBarrierSetAssembler_{x86|aarch64} changes?
>
From zgu at openjdk.java.net Mon Nov 23 16:15:11 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 23 Nov 2020 16:15:11 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v15]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <uZxSXx58P-qBZ-x7Ec1Zod4J_-GKSbH7tAGWpHBjssA=.31ce4f11-9709-4fcc-91ab-cc1fef18e856@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 21 commits:
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge
- Moved task queues to marking context
- Merge
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- Removed obsoleted class
- Merge branch 'master' into JDK-8255019-sh-mark
- Merge branch 'master' into JDK-8255019-sh-mark
- ... and 11 more: https://git.openjdk.java.net/jdk/compare/895cc4f0...9301167f
-------------
Changes: https://git.openjdk.java.net/jdk/pull/1009/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=14
Stats: 1978 lines in 23 files changed: 1071 ins; 740 del; 167 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From github.com+71722661+earthling-amzn at openjdk.java.net Mon Nov 23 17:22:14 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 23 Nov 2020 17:22:14 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v8]
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <kptM0pR1K1pYjeQVVAFrKTO0CPAG4MIDK4kyba4xKe8=.b6a75912-08c2-4325-b46e-fd9d3ed88d9b@github.com>
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
earthling-amzn has updated the pull request incrementally with one additional commit since the last revision:
Factor rate check into is_spiking
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1099/files
- new: https://git.openjdk.java.net/jdk/pull/1099/files/8b58ec85..72f15a62
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=07
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1099&range=06-07
Stats: 15 lines in 1 file changed: 4 ins; 2 del; 9 mod
Patch: https://git.openjdk.java.net/jdk/pull/1099.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1099/head:pull/1099
PR: https://git.openjdk.java.net/jdk/pull/1099
From github.com+71722661+earthling-amzn at openjdk.java.net Mon Nov 23 18:57:02 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 23 Nov 2020 18:57:02 GMT
Subject: Integrated: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes
In-Reply-To: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
Message-ID: <_C6hl32-A6sRynUa-665MaPrbDIV0ZCFYLiOc1OB6Ug=.b5531d78-5e30-4e98-a1ab-75b79dccf989@github.com>
On Fri, 6 Nov 2020 20:00:25 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:
> This change adds a "reactive" heuristic for triggering concurrent GC cycles.
>
> The reactive heuristic maintains a margin of error and an allocation spike detection mechanism to trigger cycles somewhat more aggressively than the 'adaptive' heuristic. This heuristic 'reacts' to the outcome of GC cycles by adjusting the sensitivity of the triggers.
>
> JBS ticket is here: https://bugs.openjdk.java.net/browse/JDK-8255984
>
> The "adaptive" heuristic remains the default.
>
> Steps to reproduce and test will follow shortly (there are no new jtreg test failures for Shenandoah with this change).
This pull request has now been integrated.
Changeset: aac5c2a8
Author: William Kemper <kemperw at amazon.com>
Committer: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/aac5c2a8
Stats: 348 lines in 13 files changed: 314 ins; 9 del; 25 mod
8255984: Shenandoah: "adaptive" heuristic is prone to missing load spikes
Reviewed-by: shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From shade at openjdk.java.net Mon Nov 23 18:57:00 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 23 Nov 2020 18:57:00 GMT
Subject: RFR: 8255984: Shenandoah: "adaptive" heuristic is prone to
missing load spikes [v7]
In-Reply-To: <TUHhUfdFSEVtWe9CbwDH4UbjJAXq6zeOK5y-anh9Flg=.7b1d84a9-73ae-4243-9054-b0f4f95a37a5@github.com>
References: <_jg5m8J6AekSE-9o4KEhgja118v1aWlscxzVri0ZSAM=.133b188a-5d9d-421d-b0eb-a341642bf3fc@github.com>
<0KF9L60_nXXLJPDF6jEGrVKVeWjt9GkodTGMLDfAJ7w=.243d156a-2d15-404c-ac8b-9ac4890bc6e1@github.com>
<TUHhUfdFSEVtWe9CbwDH4UbjJAXq6zeOK5y-anh9Flg=.7b1d84a9-73ae-4243-9054-b0f4f95a37a5@github.com>
Message-ID: <Y_upWEgzT5ySh6ELFChakuMghOUaVGo9iPC0BxOtVgo=.4c004b89-6983-49fb-9c14-038eef388592@github.com>
On Sun, 22 Nov 2020 10:02:13 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> earthling-amzn has updated the pull request incrementally with two additional commits since the last revision:
>>
>> - Do not re-sample allocation rate with higher frequency
>>
>> This causes too many false positives and triggers unnecessary cycles.
>> - Sample allocation rate even if cumulative bytes allocated hasn't changed
>>
>> Without this, the average allocation rate will be much higher than it should be.
>
> Okay, I have a minor suggestion that seems still preserve the semantics. It is generally looks good, the testing passes, and benchmarks look okay (some regressions, but expected). Feel free to integrate.
Congratulations! If you have time, please work on removing old alloc spike threshold and penalties handling.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1099
From rkennke at redhat.com Mon Nov 23 19:25:41 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 23 Nov 2020 20:25:41 +0100
Subject: RFR(sh/jdk8): [backport] 8228369: Shenandoah: Refactor LRB C1 stubs
Message-ID: <86b0d234-15e6-fb68-4d7a-d83c13296745@redhat.com>
This backports the following issue to jdk8:
https://bugs.openjdk.java.net/browse/JDK-8228369
This is in preparation for the SFX barrier backport.
Due to missing GC interfaces, I had to put some code in c1_Runtime*, it
is following the same pattern as other GC runtime stubs.
http://cr.openjdk.java.net/~rkennke/JDK-8228369-jdk8/webrev.00/
Let's also include a follow-ups:
https://bugs.openjdk.java.net/browse/JDK-8229709
http://cr.openjdk.java.net/~rkennke/JDK-8229709-jdk8/webrev.00/
The other two follow-ups ( JDK-8229470 and JDK-8229977) don't apply to
jdk8 because of different implementations.
Can I please get a review?
Thanks,
Roman
From zgu at openjdk.java.net Mon Nov 23 19:35:18 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 23 Nov 2020 19:35:18 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v16]
In-Reply-To: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
Message-ID: <U5ULwmL45yPORSs3HGASY7q4VPHdSg3y5OSw41KErHY=.8fbaf90b-13a6-4fe3-a06e-e6ae7a801ab5@github.com>
> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>
> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>
> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>
> First step, I would like to split STW and concurrent mark, so that:
> 1) Code has to special case for STW and concurrent mark.
> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
> 4) STW mark does not need to remark some of roots.
>
> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>
> A few changes:
> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
Removed ShenandoahConcurrentMark parameter from concurrent GC entry/op, etc.
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1009/files
- new: https://git.openjdk.java.net/jdk/pull/1009/files/9301167f..c9842ef0
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=15
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1009&range=14-15
Stats: 71 lines in 5 files changed: 4 ins; 19 del; 48 mod
Patch: https://git.openjdk.java.net/jdk/pull/1009.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
PR: https://git.openjdk.java.net/jdk/pull/1009
From rkennke at redhat.com Mon Nov 23 19:40:48 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 23 Nov 2020 20:40:48 +0100
Subject: RFR(sh/jdk8): [backport] 8231087: Shenandoah: Self-fixing load
reference barriers for C1/C2
Message-ID: <e9521eb3-9ad6-a7ff-72f9-eb5cc63341e3@redhat.com>
2nd attempt:
Issue:
https://bugs.openjdk.java.net/browse/JDK-8231087
Backport:
http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.01/
This time includes the actual C1 stuff (depended on JDK-8228369).
Let's include a few follow-ups:
https://bugs.openjdk.java.net/browse/JDK-8233165
http://cr.openjdk.java.net/~rkennke/JDK-8233165-jdk8/webrev.00/
and
https://bugs.openjdk.java.net/browse/JDK-8233021
http://cr.openjdk.java.net/~rkennke/JDK-8233021-jdk8/webrev.00/
and
https://bugs.openjdk.java.net/browse/JDK-8238153
http://cr.openjdk.java.net/~rkennke/JDK-8238153-jdk8/webrev.00/
Can I please get a review?
Thanks,
Roman
From zgu at openjdk.java.net Mon Nov 23 19:54:59 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 23 Nov 2020 19:54:59 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v14]
In-Reply-To: <WfC3iKU1scKESjXx5b7AjCVoAo5bqPestJBgWyTcncY=.b1c7cd8a-cfe6-4008-afed-778d16217a31@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
<nfffzhT__khJIRza_GCMoyTVS2iElHeHZRImvplBfDU=.6b782457-d36a-46fc-9df6-f0270207d9be@github.com>
<WfC3iKU1scKESjXx5b7AjCVoAo5bqPestJBgWyTcncY=.b1c7cd8a-cfe6-4008-afed-778d16217a31@github.com>
Message-ID: <3311iz-Hr7WaPKoY0elXzr-OVl4CflpxSPyTWxPSrHw=.59a7679b-59c0-4233-8282-f53d8fa8d7cc@github.com>
On Fri, 20 Nov 2020 20:20:14 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits:
>>
>> - Merge
>> - Moved task queues to marking context
>> - Merge
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Removed obsoleted class
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - ... and 10 more: https://git.openjdk.java.net/jdk/compare/98a5d5a6...8c58f6f4
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 373:
>
>> 371: // call the entry method below
>> 372: void vmop_entry_init_mark(ShenandoahConcurrentMark* mark);
>> 373: void vmop_entry_final_mark(ShenandoahConcurrentMark* mark);
>
> I find it odd that those entries now take ShenandoahConcurrentMark instance as argument, while all others don't. In particular because it implies it is the concurrent version anyway. Why is this necessary?
ShenandoahConcurrentMark is stateless, can be created in place. The parameters are removed.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1009
From zgu at openjdk.java.net Mon Nov 23 20:05:04 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 23 Nov 2020 20:05:04 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v14]
In-Reply-To: <WfC3iKU1scKESjXx5b7AjCVoAo5bqPestJBgWyTcncY=.b1c7cd8a-cfe6-4008-afed-778d16217a31@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
<nfffzhT__khJIRza_GCMoyTVS2iElHeHZRImvplBfDU=.6b782457-d36a-46fc-9df6-f0270207d9be@github.com>
<WfC3iKU1scKESjXx5b7AjCVoAo5bqPestJBgWyTcncY=.b1c7cd8a-cfe6-4008-afed-778d16217a31@github.com>
Message-ID: <JC8nNBu7Z7zq-KKWV2t9H3wCqVZljlLGmKodTdDn5Ac=.356c595d-7d25-45b5-8540-177df5e5086d@github.com>
On Fri, 20 Nov 2020 20:25:04 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits:
>>
>> - Merge
>> - Moved task queues to marking context
>> - Merge
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Removed obsoleted class
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - Merge branch 'master' into JDK-8255019-sh-mark
>> - ... and 10 more: https://git.openjdk.java.net/jdk/compare/98a5d5a6...8c58f6f4
>
> src/hotspot/share/gc/shenandoah/shenandoahMarkCompact.cpp line 126:
>
>> 124: // c. Update roots if this full GC is due to evac-oom, which may carry from-space pointers in roots.
>> 125: if (has_forwarded_objects) {
>> 126: ShenandoahConcurrentMark::update_roots(ShenandoahPhaseTimings::full_gc_update_roots);
>
> Same as with cancel() above.
ShenandoahConcurrentMark is a temporary placement for update_roots(), it will be moved to ShenandoahGC in later refactoring, so this call becomes instance method call, as ShenandoahMarkCompact is a subclass of ShenandoahGC.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1009
From shade at openjdk.java.net Mon Nov 23 22:04:08 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 23 Nov 2020 22:04:08 GMT
Subject: RFR: 8256912: Zero builds fail after JDK-8255984
Message-ID: <ciWUdoAvSJYi2zs5Zmx8uRP-yU_hMG3gVNXFqGUeGy4=.8534e30e-3aae-4cff-9882-deb9a32d28db@github.com>
This went through testing cracks, as JDK-8255796 was integrated at the same time. Zero x86_64 fails with:
In file included from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.hpp:30,
from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:28:
348 | inline size_t get_live_data_bytes() const;
| ^~~~~~~~~~~~~~~~~~~
351 | inline size_t garbage() const;
| ^~~~~~~
The fix is trivial: reinstating the `#include`-s.
-------------
Commit messages:
- 8256912: Zero builds fail after JDK-8255984
Changes: https://git.openjdk.java.net/jdk/pull/1401/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1401&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256912
Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
Patch: https://git.openjdk.java.net/jdk/pull/1401.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1401/head:pull/1401
PR: https://git.openjdk.java.net/jdk/pull/1401
From zgu at openjdk.java.net Mon Nov 23 22:15:57 2020
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 23 Nov 2020 22:15:57 GMT
Subject: RFR: 8256912: Zero builds fail after JDK-8255984
In-Reply-To: <ciWUdoAvSJYi2zs5Zmx8uRP-yU_hMG3gVNXFqGUeGy4=.8534e30e-3aae-4cff-9882-deb9a32d28db@github.com>
References: <ciWUdoAvSJYi2zs5Zmx8uRP-yU_hMG3gVNXFqGUeGy4=.8534e30e-3aae-4cff-9882-deb9a32d28db@github.com>
Message-ID: <AkL6WLQjnvhRZ8LWyJMFDcVbh6I_YFm5-IBSV-7mmw8=.18852b04-bc0e-4b44-8e8d-f5bcaefe638e@github.com>
On Mon, 23 Nov 2020 21:58:21 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> This went through testing cracks, as JDK-8255796 was integrated at the same time. Zero x86_64 fails with:
>
> In file included from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.hpp:30,
> from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:28:
>
> 348 | inline size_t get_live_data_bytes() const;
> | ^~~~~~~~~~~~~~~~~~~
>
> 351 | inline size_t garbage() const;
> | ^~~~~~~
>
> The fix is trivial: reinstating the `#include`-s.
Looks good.
-------------
Marked as reviewed by zgu (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1401
From rkennke at openjdk.java.net Mon Nov 23 22:27:54 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 23 Nov 2020 22:27:54 GMT
Subject: RFR: 8256912: Zero builds fail after JDK-8255984
In-Reply-To: <ciWUdoAvSJYi2zs5Zmx8uRP-yU_hMG3gVNXFqGUeGy4=.8534e30e-3aae-4cff-9882-deb9a32d28db@github.com>
References: <ciWUdoAvSJYi2zs5Zmx8uRP-yU_hMG3gVNXFqGUeGy4=.8534e30e-3aae-4cff-9882-deb9a32d28db@github.com>
Message-ID: <XObExoyNB6FXf_0bBQFt9qKoU-w2N26StAXUhakteEY=.283099a8-0956-4a18-92ba-c30d96673b35@github.com>
On Mon, 23 Nov 2020 21:58:21 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> This went through testing cracks, as JDK-8256497 was integrated at the same time. Zero x86_64 fails with:
>
> In file included from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.hpp:30,
> from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:28:
>
> 348 | inline size_t get_live_data_bytes() const;
> | ^~~~~~~~~~~~~~~~~~~
>
> 351 | inline size_t garbage() const;
> | ^~~~~~~
>
> The fix is trivial: reinstating the `#include`-s.
Looks good and trivial! Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1401
From shade at openjdk.java.net Mon Nov 23 22:33:56 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 23 Nov 2020 22:33:56 GMT
Subject: Integrated: 8256912: Zero builds fail after JDK-8255984
In-Reply-To: <ciWUdoAvSJYi2zs5Zmx8uRP-yU_hMG3gVNXFqGUeGy4=.8534e30e-3aae-4cff-9882-deb9a32d28db@github.com>
References: <ciWUdoAvSJYi2zs5Zmx8uRP-yU_hMG3gVNXFqGUeGy4=.8534e30e-3aae-4cff-9882-deb9a32d28db@github.com>
Message-ID: <ud9ddENqAjVOtpfS1pBkdgRW-Vd5f3zHMZL7o8rK-F8=.348ea649-55fa-4d20-b4a8-3163d3b5d317@github.com>
On Mon, 23 Nov 2020 21:58:21 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> This went through testing cracks, as JDK-8256497 was integrated at the same time. Zero x86_64 fails with:
>
> In file included from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.hpp:30,
> from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:28:
>
> 348 | inline size_t get_live_data_bytes() const;
> | ^~~~~~~~~~~~~~~~~~~
>
> 351 | inline size_t garbage() const;
> | ^~~~~~~
>
> The fix is trivial: reinstating the `#include`-s.
This pull request has now been integrated.
Changeset: 1df94c9f
Author: Aleksey Shipilev <shade at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/1df94c9f
Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
8256912: Zero builds fail after JDK-8255984
Reviewed-by: zgu, rkennke
-------------
PR: https://git.openjdk.java.net/jdk/pull/1401
From shade at openjdk.java.net Mon Nov 23 22:33:54 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 23 Nov 2020 22:33:54 GMT
Subject: RFR: 8256912: Zero builds fail after JDK-8255984
In-Reply-To: <XObExoyNB6FXf_0bBQFt9qKoU-w2N26StAXUhakteEY=.283099a8-0956-4a18-92ba-c30d96673b35@github.com>
References: <ciWUdoAvSJYi2zs5Zmx8uRP-yU_hMG3gVNXFqGUeGy4=.8534e30e-3aae-4cff-9882-deb9a32d28db@github.com>
<XObExoyNB6FXf_0bBQFt9qKoU-w2N26StAXUhakteEY=.283099a8-0956-4a18-92ba-c30d96673b35@github.com>
Message-ID: <VT8CbCu6SeP9_fq_kcD2Y7BSVXCu1UXh0BeecUv5AJU=.ede9fd8a-2b8a-4fdb-b831-00cee11dee54@github.com>
On Mon, 23 Nov 2020 22:25:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> This went through testing cracks, as JDK-8256497 was integrated at the same time. Zero x86_64 fails with:
>>
>> In file included from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.hpp:30,
>> from /home/runner/work/jdk/jdk/jdk/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:28:
>>
>> 348 | inline size_t get_live_data_bytes() const;
>> | ^~~~~~~~~~~~~~~~~~~
>>
>> 351 | inline size_t garbage() const;
>> | ^~~~~~~
>>
>> The fix is trivial: reinstating the `#include`-s.
>
> Looks good and trivial! Thanks!
Thanks!
-------------
PR: https://git.openjdk.java.net/jdk/pull/1401
From shade at openjdk.java.net Tue Nov 24 12:59:00 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 24 Nov 2020 12:59:00 GMT
Subject: RFR: 8256949: Shenandoah: ditch allocation spike and GC penalties
handling
Message-ID: <CER9xm59zAO5wLmGPzW4sm4yGB-N3fGYHd4BXx6TJAo=.7d356a77-cb9c-474d-8f82-97b49653dfe1@github.com>
Following the improvements in JDK-8255984, I think we can dispense with old-style allocation spike and GC penalties handling. JDK-8255984 is supposed to accommodate both cases now. This issue is to have a base for performance evaluation.
Additional testing:
- [x] Linux x86_64 `hotspot_gc_shenandoah`
-------------
Commit messages:
- 8256949: Shenandoah: ditch allocation spike and GC penalties handling
Changes: https://git.openjdk.java.net/jdk/pull/1409/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1409&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256949
Stats: 59 lines in 4 files changed: 0 ins; 55 del; 4 mod
Patch: https://git.openjdk.java.net/jdk/pull/1409.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1409/head:pull/1409
PR: https://git.openjdk.java.net/jdk/pull/1409
From zgu at redhat.com Tue Nov 24 15:31:08 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Tue, 24 Nov 2020 10:31:08 -0500
Subject: [sh/jdk8u] RFR 8221507: Implement JFR Events for Shenandoah
Message-ID: <6a76dbc7-cc95-c68d-c4a9-04a8947753eb@redhat.com>
Hi,
Please review this 8u backport of JFR events for Shenandoah.
The original patch does not apply cleanly.
1) The patch has to be split into hotspot and jdk parts.
2) Adjusted @library and removed @requires vm.hasJFR tag(not supported
in 8u) for tests
3) Removed INCLUDE_SHENANDOAHGC, sh/8u always builds with Shenandoah on.
4) Adjusted imports
5) ShenandoahHeapRegion::region_number() -> ShenandoahHeapRegion::index()
6) Manually fixed some merge conflicts
The original webrev: http://hg.openjdk.java.net/jdk/jdk/rev/43340a79840d
sh/8u webrev:
hotspot:
http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/hotspot/webrev.00/
jdk:
http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/jdk/webrev.00/
Test:
Passed new tests in patch on Linux x86_64
Thanks,
-Zhengyu
From rkennke at redhat.com Tue Nov 24 15:44:29 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 24 Nov 2020 16:44:29 +0100
Subject: [sh/jdk8u] RFR 8221507: Implement JFR Events for Shenandoah
In-Reply-To: <6a76dbc7-cc95-c68d-c4a9-04a8947753eb@redhat.com>
References: <6a76dbc7-cc95-c68d-c4a9-04a8947753eb@redhat.com>
Message-ID: <754ddbd8-b545-865c-59a4-279d915735e5@redhat.com>
Hi Zhengyu,
Looks good so far. A few things:
- Prepend commit message with [backport]
- Do not entirely remove INCLUDE_SHENANDOAHGC, replace it with
INCLUDE_ALL_GCS. Do this for the #include
"gc_implementation/shenandoah/shenandoahJfrSupport.hpp" too.
- In src/share/vm/jfr/periodic/jfrPeriodic.cpp you are removing a
newline between the includes and the first block of code. Don't do that.
Thanks for backporting this!
Please also take care of follow-up issues JDK-8224573 and JDK-8224529,
and push them together, ok?
Thanks, Roman
On 24/11/2020 16:31, Zhengyu Gu wrote:
> Hi,
>
> Please review this 8u backport of JFR events for Shenandoah.
>
> The original patch does not apply cleanly.
>
> 1) The patch has to be split into hotspot and jdk parts.
> 2) Adjusted @library and removed @requires vm.hasJFR tag(not supported
> in 8u) for tests
> 3) Removed INCLUDE_SHENANDOAHGC, sh/8u always builds with Shenandoah on.
> 4) Adjusted imports
> 5) ShenandoahHeapRegion::region_number() -> ShenandoahHeapRegion::index()
> 6) Manually fixed some merge conflicts
>
>
> The original webrev: http://hg.openjdk.java.net/jdk/jdk/rev/43340a79840d
>
> sh/8u webrev:
> ? hotspot:
> http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/hotspot/webrev.00/
>
> ? jdk:
> http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/jdk/webrev.00/
>
> Test:
> ? Passed new tests in patch on Linux x86_64
>
> Thanks,
>
> -Zhengyu
>
>
From rkennke at redhat.com Tue Nov 24 16:11:31 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 24 Nov 2020 17:11:31 +0100
Subject: RFR(sh/jdk8): [backport] 8231087: Shenandoah: Self-fixing load
reference barriers for C1/C2
In-Reply-To: <e9521eb3-9ad6-a7ff-72f9-eb5cc63341e3@redhat.com>
References: <e9521eb3-9ad6-a7ff-72f9-eb5cc63341e3@redhat.com>
Message-ID: <6b6e32e9-d842-6350-d938-32988a46e1bf@redhat.com>
I made a mistake in the aarch64 parts. Please review this webrev instead:
http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.02/
> 2nd attempt:
>
> Issue:
> https://bugs.openjdk.java.net/browse/JDK-8231087
>
> Backport:
> http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.01/
>
> This time includes the actual C1 stuff (depended on JDK-8228369).
>
> Let's include a few follow-ups:
>
> https://bugs.openjdk.java.net/browse/JDK-8233165
> http://cr.openjdk.java.net/~rkennke/JDK-8233165-jdk8/webrev.00/
>
> and
>
> https://bugs.openjdk.java.net/browse/JDK-8233021
> http://cr.openjdk.java.net/~rkennke/JDK-8233021-jdk8/webrev.00/
>
> and
>
> https://bugs.openjdk.java.net/browse/JDK-8238153
> http://cr.openjdk.java.net/~rkennke/JDK-8238153-jdk8/webrev.00/
>
> Can I please get a review?
>
> Thanks,
> Roman
From github.com+71722661+earthling-amzn at openjdk.java.net Tue Nov 24 17:34:57 2020
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Tue, 24 Nov 2020 17:34:57 GMT
Subject: RFR: 8256949: Shenandoah: ditch allocation spike and GC penalties
handling
In-Reply-To: <CER9xm59zAO5wLmGPzW4sm4yGB-N3fGYHd4BXx6TJAo=.7d356a77-cb9c-474d-8f82-97b49653dfe1@github.com>
References: <CER9xm59zAO5wLmGPzW4sm4yGB-N3fGYHd4BXx6TJAo=.7d356a77-cb9c-474d-8f82-97b49653dfe1@github.com>
Message-ID: <K6aKSu1nQJ_qBVdHo5spGO45sPzEuqbDNTfyOJDpBm8=.b131108e-6371-432c-ad3f-2177f5864d4f@github.com>
On Tue, 24 Nov 2020 12:53:29 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
> Following the improvements in JDK-8255984, I think we can dispense with old-style allocation spike and GC penalties handling. JDK-8255984 is supposed to accommodate both cases now. This issue is to have a base for performance evaluation.
>
> Additional testing:
> - [x] Linux x86_64 `hotspot_gc_shenandoah`
Might I suggest we instead lower the defaults but keep this safety net as an available option? Maybe remove the penalties, but keep the `ShenandoahAllocSpikeFactor`?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1409
From zgu at redhat.com Wed Nov 25 16:21:07 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Wed, 25 Nov 2020 11:21:07 -0500
Subject: [sh/jdk8u] RFR 8221507: Implement JFR Events for Shenandoah
In-Reply-To: <754ddbd8-b545-865c-59a4-279d915735e5@redhat.com>
References: <6a76dbc7-cc95-c68d-c4a9-04a8947753eb@redhat.com>
<754ddbd8-b545-865c-59a4-279d915735e5@redhat.com>
Message-ID: <74ff2bdc-9633-a579-3561-da6dc28aedde@redhat.com>
Hi Roman,
On 11/24/20 10:44 AM, Roman Kennke wrote:
> Hi Zhengyu,
>
> Looks good so far. A few things:
>
> - Prepend commit message with [backport]
> - Do not entirely remove INCLUDE_SHENANDOAHGC, replace it with
> INCLUDE_ALL_GCS. Do this for the #include
> "gc_implementation/shenandoah/shenandoahJfrSupport.hpp" too.
> - In src/share/vm/jfr/periodic/jfrPeriodic.cpp you are removing a
> newline between the includes and the first block of code. Don't do that.
Thanks for reviewing.
Updated:
http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/hotspot/webrev.01/
Thanks,
-Zhengyu
>
> Thanks for backporting this!
>
> Please also take care of follow-up issues JDK-8224573 and JDK-8224529,
> and push them together, ok?
>
> Thanks, Roman
>
>
> On 24/11/2020 16:31, Zhengyu Gu wrote:
>> Hi,
>>
>> Please review this 8u backport of JFR events for Shenandoah.
>>
>> The original patch does not apply cleanly.
>>
>> 1) The patch has to be split into hotspot and jdk parts.
>> 2) Adjusted @library and removed @requires vm.hasJFR tag(not supported
>> in 8u) for tests
>> 3) Removed INCLUDE_SHENANDOAHGC, sh/8u always builds with Shenandoah on.
>> 4) Adjusted imports
>> 5) ShenandoahHeapRegion::region_number() -> ShenandoahHeapRegion::index()
>> 6) Manually fixed some merge conflicts
>>
>>
>> The original webrev: http://hg.openjdk.java.net/jdk/jdk/rev/43340a79840d
>>
>> sh/8u webrev:
>> ?? hotspot:
>> http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/hotspot/webrev.00/
>>
>> ?? jdk:
>> http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/jdk/webrev.00/
>>
>> Test:
>> ?? Passed new tests in patch on Linux x86_64
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>
>
From zgu at redhat.com Wed Nov 25 16:28:32 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Wed, 25 Nov 2020 11:28:32 -0500
Subject: [sh/jdk8u] RFR 8224573: Fix windows build after JDK-8221507
Message-ID: <e1b3d21c-2177-e2af-f533-b6b3bf334c02@redhat.com>
Hi,
I would like to backport this patch to Sh/jdk8u as followup of
JDK-8221507 backport, to fix build problem on Windows.
The original patch does not apply cleanly, due to
ShenandoahHeapRegion::region_number() -> ShenandoahHeapRegion::index()
renaming.
Original patch: http://hg.openjdk.java.net/jdk/jdk/rev/6b976a59ee87
Sh/jdk8u patch:
# HG changeset patch
# User clanger
# Date 1558510920 -3600
# Node ID 6b976a59ee8783bb2d37a7ae1309f56be8636c10
# Parent a8be2165f87c1faf9e445f67f8b0a16d38844f56
[backport] 8224573: Fix windows build after JDK-8221507
Reviewed-by: shade, stuefe
diff -r a8be2165f87c
src/share/vm/gc_implementation/shenandoah/shenandoahHeapRegion.cpp
--- a/src/share/vm/gc_implementation/shenandoah/shenandoahHeapRegion.cpp
Tue May 21 10:36:23 2019 +0200
+++ b/src/share/vm/gc_implementation/shenandoah/shenandoahHeapRegion.cpp
Wed Nov 25 11:20:30 2020 -0500
@@ -641,7 +641,7 @@
void ShenandoahHeapRegion::set_state(RegionState to) {
EventShenandoahHeapRegionStateChange evt;
if (evt.should_commit()){
- evt.set_index(index());
+ evt.set_index((unsigned)index());
evt.set_start((uintptr_t)bottom());
evt.set_used(used());
evt.set_from(_state);
diff -r a8be2165f87c
src/share/vm/gc_implementation/shenandoah/shenandoahJfrSupport.cpp
--- a/src/share/vm/gc_implementation/shenandoah/shenandoahJfrSupport.cpp
Tue May 21 10:36:23 2019 +0200
+++ b/src/share/vm/gc_implementation/shenandoah/shenandoahJfrSupport.cpp
Wed Nov 25 11:20:30 2020 -0500
@@ -57,7 +57,7 @@
public:
virtual void heap_region_do(ShenandoahHeapRegion* r) {
EventShenandoahHeapRegionInformation evt;
- evt.set_index(r->index());
+ evt.set_index((unsigned)r->index());
evt.set_state((u8)r->state());
evt.set_start((uintptr_t)r->bottom());
evt.set_used(r->used());
Thanks,
-Zhengyu
From rkennke at redhat.com Wed Nov 25 18:05:53 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 25 Nov 2020 19:05:53 +0100
Subject: [sh/jdk8u] RFR 8221507: Implement JFR Events for Shenandoah
In-Reply-To: <74ff2bdc-9633-a579-3561-da6dc28aedde@redhat.com>
References: <6a76dbc7-cc95-c68d-c4a9-04a8947753eb@redhat.com>
<754ddbd8-b545-865c-59a4-279d915735e5@redhat.com>
<74ff2bdc-9633-a579-3561-da6dc28aedde@redhat.com>
Message-ID: <ec060204-a88c-fef8-32b6-cb0b861b71b4@redhat.com>
Looks good to me!
Thanks,
Roman
> Hi Roman,
>
> On 11/24/20 10:44 AM, Roman Kennke wrote:
>> Hi Zhengyu,
>>
>> Looks good so far. A few things:
>>
>> - Prepend commit message with [backport]
>> - Do not entirely remove INCLUDE_SHENANDOAHGC, replace it with
>> INCLUDE_ALL_GCS. Do this for the #include
>> "gc_implementation/shenandoah/shenandoahJfrSupport.hpp" too.
>> - In src/share/vm/jfr/periodic/jfrPeriodic.cpp you are removing a
>> newline between the includes and the first block of code. Don't do that.
>
> Thanks for reviewing.
>
> Updated:
> http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/hotspot/webrev.01/
>
>
> Thanks,
>
> -Zhengyu
>
>>
>> Thanks for backporting this!
>>
>> Please also take care of follow-up issues JDK-8224573 and JDK-8224529,
>> and push them together, ok?
>>
>> Thanks, Roman
>>
>>
>> On 24/11/2020 16:31, Zhengyu Gu wrote:
>>> Hi,
>>>
>>> Please review this 8u backport of JFR events for Shenandoah.
>>>
>>> The original patch does not apply cleanly.
>>>
>>> 1) The patch has to be split into hotspot and jdk parts.
>>> 2) Adjusted @library and removed @requires vm.hasJFR tag(not
>>> supported in 8u) for tests
>>> 3) Removed INCLUDE_SHENANDOAHGC, sh/8u always builds with Shenandoah on.
>>> 4) Adjusted imports
>>> 5) ShenandoahHeapRegion::region_number() ->
>>> ShenandoahHeapRegion::index()
>>> 6) Manually fixed some merge conflicts
>>>
>>>
>>> The original webrev: http://hg.openjdk.java.net/jdk/jdk/rev/43340a79840d
>>>
>>> sh/8u webrev:
>>> ?? hotspot:
>>> http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/hotspot/webrev.00/
>>>
>>> ?? jdk:
>>> http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8221507-8u/jdk/webrev.00/
>>>
>>> Test:
>>> ?? Passed new tests in patch on Linux x86_64
>>>
>>> Thanks,
>>>
>>> -Zhengyu
>>>
>>>
>>
>
From shade at redhat.com Wed Nov 25 18:51:16 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 25 Nov 2020 19:51:16 +0100
Subject: [sh/jdk8u] RFR 8224573: Fix windows build after JDK-8221507
In-Reply-To: <e1b3d21c-2177-e2af-f533-b6b3bf334c02@redhat.com>
References: <e1b3d21c-2177-e2af-f533-b6b3bf334c02@redhat.com>
Message-ID: <b31f316c-c1b3-993e-df33-cb2431718252@redhat.com>
On 11/25/20 5:28 PM, Zhengyu Gu wrote:
> Original patch: http://hg.openjdk.java.net/jdk/jdk/rev/6b976a59ee87
> Sh/jdk8u patch:
Looks fine.
--
Thanks,
-Aleksey
From zgu at redhat.com Wed Nov 25 18:53:01 2020
From: zgu at redhat.com (zgu at redhat.com)
Date: Wed, 25 Nov 2020 18:53:01 +0000
Subject: hg: shenandoah/jdk8/hotspot: [backport] 8221507: Implement JFR Events
for Shenandoah
Message-ID: <202011251853.0APIr15K019014@aojmv0008.oracle.com>
Changeset: a8be2165f87c
Author: kdobson
Date: 2019-05-21 10:36 +0200
URL: https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/a8be2165f87c
[backport] 8221507: Implement JFR Events for Shenandoah
Reviewed-by: rkennke, shade, egahlin, mseledtsov
Contributed-by: Ken Dobson <kdobson at redhat.com>
! src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp
! src/share/vm/gc_implementation/shenandoah/shenandoahHeapRegion.cpp
! src/share/vm/gc_implementation/shenandoah/shenandoahHeapRegion.hpp
+ src/share/vm/gc_implementation/shenandoah/shenandoahJfrSupport.cpp
+ src/share/vm/gc_implementation/shenandoah/shenandoahJfrSupport.hpp
! src/share/vm/jfr/metadata/metadata.xml
! src/share/vm/jfr/periodic/jfrPeriodic.cpp
From rkennke at redhat.com Wed Nov 25 20:39:57 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 25 Nov 2020 21:39:57 +0100
Subject: RFR: [backport] 8202976: Add C1 lea patching support for x86
In-Reply-To: <3f28dd8b-5e4c-c285-0a99-ba177389683a@redhat.com>
References: <02bae8a4-04ff-f6f4-5cb0-170bd82d3511@redhat.com>
<3f28dd8b-5e4c-c285-0a99-ba177389683a@redhat.com>
Message-ID: <1f2f075a-8c5c-36f6-047d-137c7201aaac@redhat.com>
This patch was actually missing the aarch64 parts. The original change
did not have it either, it only came much later with the ZGC port to
aarch64. I would like to add it here.
http://cr.openjdk.java.net/~rkennke/JDK-8202976-jdk8/webrev.01/
Testing: hotspot_gc_shenandoah (also, with the following SFX patches
that actually use it)
Ok too?
Thanks,
Roman
On 23/11/2020 13:11, Aleksey Shipilev wrote:
> On 11/20/20 8:17 PM, Roman Kennke wrote:
>> http://cr.openjdk.java.net/~rkennke/JDK-8202976-jdk8/webrev.00/
>
> Looks fine.
>
From rkennke at redhat.com Wed Nov 25 21:23:53 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 25 Nov 2020 22:23:53 +0100
Subject: RFR(sh/jdk8): [backport] 8228369: Shenandoah: Refactor LRB C1
stubs
In-Reply-To: <86b0d234-15e6-fb68-4d7a-d83c13296745@redhat.com>
References: <86b0d234-15e6-fb68-4d7a-d83c13296745@redhat.com>
Message-ID: <83dfb812-4aaa-1b8f-9e37-82f1cc9df748@redhat.com>
It turns out that the aarch64 parts have been incorrect. Please review
this webrev instead:
http://cr.openjdk.java.net/~rkennke/JDK-8228369-jdk8/webrev.01/
Testing: hotspot_gc_shenandoah, specjvm, both on x86 and aarch64, also
with upcoming SFX barriers change.
Ok?
Thanks,
Roman
On 23/11/2020 20:25, Roman Kennke wrote:
> This backports the following issue to jdk8:
>
> https://bugs.openjdk.java.net/browse/JDK-8228369
>
> This is in preparation for the SFX barrier backport.
>
> Due to missing GC interfaces, I had to put some code in c1_Runtime*, it
> is following the same pattern as other GC runtime stubs.
>
> http://cr.openjdk.java.net/~rkennke/JDK-8228369-jdk8/webrev.00/
>
> Let's also include a follow-ups:
>
> https://bugs.openjdk.java.net/browse/JDK-8229709
> http://cr.openjdk.java.net/~rkennke/JDK-8229709-jdk8/webrev.00/
>
> The other two follow-ups ( JDK-8229470 and JDK-8229977) don't apply to
> jdk8 because of different implementations.
>
> Can I please get a review?
>
> Thanks,
> Roman
From rkennke at redhat.com Wed Nov 25 21:28:44 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 25 Nov 2020 22:28:44 +0100
Subject: RFR(sh/jdk8): [backport] 8231087: Shenandoah: Self-fixing load
reference barriers for C1/C2
In-Reply-To: <6b6e32e9-d842-6350-d938-32988a46e1bf@redhat.com>
References: <e9521eb3-9ad6-a7ff-72f9-eb5cc63341e3@redhat.com>
<6b6e32e9-d842-6350-d938-32988a46e1bf@redhat.com>
Message-ID: <5c476212-75c1-aaea-53b7-4f750a531335@redhat.com>
Rebased on previous changes. Please review this instead:
http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.03/
Thanks,
Roman
On 24/11/2020 17:11, Roman Kennke wrote:
> I made a mistake in the aarch64 parts. Please review this webrev instead:
>
> http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.02/
>
>> 2nd attempt:
>>
>> Issue:
>> https://bugs.openjdk.java.net/browse/JDK-8231087
>>
>> Backport:
>> http://cr.openjdk.java.net/~rkennke/JDK-8231087-jdk8/webrev.01/
>>
>> This time includes the actual C1 stuff (depended on JDK-8228369).
>>
>> Let's include a few follow-ups:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8233165
>> http://cr.openjdk.java.net/~rkennke/JDK-8233165-jdk8/webrev.00/
>>
>> and
>>
>> https://bugs.openjdk.java.net/browse/JDK-8233021
>> http://cr.openjdk.java.net/~rkennke/JDK-8233021-jdk8/webrev.00/
>>
>> and
>>
>> https://bugs.openjdk.java.net/browse/JDK-8238153
>> http://cr.openjdk.java.net/~rkennke/JDK-8238153-jdk8/webrev.00/
>>
>> Can I please get a review?
>>
>> Thanks,
>> Roman
From kvn at openjdk.java.net Wed Nov 25 23:35:14 2020
From: kvn at openjdk.java.net (Vladimir Kozlov)
Date: Wed, 25 Nov 2020 23:35:14 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v2]
In-Reply-To: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
Message-ID: <lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
> JDK-8188055 added the function Reference.refersTo. For performance, the supporting native methods Reference.refersTo0 and PhantomReference.refersTo0 should be intrinsified by C2.
>
> Initial patch was prepared by @fisk.
>
> Tested hs-tier1-4. Added new compiler tests to test intrinsics.
>
> Ran new test with Shenandoah. Found only one issue. As result I disable PhantomReference::refersTo intrinsic for COOP+ Shenandoah combination. Someone from Shenandoah team have to test changes if that is enough.
Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
- Merge branch 'master' into JDK-8256999
- Added ZLoadBarrierElided = 0 definition.
Removed is_exact argument in load_field_from_object().
Added Shenandoah support for narrow phantom accesses.
- 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1425/files
- new: https://git.openjdk.java.net/jdk/pull/1425/files/7bfec378..08bdd307
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1425&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1425&range=00-01
Stats: 8771 lines in 206 files changed: 2656 ins; 872 del; 5243 mod
Patch: https://git.openjdk.java.net/jdk/pull/1425.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1425/head:pull/1425
PR: https://git.openjdk.java.net/jdk/pull/1425
From vlivanov at openjdk.java.net Thu Nov 26 11:33:58 2020
From: vlivanov at openjdk.java.net (Vladimir Ivanov)
Date: Thu, 26 Nov 2020 11:33:58 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v2]
In-Reply-To: <lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
<lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
Message-ID: <yHVzVgrZavafxaWMHuZosSesJQiOwvFddAxrDRv7V7M=.41edbc53-bcd7-450b-84ae-8ae5bd778350@github.com>
On Wed, 25 Nov 2020 23:35:14 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:
>> JDK-8188055 added the function Reference.refersTo. For performance, the supporting native methods Reference.refersTo0 and PhantomReference.refersTo0 should be intrinsified by C2.
>>
>> Initial patch was prepared by @fisk.
>>
>> Tested hs-tier1-4. Added new compiler tests to test intrinsics.
>>
>> Ran new test with Shenandoah. Found only one issue. As result I disable PhantomReference::refersTo intrinsic for COOP+ Shenandoah combination. Someone from Shenandoah team have to test changes if that is enough.
>
> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
>
> - Merge branch 'master' into JDK-8256999
> - Added ZLoadBarrierElided = 0 definition.
> Removed is_exact argument in load_field_from_object().
> Added Shenandoah support for narrow phantom accesses.
> - 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
Marked as reviewed by vlivanov (Reviewer).
-------------
PR: https://git.openjdk.java.net/jdk/pull/1425
From rkennke at openjdk.java.net Thu Nov 26 13:53:59 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 26 Nov 2020 13:53:59 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v2]
In-Reply-To: <lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
<lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
Message-ID: <pg7lETjxXQK3qQ8EoVvimXfB00tTIeHEqPsg0OLRK2c=.28d436a8-0911-4f82-9518-6d45075eb837@github.com>
On Wed, 25 Nov 2020 23:35:14 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:
>> JDK-8188055 added the function Reference.refersTo. For performance, the supporting native methods Reference.refersTo0 and PhantomReference.refersTo0 should be intrinsified by C2.
>>
>> Initial patch was prepared by @fisk.
>>
>> Tested hs-tier1-4. Added new compiler tests to test intrinsics.
>>
>> Ran new test with Shenandoah. Found only one issue. As result I disable PhantomReference::refersTo intrinsic for COOP+ Shenandoah combination. Someone from Shenandoah team have to test changes if that is enough.
>
> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
>
> - Merge branch 'master' into JDK-8256999
> - Added ZLoadBarrierElided = 0 definition.
> Removed is_exact argument in load_field_from_object().
> Added Shenandoah support for narrow phantom accesses.
> - 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
Shenandoah parts look good to me! Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1425
From rkennke at redhat.com Thu Nov 26 20:00:14 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 26 Nov 2020 21:00:14 +0100
Subject: RFR(sh/jdk8): [backport] 8232992: Shenandoah: Implement self-fixing
interpreter LRB
Message-ID: <bdcb378b-bfc8-7a99-2f61-bde6f0ce77df@redhat.com>
This backports the issue:
https://bugs.openjdk.java.net/browse/JDK-8257180
It is different from later incarnations. The JDK8 interpreter code does
not call through stub as we did when we originally implemented the sfx
barriers, but instead calls the runtime entries directly. This was
because we have no simple GC interface for interpreter stub generation.
However, we switched to that runtime calling model in tip later. So what
I did was adapt the *current* interpreter code, take out handling of
weak access, and adjust a little to jdk8.
Another change was, instead of adding our LRB into
MacroAssembler::load_heap_oop(), we now fully replace it. That's because
we need to preserve registers of the src address. This would have been
much too messy otherwise.
Testing: hotspot_gc_shenandoah (x86 and aarch64)
http://cr.openjdk.java.net/~rkennke/JDK-8232992-jdk8/webrev.01/
Ok?
From rkennke at redhat.com Fri Nov 27 13:55:44 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 27 Nov 2020 14:55:44 +0100
Subject: RFR(sh/jdk8): [backport] 8238851: Shenandoah: C1: Resolve into
registers of correct type
Message-ID: <8e87d1bb-aeaa-303f-8dbf-18a49f45828b@redhat.com>
This backports:
https://bugs.openjdk.java.net/browse/JDK-8238851
It is relatively straightforward, mostly file location and few context
changes.
Testing: hotspot_gc_shenandoah
Ok?
Roman
From kvn at openjdk.java.net Fri Nov 27 19:05:00 2020
From: kvn at openjdk.java.net (Vladimir Kozlov)
Date: Fri, 27 Nov 2020 19:05:00 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v2]
In-Reply-To: <pg7lETjxXQK3qQ8EoVvimXfB00tTIeHEqPsg0OLRK2c=.28d436a8-0911-4f82-9518-6d45075eb837@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
<lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
<pg7lETjxXQK3qQ8EoVvimXfB00tTIeHEqPsg0OLRK2c=.28d436a8-0911-4f82-9518-6d45075eb837@github.com>
Message-ID: <SOohAnwuR42x60H8-f6ZAo_iNnZxbNCo2CdPQEiKrEU=.4b82aa52-80c9-4373-88bf-06bfcbb26c81@github.com>
On Thu, 26 Nov 2020 13:51:05 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
>>
>> - Merge branch 'master' into JDK-8256999
>> - Added ZLoadBarrierElided = 0 definition.
>> Removed is_exact argument in load_field_from_object().
>> Added Shenandoah support for narrow phantom accesses.
>> - 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
>
> Shenandoah parts look good to me! Thanks!
@fisk and @shipilev are you fine with updated version?
-------------
PR: https://git.openjdk.java.net/jdk/pull/1425
From eosterlund at openjdk.java.net Sat Nov 28 09:04:03 2020
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Sat, 28 Nov 2020 09:04:03 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v2]
In-Reply-To: <lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
<lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
Message-ID: <eJl9Z5tz7m2XRNvtXMdLMA9ZVqRQ2nrpuTzaIHXakCg=.c97c75ea-2867-4eea-8792-14a90b5fae8c@github.com>
On Wed, 25 Nov 2020 23:35:14 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:
>> JDK-8188055 added the function Reference.refersTo. For performance, the supporting native methods Reference.refersTo0 and PhantomReference.refersTo0 should be intrinsified by C2.
>>
>> Initial patch was prepared by @fisk.
>>
>> Tested hs-tier1-4. Added new compiler tests to test intrinsics.
>>
>> Ran new test with Shenandoah. Found only one issue. As result I disable PhantomReference::refersTo intrinsic for COOP+ Shenandoah combination. Someone from Shenandoah team have to test changes if that is enough.
>
> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
>
> - Merge branch 'master' into JDK-8256999
> - Added ZLoadBarrierElided = 0 definition.
> Removed is_exact argument in load_field_from_object().
> Added Shenandoah support for narrow phantom accesses.
> - 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
Looks good.
-------------
Marked as reviewed by eosterlund (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1425
From shade at openjdk.java.net Sat Nov 28 09:46:01 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Sat, 28 Nov 2020 09:46:01 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v2]
In-Reply-To: <tgxA6KEmUot4HOhNKYT6O5k_DoYjQ_XPE4_8ev09Gik=.85e47193-644f-4f09-921d-32c922d48bae@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
<lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
<tgxA6KEmUot4HOhNKYT6O5k_DoYjQ_XPE4_8ev09Gik=.85e47193-644f-4f09-921d-32c922d48bae@github.com>
Message-ID: <yP9dZ_ws9OWKrw0yHvnUABs6pQfCF_JQG5VFLly8mv8=.741dfc53-efc9-4302-9cdb-ed5065682a23@github.com>
On Sat, 28 Nov 2020 09:43:15 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
>>
>> - Merge branch 'master' into JDK-8256999
>> - Added ZLoadBarrierElided = 0 definition.
>> Removed is_exact argument in load_field_from_object().
>> Added Shenandoah support for narrow phantom accesses.
>> - 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
>
> Yes, still looks good for me.
You might want to pull from master to get clean `x86_32` test runs.
-------------
PR: https://git.openjdk.java.net/jdk/pull/1425
From shade at openjdk.java.net Sat Nov 28 09:45:59 2020
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Sat, 28 Nov 2020 09:45:59 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v2]
In-Reply-To: <lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
<lRizv2JGrdsVw9AZ01ORWRlupCj4EkoGQ2phRMNUQEc=.9dd769f2-a3ae-4673-89cf-1959f984225c@github.com>
Message-ID: <tgxA6KEmUot4HOhNKYT6O5k_DoYjQ_XPE4_8ev09Gik=.85e47193-644f-4f09-921d-32c922d48bae@github.com>
On Wed, 25 Nov 2020 23:35:14 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:
>> JDK-8188055 added the function Reference.refersTo. For performance, the supporting native methods Reference.refersTo0 and PhantomReference.refersTo0 should be intrinsified by C2.
>>
>> Initial patch was prepared by @fisk.
>>
>> Tested hs-tier1-4. Added new compiler tests to test intrinsics.
>>
>> Ran new test with Shenandoah. Found only one issue. As result I disable PhantomReference::refersTo intrinsic for COOP+ Shenandoah combination. Someone from Shenandoah team have to test changes if that is enough.
>
> Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
>
> - Merge branch 'master' into JDK-8256999
> - Added ZLoadBarrierElided = 0 definition.
> Removed is_exact argument in load_field_from_object().
> Added Shenandoah support for narrow phantom accesses.
> - 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
Yes, still looks good for me.
-------------
Marked as reviewed by shade (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1425
From kvn at openjdk.java.net Sat Nov 28 23:30:13 2020
From: kvn at openjdk.java.net (Vladimir Kozlov)
Date: Sat, 28 Nov 2020 23:30:13 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v3]
In-Reply-To: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
Message-ID: <amcTT5-ov4DA74W4AXS1ouaeqzyM8kBT3a8irogNkcQ=.e5a52b6a-29a1-44d9-9466-1a5713aaf7be@github.com>
> JDK-8188055 added the function Reference.refersTo. For performance, the supporting native methods Reference.refersTo0 and PhantomReference.refersTo0 should be intrinsified by C2.
>
> Initial patch was prepared by @fisk.
>
> Tested hs-tier1-4. Added new compiler tests to test intrinsics.
>
> Ran new test with Shenandoah. Found only one issue. As result I disable PhantomReference::refersTo intrinsic for COOP+ Shenandoah combination. Someone from Shenandoah team have to test changes if that is enough.
Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
- Merge branch 'master' into JDK-8256999
- Merge branch 'master' into JDK-8256999
- Added ZLoadBarrierElided = 0 definition.
Removed is_exact argument in load_field_from_object().
Added Shenandoah support for narrow phantom accesses.
- 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1425/files
- new: https://git.openjdk.java.net/jdk/pull/1425/files/08bdd307..962d54d5
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1425&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1425&range=01-02
Stats: 5258 lines in 151 files changed: 2571 ins; 1881 del; 806 mod
Patch: https://git.openjdk.java.net/jdk/pull/1425.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1425/head:pull/1425
PR: https://git.openjdk.java.net/jdk/pull/1425
From kvn at openjdk.java.net Sun Nov 29 16:52:15 2020
From: kvn at openjdk.java.net (Vladimir Kozlov)
Date: Sun, 29 Nov 2020 16:52:15 GMT
Subject: RFR: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo [v4]
In-Reply-To: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
Message-ID: <u4S6Ff8wqORs8QXge6dHp9L6gEKjQ5XR7cuycqB-4vc=.e7589b4b-a7aa-4c67-ba98-748405f51500@github.com>
> JDK-8188055 added the function Reference.refersTo. For performance, the supporting native methods Reference.refersTo0 and PhantomReference.refersTo0 should be intrinsified by C2.
>
> Initial patch was prepared by @fisk.
>
> Tested hs-tier1-4. Added new compiler tests to test intrinsics.
>
> Ran new test with Shenandoah. Found only one issue. As result I disable PhantomReference::refersTo intrinsic for COOP+ Shenandoah combination. Someone from Shenandoah team have to test changes if that is enough.
Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
- Merge branch 'master' into JDK-8256999
- Merge branch 'master' into JDK-8256999
- Merge branch 'master' into JDK-8256999
- Added ZLoadBarrierElided = 0 definition.
Removed is_exact argument in load_field_from_object().
Added Shenandoah support for narrow phantom accesses.
- 8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
-------------
Changes:
- all: https://git.openjdk.java.net/jdk/pull/1425/files
- new: https://git.openjdk.java.net/jdk/pull/1425/files/962d54d5..0e4596ce
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1425&range=03
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1425&range=02-03
Stats: 115 lines in 5 files changed: 8 ins; 98 del; 9 mod
Patch: https://git.openjdk.java.net/jdk/pull/1425.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1425/head:pull/1425
PR: https://git.openjdk.java.net/jdk/pull/1425
From kvn at openjdk.java.net Sun Nov 29 20:31:58 2020
From: kvn at openjdk.java.net (Vladimir Kozlov)
Date: Sun, 29 Nov 2020 20:31:58 GMT
Subject: Integrated: 8256999: Add C2 intrinsic for Reference.refersTo and
PhantomReference::refersTo
In-Reply-To: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
References: <lfK3_B12A6FMofYIJXvJ2jWlEEd7LcXV_brCr2yctuk=.42dee9ca-0924-44e8-beda-42bbf3e26535@github.com>
Message-ID: <aiRBEGvI3kLeDEWX6C2uMe-P_F3LpY9ijH0XediyOSA=.0b977e6c-ffbd-47a8-96d1-b678d5a1926d@github.com>
On Wed, 25 Nov 2020 03:31:36 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:
> JDK-8188055 added the function Reference.refersTo. For performance, the supporting native methods Reference.refersTo0 and PhantomReference.refersTo0 should be intrinsified by C2.
>
> Initial patch was prepared by @fisk.
>
> Tested hs-tier1-4. Added new compiler tests to test intrinsics.
>
> Ran new test with Shenandoah. Found only one issue. As result I disable PhantomReference::refersTo intrinsic for COOP+ Shenandoah combination. Someone from Shenandoah team have to test changes if that is enough.
This pull request has now been integrated.
Changeset: 816e8f83
Author: Vladimir Kozlov <kvn at openjdk.org>
URL: https://git.openjdk.java.net/jdk/commit/816e8f83
Stats: 381 lines in 20 files changed: 248 ins; 62 del; 71 mod
8256999: Add C2 intrinsic for Reference.refersTo and PhantomReference::refersTo
Reviewed-by: pliden, vlivanov, rkennke, eosterlund, shade
-------------
PR: https://git.openjdk.java.net/jdk/pull/1425
From rkennke at openjdk.java.net Mon Nov 30 14:48:59 2020
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 30 Nov 2020 14:48:59 GMT
Subject: RFR: 8255019: Shenandoah: Split STW and concurrent mark into
separate classes [v16]
In-Reply-To: <U5ULwmL45yPORSs3HGASY7q4VPHdSg3y5OSw41KErHY=.8fbaf90b-13a6-4fe3-a06e-e6ae7a801ab5@github.com>
References: <T9KtzqPK9rEjIUcR_aNy4dmt_0E_3zI57Ricct56lHE=.1c6e85b4-c359-4e4c-b1d5-b9aae902d49c@github.com>
<U5ULwmL45yPORSs3HGASY7q4VPHdSg3y5OSw41KErHY=.8fbaf90b-13a6-4fe3-a06e-e6ae7a801ab5@github.com>
Message-ID: <mznIXUuJHlhvCHleEQLcUhWklbt8hhUYj5WvVNGEglM=.e8a46bcc-781a-4132-9fac-bbe4c3a7bf9a@github.com>
On Mon, 23 Nov 2020 19:35:18 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:
>> This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).
>>
>> Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.
>>
>> It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.
>>
>> First step, I would like to split STW and concurrent mark, so that:
>> 1) Code has to special case for STW and concurrent mark.
>> 2) STW mark does not need to rendezvous workers between root mark and the rest of mark
>> 3) STW mark does not need to activate SATB barrier and drain SATB buffers.
>> 4) STW mark does not need to remark some of roots.
>>
>> The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.
>>
>> A few changes:
>> 1) Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
>> 2) Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
>> 3) Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
>> 4) Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>
> Removed ShenandoahConcurrentMark parameter from concurrent GC entry/op, etc.
Looks ok to me. @shipilev should also have a look. Thanks!
-------------
Marked as reviewed by rkennke (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/1009