From vlivanov at openjdk.java.net Sun Nov 1 19:10:58 2020 From: vlivanov at openjdk.java.net (Vladimir Ivanov) Date: Sun, 1 Nov 2020 19:10:58 GMT Subject: RFR: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 17:40:51 GMT, Vladimir Kozlov wrote: > We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. > > We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. > > We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. > > Tested changes in all tiers. > > I verified that with these changes I still able to build Graal in open repo and run graalunit testing: > > `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` > `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` > `open$ make jdk-image` > `open$ make test-image` > `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` Looks good. ------------- Marked as reviewed by vlivanov (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/960 From iveresov at openjdk.java.net Sun Nov 1 20:17:55 2020 From: iveresov at openjdk.java.net (Igor Veresov) Date: Sun, 1 Nov 2020 20:17:55 GMT Subject: RFR: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 17:40:51 GMT, Vladimir Kozlov wrote: > We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. > > We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. > > We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. > > Tested changes in all tiers. > > I verified that with these changes I still able to build Graal in open repo and run graalunit testing: > > `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` > `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` > `open$ make jdk-image` > `open$ make test-image` > `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` Marked as reviewed by iveresov (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/960 From kvn at openjdk.java.net Sun Nov 1 21:02:53 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Sun, 1 Nov 2020 21:02:53 GMT Subject: RFR: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: On Sun, 1 Nov 2020 20:15:01 GMT, Igor Veresov wrote: >> We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. >> >> We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. >> >> We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. >> >> Tested changes in all tiers. >> >> I verified that with these changes I still able to build Graal in open repo and run graalunit testing: >> >> `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` >> `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` >> `open$ make jdk-image` >> `open$ make test-image` >> `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` > > Marked as reviewed by iveresov (Reviewer). Thank you for reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/960 From ngasson at openjdk.java.net Mon Nov 2 04:11:55 2020 From: ngasson at openjdk.java.net (Nick Gasson) Date: Mon, 2 Nov 2020 04:11:55 GMT Subject: RFR: 8254723: add diagnostic command to write Linux perf map file [v5] In-Reply-To: References: <7T_M6C-3WpLwXYH3RuRCuDQUW0qMyKIWAs8RaPW7D0s=.d659e5a0-e8a2-4816-8f60-1dd7653f4c7b@github.com> Message-ID: On Fri, 30 Oct 2020 04:34:19 GMT, Yasumasa Suenaga wrote: > > Sure, the change looks good to me. However I don't understand why CSR is not needed. It introduces new dcmd for Linux. I think because interfaces that are for diagnostic purposes don't require a CSR. See question 4 on the CSR FAQs: https://wiki.openjdk.java.net/display/csr/CSR+FAQs ------------- PR: https://git.openjdk.java.net/jdk/pull/760 From ysuenaga at openjdk.java.net Mon Nov 2 11:03:59 2020 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 2 Nov 2020 11:03:59 GMT Subject: RFR: 8254723: add diagnostic command to write Linux perf map file [v5] In-Reply-To: <1IQqVGMwEJYYpHWOQXdtJvIp3hs6mjEJkaMfTb3lwWo=.5cb02e47-d773-41ba-b206-b2657417124c@github.com> References: <7T_M6C-3WpLwXYH3RuRCuDQUW0qMyKIWAs8RaPW7D0s=.d659e5a0-e8a2-4816-8f60-1dd7653f4c7b@github.com> <1IQqVGMwEJYYpHWOQXdtJvIp3hs6mjEJkaMfTb3lwWo=.5cb02e47-d773-41ba-b206-b2657417124c@github.com> Message-ID: On Tue, 27 Oct 2020 04:21:33 GMT, Nick Gasson wrote: >> When using the Linux "perf" tool to do system profiling, symbol names of >> running Java methods cannot be decoded, resulting in unhelpful output >> such as: >> >> 10.52% [JIT] tid 236748 [.] 0x00007f6fdb75d223 >> >> Perf can read a simple text file format describing the mapping between >> address ranges and symbol names for a particular process [1]. >> >> It's possible to generate this already for Java processes using a JVMTI >> plugin such as perf-map-agent [2]. However this requires compiling >> third-party code and then loading the agent into your Java process. It >> would be more convenient if Hotspot could write this file directly using >> a diagnostic command. The information required is almost identical to >> that of the existing Compiler.codelist command. >> >> This patch adds a Compiler.perfmap diagnostic command on Linux only. To >> use, first run "jcmd Compiler.perfmap" and then "perf top" or >> "perf record" and the report should show decoded Java symbol names for >> that process. >> >> As this just writes a snapshot of the code cache when the command is >> run, it will become stale if methods are compiled later or unloaded. >> However this shouldn't be a big problem in practice if the map file is >> generated after the application has warmed up. >> >> [1] https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/jit-interface.txt >> [2] https://github.com/jvm-profiling-tools/perf-map-agent > > Nick Gasson has updated the pull request incrementally with one additional commit since the last revision: > > Make DumpPerfMapAtExit a diagnostic option Ok, I reviewed this change. I guess you should close [JDK-8254723](https://bugs.openjdk.java.net/browse/JDK-8254723). ------------- Marked as reviewed by ysuenaga (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/760 From eosterlund at openjdk.java.net Mon Nov 2 11:14:10 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 11:14:10 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: > The imasm::remove_activation() call does not deal with safepoints very well. However, when the MethodExit JVMTI event is being called, we call into the runtime in the middle of remove_activation(). If the value being returned is an object type, then the top-of-stack contains the oop. However, the GC does not traverse said oop in any oop map, because it is simply not expected that we safepoint in the middle of remove_activation(). > > The JvmtiExport::post_method_exit() function we end up calling, reads the top-of-stack oop, and puts it in a handle. Then it calls JVMTI callbacks, that eventually call Java and a bunch of stuff that safepoints. So after the JVMTI callback, we can expect the top-of-stack oop to be broken. Unfortunately, when we continue, we therefore end up returning a broken oop. > > Notably, the fact that InterpreterRuntime::post_method_exit is a JRT_ENTRY, is wrong, as we can safepoint on the way back to Java, which will break the return oop in a similar way. So this patch makes it a JRT_BLOCK_ENTRY, moving the transition to VM and back, into a block of code that is protected against GC. Before the JRT_BLOCK is called, we stash away the return oop, and after the JRT_BLOCK_END, we restore the top-of-stack oop. In the path when InterpreterRuntime::post_method_exit is called when throwing an exception, we don't have the same problem of retaining an oop result, and hence the JRT_BLOCK/JRT_BLOCK_END section is not performed in this case; the logic is the same as before for this path. > > This is a JVMTI bug that has probably been around for a long time. It crashes with all GCs, but was discovered recently after concurrent stack processing, as StefanK has been running better GC stressing code in JVMTI, and the bug reproduced more easily with concurrent stack processing, as the timings were a bit different. The following reproducer failed pretty much 100% of the time: > while true; do make test JTREG="RETAIN=all" TEST=test/hotspot/jtreg/vmTestbase/nsk/jdi/MethodExitEvent/returnValue/returnValue003/returnValue003.java TEST_OPTS_JAVA_OPTIONS="-XX:+UseZGC -Xmx2g -XX:ZCollectionInterval=0.0001 -XX:ZFragmentationLimit=0.01 -XX:+VerifyOops -XX:+ZVerifyViews -Xint" ; done > > With my fix I can run this repeatedly without any more failures. I have also sanity checked the patch by running tier 1-5, so that it does not introduces any new issues on its own. I have also used Stefan's nice external GC stressing with jcmd technique that was used to trigger crashes with other GCs, to make sure said crashes no longer reproduce either. Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: Coleen CR1: Refactoring ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/930/files - new: https://git.openjdk.java.net/jdk/pull/930/files/d7500082..ae6355fd Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=930&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=930&range=00-01 Stats: 41 lines in 2 files changed: 13 ins; 19 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/930.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/930/head:pull/930 PR: https://git.openjdk.java.net/jdk/pull/930 From eosterlund at openjdk.java.net Mon Nov 2 11:19:59 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 11:19:59 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 14:09:46 GMT, Erik ?sterlund wrote: >> Oh that's actually horrible. I wonder if it's possible to hoist saving the result oop into the InterpreterRuntime entry. And pass the Handle into JvmtiExport::post_method_exit(). > > I tried that first, and ended up with a bunch of non-trivial code duplication instead, as reading the oop is done in both paths but for different reasons. One to preserve/restore it (interpreter remove_activation entry), but also inside of JvmtiExport::post_method_exit() so that it can be passed into the MethodExit. I will give it another shot and see if it is possible to refactor it in a better way. I uploaded a CR that does pretty much what you suggested, ish. Hope you like it! ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From eosterlund at openjdk.java.net Mon Nov 2 11:19:59 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 11:19:59 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: <5-JnsHKeZztip64W88tRwA9_pcuWze2jftUq0C6oYSM=.79916a5b-5dd1-402f-b1ba-0caa38a39c1b@github.com> References: <5-JnsHKeZztip64W88tRwA9_pcuWze2jftUq0C6oYSM=.79916a5b-5dd1-402f-b1ba-0caa38a39c1b@github.com> Message-ID: On Sat, 31 Oct 2020 09:54:09 GMT, Serguei Spitsyn wrote: >> Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: >> >> Coleen CR1: Refactoring > > Hi Erik, > > Nice discovery! Indeed, this is a long standing issue. It looks good in general. > I agree with Coleen, it would be nice if there is an elegant way to move the oop_result saving/restoring code to InterpreterRuntime::post_method_exit. Otherwise, I'm okay with what you have now. > It is also nice discovery of the issue with clearing the expression stack. I think, it was my mistake in the initial implementation of the ForceEarlyReturn when I followed the PopFrame implementation pattern. It is good to separate it from the current fix. > > Thanks, > Serguei I uploaded a new commit to perform some refactoring as requested by Coleen and Serguei. I made the oop save/restore + JRT_BLOCK logic belong only to the path taken from InterpreterRuntime::post_method_exit. An inner posting method is called both from that path and from JvmtiExport::notice_unwind_due_to_exception. I think the result is an improvement in terms of how clear it is. I didn't want to move logic all the way back to InterpreterRuntime::post_method_exit though, as I don't think it looks pretty to have large chunks of JVMTI implementation details in the interpreterRuntime.cpp file. So I did basically what you suggested, with the slight difference of moving all the JVMTI implementation into the JVMTI file instead, which is just called from InterpreterRuntime::post_method_exit. Hope you are okay with this! ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From eosterlund at openjdk.java.net Mon Nov 2 11:25:56 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 11:25:56 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: <5-JnsHKeZztip64W88tRwA9_pcuWze2jftUq0C6oYSM=.79916a5b-5dd1-402f-b1ba-0caa38a39c1b@github.com> References: <5-JnsHKeZztip64W88tRwA9_pcuWze2jftUq0C6oYSM=.79916a5b-5dd1-402f-b1ba-0caa38a39c1b@github.com> Message-ID: On Sat, 31 Oct 2020 09:54:09 GMT, Serguei Spitsyn wrote: > Hi Erik, > > Nice discovery! Indeed, this is a long standing issue. It looks good in general. > I agree with Coleen, it would be nice if there is an elegant way to move the oop_result saving/restoring code to InterpreterRuntime::post_method_exit. Otherwise, I'm okay with what you have now. > It is also nice discovery of the issue with clearing the expression stack. I think, it was my mistake in the initial implementation of the ForceEarlyReturn when I followed the PopFrame implementation pattern. It is good to separate it from the current fix. > > Thanks, > Serguei Thanks for reviewing this Serguei. And thanks for confirming our suspicions regarding clearing of the expression stack. I wasn't sure if anyone would be around that knew how it ended up there! I made the refactoring that you and Coleen wanted, I think. Hope you like it! ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From stefank at openjdk.java.net Mon Nov 2 11:40:04 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Mon, 2 Nov 2020 11:40:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 08:25:28 GMT, Stefan Karlsson wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > src/hotspot/share/prims/jvmtiTagMap.cpp line 126: > >> 124: // concurrent GCs. So fix it here once we have a lock or are >> 125: // at a safepoint. >> 126: // SetTag and GetTag should not post events! > > I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case. Maybe even move this comment to the set_tag/get_tag code. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From stefank at openjdk.java.net Mon Nov 2 11:40:04 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Mon, 2 Nov 2020 11:40:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore wrote: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Commented on nits, and reviewed GC code and tag map code. Didn't look closely on the hashmap changes. src/hotspot/share/gc/shared/oopStorageSet.hpp line 41: > 39: // Must be updated when new OopStorages are introduced > 40: static const uint strong_count = 4 JVMTI_ONLY(+ 1); > 41: static const uint weak_count = 5 JVMTI_ONLY(+1) JFR_ONLY(+ 1); All other uses `+ 1` instead of `+1`. src/hotspot/share/gc/shared/weakProcessorPhaseTimes.hpp line 49: > 47: double _phase_times_sec[1]; > 48: size_t _phase_dead_items[1]; > 49: size_t _phase_total_items[1]; This should be removed and the associated reset_items src/hotspot/share/gc/z/zOopClosures.hpp line 64: > 62: }; > 63: > 64: class ZPhantomKeepAliveOopClosure : public ZRootsIteratorClosure { Seems like you flipped the location of these two. Maybe revert? src/hotspot/share/prims/jvmtiExport.hpp line 405: > 403: > 404: // Delete me and all my callers! > 405: static void weak_oops_do(BoolObjectClosure* b, OopClosure* f) {} Maybe delete? src/hotspot/share/prims/jvmtiTagMap.cpp line 126: > 124: // concurrent GCs. So fix it here once we have a lock or are > 125: // at a safepoint. > 126: // SetTag and GetTag should not post events! I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case. src/hotspot/share/prims/jvmtiTagMap.cpp line 131: > 129: // Operating on the hashmap must always be locked, since concurrent GC threads may > 130: // notify while running through a safepoint. > 131: assert(is_locked(), "checking"); Maybe move this to the top of the function to make it very clear. src/hotspot/share/prims/jvmtiTagMap.cpp line 133: > 131: assert(is_locked(), "checking"); > 132: if (post_events && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { > 133: log_info(jvmti, table)("TagMap table needs posting before heap walk"); Not sure about the "before heap walk" since this is also done from GetObjectsWithTags, which does *not* do a heap walk but still requires posting. src/hotspot/share/prims/jvmtiTagMap.cpp line 140: > 138: hashmap()->rehash(); > 139: _needs_rehashing = false; > 140: } It's not clear to me that it's correct to rehash *after* posting. I think it is, because unlink_and_post will use load barriers to fixup old pointers. src/hotspot/share/prims/jvmtiTagMap.cpp line 146: > 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed. > 145: void JvmtiTagMap::check_hashmaps_for_heapwalk() { > 146: Extra white space. (Also double whitespace after this function) src/hotspot/share/prims/jvmtiTagMap.cpp line 144: > 142: > 143: // This checks for posting and rehashing and is called from the heap walks. > 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed. Should this comment be moved down to the lock taking? src/hotspot/share/prims/jvmtiTagMap.cpp line 377: > 375: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag); > 376: > 377: // Check if we have to processing for concurrent GCs. Sentence seems to be missing a few words. src/hotspot/share/prims/jvmtiTagMap.cpp line 954: > 952: o->klass()->external_name()); > 953: return; > 954: } Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch? src/hotspot/share/prims/jvmtiTagMap.cpp line 1152: > 1150: void JvmtiTagMap::unlink_and_post_locked() { > 1151: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag); > 1152: log_info(jvmti, table)("TagMap table needs posting before GetObjectTags"); There's no function called GetObjectTags. This log line needs to be adjusted. src/hotspot/share/prims/jvmtiTagMap.cpp line 1162: > 1160: VMOp_Type type() const { return VMOp_Cleanup; } > 1161: void doit() { > 1162: _tag_map->unlink_and_post_locked(); Either inline unlink_and_post_locked() or updated gc_notification to use it? src/hotspot/share/prims/jvmtiTagMap.cpp line 1279: > 1277: // Can't post ObjectFree events here from a JavaThread, so this > 1278: // will race with the gc_notification thread in the tiny > 1279: // window where the oop is not marked but hasn't been notified that Please don't use "oop" when referring to "objects". src/hotspot/share/prims/jvmtiTagMap.cpp line 2975: > 2973: } > 2974: > 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved oops => objects src/hotspot/share/prims/jvmtiTagMap.cpp line 2977: > 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved > 2976: // and have their new addresses, the table can be rehashed. > 2977: void JvmtiTagMap::set_needs_processing() { Maybe rename to set_needs_rehashing? src/hotspot/share/prims/jvmtiTagMap.cpp line 2985: > 2983: > 2984: JvmtiEnvIterator it; > 2985: for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { The iterator seems fast enough, so it seems unnecessary to have the environments_might_exist check. src/hotspot/share/prims/jvmtiTagMap.cpp line 2998: > 2996: // thread creation and before VMThread creation (1 thread); initial GC > 2997: // verification can happen in that window which gets to here. > 2998: if (!JvmtiEnv::environments_might_exist()) { return; } I don't know what this comment is saying, and why the code is needed. src/hotspot/share/prims/jvmtiTagMap.cpp line 3020: > 3018: JvmtiTagMap* tag_map = env->tag_map_acquire(); > 3019: if (tag_map != NULL && !tag_map->is_empty()) { > 3020: if (num_dead_entries > 0) { The other num_dead_entries check for != 0. Maybe use the same in the two branches? src/hotspot/share/prims/jvmtiTagMap.cpp line 3023: > 3021: tag_map->hashmap()->unlink_and_post(tag_map->env()); > 3022: } > 3023: tag_map->_needs_rehashing = true; Maybe add a small comment why this is deferred. src/hotspot/share/prims/jvmtiTagMap.hpp line 56: > 54: void entry_iterate(JvmtiTagMapEntryClosure* closure); > 55: void post_dead_object_on_vm_thread(); > 56: public: Looked nicer when there was a blank line before public. Now it looks like public "relates" more to the code before than after. src/hotspot/share/prims/jvmtiTagMap.hpp line 114: > 112: static void check_hashmaps_for_heapwalk(); > 113: static void set_needs_processing() NOT_JVMTI_RETURN; > 114: static void gc_notification(size_t num_dead_entries) NOT_JVMTI_RETURN; Have you verified that this builds without JVMTI? src/hotspot/share/prims/jvmtiTagMapTable.cpp line 50: > 48: // A subsequent oop_load without AS_NO_KEEPALIVE (the object() accessor) > 49: // keeps the oop alive before doing so. > 50: return literal().peek(); I'm not sure we should be talking about the low-level Access names. Maybe reword in terms of WeakHandle operations? src/hotspot/share/prims/jvmtiTagMapTable.cpp line 81: > 79: void JvmtiTagMapTable::free_entry(JvmtiTagMapEntry* entry) { > 80: unlink_entry(entry); > 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage release *to* OopStorage? src/hotspot/share/prims/jvmtiTagMapTable.cpp line 82: > 80: unlink_entry(entry); > 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage > 82: FREE_C_HEAP_ARRAY(char, entry); // C_Heap free. Seems excessively redundant: // C_Heap free. src/hotspot/share/prims/jvmtiTagMapTable.cpp line 98: > 96: > 97: // The obj is in the table as a target already > 98: if (target != NULL && target == obj) { Wonder if we could assert that obj is not NULL at the entry of this function, and then change this to simply target == obj? src/hotspot/share/prims/jvmtiTagMapTable.cpp line 122: > 120: int index = hash_to_index(hash); > 121: // One was added while acquiring the lock > 122: JvmtiTagMapEntry* entry = find(index, hash, obj); Should this be done inside ASSERT? test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 64: > 62: static jclass klass = NULL; > 63: static jobject testedObject = NULL; > 64: const jlong TESTED_TAG_VALUE = (5555555L); Remove parenthesis? ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From ihse at openjdk.java.net Mon Nov 2 12:30:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 12:30:56 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: <4yShPUjSyZSPGnbVh-jr58lGT9psXrg88HZIk5Wa-40=.c2f77dae-d582-4fb4-9f6e-bdcb5ced63b0@github.com> On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore wrote: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Build changes look good. Not reviewed hotspot changes. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From ihse at openjdk.java.net Mon Nov 2 12:31:59 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 12:31:59 GMT Subject: RFR: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 17:40:51 GMT, Vladimir Kozlov wrote: > We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. > > We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. > > We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. > > Tested changes in all tiers. > > I verified that with these changes I still able to build Graal in open repo and run graalunit testing: > > `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` > `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` > `open$ make jdk-image` > `open$ make test-image` > `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` Build changes look good. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/960 From coleenp at openjdk.java.net Mon Nov 2 13:22:15 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 13:22:15 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 08:08:53 GMT, Stefan Karlsson wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > src/hotspot/share/gc/shared/oopStorageSet.hpp line 41: > >> 39: // Must be updated when new OopStorages are introduced >> 40: static const uint strong_count = 4 JVMTI_ONLY(+ 1); >> 41: static const uint weak_count = 5 JVMTI_ONLY(+1) JFR_ONLY(+ 1); > > All other uses `+ 1` instead of `+1`. Fixed, although I think the space looks strange there but I'll go along. > src/hotspot/share/gc/shared/weakProcessorPhaseTimes.hpp line 49: > >> 47: double _phase_times_sec[1]; >> 48: size_t _phase_dead_items[1]; >> 49: size_t _phase_total_items[1]; > > This should be removed and the associated reset_items Removed. > src/hotspot/share/gc/z/zOopClosures.hpp line 64: > >> 62: }; >> 63: >> 64: class ZPhantomKeepAliveOopClosure : public ZRootsIteratorClosure { > > Seems like you flipped the location of these two. Maybe revert? Reverted. There was a rebasing conflict here so this was unintentional. > src/hotspot/share/prims/jvmtiExport.hpp line 405: > >> 403: >> 404: // Delete me and all my callers! >> 405: static void weak_oops_do(BoolObjectClosure* b, OopClosure* f) {} > > Maybe delete? Yes, meant to do that. > src/hotspot/share/prims/jvmtiTagMap.cpp line 131: > >> 129: // Operating on the hashmap must always be locked, since concurrent GC threads may >> 130: // notify while running through a safepoint. >> 131: assert(is_locked(), "checking"); > > Maybe move this to the top of the function to make it very clear. ok. > src/hotspot/share/prims/jvmtiTagMap.cpp line 133: > >> 131: assert(is_locked(), "checking"); >> 132: if (post_events && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { >> 133: log_info(jvmti, table)("TagMap table needs posting before heap walk"); > > Not sure about the "before heap walk" since this is also done from GetObjectsWithTags, which does *not* do a heap walk but still requires posting. I don't call check_hashmap for GetObjectsWithTags. > src/hotspot/share/prims/jvmtiTagMap.cpp line 140: > >> 138: hashmap()->rehash(); >> 139: _needs_rehashing = false; >> 140: } > > It's not clear to me that it's correct to rehash *after* posting. I think it is, because unlink_and_post will use load barriers to fixup old pointers. I think it's better that the rehashing doesn't encounter null entries and the WeakHandle.peek() operation is used for both so I hope it would get the same answer. If not, which seems bad, the last answer should be what we hash on. > src/hotspot/share/prims/jvmtiTagMap.cpp line 144: > >> 142: >> 143: // This checks for posting and rehashing and is called from the heap walks. >> 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed. > > Should this comment be moved down to the lock taking? ok, I also made it singular since Erik pointed out that we don't need the other lock. > src/hotspot/share/prims/jvmtiTagMap.cpp line 146: > >> 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed. >> 145: void JvmtiTagMap::check_hashmaps_for_heapwalk() { >> 146: > > Extra white space. (Also double whitespace after this function) ? I removed the double whitespace after the function and put the whitespace here. It needs some whitespace. // This checks for posting and rehashing and is called from the heap walks. void JvmtiTagMap::check_hashmaps_for_heapwalk() { assert(SafepointSynchronize::is_at_safepoint(), "called from safepoints"); // Verify that the tag map tables are valid and unconditionally post events // that are expected to be posted before gc_notification. JvmtiEnvIterator it; > src/hotspot/share/prims/jvmtiTagMap.cpp line 377: > >> 375: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag); >> 376: >> 377: // Check if we have to processing for concurrent GCs. > > Sentence seems to be missing a few words. removed the sentence, because non concurrent GCs also defer rehashing to next use. > src/hotspot/share/prims/jvmtiTagMap.cpp line 954: > >> 952: o->klass()->external_name()); >> 953: return; >> 954: } > > Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch? Because it crashed with my changes and didn't without. I cannot recollect why. > src/hotspot/share/prims/jvmtiTagMap.cpp line 1152: > >> 1150: void JvmtiTagMap::unlink_and_post_locked() { >> 1151: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag); >> 1152: log_info(jvmti, table)("TagMap table needs posting before GetObjectTags"); > > There's no function called GetObjectTags. This log line needs to be adjusted. GetObjectsWithTags fixed. > src/hotspot/share/prims/jvmtiTagMap.cpp line 1162: > >> 1160: VMOp_Type type() const { return VMOp_Cleanup; } >> 1161: void doit() { >> 1162: _tag_map->unlink_and_post_locked(); > > Either inline unlink_and_post_locked() or updated gc_notification to use it? I thought of trying to share it but one logs and the other doesn't and it only saves 1 lines of code. > src/hotspot/share/prims/jvmtiTagMap.cpp line 1279: > >> 1277: // Can't post ObjectFree events here from a JavaThread, so this >> 1278: // will race with the gc_notification thread in the tiny >> 1279: // window where the oop is not marked but hasn't been notified that > > Please don't use "oop" when referring to "objects". fixed. > src/hotspot/share/prims/jvmtiTagMap.cpp line 2975: > >> 2973: } >> 2974: >> 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved > > oops => objects fixed. > src/hotspot/share/prims/jvmtiTagMap.cpp line 2977: > >> 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved >> 2976: // and have their new addresses, the table can be rehashed. >> 2977: void JvmtiTagMap::set_needs_processing() { > > Maybe rename to set_needs_rehashing? Since I went back and forth about what this function did (it posted events at one time), I thought the generic _processing name would be better. GC callers shouldn't really have to know what processing we're doing here. Hopefully it won't change from rehashing. That's why I like processing. > src/hotspot/share/prims/jvmtiTagMap.cpp line 2985: > >> 2983: >> 2984: JvmtiEnvIterator it; >> 2985: for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > > The iterator seems fast enough, so it seems unnecessary to have the environments_might_exist check. yes, it looks like it does the same thing. I'll remove it. > src/hotspot/share/prims/jvmtiTagMap.cpp line 2998: > >> 2996: // thread creation and before VMThread creation (1 thread); initial GC >> 2997: // verification can happen in that window which gets to here. >> 2998: if (!JvmtiEnv::environments_might_exist()) { return; } > > I don't know what this comment is saying, and why the code is needed. I've spent tons of time trying to understand this comment too. I think gc verification used to call oops do on the tagmap table. This comments obsolete now, and I'll remove it. > src/hotspot/share/prims/jvmtiTagMap.cpp line 3020: > >> 3018: JvmtiTagMap* tag_map = env->tag_map_acquire(); >> 3019: if (tag_map != NULL && !tag_map->is_empty()) { >> 3020: if (num_dead_entries > 0) { > > The other num_dead_entries check for != 0. Maybe use the same in the two branches? ok. > src/hotspot/share/prims/jvmtiTagMap.cpp line 3023: > >> 3021: tag_map->hashmap()->unlink_and_post(tag_map->env()); >> 3022: } >> 3023: tag_map->_needs_rehashing = true; > > Maybe add a small comment why this is deferred. // Later GC code will relocate the oops, so defer rehashing until then. ? > src/hotspot/share/prims/jvmtiTagMap.hpp line 56: > >> 54: void entry_iterate(JvmtiTagMapEntryClosure* closure); >> 55: void post_dead_object_on_vm_thread(); >> 56: public: > > Looked nicer when there was a blank line before public. Now it looks like public "relates" more to the code before than after. ok > src/hotspot/share/prims/jvmtiTagMap.hpp line 114: > >> 112: static void check_hashmaps_for_heapwalk(); >> 113: static void set_needs_processing() NOT_JVMTI_RETURN; >> 114: static void gc_notification(size_t num_dead_entries) NOT_JVMTI_RETURN; > > Have you verified that this builds without JVMTI? I will do (might have already done) that. Building non-oracle platforms builds minimal vm. > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 50: > >> 48: // A subsequent oop_load without AS_NO_KEEPALIVE (the object() accessor) >> 49: // keeps the oop alive before doing so. >> 50: return literal().peek(); > > I'm not sure we should be talking about the low-level Access names. Maybe reword in terms of WeakHandle operations? I'm going to say: // Just peek at the object without keeping it alive. > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 81: > >> 79: void JvmtiTagMapTable::free_entry(JvmtiTagMapEntry* entry) { >> 80: unlink_entry(entry); >> 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage > > release *to* OopStorage? fixed > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 98: > >> 96: >> 97: // The obj is in the table as a target already >> 98: if (target != NULL && target == obj) { > > Wonder if we could assert that obj is not NULL at the entry of this function, and then change this to simply target == obj? makes sense. > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 122: > >> 120: int index = hash_to_index(hash); >> 121: // One was added while acquiring the lock >> 122: JvmtiTagMapEntry* entry = find(index, hash, obj); > > Should this be done inside ASSERT? yes. > test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 64: > >> 62: static jclass klass = NULL; >> 63: static jobject testedObject = NULL; >> 64: const jlong TESTED_TAG_VALUE = (5555555L); > > Remove parenthesis? I copied this from some other place that had parenthesis. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 13:22:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 13:22:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore wrote: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. I think I addressed your comments, retesting now. Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 13:22:15 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 13:22:15 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 08:34:17 GMT, Stefan Karlsson wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 126: >> >>> 124: // concurrent GCs. So fix it here once we have a lock or are >>> 125: // at a safepoint. >>> 126: // SetTag and GetTag should not post events! >> >> I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case. > > Maybe even move this comment to the set_tag/get_tag code. I was trying to explain why there's a boolean there but I can put this comment at both get_tag and set_tag. // Check if we have to processing for concurrent GCs. // GetTag should not post events because the JavaThread has to // transition to native for the callback and this cannot stop for // safepoints with the hashmap lock held. check_hashmap(false); ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 13:28:59 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 13:28:59 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:46:31 GMT, Erik Joelsson wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Build changes look ok. There should be a thank you emoji that doesn't send email except maybe to the person reviewing the code. Thank you @erikj79 and @magicus for reviewing the build changes. There should also be a 'fixed' emoji. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 15:58:15 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 15:58:15 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Code review comments from StefanK. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/534326da..cb4c83e0 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=00-01 Stats: 75 lines in 9 files changed: 12 ins; 48 del; 15 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From zgu at openjdk.java.net Mon Nov 2 16:07:02 2020 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Mon, 2 Nov 2020 16:07:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> Message-ID: On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from StefanK. Shenandoah part looks good. ------------- Marked as reviewed by zgu (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From kvn at openjdk.java.net Mon Nov 2 16:09:55 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 2 Nov 2020 16:09:55 GMT Subject: Integrated: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: <-4ETgOIXCz-lGU-CQQTpYtgUZ5gs6TYaeAZtBRqXmmg=.16b603f0-0af9-425f-aa62-b3ae29e41e56@github.com> On Fri, 30 Oct 2020 17:40:51 GMT, Vladimir Kozlov wrote: > We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. > > We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. > > We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. > > Tested changes in all tiers. > > I verified that with these changes I still able to build Graal in open repo and run graalunit testing: > > `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` > `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` > `open$ make jdk-image` > `open$ make test-image` > `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` This pull request has now been integrated. Changeset: 2f7d34f2 Author: Vladimir Kozlov URL: https://git.openjdk.java.net/jdk/commit/2f7d34f2 Stats: 36 lines in 4 files changed: 21 ins; 11 del; 4 mod 8255616: Disable AOT and Graal in Oracle OpenJDK Reviewed-by: iignatyev, vlivanov, iveresov, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/960 From eosterlund at openjdk.java.net Mon Nov 2 16:14:06 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 16:14:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> Message-ID: <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from StefanK. Looks great in general. Great work Coleen, and thanks again for fixing this. I like all the red lines in the GC code. I added a few nits/questions. test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 656: > 654: result = NSK_FALSE; > 655: > 656: printf("Object free events %d\n", ObjectFreeEventsCount); Is this old debug info you forgot to remove? Other code seems to use NSK_DISPLAY macros instead. src/hotspot/share/prims/jvmtiTagMap.cpp line 345: > 343: > 344: // Check if we have to process for concurrent GCs. > 345: check_hashmap(false); Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap. src/hotspot/share/prims/jvmtiTagMap.cpp line 3009: > 3007: // Lock each hashmap from concurrent posting and cleaning > 3008: MutexLocker ml(tag_map->lock(), Mutex::_no_safepoint_check_flag); > 3009: tag_map->hashmap()->unlink_and_post(tag_map->env()); This could call unlink_and_post_locked instead of manually locking. ------------- Changes requested by eosterlund (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Mon Nov 2 16:14:06 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 16:14:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 12:50:23 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 954: >> >>> 952: o->klass()->external_name()); >>> 953: return; >>> 954: } >> >> Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch? > > Because it crashed with my changes and didn't without. I cannot recollect why. I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 16:23:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 16:23:03 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 11:14:10 GMT, Erik ?sterlund wrote: >> The imasm::remove_activation() call does not deal with safepoints very well. However, when the MethodExit JVMTI event is being called, we call into the runtime in the middle of remove_activation(). If the value being returned is an object type, then the top-of-stack contains the oop. However, the GC does not traverse said oop in any oop map, because it is simply not expected that we safepoint in the middle of remove_activation(). >> >> The JvmtiExport::post_method_exit() function we end up calling, reads the top-of-stack oop, and puts it in a handle. Then it calls JVMTI callbacks, that eventually call Java and a bunch of stuff that safepoints. So after the JVMTI callback, we can expect the top-of-stack oop to be broken. Unfortunately, when we continue, we therefore end up returning a broken oop. >> >> Notably, the fact that InterpreterRuntime::post_method_exit is a JRT_ENTRY, is wrong, as we can safepoint on the way back to Java, which will break the return oop in a similar way. So this patch makes it a JRT_BLOCK_ENTRY, moving the transition to VM and back, into a block of code that is protected against GC. Before the JRT_BLOCK is called, we stash away the return oop, and after the JRT_BLOCK_END, we restore the top-of-stack oop. In the path when InterpreterRuntime::post_method_exit is called when throwing an exception, we don't have the same problem of retaining an oop result, and hence the JRT_BLOCK/JRT_BLOCK_END section is not performed in this case; the logic is the same as before for this path. >> >> This is a JVMTI bug that has probably been around for a long time. It crashes with all GCs, but was discovered recently after concurrent stack processing, as StefanK has been running better GC stressing code in JVMTI, and the bug reproduced more easily with concurrent stack processing, as the timings were a bit different. The following reproducer failed pretty much 100% of the time: >> while true; do make test JTREG="RETAIN=all" TEST=test/hotspot/jtreg/vmTestbase/nsk/jdi/MethodExitEvent/returnValue/returnValue003/returnValue003.java TEST_OPTS_JAVA_OPTIONS="-XX:+UseZGC -Xmx2g -XX:ZCollectionInterval=0.0001 -XX:ZFragmentationLimit=0.01 -XX:+VerifyOops -XX:+ZVerifyViews -Xint" ; done >> >> With my fix I can run this repeatedly without any more failures. I have also sanity checked the patch by running tier 1-5, so that it does not introduces any new issues on its own. I have also used Stefan's nice external GC stressing with jcmd technique that was used to trigger crashes with other GCs, to make sure said crashes no longer reproduce either. > > Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: > > Coleen CR1: Refactoring This looks better. Just to have the JRT_BLOCK be unconditional is an improvement. src/hotspot/share/prims/jvmtiExport.cpp line 1570: > 1568: // return a flag when a method terminates by throwing an exception > 1569: // i.e. if an exception is thrown and it's not caught by the current method > 1570: bool exception_exit = state->is_exception_detected() && !state->is_exception_caught(); So this only applies to the case where the post_method_exit comes from remove_activation? Good to have it only on this path in this case. ------------- Marked as reviewed by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/930 From coleenp at openjdk.java.net Mon Nov 2 16:27:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 16:27:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> Message-ID: On Mon, 2 Nov 2020 15:18:43 GMT, Erik ?sterlund wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Code review comments from StefanK. > > test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 656: > >> 654: result = NSK_FALSE; >> 655: >> 656: printf("Object free events %d\n", ObjectFreeEventsCount); > > Is this old debug info you forgot to remove? Other code seems to use NSK_DISPLAY macros instead. yes, removed it. > src/hotspot/share/prims/jvmtiTagMap.cpp line 345: > >> 343: >> 344: // Check if we have to process for concurrent GCs. >> 345: check_hashmap(false); > > Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap. Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before? > src/hotspot/share/prims/jvmtiTagMap.cpp line 3009: > >> 3007: // Lock each hashmap from concurrent posting and cleaning >> 3008: MutexLocker ml(tag_map->lock(), Mutex::_no_safepoint_check_flag); >> 3009: tag_map->hashmap()->unlink_and_post(tag_map->env()); > > This could call unlink_and_post_locked instead of manually locking. Ok, 2 requests. I can call that then and move the logging. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 16:57:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 16:57:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 16:52:02 GMT, Coleen Phillimore wrote: >> I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference. > > Now I remember. I added an assert in JvmtiTagMapTable::find() for oop != NULL which didn't exist in the current hashmap code. The current hashmap code just didn't find a null oop. I tracked it down to the fact that we're finding dormant objects whose class hasn't been loaded yet. So I think we do load the archived heap from CDS. The heap walker capabilities can be added dynamically. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 16:57:13 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 16:57:13 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 15:45:18 GMT, Erik ?sterlund wrote: >> Because it crashed with my changes and didn't without. I cannot recollect why. > > I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference. Now I remember. I added an assert in JvmtiTagMapTable::find() for oop != NULL which didn't exist in the current hashmap code. The current hashmap code just didn't find a null oop. I tracked it down to the fact that we're finding dormant objects whose class hasn't been loaded yet. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Mon Nov 2 17:56:07 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 17:56:07 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 16:19:59 GMT, Coleen Phillimore wrote: >> Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: >> >> Coleen CR1: Refactoring > > src/hotspot/share/prims/jvmtiExport.cpp line 1570: > >> 1568: // return a flag when a method terminates by throwing an exception >> 1569: // i.e. if an exception is thrown and it's not caught by the current method >> 1570: bool exception_exit = state->is_exception_detected() && !state->is_exception_caught(); > > So this only applies to the case where the post_method_exit comes from remove_activation? Good to have it only on this path in this case. I'm not sure. There might be other cases, when remove_activation is called by the exception code. That's why I didn't want to change it to just true in this path. ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From cjplummer at openjdk.java.net Mon Nov 2 17:59:17 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 2 Nov 2020 17:59:17 GMT Subject: RFR: 8255694: memory leak in JDWP debug agent after calling JVMTI GetAllThreads [v2] In-Reply-To: References: Message-ID: <_WIfwyonLbTH7Qyx1_OhuOSj7Mh-VLfmofnY-UFzgx8=.67214137-4320-4c37-b8ee-123449438722@github.com> > The debug agent needs to deallocate memory that is allocated by calls to GetAllThreads. There were two places were this was not being done, resulting in a memory leak. Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: Remove unecessary semicolon ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/968/files - new: https://git.openjdk.java.net/jdk/pull/968/files/06063b7d..4e4dcae6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=968&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=968&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/968.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/968/head:pull/968 PR: https://git.openjdk.java.net/jdk/pull/968 From cjplummer at openjdk.java.net Mon Nov 2 18:00:02 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 2 Nov 2020 18:00:02 GMT Subject: RFR: 8255695: Some JVMTI calls in the jdwp debug agent are using FUNC_PTR instead of JVMTI_FUNC_PTR In-Reply-To: References: <9EOh170kLoAkCerX-ljRzRtlPDWr37m87C_ss2bVXDQ=.bb5632dc-97d4-4470-817e-a52a74964ba6@github.com> Message-ID: On Sat, 31 Oct 2020 08:12:50 GMT, Serguei Spitsyn wrote: >> Use JVMTI_FUNC_PTR instead of FUNC_PTR for most JVMTI calls. See CR for details. > > Looks good. > Thanks, > Serguei Ping! I could use one more review. The changes are pretty trivial. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/971 From amenkov at openjdk.java.net Mon Nov 2 19:39:00 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Mon, 2 Nov 2020 19:39:00 GMT Subject: RFR: 8255695: Some JVMTI calls in the jdwp debug agent are using FUNC_PTR instead of JVMTI_FUNC_PTR In-Reply-To: <9EOh170kLoAkCerX-ljRzRtlPDWr37m87C_ss2bVXDQ=.bb5632dc-97d4-4470-817e-a52a74964ba6@github.com> References: <9EOh170kLoAkCerX-ljRzRtlPDWr37m87C_ss2bVXDQ=.bb5632dc-97d4-4470-817e-a52a74964ba6@github.com> Message-ID: <-Z3p87v-S6xu1UinD_gNrxboOMjCa5uOJyMEias5NhY=.05df8db1-eaef-4fa6-8ab2-bab0ba548446@github.com> On Fri, 30 Oct 2020 21:11:09 GMT, Chris Plummer wrote: > Use JVMTI_FUNC_PTR instead of FUNC_PTR for most JVMTI calls. See CR for details. Marked as reviewed by amenkov (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/971 From cjplummer at openjdk.java.net Mon Nov 2 20:18:55 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 2 Nov 2020 20:18:55 GMT Subject: Integrated: 8255694: memory leak in JDWP debug agent after calling JVMTI GetAllThreads In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:30:49 GMT, Chris Plummer wrote: > The debug agent needs to deallocate memory that is allocated by calls to GetAllThreads. There were two places were this was not being done, resulting in a memory leak. This pull request has now been integrated. Changeset: a250716a Author: Chris Plummer URL: https://git.openjdk.java.net/jdk/commit/a250716a Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod 8255694: memory leak in JDWP debug agent after calling JVMTI GetAllThreads Reviewed-by: amenkov, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/968 From cjplummer at openjdk.java.net Mon Nov 2 20:29:56 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 2 Nov 2020 20:29:56 GMT Subject: Integrated: 8255696: JDWP debug agent's canSuspendResumeThreadLists() should be removed In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:42:24 GMT, Chris Plummer wrote: > This RFR removes the debug agent's canSuspendResumeThreadLists() function and code that depends on it. Please see the CR for a description of why this is being done. This pull request has now been integrated. Changeset: ceba2f85 Author: Chris Plummer URL: https://git.openjdk.java.net/jdk/commit/ceba2f85 Stats: 33 lines in 3 files changed: 0 ins; 29 del; 4 mod 8255696: JDWP debug agent's canSuspendResumeThreadLists() should be removed Reviewed-by: amenkov, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/970 From cjplummer at openjdk.java.net Mon Nov 2 20:36:59 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 2 Nov 2020 20:36:59 GMT Subject: Integrated: 8255695: Some JVMTI calls in the jdwp debug agent are using FUNC_PTR instead of JVMTI_FUNC_PTR In-Reply-To: <9EOh170kLoAkCerX-ljRzRtlPDWr37m87C_ss2bVXDQ=.bb5632dc-97d4-4470-817e-a52a74964ba6@github.com> References: <9EOh170kLoAkCerX-ljRzRtlPDWr37m87C_ss2bVXDQ=.bb5632dc-97d4-4470-817e-a52a74964ba6@github.com> Message-ID: On Fri, 30 Oct 2020 21:11:09 GMT, Chris Plummer wrote: > Use JVMTI_FUNC_PTR instead of FUNC_PTR for most JVMTI calls. See CR for details. This pull request has now been integrated. Changeset: c7747416 Author: Chris Plummer URL: https://git.openjdk.java.net/jdk/commit/c7747416 Stats: 14 lines in 1 file changed: 0 ins; 0 del; 14 mod 8255695: Some JVMTI calls in the jdwp debug agent are using FUNC_PTR instead of JVMTI_FUNC_PTR Reviewed-by: sspitsyn, amenkov ------------- PR: https://git.openjdk.java.net/jdk/pull/971 From sspitsyn at openjdk.java.net Mon Nov 2 21:02:56 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Mon, 2 Nov 2020 21:02:56 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 16:20:09 GMT, Coleen Phillimore wrote: >> Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: >> >> Coleen CR1: Refactoring > > This looks better. Just to have the JRT_BLOCK be unconditional is an improvement. Erik, Thank you for the update! It looks more elegant. One concern is that after the move of this fragment to the post_method_exit_inner: 1614 if (state == NULL || !state->is_interp_only_mode()) { 1615 // for any thread that actually wants method exit, interp_only_mode is set 1616 return; 1617 } there is no guarantee that the current frame is interpreted below: 1580 if (!exception_exit) { 1581 oop oop_result; 1582 BasicType type = current_frame.interpreter_frame_result(&oop_result, &value); . . . 1597 if (result.not_null() && !mh->is_native()) { 1598 // We have to restore the oop on the stack for interpreter frames 1599 *(oop*)current_frame.interpreter_frame_tos_address() = result(); 1600 } Probably, extra checks for current_frame.is_interpreted_frame() in these fragments will be sufficient. ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From sspitsyn at openjdk.java.net Mon Nov 2 21:23:00 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Mon, 2 Nov 2020 21:23:00 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 17:52:58 GMT, Erik ?sterlund wrote: >> src/hotspot/share/prims/jvmtiExport.cpp line 1570: >> >>> 1568: // return a flag when a method terminates by throwing an exception >>> 1569: // i.e. if an exception is thrown and it's not caught by the current method >>> 1570: bool exception_exit = state->is_exception_detected() && !state->is_exception_caught(); >> >> So this only applies to the case where the post_method_exit comes from remove_activation? Good to have it only on this path in this case. > > I'm not sure. There might be other cases, when remove_activation is called by the exception code. That's why I didn't want to change it to just true in this path. The post_method_exit can come from Zero interpreter: src/hotspot/share/interpreter/zero/bytecodeInterpreter.cpp: CALL_VM_NOCHECK(InterpreterRuntime::post_method_exit(THREAD)); ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From cjplummer at openjdk.java.net Mon Nov 2 22:28:02 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 2 Nov 2020 22:28:02 GMT Subject: RFR: 8255706: The JDWP debug agent unecessarily checks for JVMTI_ERROR_INTERRUPT after calling RawMonitorEnter Message-ID: Remove code that retries if RawMonitorEnter is interrupted since that can't happen: https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#RawMonitorEnter ------------- Commit messages: - RawMonitorEnter will never return JVMTI_ERROR_INTERRUPT, so don't check for it. Changes: https://git.openjdk.java.net/jdk/pull/1022/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1022&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255706 Stats: 10 lines in 1 file changed: 0 ins; 7 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1022.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1022/head:pull/1022 PR: https://git.openjdk.java.net/jdk/pull/1022 From ngasson at openjdk.java.net Tue Nov 3 01:59:57 2020 From: ngasson at openjdk.java.net (Nick Gasson) Date: Tue, 3 Nov 2020 01:59:57 GMT Subject: Integrated: 8254723: add diagnostic command to write Linux perf map file In-Reply-To: <7T_M6C-3WpLwXYH3RuRCuDQUW0qMyKIWAs8RaPW7D0s=.d659e5a0-e8a2-4816-8f60-1dd7653f4c7b@github.com> References: <7T_M6C-3WpLwXYH3RuRCuDQUW0qMyKIWAs8RaPW7D0s=.d659e5a0-e8a2-4816-8f60-1dd7653f4c7b@github.com> Message-ID: On Tue, 20 Oct 2020 09:27:45 GMT, Nick Gasson wrote: > When using the Linux "perf" tool to do system profiling, symbol names of > running Java methods cannot be decoded, resulting in unhelpful output > such as: > > 10.52% [JIT] tid 236748 [.] 0x00007f6fdb75d223 > > Perf can read a simple text file format describing the mapping between > address ranges and symbol names for a particular process [1]. > > It's possible to generate this already for Java processes using a JVMTI > plugin such as perf-map-agent [2]. However this requires compiling > third-party code and then loading the agent into your Java process. It > would be more convenient if Hotspot could write this file directly using > a diagnostic command. The information required is almost identical to > that of the existing Compiler.codelist command. > > This patch adds a Compiler.perfmap diagnostic command on Linux only. To > use, first run "jcmd Compiler.perfmap" and then "perf top" or > "perf record" and the report should show decoded Java symbol names for > that process. > > As this just writes a snapshot of the code cache when the command is > run, it will become stale if methods are compiled later or unloaded. > However this shouldn't be a big problem in practice if the map file is > generated after the application has warmed up. > > [1] https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/jit-interface.txt > [2] https://github.com/jvm-profiling-tools/perf-map-agent This pull request has now been integrated. Changeset: 50357d13 Author: Nick Gasson URL: https://git.openjdk.java.net/jdk/commit/50357d13 Stats: 171 lines in 8 files changed: 169 ins; 1 del; 1 mod 8254723: add diagnostic command to write Linux perf map file Reviewed-by: ysuenaga, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/760 From alanb at openjdk.java.net Tue Nov 3 07:22:56 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 3 Nov 2020 07:22:56 GMT Subject: RFR: 8255706: The JDWP debug agent unecessarily checks for JVMTI_ERROR_INTERRUPT after calling RawMonitorEnter In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 22:11:48 GMT, Chris Plummer wrote: > Remove code that retries if RawMonitorEnter is interrupted since that can't happen: > > https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#RawMonitorEnter Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1022 From eosterlund at openjdk.java.net Tue Nov 3 10:25:00 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 3 Nov 2020 10:25:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> Message-ID: On Mon, 2 Nov 2020 16:22:57 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 345: >> >>> 343: >>> 344: // Check if we have to process for concurrent GCs. >>> 345: check_hashmap(false); >> >> Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap. > > Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before? ZGC people passionately place the comment after the argument value. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From stefank at openjdk.java.net Tue Nov 3 10:36:57 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Tue, 3 Nov 2020 10:36:57 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 12:58:49 GMT, Coleen Phillimore wrote: > GC callers shouldn't really have to know what processing we're doing here. I completely disagree with this. It's extremely important that the GC and Runtime code agrees on what this code does and where the GC *must* call it. Knowing the details allows us to skip calling this after mark end, but forces us to call it in relocate start, when objects should start to move. Though, I don't want to block this review because of this point, so if you still thinks that a non-descriptive name is better then we can argue that separately. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From stefank at openjdk.java.net Tue Nov 3 10:44:05 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Tue, 3 Nov 2020 10:44:05 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> Message-ID: <0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com> On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from StefanK. Some more nit-picking to make the code more consistent. src/hotspot/share/prims/jvmtiTagMapTable.cpp line 52: > 50: : Hashtable(_table_size, sizeof(JvmtiTagMapEntry)) {} > 51: > 52: Double whitespace src/hotspot/share/prims/jvmtiTagMapTable.cpp line 185: > 183: // Serially remove unused oops from the table, and notify jvmti. > 184: void JvmtiTagMapTable::unlink_and_post(JvmtiEnv* env) { > 185: Stray newline src/hotspot/share/prims/jvmtiTagMapTable.cpp line 224: > 222: // Rehash oops in the table > 223: void JvmtiTagMapTable::rehash() { > 224: Stray newline src/hotspot/share/prims/jvmtiTagMapTable.hpp line 75: > 73: > 74: void resize_if_needed(); > 75: public: Newline between src/hotspot/share/prims/jvmtiTagMapTable.hpp line 100: > 98: }; > 99: > 100: Double newline src/hotspot/share/prims/jvmtiTagMapTable.cpp line 258: > 256: int rehash_len = moved_entries.length(); > 257: // Now add back in the entries that were removed. > 258: for (int i = 0; i < moved_entries.length(); i++) { rehash_len is read, but not used in for loop condition. src/hotspot/share/prims/jvmtiTagMapTable.cpp line 165: > 163: } > 164: } > 165: const int _resize_load_trigger = 5; // load factor that will trigger the resize Newline between ------------- Marked as reviewed by stefank (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 12:22:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 12:22:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> Message-ID: On Tue, 3 Nov 2020 10:22:09 GMT, Erik ?sterlund wrote: >> Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before? > > ZGC people passionately place the comment after the argument value. I see that but in the non-zgc code it's the opposite and this is non-zgc code. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 12:36:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 12:36:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> <0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com> Message-ID: On Tue, 3 Nov 2020 10:36:21 GMT, Stefan Karlsson wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Code review comments from StefanK. > > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 185: > >> 183: // Serially remove unused oops from the table, and notify jvmti. >> 184: void JvmtiTagMapTable::unlink_and_post(JvmtiEnv* env) { >> 185: > > Stray newline that's a useful newline. > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 258: > >> 256: int rehash_len = moved_entries.length(); >> 257: // Now add back in the entries that were removed. >> 258: for (int i = 0; i < moved_entries.length(); i++) { > > rehash_len is read, but not used in for loop condition. It's in the logging message below. I'll use it in the loop too. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 12:40:06 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 12:40:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 10:33:16 GMT, Stefan Karlsson wrote: >> Since I went back and forth about what this function did (it posted events at one time), I thought the generic _processing name would be better. GC callers shouldn't really have to know what processing we're doing here. Hopefully it won't change from rehashing. That's why I like processing. > >> GC callers shouldn't really have to know what processing we're doing here. > > I completely disagree with this. It's extremely important that the GC and Runtime code agrees on what this code does and where the GC *must* call it. Knowing the details allows us to skip calling this after mark end, but forces us to call it in relocate start, when objects should start to move. Though, I don't want to block this review because of this point, so if you still thinks that a non-descriptive name is better then we can argue that separately. Ok, I'll rename it to needs_hashing. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From rrich at openjdk.java.net Tue Nov 3 12:41:54 2020 From: rrich at openjdk.java.net (Richard Reingruber) Date: Tue, 3 Nov 2020 12:41:54 GMT Subject: Integrated: 8255072: [TESTBUG] com/sun/jdi/EATests.java should not fail if expected VMOutOfMemoryException is not thrown In-Reply-To: References: Message-ID: On Tue, 20 Oct 2020 21:53:10 GMT, Richard Reingruber wrote: > The following test cases try to provoke VMOutOfMemoryException during object reallocation because of JVMTI PopFrame / ForceEarlyReturn: > > EAPopFrameNotInlinedReallocFailure > EAPopInlinedMethodWithScalarReplacedObjectsReallocFailure > EAForceEarlyReturnOfInlinedMethodWithScalarReplacedObjectsReallocFailure > > For ZGC (so far) this is not 100% reliable. > > Just ignoring the runs where the expected OOME was not raised was not accepted. > > Summary of the now accepted solution: > > - The 3 problematic test cases are skipped if ZGC is selected. > > - They are also skipped if no OOME during object reallocation can be expected because allocations are not eliminated. > > - In consumeAllMemory, as a last step, empty LinkedList nodes are created without long array to fill up small blocks of free memory. > > - EATests.java is removed from the problem list for ZGC. This pull request has now been integrated. Changeset: 63461d59 Author: Richard Reingruber URL: https://git.openjdk.java.net/jdk/commit/63461d59 Stats: 65 lines in 2 files changed: 37 ins; 8 del; 20 mod 8255072: [TESTBUG] com/sun/jdi/EATests.java should not fail if expected VMOutOfMemoryException is not thrown Reviewed-by: cjplummer, sspitsyn, kvn ------------- PR: https://git.openjdk.java.net/jdk/pull/775 From coleenp at openjdk.java.net Tue Nov 3 12:58:22 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 12:58:22 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: Message-ID: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: More review comments from Stefan and ErikO ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/cb4c83e0..eeaf9aed Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=01-02 Stats: 18 lines in 7 files changed: 3 ins; 6 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 13:06:53 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 13:06:53 GMT Subject: RFR: 8255810: Zero: build fails without JVMTI In-Reply-To: References: Message-ID: <8RuFCVYY-9hEzt8yw9K_PUuMLx74KRXeihjGMPZnKtA=.8da9b0de-e209-4f10-807e-5388680dd3c1@github.com> On Tue, 3 Nov 2020 11:22:50 GMT, Aleksey Shipilev wrote: > Zero interpreter has two modes: with JVMTI built-in, and without. But we cannot test it properly, because the build fails without JVMTI in shared code. Mostly due to JDK-8147388. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug builds `--with-jvm-features=-jvmti` Looks good. ------------- Marked as reviewed by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1036 From eosterlund at openjdk.java.net Tue Nov 3 13:12:02 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 3 Nov 2020 13:12:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> Message-ID: <4rQSjnt6O05gF9AFgc-nOxXIPehn_TKL0jJj85fvbr4=.c80aafac-a743-47d0-9d1a-31caa35d54ec@github.com> On Tue, 3 Nov 2020 12:58:22 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > More review comments from Stefan and ErikO Marked as reviewed by eosterlund (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Tue Nov 3 13:50:11 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 3 Nov 2020 13:50:11 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v3] In-Reply-To: References: Message-ID: <78WEYNz8e_RDtz43fTct4YdAh70TYGSM-x6Y0JQwgqs=.9c01b9c1-c8c2-420f-92f9-bb3cd95d2589@github.com> > The imasm::remove_activation() call does not deal with safepoints very well. However, when the MethodExit JVMTI event is being called, we call into the runtime in the middle of remove_activation(). If the value being returned is an object type, then the top-of-stack contains the oop. However, the GC does not traverse said oop in any oop map, because it is simply not expected that we safepoint in the middle of remove_activation(). > > The JvmtiExport::post_method_exit() function we end up calling, reads the top-of-stack oop, and puts it in a handle. Then it calls JVMTI callbacks, that eventually call Java and a bunch of stuff that safepoints. So after the JVMTI callback, we can expect the top-of-stack oop to be broken. Unfortunately, when we continue, we therefore end up returning a broken oop. > > Notably, the fact that InterpreterRuntime::post_method_exit is a JRT_ENTRY, is wrong, as we can safepoint on the way back to Java, which will break the return oop in a similar way. So this patch makes it a JRT_BLOCK_ENTRY, moving the transition to VM and back, into a block of code that is protected against GC. Before the JRT_BLOCK is called, we stash away the return oop, and after the JRT_BLOCK_END, we restore the top-of-stack oop. In the path when InterpreterRuntime::post_method_exit is called when throwing an exception, we don't have the same problem of retaining an oop result, and hence the JRT_BLOCK/JRT_BLOCK_END section is not performed in this case; the logic is the same as before for this path. > > This is a JVMTI bug that has probably been around for a long time. It crashes with all GCs, but was discovered recently after concurrent stack processing, as StefanK has been running better GC stressing code in JVMTI, and the bug reproduced more easily with concurrent stack processing, as the timings were a bit different. The following reproducer failed pretty much 100% of the time: > while true; do make test JTREG="RETAIN=all" TEST=test/hotspot/jtreg/vmTestbase/nsk/jdi/MethodExitEvent/returnValue/returnValue003/returnValue003.java TEST_OPTS_JAVA_OPTIONS="-XX:+UseZGC -Xmx2g -XX:ZCollectionInterval=0.0001 -XX:ZFragmentationLimit=0.01 -XX:+VerifyOops -XX:+ZVerifyViews -Xint" ; done > > With my fix I can run this repeatedly without any more failures. I have also sanity checked the patch by running tier 1-5, so that it does not introduces any new issues on its own. I have also used Stefan's nice external GC stressing with jcmd technique that was used to trigger crashes with other GCs, to make sure said crashes no longer reproduce either. Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: Serguei CR1: Check interpreted only ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/930/files - new: https://git.openjdk.java.net/jdk/pull/930/files/ae6355fd..4d68c624 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=930&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=930&range=01-02 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/930.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/930/head:pull/930 PR: https://git.openjdk.java.net/jdk/pull/930 From eosterlund at openjdk.java.net Tue Nov 3 13:54:57 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 3 Nov 2020 13:54:57 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 21:00:23 GMT, Serguei Spitsyn wrote: > Erik, > > Thank you for the update! It looks more elegant. > > One concern is that after the move of this fragment to the post_method_exit_inner: > > ``` > 1614 if (state == NULL || !state->is_interp_only_mode()) { > 1615 // for any thread that actually wants method exit, interp_only_mode is set > 1616 return; > 1617 } > ``` > > there is no guarantee that the current frame is interpreted below: > > ``` > 1580 if (!exception_exit) { > 1581 oop oop_result; > 1582 BasicType type = current_frame.interpreter_frame_result(&oop_result, &value); > . . . > 1597 if (result.not_null() && !mh->is_native()) { > 1598 // We have to restore the oop on the stack for interpreter frames > 1599 *(oop*)current_frame.interpreter_frame_tos_address() = result(); > 1600 } > ``` > > Probably, extra checks for current_frame.is_interpreted_frame() in these fragments will be sufficient. That makes sense. Added a check in the latest version that we are in interp only mode. ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From shade at openjdk.java.net Tue Nov 3 14:00:59 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 3 Nov 2020 14:00:59 GMT Subject: RFR: 8255810: Zero: build fails without JVMTI In-Reply-To: <8RuFCVYY-9hEzt8yw9K_PUuMLx74KRXeihjGMPZnKtA=.8da9b0de-e209-4f10-807e-5388680dd3c1@github.com> References: <8RuFCVYY-9hEzt8yw9K_PUuMLx74KRXeihjGMPZnKtA=.8da9b0de-e209-4f10-807e-5388680dd3c1@github.com> Message-ID: On Tue, 3 Nov 2020 13:04:08 GMT, Coleen Phillimore wrote: >> Zero interpreter has two modes: with JVMTI built-in, and without. But we cannot test it properly, because the build fails without JVMTI in shared code. Mostly due to JDK-8147388. >> >> Additional testing: >> - [x] Linux x86_64 Zero fastdebug builds `--with-jvm-features=-jvmti` > > Looks good. Thanks @coleenp! Would you say it is trivial? ------------- PR: https://git.openjdk.java.net/jdk/pull/1036 From coleenp at openjdk.java.net Tue Nov 3 14:46:58 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 14:46:58 GMT Subject: RFR: 8255810: Zero: build fails without JVMTI In-Reply-To: References: <8RuFCVYY-9hEzt8yw9K_PUuMLx74KRXeihjGMPZnKtA=.8da9b0de-e209-4f10-807e-5388680dd3c1@github.com> Message-ID: On Tue, 3 Nov 2020 13:58:36 GMT, Aleksey Shipilev wrote: >> Looks good. > > Thanks @coleenp! Would you say it is trivial? Yes, trivial! ------------- PR: https://git.openjdk.java.net/jdk/pull/1036 From coleenp at openjdk.java.net Tue Nov 3 14:53:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 14:53:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - Merge branch 'master' into jvmti-table - Merge branch 'master' into jvmti-table - More review comments from Stefan and ErikO - Code review comments from StefanK. - 8212879: Make JVMTI TagMap table not hash on oop address ------------- Changes: https://git.openjdk.java.net/jdk/pull/967/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=03 Stats: 1749 lines in 41 files changed: 627 ins; 1020 del; 102 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From shade at openjdk.java.net Tue Nov 3 17:17:57 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 3 Nov 2020 17:17:57 GMT Subject: RFR: 8255810: Zero: build fails without JVMTI In-Reply-To: References: <8RuFCVYY-9hEzt8yw9K_PUuMLx74KRXeihjGMPZnKtA=.8da9b0de-e209-4f10-807e-5388680dd3c1@github.com> Message-ID: <7hMYf7a4gV9wNZ-EyvZaMI4scxKjVqv2s55SUDiO8hg=.8f863caa-86c0-4512-bd82-284e9cc324c7@github.com> On Tue, 3 Nov 2020 14:43:52 GMT, Coleen Phillimore wrote: >> Thanks @coleenp! Would you say it is trivial? > > Yes, trivial! Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1036 From shade at openjdk.java.net Tue Nov 3 17:27:59 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 3 Nov 2020 17:27:59 GMT Subject: Integrated: 8255810: Zero: build fails without JVMTI In-Reply-To: References: Message-ID: <0wJLrYNFynBtocdY_3sIeDuDcj8D3xNbO2AmNXvUBP4=.cb9116da-e9ab-4b3e-ba48-a2662c6b60b3@github.com> On Tue, 3 Nov 2020 11:22:50 GMT, Aleksey Shipilev wrote: > Zero interpreter has two modes: with JVMTI built-in, and without. But we cannot test it properly, because the build fails without JVMTI in shared code. Mostly due to JDK-8147388. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug builds `--with-jvm-features=-jvmti` This pull request has now been integrated. Changeset: ca216bae Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/ca216bae Stats: 4 lines in 2 files changed: 2 ins; 0 del; 2 mod 8255810: Zero: build fails without JVMTI Reviewed-by: coleenp ------------- PR: https://git.openjdk.java.net/jdk/pull/1036 From sspitsyn at openjdk.java.net Tue Nov 3 17:53:54 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Tue, 3 Nov 2020 17:53:54 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 13:52:24 GMT, Erik ?sterlund wrote: >> Erik, >> >> Thank you for the update! It looks more elegant. >> >> One concern is that after the move of this fragment to the post_method_exit_inner: >> 1614 if (state == NULL || !state->is_interp_only_mode()) { >> 1615 // for any thread that actually wants method exit, interp_only_mode is set >> 1616 return; >> 1617 } >> there is no guarantee that the current frame is interpreted below: >> 1580 if (!exception_exit) { >> 1581 oop oop_result; >> 1582 BasicType type = current_frame.interpreter_frame_result(&oop_result, &value); >> . . . >> 1597 if (result.not_null() && !mh->is_native()) { >> 1598 // We have to restore the oop on the stack for interpreter frames >> 1599 *(oop*)current_frame.interpreter_frame_tos_address() = result(); >> 1600 } >> Probably, extra checks for current_frame.is_interpreted_frame() in these fragments will be sufficient. > >> Erik, >> >> Thank you for the update! It looks more elegant. >> >> One concern is that after the move of this fragment to the post_method_exit_inner: >> >> ``` >> 1614 if (state == NULL || !state->is_interp_only_mode()) { >> 1615 // for any thread that actually wants method exit, interp_only_mode is set >> 1616 return; >> 1617 } >> ``` >> >> there is no guarantee that the current frame is interpreted below: >> >> ``` >> 1580 if (!exception_exit) { >> 1581 oop oop_result; >> 1582 BasicType type = current_frame.interpreter_frame_result(&oop_result, &value); >> . . . >> 1597 if (result.not_null() && !mh->is_native()) { >> 1598 // We have to restore the oop on the stack for interpreter frames >> 1599 *(oop*)current_frame.interpreter_frame_tos_address() = result(); >> 1600 } >> ``` >> >> Probably, extra checks for current_frame.is_interpreted_frame() in these fragments will be sufficient. > > That makes sense. Added a check in the latest version that we are in interp only mode. Hi Erik, I'm not sure, if this fragment is still needed: 1620 if (state == NULL || !state->is_interp_only_mode()) { 1621 // for any thread that actually wants method exit, interp_only_mode is set 1622 return; 1623 } Also, can it be that this condition is true: ` (state == NULL || !state->is_interp_only_mode())` but the top frame is interpreted? If so, then should we still safe/restore the result oop over a possible safepoint? Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From kbarrett at openjdk.java.net Tue Nov 3 18:12:10 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 3 Nov 2020 18:12:10 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:53:12 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Merge branch 'master' into jvmti-table > - Merge branch 'master' into jvmti-table > - More review comments from Stefan and ErikO > - Code review comments from StefanK. > - 8212879: Make JVMTI TagMap table not hash on oop address src/hotspot/share/prims/jvmtiTagMap.cpp line 3018: > 3016: } > 3017: // Later GC code will relocate the oops, so defer rehashing until then. > 3018: tag_map->_needs_rehashing = true; This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.) For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing. (I think CMS oldgen remark may have been similar, for what that's worth.) src/hotspot/share/prims/jvmtiTagMap.cpp line 3015: > 3013: if (tag_map != NULL && !tag_map->is_empty()) { > 3014: if (num_dead_entries != 0) { > 3015: tag_map->hashmap()->unlink_and_post(tag_map->env()); Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup. src/hotspot/share/prims/jvmtiTagMap.cpp line 2979: > 2977: > 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved > 2979: // and have their new addresses, the table can be rehashed. I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that). src/hotspot/share/prims/jvmtiTagMap.cpp line 127: > 125: // The table cleaning, posting and rehashing can race for > 126: // concurrent GCs. So fix it here once we have a lock or are > 127: // at a safepoint. I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time). ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Tue Nov 3 18:12:07 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 3 Nov 2020 18:12:07 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> Message-ID: On Tue, 3 Nov 2020 12:58:22 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > More review comments from Stefan and ErikO src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 41: > 39: class Iterator; > 40: > 41: typedef void (*Processor)(BoolObjectClosure*, OopClosure*); I think this typedef is to support serial phases and that it is probably no longer used. src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50: > 48: }; > 49: > 50: typedef uint WeakProcessorPhase; This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes. But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here: src/hotspot/share/gc/shared/weakProcessor.inline.hpp 116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase); 117 StorageState* cur_state = _storage_states.par_state(oopstorage_index); => 103 StorageState* cur_state = _storage_states.par_state(phase); I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet. enum class : uint WeakProcessorPhase {}; ENUMERATOR_RANGE(WeakProcessorPhase, static_cast(0), static_cast(OopStorageSet::weak_count)); and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator (which involves more than a type alias). Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:17:05 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:17:05 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:47:35 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/prims/jvmtiTagMap.cpp line 3018: > >> 3016: } >> 3017: // Later GC code will relocate the oops, so defer rehashing until then. >> 3018: tag_map->_needs_rehashing = true; > > This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.) > > For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing. > > (I think CMS oldgen remark may have been similar, for what that's worth.) Ok, so I'm going to need help to know where in all the different GCs to make this call. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From ayang at openjdk.java.net Tue Nov 3 21:17:03 2020 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 3 Nov 2020 21:17:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:53:12 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Merge branch 'master' into jvmti-table > - Merge branch 'master' into jvmti-table > - More review comments from Stefan and ErikO > - Code review comments from StefanK. > - 8212879: Make JVMTI TagMap table not hash on oop address src/hotspot/share/utilities/hashtable.cpp line 164: > 162: } > 163: } > 164: return newsize; It is existing code, but could be made clearer as part of this PR: int newsize; for (int i=0; i= requested) break; } return newsize; Additionally, this method could be made `const`, right? PS: not a review, just a comment in passing ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From daniel.daugherty at oracle.com Tue Nov 3 21:18:23 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 3 Nov 2020 16:18:23 -0500 Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: <3fb77c6e-a99b-c90b-9170-1ce5fca6df1b@oracle.com> On 11/2/20 8:22 AM, Coleen Phillimore wrote: > On Mon, 2 Nov 2020 08:34:17 GMT, Stefan Karlsson wrote: > >>> src/hotspot/share/prims/jvmtiTagMap.cpp line 126: >>> >>>> 124: // concurrent GCs. So fix it here once we have a lock or are >>>> 125: // at a safepoint. >>>> 126: // SetTag and GetTag should not post events! >>> I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case. >> Maybe even move this comment to the set_tag/get_tag code. > I was trying to explain why there's a boolean there but I can put this comment at both get_tag and set_tag. > > // Check if we have to processing for concurrent GCs. Typo: s/we have to processing/we have to do processing/ > // GetTag should not post events because the JavaThread has to > // transition to native for the callback and this cannot stop for > // safepoints with the hashmap lock held. > check_hashmap(false); > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:28:08 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:28:08 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:50:36 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/prims/jvmtiTagMap.cpp line 3015: > >> 3013: if (tag_map != NULL && !tag_map->is_empty()) { >> 3014: if (num_dead_entries != 0) { >> 3015: tag_map->hashmap()->unlink_and_post(tag_map->env()); > > Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup. The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From dcubed at openjdk.java.net Tue Nov 3 21:28:06 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 3 Nov 2020 21:28:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: <6uV_hLuxf7fu90EgzZckv9LwT-jAVboukH35MKaTtcU=.40efbbd6-fd6e-427a-a940-784ec86ddb15@github.com> On Mon, 2 Nov 2020 13:19:08 GMT, Coleen Phillimore wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > I think I addressed your comments, retesting now. Thank you! @coleenp - please make sure you hear from someone on the Serviceability team for this PR... ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:33:59 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:33:59 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 16:12:21 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/prims/jvmtiTagMap.cpp line 2979: > >> 2977: >> 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved >> 2979: // and have their new addresses, the table can be rehashed. > > I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that). So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case? ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:46:04 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:46:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 16:17:58 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/prims/jvmtiTagMap.cpp line 127: > >> 125: // The table cleaning, posting and rehashing can race for >> 126: // concurrent GCs. So fix it here once we have a lock or are >> 127: // at a safepoint. > > I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time). The comment is trying to describe the situation like: 1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked) 2. safepoint for heap walk 2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A. 3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable. Both operations require the table to be locked. The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:46:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:46:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 13:19:08 GMT, Coleen Phillimore wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > I think I addressed your comments, retesting now. Thank you! > @coleenp - please make sure you hear from someone on the Serviceability team > for this PR... I've asked @sspitsyn to review this. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:46:05 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:46:05 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> Message-ID: On Tue, 3 Nov 2020 13:43:32 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> More review comments from Stefan and ErikO > > src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 41: > >> 39: class Iterator; >> 40: >> 41: typedef void (*Processor)(BoolObjectClosure*, OopClosure*); > > I think this typedef is to support serial phases and that it is probably no longer used. ok, removed. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:51:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:51:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> Message-ID: <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> On Tue, 3 Nov 2020 13:45:57 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> More review comments from Stefan and ErikO > > src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50: > >> 48: }; >> 49: >> 50: typedef uint WeakProcessorPhase; > > This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes. > > But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here: > > src/hotspot/share/gc/shared/weakProcessor.inline.hpp > 116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase); > 117 StorageState* cur_state = _storage_states.par_state(oopstorage_index); > => > 103 StorageState* cur_state = _storage_states.par_state(phase); > > I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet. > enum class : uint WeakProcessorPhase {}; > > ENUMERATOR_RANGE(WeakProcessorPhase, > static_cast(0), > static_cast(OopStorageSet::weak_count)); > and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator (which involves more than a type alias). > > Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. Ok, so I'm not sure what to do with this: enum Phase { // Serial phase. JVMTI_ONLY(jvmti) // Additional implicit phase values follow for oopstorages. `};` I've removed the only thing in this enum. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From sspitsyn at openjdk.java.net Tue Nov 3 22:23:00 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Tue, 3 Nov 2020 22:23:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 21:41:55 GMT, Coleen Phillimore wrote: > > @coleenp - please make sure you hear from someone on the Serviceability team > > for this PR... > > I've asked @sspitsyn to review this. Yes, I'm reviewing this. Still need another pass. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From amenkov at openjdk.java.net Tue Nov 3 23:22:59 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Tue, 3 Nov 2020 23:22:59 GMT Subject: RFR: 8254864: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java timed out Message-ID: - Fixed synchronization logic in resexhausted001; - On Windows JVM cannot produce OOM caused by thread creation failure. Massive thread creation causes big system (not VM) memory consumption, as a result test host dramatically slows down up to hang. Specifying small heap size we can get RESOURCE_EXHAUSTED_JAVA_HEAP, but this is not the goal of resexhausted001. So I disabled resexhausted001 on Windows. Had to implement runtime check (instead of using @requires) because resexhausted001 is also called by resexhausted004. - Additionally resexhausted001-resexhausted003 test were improved to verify JVMTI generates ResourceExhausted event with correct flag. ------------- Commit messages: - 8254864: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java timed out Changes: https://git.openjdk.java.net/jdk/pull/1046/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1046&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8254864 Stats: 47 lines in 6 files changed: 26 ins; 1 del; 20 mod Patch: https://git.openjdk.java.net/jdk/pull/1046.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1046/head:pull/1046 PR: https://git.openjdk.java.net/jdk/pull/1046 From coleenp at openjdk.java.net Tue Nov 3 23:41:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 23:41:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> Message-ID: <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> On Tue, 3 Nov 2020 21:47:24 GMT, Coleen Phillimore wrote: >> src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50: >> >>> 48: }; >>> 49: >>> 50: typedef uint WeakProcessorPhase; >> >> This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes. >> >> But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here: >> >> src/hotspot/share/gc/shared/weakProcessor.inline.hpp >> 116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase); >> 117 StorageState* cur_state = _storage_states.par_state(oopstorage_index); >> => >> 103 StorageState* cur_state = _storage_states.par_state(phase); >> >> I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet. >> enum class : uint WeakProcessorPhase {}; >> >> ENUMERATOR_RANGE(WeakProcessorPhase, >> static_cast(0), >> static_cast(OopStorageSet::weak_count)); >> and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator (which involves more than a type alias). >> >> Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. > > Ok, so I'm not sure what to do with this: > > enum Phase { > // Serial phase. > JVMTI_ONLY(jvmti) > // Additional implicit phase values follow for oopstorages. > `};` > > I've removed the only thing in this enum. >Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 23:41:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 23:41:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 21:12:49 GMT, Albert Mingkun Yang wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/utilities/hashtable.cpp line 164: > >> 162: } >> 163: } >> 164: return newsize; > > It is existing code, but could be made clearer as part of this PR: > int newsize; > for (int i=0; i newsize = primelist[i]; > if (newsize >= requested) > break; > } > return newsize; > Additionally, this method could be made `const`, right? > > PS: not a review, just a comment in passing Yes, that is a lot simpler and better. I'd copied that code from another file without changing it that much. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From sspitsyn at openjdk.java.net Tue Nov 3 23:57:55 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Tue, 3 Nov 2020 23:57:55 GMT Subject: RFR: 8254864: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java timed out In-Reply-To: References: Message-ID: <4fKsZCZiosTTFNuG9000ATfHHxklYzjKTCzUVEiEoC0=.e5da3766-aa42-4c40-b667-83633550316c@github.com> On Tue, 3 Nov 2020 23:16:50 GMT, Alex Menkov wrote: > - Fixed synchronization logic in resexhausted001; > - On Windows JVM cannot produce OOM caused by thread creation failure. Massive thread creation causes big system (not VM) memory consumption, as a result test host dramatically slows down up to hang. Specifying small heap size we can get RESOURCE_EXHAUSTED_JAVA_HEAP, but this is not the goal of resexhausted001. > So I disabled resexhausted001 on Windows. > Had to implement runtime check (instead of using @requires) because resexhausted001 is also called by resexhausted004. > - Additionally resexhausted001-resexhausted003 test were improved to verify JVMTI generates ResourceExhausted event with correct flag. Hi Alex, The fix looks good to me. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1046 From coleenp at openjdk.java.net Wed Nov 4 00:08:10 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 4 Nov 2020 00:08:10 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Code review comments from Kim and Albert. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/1c3f2e1e..f66ea839 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=03-04 Stats: 10 lines in 3 files changed: 0 ins; 4 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From cjplummer at openjdk.java.net Wed Nov 4 01:30:04 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 4 Nov 2020 01:30:04 GMT Subject: RFR: 8255858: Add debug agent support for storing thread names Message-ID: Simple change to add a `name` field to `ThreadNode` to aide when debugging the debug agent. It's off by default. You need to add a `#define DEBUG_THREADNAME` to enable it. ------------- Commit messages: - Add support for tracking a thread's name. Changes: https://git.openjdk.java.net/jdk/pull/1047/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1047&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255858 Stats: 19 lines in 1 file changed: 19 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1047.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1047/head:pull/1047 PR: https://git.openjdk.java.net/jdk/pull/1047 From sspitsyn at openjdk.java.net Wed Nov 4 02:19:00 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 4 Nov 2020 02:19:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 00:08:10 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from Kim and Albert. Hi Coleen, Wow, there are a lot of simplifications and code removal with this fix! It looks great in general, just some nits below. I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :) src/hotspot/share/prims/jvmtiTagMap.hpp: Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? : `+ void post_dead_object_on_vm_thread();` src/hotspot/share/prims/jvmtiTagMap.cpp: Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk: 122 void JvmtiTagMap::check_hashmap(bool post_events) { . . . 143 // This checks for posting and rehashing and is called from the heap walks. 144 void JvmtiTagMap::check_hashmaps_for_heapwalk() { I'm just curious how this fragment was added. Did you get any failures in testing? : 1038 // skip if object is a dormant shared object whose mirror hasn't been loaded 1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) { 1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj), 1041 obj->klass()->external_name()); 1042 return; 1043 } Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? : `1186 bool _some_dead;` Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts. 2996 if (!is_vm_thread) { 2997 if (num_dead_entries != 0) { 2998 JvmtiEnvIterator it; 2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { 3000 JvmtiTagMap* tag_map = env->tag_map_acquire(); 3001 if (tag_map != NULL) { 3002 // Lock each hashmap from concurrent posting and cleaning 3003 tag_map->unlink_and_post_locked(); 3004 } 3005 } 3006 // there's another callback for needs_rehashing 3007 } 3008 } else { 3009 assert(SafepointSynchronize::is_at_safepoint(), "must be"); 3010 JvmtiEnvIterator it; 3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { 3012 JvmtiTagMap* tag_map = env->tag_map_acquire(); 3013 if (tag_map != NULL && !tag_map->is_empty()) { 3014 if (num_dead_entries != 0) { 3015 tag_map->hashmap()->unlink_and_post(tag_map->env()); 3016 } 3017 // Later GC code will relocate the oops, so defer rehashing until then. 3018 tag_map->_needs_rehashing = true; 3019 } It feels like it can be refactored/simplified, at least, a little bit. Is it possible to check and just return if (num_dead_entries == 0)? If not, then, at least, it can be done the same way (except of locking). Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts. src/hotspot/share/utilities/hashtable.cpp: Nit: Need space after the '{' : `+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;` src/hotspot/share/prims/jvmtiTagMapTable.cpp: Nit: Extra space after assert: `119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");` Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From sspitsyn at openjdk.java.net Wed Nov 4 04:38:58 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 4 Nov 2020 04:38:58 GMT Subject: RFR: 8255858: Add debug agent support for storing thread names In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 23:30:38 GMT, Chris Plummer wrote: > Simple change to add a `name` field to `ThreadNode` to aide when debugging the debug agent. It's off by default. You need to add a `#define DEBUG_THREADNAME` to enable it. Chris, It looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1047 From sspitsyn at openjdk.java.net Wed Nov 4 05:40:03 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 4 Nov 2020 05:40:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 02:15:52 GMT, Serguei Spitsyn wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Code review comments from Kim and Albert. > > Hi Coleen, > > Wow, there are a lot of simplifications and code removal with this fix! > It looks great in general, just some nits below. > I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :) > > src/hotspot/share/prims/jvmtiTagMap.hpp: > > Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? : > `+ void post_dead_object_on_vm_thread();` > > src/hotspot/share/prims/jvmtiTagMap.cpp: > > Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk: > 122 void JvmtiTagMap::check_hashmap(bool post_events) { > . . . > 143 // This checks for posting and rehashing and is called from the heap walks. > 144 void JvmtiTagMap::check_hashmaps_for_heapwalk() { > > I'm just curious how this fragment was added. Did you get any failures in testing? : > 1038 // skip if object is a dormant shared object whose mirror hasn't been loaded > 1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) { > 1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj), > 1041 obj->klass()->external_name()); > 1042 return; > 1043 } > > Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? : > `1186 bool _some_dead;` > > Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts. > 2996 if (!is_vm_thread) { > 2997 if (num_dead_entries != 0) { > 2998 JvmtiEnvIterator it; > 2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > 3000 JvmtiTagMap* tag_map = env->tag_map_acquire(); > 3001 if (tag_map != NULL) { > 3002 // Lock each hashmap from concurrent posting and cleaning > 3003 tag_map->unlink_and_post_locked(); > 3004 } > 3005 } > 3006 // there's another callback for needs_rehashing > 3007 } > 3008 } else { > 3009 assert(SafepointSynchronize::is_at_safepoint(), "must be"); > 3010 JvmtiEnvIterator it; > 3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > 3012 JvmtiTagMap* tag_map = env->tag_map_acquire(); > 3013 if (tag_map != NULL && !tag_map->is_empty()) { > 3014 if (num_dead_entries != 0) { > 3015 tag_map->hashmap()->unlink_and_post(tag_map->env()); > 3016 } > 3017 // Later GC code will relocate the oops, so defer rehashing until then. > 3018 tag_map->_needs_rehashing = true; > 3019 } > It feels like it can be refactored/simplified, at least, a little bit. > Is it possible to check and just return if (num_dead_entries == 0)? > If not, then, at least, it can be done the same way (except of locking). > Q: Should the _needs_rehashing be set in both contexts? > > Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts. > > src/hotspot/share/utilities/hashtable.cpp: > > Nit: Need space after the '{' : > `+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;` > > src/hotspot/share/prims/jvmtiTagMapTable.cpp: > > Nit: Extra space after assert: > `119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");` > > Thanks, > Serguei More about possible refactoring of the JvmtiTagMap::gc_notification(). I'm thinking about something like below: void JvmtiTagMap::unlink_and_post_for_all_envs() { if (num_dead_entries == 0) { return; // nothing to unlink and post } JvmtiEnvIterator it; for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { JvmtiTagMap* tag_map = env->tag_map_acquire(); if (tag_map != NULL && !tag_map->is_empty()) { tag_map->unlink_and_post(); } } } void JvmtiTagMap::gc_notification(size_t num_dead_entries) { if (Thread::current()->is_VM_thread()) { assert(SafepointSynchronize::is_at_safepoint(), "must be"); unlink_and_post_for_all_envs(); set_needs_rehashing(); } else { MutexLocker ml(JvmtiTagMap_lock(), Mutex::_no_safepoint_check_flag); unlink_and_post_for_all_envs(); // there's another callback for needs_rehashing } } If we still need a lock per each JvmtiTagMap then it is possible to add this fragment to the unlink_and_post_for_all_envs: bool is_vm_thread = Thread::current()->is_VM_thread() MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag); Then the code above could look like below: void JvmtiTagMap::unlink_and_post_for_all_envs() { if (num_dead_entries == 0) { return; // nothing to unlink and post } bool is_vm_thread = Thread::current()->is_VM_thread() JvmtiEnvIterator it; for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { JvmtiTagMap* tag_map = env->tag_map_acquire(); if (tag_map != NULL && !tag_map->is_empty()) { MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag); tag_map->unlink_and_post(); } } } void JvmtiTagMap::gc_notification(size_t num_dead_entries) { if (Thread::current()->is_VM_thread()) { assert(SafepointSynchronize::is_at_safepoint(), "must be"); set_needs_rehashing(); } unlink_and_post_for_all_envs(); } ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Wed Nov 4 09:06:07 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Wed, 4 Nov 2020 09:06:07 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v4] In-Reply-To: References: Message-ID: > The imasm::remove_activation() call does not deal with safepoints very well. However, when the MethodExit JVMTI event is being called, we call into the runtime in the middle of remove_activation(). If the value being returned is an object type, then the top-of-stack contains the oop. However, the GC does not traverse said oop in any oop map, because it is simply not expected that we safepoint in the middle of remove_activation(). > > The JvmtiExport::post_method_exit() function we end up calling, reads the top-of-stack oop, and puts it in a handle. Then it calls JVMTI callbacks, that eventually call Java and a bunch of stuff that safepoints. So after the JVMTI callback, we can expect the top-of-stack oop to be broken. Unfortunately, when we continue, we therefore end up returning a broken oop. > > Notably, the fact that InterpreterRuntime::post_method_exit is a JRT_ENTRY, is wrong, as we can safepoint on the way back to Java, which will break the return oop in a similar way. So this patch makes it a JRT_BLOCK_ENTRY, moving the transition to VM and back, into a block of code that is protected against GC. Before the JRT_BLOCK is called, we stash away the return oop, and after the JRT_BLOCK_END, we restore the top-of-stack oop. In the path when InterpreterRuntime::post_method_exit is called when throwing an exception, we don't have the same problem of retaining an oop result, and hence the JRT_BLOCK/JRT_BLOCK_END section is not performed in this case; the logic is the same as before for this path. > > This is a JVMTI bug that has probably been around for a long time. It crashes with all GCs, but was discovered recently after concurrent stack processing, as StefanK has been running better GC stressing code in JVMTI, and the bug reproduced more easily with concurrent stack processing, as the timings were a bit different. The following reproducer failed pretty much 100% of the time: > while true; do make test JTREG="RETAIN=all" TEST=test/hotspot/jtreg/vmTestbase/nsk/jdi/MethodExitEvent/returnValue/returnValue003/returnValue003.java TEST_OPTS_JAVA_OPTIONS="-XX:+UseZGC -Xmx2g -XX:ZCollectionInterval=0.0001 -XX:ZFragmentationLimit=0.01 -XX:+VerifyOops -XX:+ZVerifyViews -Xint" ; done > > With my fix I can run this repeatedly without any more failures. I have also sanity checked the patch by running tier 1-5, so that it does not introduces any new issues on its own. I have also used Stefan's nice external GC stressing with jcmd technique that was used to trigger crashes with other GCs, to make sure said crashes no longer reproduce either. Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: Serguei CR2: Don't check interpreted only ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/930/files - new: https://git.openjdk.java.net/jdk/pull/930/files/4d68c624..95306514 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=930&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=930&range=02-03 Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/930.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/930/head:pull/930 PR: https://git.openjdk.java.net/jdk/pull/930 From eosterlund at openjdk.java.net Wed Nov 4 09:06:07 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Wed, 4 Nov 2020 09:06:07 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 17:49:38 GMT, Serguei Spitsyn wrote: > Hi Erik, > > I'm not sure, if this fragment is still needed: > > ``` > 1620 if (state == NULL || !state->is_interp_only_mode()) { > 1621 // for any thread that actually wants method exit, interp_only_mode is set > 1622 return; > 1623 } > ``` Seems like it is not needed. I removed it. > Also, can it be that this condition is true: > ` (state == NULL || !state->is_interp_only_mode())` > but the top frame is interpreted? > If so, then should we still safe/restore the result oop over a possible safepoint? It could definitely be that the top frame is interpreted even though that condition is true. However, we now enter InterpreterRuntime::post_method_exit as a JRT_BLOCK_ENTRY call, which performs no transition (similar to JRT_LEAF). So if so we should just return back to the caller without doing anything, and no GC will happen in this path then. It is only when we perform the JRT_BLOCK and JRT_BLOCK_END that we allow GCs to happen, and we save/restore the result across that section. Thanks, > Thanks, > Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From kbarrett at openjdk.java.net Wed Nov 4 09:36:02 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> Message-ID: On Wed, 4 Nov 2020 07:41:39 GMT, Kim Barrett wrote: >>>Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. >> >> This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. > >> Ok, so I'm not sure what to do with this: >> >> enum Phase { >> // Serial phase. >> JVMTI_ONLY(jvmti) >> // Additional implicit phase values follow for oopstorages. >> `};` >> >> I've removed the only thing in this enum. > > Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums. > > Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. > > This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. I think the deletion stopped at the wrong place; it either went too far, or not far enough. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:36:02 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> Message-ID: On Tue, 3 Nov 2020 23:38:08 GMT, Coleen Phillimore wrote: >> Ok, so I'm not sure what to do with this: >> >> enum Phase { >> // Serial phase. >> JVMTI_ONLY(jvmti) >> // Additional implicit phase values follow for oopstorages. >> `};` >> >> I've removed the only thing in this enum. > >>Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. > > This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. > Ok, so I'm not sure what to do with this: > > enum Phase { > // Serial phase. > JVMTI_ONLY(jvmti) > // Additional implicit phase values follow for oopstorages. > `};` > > I've removed the only thing in this enum. Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:36:03 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 21:31:35 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 2979: >> >>> 2977: >>> 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved >>> 2979: // and have their new addresses, the table can be rehashed. >> >> I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that). > > So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case? I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.) >> src/hotspot/share/prims/jvmtiTagMap.cpp line 3015: >> >>> 3013: if (tag_map != NULL && !tag_map->is_empty()) { >>> 3014: if (num_dead_entries != 0) { >>> 3015: tag_map->hashmap()->unlink_and_post(tag_map->env()); >> >> Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup. > > The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table. I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:36:04 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 00:08:10 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from Kim and Albert. src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36: > 34: class JvmtiTagMapEntryClosure; > 35: > 36: class JvmtiTagMapEntry : public HashtableEntry { By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%). It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:36:04 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 07:52:12 GMT, Kim Barrett wrote: >> So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case? > > I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.) It used to be that jvmti tagmap processing was all-in-one (in GC weak reference processing, with the weak clearing, dead table entry removal, and rehashing all done in one pass. This change has split that up, with the weak clearing happening in a different place (still as part of the GC's weak reference processing) than the others (which I claim can now be part of the mutator, whether further separated or not). "Concurrent GC" has nothing to do with whether tagmaps need rehashing. Any copying collector needs to do so. A non-copying collector (whether concurrent or not) would not. (We don't have any of those in HotSpot today.) And weak reference clearing (whether concurrent or not) has nothing to do with whether objects have been moved and so the hashing has been invalidated. There's also a "well known" issue with address-based hashing and generational or similar collectors, where a simple boolean "objects have moved" flag can be problematic, and which tagmaps seem likely to be prone to. The old do_weak_oops tries to mitigate it by recognizing when the object didn't move. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 10:08:00 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 10:08:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> On Tue, 3 Nov 2020 21:40:39 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 127: >> >>> 125: // The table cleaning, posting and rehashing can race for >>> 126: // concurrent GCs. So fix it here once we have a lock or are >>> 127: // at a safepoint. >> >> I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time). > > The comment is trying to describe the situation like: > 1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked) > 2. safepoint for heap walk > 2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A. > 3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened > > The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable. > > Both operations require the table to be locked. > > The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints. For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing, an d so still effectively be a serial subtask after all the parallel subtasks are done or mostly done. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Wed Nov 4 12:21:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 4 Nov 2020 12:21:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 05:37:00 GMT, Serguei Spitsyn wrote: >> Hi Coleen, >> >> Wow, there are a lot of simplifications and code removal with this fix! >> It looks great in general, just some nits below. >> I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :) >> >> src/hotspot/share/prims/jvmtiTagMap.hpp: >> >> Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? : >> `+ void post_dead_object_on_vm_thread();` >> >> src/hotspot/share/prims/jvmtiTagMap.cpp: >> >> Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk: >> 122 void JvmtiTagMap::check_hashmap(bool post_events) { >> . . . >> 143 // This checks for posting and rehashing and is called from the heap walks. >> 144 void JvmtiTagMap::check_hashmaps_for_heapwalk() { >> >> I'm just curious how this fragment was added. Did you get any failures in testing? : >> 1038 // skip if object is a dormant shared object whose mirror hasn't been loaded >> 1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) { >> 1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj), >> 1041 obj->klass()->external_name()); >> 1042 return; >> 1043 } >> >> Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? : >> `1186 bool _some_dead;` >> >> Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts. >> 2996 if (!is_vm_thread) { >> 2997 if (num_dead_entries != 0) { >> 2998 JvmtiEnvIterator it; >> 2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { >> 3000 JvmtiTagMap* tag_map = env->tag_map_acquire(); >> 3001 if (tag_map != NULL) { >> 3002 // Lock each hashmap from concurrent posting and cleaning >> 3003 tag_map->unlink_and_post_locked(); >> 3004 } >> 3005 } >> 3006 // there's another callback for needs_rehashing >> 3007 } >> 3008 } else { >> 3009 assert(SafepointSynchronize::is_at_safepoint(), "must be"); >> 3010 JvmtiEnvIterator it; >> 3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { >> 3012 JvmtiTagMap* tag_map = env->tag_map_acquire(); >> 3013 if (tag_map != NULL && !tag_map->is_empty()) { >> 3014 if (num_dead_entries != 0) { >> 3015 tag_map->hashmap()->unlink_and_post(tag_map->env()); >> 3016 } >> 3017 // Later GC code will relocate the oops, so defer rehashing until then. >> 3018 tag_map->_needs_rehashing = true; >> 3019 } >> It feels like it can be refactored/simplified, at least, a little bit. >> Is it possible to check and just return if (num_dead_entries == 0)? >> If not, then, at least, it can be done the same way (except of locking). >> Q: Should the _needs_rehashing be set in both contexts? >> >> Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts. >> >> src/hotspot/share/utilities/hashtable.cpp: >> >> Nit: Need space after the '{' : >> `+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;` >> >> src/hotspot/share/prims/jvmtiTagMapTable.cpp: >> >> Nit: Extra space after assert: >> `119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");` >> >> Thanks, >> Serguei > > More about possible refactoring of the JvmtiTagMap::gc_notification(). > I'm thinking about something like below: > > void JvmtiTagMap::unlink_and_post_for_all_envs() { > if (num_dead_entries == 0) { > return; // nothing to unlink and post > } > JvmtiEnvIterator it; > for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > JvmtiTagMap* tag_map = env->tag_map_acquire(); > if (tag_map != NULL && !tag_map->is_empty()) { > tag_map->unlink_and_post(); > } > } > } > > void JvmtiTagMap::gc_notification(size_t num_dead_entries) { > if (Thread::current()->is_VM_thread()) { > assert(SafepointSynchronize::is_at_safepoint(), "must be"); > unlink_and_post_for_all_envs(); > set_needs_rehashing(); > } else { > MutexLocker ml(JvmtiTagMap_lock(), Mutex::_no_safepoint_check_flag); > unlink_and_post_for_all_envs(); > // there's another callback for needs_rehashing > } > } > > If we still need a lock per each JvmtiTagMap then it is possible to add this fragment to the unlink_and_post_for_all_envs: > bool is_vm_thread = Thread::current()->is_VM_thread() > MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag); > > Then the code above could look like below: > > void JvmtiTagMap::unlink_and_post_for_all_envs() { > if (num_dead_entries == 0) { > return; // nothing to unlink and post > } > bool is_vm_thread = Thread::current()->is_VM_thread() > JvmtiEnvIterator it; > for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > JvmtiTagMap* tag_map = env->tag_map_acquire(); > if (tag_map != NULL && !tag_map->is_empty()) { > MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag); > tag_map->unlink_and_post(); > } > } > } > > void JvmtiTagMap::gc_notification(size_t num_dead_entries) { > if (Thread::current()->is_VM_thread()) { > assert(SafepointSynchronize::is_at_safepoint(), "must be"); > set_needs_rehashing(); > } > unlink_and_post_for_all_envs(); > } @sspitsyn Thank you for reviewing the code. The gc_notification refactoring is awkward because each table needs a lock and not a global lock. If we find another place in the GCs to call set_needs_rehashing() it might be possible to make gc_notification call two functions with a boolean to decide whether to take the lock. We're still working on @kimbarrett comments so maybe the notification will change to some new thread and be refactored that way if necessary. I fixed the code for your other comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Wed Nov 4 12:21:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 4 Nov 2020 12:21:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision: - Add back WeakProcessorPhases::Phase enum. - Serguei 1. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/f66ea839..7d3fdf68 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=04-05 Stats: 18 lines in 5 files changed: 7 ins; 0 del; 11 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From cjplummer at openjdk.java.net Wed Nov 4 21:46:54 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 4 Nov 2020 21:46:54 GMT Subject: RFR: 8254864: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java timed out In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 23:16:50 GMT, Alex Menkov wrote: > Had to implement runtime check (instead of using @requires) because resexhausted001 is also called by resexhausted004. So does that mean you do or don't want resexhausted004 disabled on windows? If yes, there's no reason why you can't use @requires in the TestDescriptions.java for each of these tests (although I have no objecting to the runtime check). If no then you have a bug because you are disabling resexhausted004 on windows, and should use @requires in TestDescriptions.java for resexhausted001 instead. ------------- PR: https://git.openjdk.java.net/jdk/pull/1046 From amenkov at openjdk.java.net Wed Nov 4 21:47:58 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 4 Nov 2020 21:47:58 GMT Subject: RFR: 8255858: Add debug agent support for storing thread names In-Reply-To: References: Message-ID: <70Ay3WumNE2UQBA0xENw7_tNGLkblEotQ6jREtPPJ74=.591a9a14-1042-4f57-9f39-bbaebd26d3eb@github.com> On Tue, 3 Nov 2020 23:30:38 GMT, Chris Plummer wrote: > Simple change to add a `name` field to `ThreadNode` to aide when debugging the debug agent. It's off by default. You need to add a `#define DEBUG_THREADNAME` to enable it. Marked as reviewed by amenkov (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1047 From sspitsyn at openjdk.java.net Wed Nov 4 22:12:06 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 4 Nov 2020 22:12:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 12:21:12 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision: > > - Add back WeakProcessorPhases::Phase enum. > - Serguei 1. Thank you for the update, Coleen! I leave it for you to decide to refactor the gc_notification or not. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From sspitsyn at openjdk.java.net Wed Nov 4 22:15:58 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 4 Nov 2020 22:15:58 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v4] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 09:06:07 GMT, Erik ?sterlund wrote: >> The imasm::remove_activation() call does not deal with safepoints very well. However, when the MethodExit JVMTI event is being called, we call into the runtime in the middle of remove_activation(). If the value being returned is an object type, then the top-of-stack contains the oop. However, the GC does not traverse said oop in any oop map, because it is simply not expected that we safepoint in the middle of remove_activation(). >> >> The JvmtiExport::post_method_exit() function we end up calling, reads the top-of-stack oop, and puts it in a handle. Then it calls JVMTI callbacks, that eventually call Java and a bunch of stuff that safepoints. So after the JVMTI callback, we can expect the top-of-stack oop to be broken. Unfortunately, when we continue, we therefore end up returning a broken oop. >> >> Notably, the fact that InterpreterRuntime::post_method_exit is a JRT_ENTRY, is wrong, as we can safepoint on the way back to Java, which will break the return oop in a similar way. So this patch makes it a JRT_BLOCK_ENTRY, moving the transition to VM and back, into a block of code that is protected against GC. Before the JRT_BLOCK is called, we stash away the return oop, and after the JRT_BLOCK_END, we restore the top-of-stack oop. In the path when InterpreterRuntime::post_method_exit is called when throwing an exception, we don't have the same problem of retaining an oop result, and hence the JRT_BLOCK/JRT_BLOCK_END section is not performed in this case; the logic is the same as before for this path. >> >> This is a JVMTI bug that has probably been around for a long time. It crashes with all GCs, but was discovered recently after concurrent stack processing, as StefanK has been running better GC stressing code in JVMTI, and the bug reproduced more easily with concurrent stack processing, as the timings were a bit different. The following reproducer failed pretty much 100% of the time: >> while true; do make test JTREG="RETAIN=all" TEST=test/hotspot/jtreg/vmTestbase/nsk/jdi/MethodExitEvent/returnValue/returnValue003/returnValue003.java TEST_OPTS_JAVA_OPTIONS="-XX:+UseZGC -Xmx2g -XX:ZCollectionInterval=0.0001 -XX:ZFragmentationLimit=0.01 -XX:+VerifyOops -XX:+ZVerifyViews -Xint" ; done >> >> With my fix I can run this repeatedly without any more failures. I have also sanity checked the patch by running tier 1-5, so that it does not introduces any new issues on its own. I have also used Stefan's nice external GC stressing with jcmd technique that was used to trigger crashes with other GCs, to make sure said crashes no longer reproduce either. > > Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: > > Serguei CR2: Don't check interpreted only Erik, Thank you for the explanation! I agree with you, so the fix is good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/930 From cjplummer at openjdk.java.net Wed Nov 4 22:45:58 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 4 Nov 2020 22:45:58 GMT Subject: Integrated: 8255858: Add debug agent support for storing thread names In-Reply-To: References: Message-ID: <2Fkh40RilVyt-5-AAXPZqqtKaWxwWcgL7CnTfyWHVw4=.d58025de-bd58-4455-9a8b-84f8bc5d1d0a@github.com> On Tue, 3 Nov 2020 23:30:38 GMT, Chris Plummer wrote: > Simple change to add a `name` field to `ThreadNode` to aide when debugging the debug agent. It's off by default. You need to add a `#define DEBUG_THREADNAME` to enable it. This pull request has now been integrated. Changeset: 166c7283 Author: Chris Plummer URL: https://git.openjdk.java.net/jdk/commit/166c7283 Stats: 19 lines in 1 file changed: 19 ins; 0 del; 0 mod 8255858: Add debug agent support for storing thread names Reviewed-by: sspitsyn, amenkov ------------- PR: https://git.openjdk.java.net/jdk/pull/1047 From coleenp at openjdk.java.net Thu Nov 5 00:06:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 00:06:01 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v4] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 09:06:07 GMT, Erik ?sterlund wrote: >> The imasm::remove_activation() call does not deal with safepoints very well. However, when the MethodExit JVMTI event is being called, we call into the runtime in the middle of remove_activation(). If the value being returned is an object type, then the top-of-stack contains the oop. However, the GC does not traverse said oop in any oop map, because it is simply not expected that we safepoint in the middle of remove_activation(). >> >> The JvmtiExport::post_method_exit() function we end up calling, reads the top-of-stack oop, and puts it in a handle. Then it calls JVMTI callbacks, that eventually call Java and a bunch of stuff that safepoints. So after the JVMTI callback, we can expect the top-of-stack oop to be broken. Unfortunately, when we continue, we therefore end up returning a broken oop. >> >> Notably, the fact that InterpreterRuntime::post_method_exit is a JRT_ENTRY, is wrong, as we can safepoint on the way back to Java, which will break the return oop in a similar way. So this patch makes it a JRT_BLOCK_ENTRY, moving the transition to VM and back, into a block of code that is protected against GC. Before the JRT_BLOCK is called, we stash away the return oop, and after the JRT_BLOCK_END, we restore the top-of-stack oop. In the path when InterpreterRuntime::post_method_exit is called when throwing an exception, we don't have the same problem of retaining an oop result, and hence the JRT_BLOCK/JRT_BLOCK_END section is not performed in this case; the logic is the same as before for this path. >> >> This is a JVMTI bug that has probably been around for a long time. It crashes with all GCs, but was discovered recently after concurrent stack processing, as StefanK has been running better GC stressing code in JVMTI, and the bug reproduced more easily with concurrent stack processing, as the timings were a bit different. The following reproducer failed pretty much 100% of the time: >> while true; do make test JTREG="RETAIN=all" TEST=test/hotspot/jtreg/vmTestbase/nsk/jdi/MethodExitEvent/returnValue/returnValue003/returnValue003.java TEST_OPTS_JAVA_OPTIONS="-XX:+UseZGC -Xmx2g -XX:ZCollectionInterval=0.0001 -XX:ZFragmentationLimit=0.01 -XX:+VerifyOops -XX:+ZVerifyViews -Xint" ; done >> >> With my fix I can run this repeatedly without any more failures. I have also sanity checked the patch by running tier 1-5, so that it does not introduces any new issues on its own. I have also used Stefan's nice external GC stressing with jcmd technique that was used to trigger crashes with other GCs, to make sure said crashes no longer reproduce either. > > Erik ?sterlund has updated the pull request incrementally with one additional commit since the last revision: > > Serguei CR2: Don't check interpreted only Still looks good. ------------- Marked as reviewed by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/930 From coleenp at openjdk.java.net Thu Nov 5 00:06:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 00:06:03 GMT Subject: RFR: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 21:20:27 GMT, Serguei Spitsyn wrote: >> I'm not sure. There might be other cases, when remove_activation is called by the exception code. That's why I didn't want to change it to just true in this path. > > The post_method_exit can come from Zero interpreter: > src/hotspot/share/interpreter/zero/bytecodeInterpreter.cpp: > CALL_VM_NOCHECK(InterpreterRuntime::post_method_exit(THREAD)); It seems like the case of exception_exit is a condition from the call from notice_unwind_due_to_exception() and not the calls from InterpreterRuntime::post_method_exit() for both callers. But maybe if the exception is set and not caught, this post_method_exit() is called when unwinding to the exception handler. I can't tell, so leave it to be safe. ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From coleenp at openjdk.java.net Thu Nov 5 00:44:00 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 00:44:00 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c Message-ID: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. thanks, Coleen ------------- Commit messages: - 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c Changes: https://git.openjdk.java.net/jdk/pull/1067/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8254270 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1067.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1067/head:pull/1067 PR: https://git.openjdk.java.net/jdk/pull/1067 From david.holmes at oracle.com Thu Nov 5 01:05:45 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 5 Nov 2020 11:05:45 +1000 Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: Hi Coleen, On 5/11/2020 10:44 am, Coleen Phillimore wrote: > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. Sorry but I don't understand the problem or the fix. IIUC the compiler is complaining about the size of the destination buffer - yet I don't see from the local context how the compiler knows what the size of that buffer is. And we pass the size of the buffer precisely to ensure we limit the size of what is written if needed. Where did the magic numbers in "%.19s.%.3d %.50s" come from? Thanks, David ----- > thanks, > Coleen > > ------------- > > Commit messages: > - 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c > > Changes: https://git.openjdk.java.net/jdk/pull/1067/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8254270 > Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod > Patch: https://git.openjdk.java.net/jdk/pull/1067.diff > Fetch: git fetch https://git.openjdk.java.net/jdk pull/1067/head:pull/1067 > > PR: https://git.openjdk.java.net/jdk/pull/1067 > From cjplummer at openjdk.java.net Thu Nov 5 02:13:55 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 5 Nov 2020 02:13:55 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 00:37:15 GMT, Coleen Phillimore wrote: > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. > > thanks, > Coleen > Where did the magic numbers in > > "%.19s.%.3d %.50s" > > come from? The first argument is a char[DT_SIZE], which is MAXLEN_DT+1, which is 20. The +1 is probably for a null. The 3rd argument is char[TZ_SIZE], which is TIMESTAMP_SIZE - MAXLEN_DT - MAXLEN_MS, which is 81 -19 - 5, which is 56. The target buffer is char[MAXLEN_TIMESTAMP+1], which is 81 (and so is ltbuf). So without any of Coleen's changes we are writing at most 19 + 3 + 1 + 56 + 1 bytes, which is 80 bytes, into an 81 byte buffer. The compiler should not be complaining here. And just to clarify, the compiler DOES see the size of the destination buffer. It's looking at the source of the argument passed in. However, it seems to have computed some lengths incorrectly and came to the conclusion that the buffer might not be big enough to handle the known sizes of all the snprintf arguments. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From david.holmes at oracle.com Thu Nov 5 02:44:28 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 5 Nov 2020 12:44:28 +1000 Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: Hi Chris, On 5/11/2020 12:13 pm, Chris Plummer wrote: > On Thu, 5 Nov 2020 00:37:15 GMT, Coleen Phillimore wrote: > >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > >> Where did the magic numbers in >> >> "%.19s.%.3d %.50s" >> >> come from? > > The first argument is a char[DT_SIZE], which is MAXLEN_DT+1, which is 20. The +1 is probably for a null. The 3rd argument is char[TZ_SIZE], which is TIMESTAMP_SIZE - MAXLEN_DT - MAXLEN_MS, which is 81 -19 - 5, which is 56. The target buffer is char[MAXLEN_TIMESTAMP+1], which is 81 (and so is ltbuf). Right, I can see all this from the caller context now - it is what was set by 8214854. A comment to that affect would be good. > So without any of Coleen's changes we are writing at most 19 + 3 + 1 + 56 + 1 bytes, which is 80 bytes, into an 81 byte buffer. The compiler should not be complaining here. I think that (compiler is wrong) is what was concluded the last time this kind of warning came up (recently, not 8214854). We've sized the buffer exactly as needed based on the components. I'm trying to find the other bug related to this but I think we just silenced the warning in that recent case. Thanks, David ----- > And just to clarify, the compiler DOES see the size of the destination buffer. It's looking at the source of the argument passed in. However, it seems to have computed some lengths incorrectly and came to the conclusion that the buffer might not be big enough to handle the known sizes of all the snprintf arguments. > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/1067 > From shade at openjdk.java.net Thu Nov 5 09:15:58 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 5 Nov 2020 09:15:58 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: <75G_ZuR5au4Gd6jRpPJE3sgs1fGt2cFa8c9AIlxwXCQ=.71bf6645-b5c3-4c40-aa99-5089d8809d60@github.com> On Thu, 5 Nov 2020 02:11:23 GMT, Chris Plummer wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > >> Where did the magic numbers in >> >> "%.19s.%.3d %.50s" >> >> come from? > > The first argument is a char[DT_SIZE], which is MAXLEN_DT+1, which is 20. The +1 is probably for a null. The 3rd argument is char[TZ_SIZE], which is TIMESTAMP_SIZE - MAXLEN_DT - MAXLEN_MS, which is 81 -19 - 5, which is 56. The target buffer is char[MAXLEN_TIMESTAMP+1], which is 81 (and so is ltbuf). So without any of Coleen's changes we are writing at most 19 + 3 + 1 + 56 + 1 bytes, which is 80 bytes, into an 81 byte buffer. The compiler should not be complaining here. > > And just to clarify, the compiler DOES see the size of the destination buffer. It's looking at the source of the argument passed in. However, it seems to have computed some lengths incorrectly and came to the conclusion that the buffer might not be big enough to handle the known sizes of all the snprintf arguments. Please make sure `linux-x86` jobs pass in GH actions. I think you can navigate to https://github.com/coleenp/jdk/runs/1355854648 -- and press "Re-run jobs" at top right. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From redestad at openjdk.java.net Thu Nov 5 11:20:55 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Thu, 5 Nov 2020 11:20:55 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: <75G_ZuR5au4Gd6jRpPJE3sgs1fGt2cFa8c9AIlxwXCQ=.71bf6645-b5c3-4c40-aa99-5089d8809d60@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <75G_ZuR5au4Gd6jRpPJE3sgs1fGt2cFa8c9AIlxwXCQ=.71bf6645-b5c3-4c40-aa99-5089d8809d60@github.com> Message-ID: <4Vox11v8YXo5pk_dGXM2-D8QYhTAOOutELBrQAb61dc=.b3a409a3-3511-4a57-abaf-9749c983a4fd@github.com> On Thu, 5 Nov 2020 09:12:54 GMT, Aleksey Shipilev wrote: >>> Where did the magic numbers in >>> >>> "%.19s.%.3d %.50s" >>> >>> come from? >> >> The first argument is a char[DT_SIZE], which is MAXLEN_DT+1, which is 20. The +1 is probably for a null. The 3rd argument is char[TZ_SIZE], which is TIMESTAMP_SIZE - MAXLEN_DT - MAXLEN_MS, which is 81 -19 - 5, which is 56. The target buffer is char[MAXLEN_TIMESTAMP+1], which is 81 (and so is ltbuf). So without any of Coleen's changes we are writing at most 19 + 3 + 1 + 56 + 1 bytes, which is 80 bytes, into an 81 byte buffer. The compiler should not be complaining here. >> >> And just to clarify, the compiler DOES see the size of the destination buffer. It's looking at the source of the argument passed in. However, it seems to have computed some lengths incorrectly and came to the conclusion that the buffer might not be big enough to handle the known sizes of all the snprintf arguments. > > Please make sure `linux-x86` jobs pass in GH actions. I think you can navigate to https://github.com/coleenp/jdk/runs/1355854648 -- and press "Re-run jobs" at top right. FWIW I don't think I intended the workaround of specifying limits to be patched in as-is, although it seems benign to go ahead and do this. While it's possible that these warnings on 32-bit builds is just a matter of the compiler being wrong, it could also be that it can't determine all arguments are null-terminated. Specifying max string length in format specifiers seem a benign and cheap safeguard. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Thu Nov 5 12:23:53 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:23:53 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: <4Vox11v8YXo5pk_dGXM2-D8QYhTAOOutELBrQAb61dc=.b3a409a3-3511-4a57-abaf-9749c983a4fd@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <75G_ZuR5au4Gd6jRpPJE3sgs1fGt2cFa8c9AIlxwXCQ=.71bf6645-b5c3-4c40-aa99-5089d8809d60@github.com> <4Vox11v8YXo5pk_dGXM2-D8QYhTAOOutELBrQAb61dc=.b3a409a3-3511-4a57-abaf-9749c983a4fd@github.com> Message-ID: <1knGMUw_BCw8O3bv0zmHhVdTFtx3Tf6rP_yIdqWhIms=.37997de0-bf1e-407d-b651-68d88eca8042@github.com> On Thu, 5 Nov 2020 11:18:04 GMT, Claes Redestad wrote: >> Please make sure `linux-x86` jobs pass in GH actions. I think you can navigate to https://github.com/coleenp/jdk/runs/1355854648 -- and press "Re-run jobs" at top right. > > FWIW I don't think I intended the workaround of specifying limits to be patched in as-is, although it seems benign to go ahead and do this. While it's possible that these warnings on 32-bit builds is just a matter of the compiler being wrong, it could also be that it can't determine all arguments are null-terminated. Specifying max string length in format specifiers seem a benign and cheap safeguard. Thanks @dholmes-ora and @plummercj I don't know why the compiler is complaining but I'd like it to stop. If this isn't the best way to do so, please suggest another workaround. I could also open a new bug to investigate whatever the bug in the 32 bit compiler is and push this workaround, if you would like. @shipilev there's something wrong with one of the github test machines (error for non-payment) that needs to be fixed. This builds on 32 bit in our mach5 configuration. And @cl4es thank you for the suggested workaround. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Thu Nov 5 12:30:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:30:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> References: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> Message-ID: On Wed, 4 Nov 2020 10:05:29 GMT, Kim Barrett wrote: >> The comment is trying to describe the situation like: >> 1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked) >> 2. safepoint for heap walk >> 2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A. >> 3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened >> >> The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable. >> >> Both operations require the table to be locked. >> >> The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints. > > For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing, and so still effectively be a serial subtask after all the parallel subtasks are done or mostly done. Yes, you are right that the processing will be done serially and not by a parallel worker thread. This is could spawn a new GC worker thread to process the posts, as you suggest. We could do that if we find a customer that has a complaint about the pause time of this processing. >> The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table. > > I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event. Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 12:30:00 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:30:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 08:56:54 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Code review comments from Kim and Albert. > > src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36: > >> 34: class JvmtiTagMapEntryClosure; >> 35: >> 36: class JvmtiTagMapEntry : public HashtableEntry { > > By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%). > > It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost. Yes, there is 64/32 bits extra per hashtable entry with the standard hashtable implementation. It wouldn't have been hard to replace the oop object, but using shared code was a goal of this change. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 12:30:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:30:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 09:27:39 GMT, Kim Barrett wrote: >> I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.) > > It used to be that jvmti tagmap processing was all-in-one (in GC weak reference processing, with the weak clearing, dead table entry removal, and rehashing all done in one pass. This change has split that up, with the weak clearing happening in a different place (still as part of the GC's weak reference processing) than the others (which I claim can now be part of the mutator, whether further separated or not). > > "Concurrent GC" has nothing to do with whether tagmaps need rehashing. Any copying collector needs to do so. A non-copying collector (whether concurrent or not) would not. (We don't have any purely non-copying collectors, but G1 concurrent oldgen collection is non-copying.) And weak reference clearing (whether concurrent or not) has nothing to do with whether objects have been moved and so the hashing has been invalidated. > > There's also a "well known" issue with address-based hashing and generational or similar collectors, where a simple boolean "objects have moved" flag can be problematic, and which tagmaps seem likely to be prone to. The old do_weak_oops tries to mitigate it by recognizing when the object didn't move. The new rehash function also doesn't move the objects either. It essentially does the same as the old weak_oops_do function. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 12:30:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:30:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> Message-ID: On Wed, 4 Nov 2020 07:42:36 GMT, Kim Barrett wrote: >>> Ok, so I'm not sure what to do with this: >>> >>> enum Phase { >>> // Serial phase. >>> JVMTI_ONLY(jvmti) >>> // Additional implicit phase values follow for oopstorages. >>> `};` >>> >>> I've removed the only thing in this enum. >> >> Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums. > >> > Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. >> >> This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. > > I think the deletion stopped at the wrong place; it either went too far, or not far enough. I restored the empty enum for Phase and can open a new RFE if there is more code to remove. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 12:45:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:45:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 22:09:21 GMT, Serguei Spitsyn wrote: >> Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision: >> >> - Add back WeakProcessorPhases::Phase enum. >> - Serguei 1. > > Thank you for the update, Coleen! > I leave it for you to decide to refactor the gc_notification or not. > Thanks, > Serguei Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Thu Nov 5 14:22:01 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Thu, 5 Nov 2020 14:22:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 21:14:04 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 3018: >> >>> 3016: } >>> 3017: // Later GC code will relocate the oops, so defer rehashing until then. >>> 3018: tag_map->_needs_rehashing = true; >> >> This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.) >> >> For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing. >> >> (I think CMS oldgen remark may have been similar, for what that's worth.) > > Ok, so I'm going to need help to know where in all the different GCs to make this call. This seemed simpler at the expense of maybe causing a rehash at some points when it might not be necessary. For what GC is this wrong? I can see that it might yield more work than required, when performing a full GC, but not that it would do too little work. In other words, I can't see how it is wrong, as opposed to inaccurate. Littering GCs with JVMTI hooks so that we can optimize away an operation we do every young GC, from a full GC, does not really seem worth it IMO. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Thu Nov 5 14:40:02 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Thu, 5 Nov 2020 14:40:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> Message-ID: On Wed, 4 Nov 2020 13:32:07 GMT, Coleen Phillimore wrote: >> I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event. > > Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times. The point of this change was not necessarily to be lazy about updating the tagmap, until someone uses it. The point was to get rid of the last annoying serial GC phase. Doing it all lazily would certainly also achieve that. But it would also lead to situations where no event is ever posted from GC to GC. So you would get the event 20 GCs later, which might come as a surprise. It did come as a surprise to some tests, so it is reasonable to assume it would come as a surprise to users too. And I don't think we want such surprises unless we couldn't deal with them. And we can. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Thu Nov 5 14:53:01 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Thu, 5 Nov 2020 14:53:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> Message-ID: On Wed, 4 Nov 2020 13:22:57 GMT, Coleen Phillimore wrote: >> For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing, and so still effectively be a serial subtask after all the parallel subtasks are done or mostly done. > > Yes, you are right that the processing will be done serially and not by a parallel worker thread. This is could spawn a new GC worker thread to process the posts, as you suggest. We could do that if we find a customer that has a complaint about the pause time of this processing. So both before and now, this task is a single threaded task. The difference is that before that single threaded task could be performed in parallel to other tasks. So if the table is small, you probably won't be able to notice any difference as small table implies not much to do. And if the table is large, you still probably won't be able to notice any difference as a large table implies it will dominate the pause with both the old and new approach. Any difference at all is bounded at 2x processing time, as it was serial both before and after. But now if we have a perfectly medium balanced table, we can at the very worst observe a theoretical 2x worse processing of this JVMTI table. I think that if we truly did care about this difference, and that it is important to keep this code as well performed as possible, then we would not have a serial phase for this at all. The fact that this has been serial suggests to me that it is not a path that is critical, and therefore I don't think op timizing the theoretical max 2x worse processing times for perfectly medium sized JVMTI tag map tables, is worth the hassle. At least I can't see why this would be of any importance. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 15:07:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 15:07:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: <_w3Wb6lCIkBeH6UxIGCZ-HoXgwx-qSYF6KpLY1txy6Y=.eeeb53b9-6b1f-41cb-b59b-109b714e8eed@github.com> On Wed, 4 Nov 2020 13:19:21 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36: >> >>> 34: class JvmtiTagMapEntryClosure; >>> 35: >>> 36: class JvmtiTagMapEntry : public HashtableEntry { >> >> By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%). >> >> It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost. > > Yes, there is 64/32 bits extra per hashtable entry with the standard hashtable implementation. It wouldn't have been hard to replace the oop object, but using shared code was a goal of this change. So looking at the concurrent hashtable, the entries can be created without saving the hashcode. I was going to use that at first but didn't want to cut/paste the boilerplate to do so and the jvmti tag map hashtable is always accessed with a lock. This could be a future RFE if necessary and would also serve to eliminate another ad-hoc hashtable. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From redestad at openjdk.java.net Thu Nov 5 15:55:56 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Thu, 5 Nov 2020 15:55:56 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 00:37:15 GMT, Coleen Phillimore wrote: > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. > > thanks, > Coleen While you might want to wait for someone to suggest a better alternative, this LGTM. As you say the x86 failures on GH look related to the setup of the build image and thus unrelated to this patch. ------------- Marked as reviewed by redestad (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1067 From eosterlund at openjdk.java.net Thu Nov 5 16:21:56 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Thu, 5 Nov 2020 16:21:56 GMT Subject: Integrated: 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop In-Reply-To: References: Message-ID: On Thu, 29 Oct 2020 12:44:58 GMT, Erik ?sterlund wrote: > The imasm::remove_activation() call does not deal with safepoints very well. However, when the MethodExit JVMTI event is being called, we call into the runtime in the middle of remove_activation(). If the value being returned is an object type, then the top-of-stack contains the oop. However, the GC does not traverse said oop in any oop map, because it is simply not expected that we safepoint in the middle of remove_activation(). > > The JvmtiExport::post_method_exit() function we end up calling, reads the top-of-stack oop, and puts it in a handle. Then it calls JVMTI callbacks, that eventually call Java and a bunch of stuff that safepoints. So after the JVMTI callback, we can expect the top-of-stack oop to be broken. Unfortunately, when we continue, we therefore end up returning a broken oop. > > Notably, the fact that InterpreterRuntime::post_method_exit is a JRT_ENTRY, is wrong, as we can safepoint on the way back to Java, which will break the return oop in a similar way. So this patch makes it a JRT_BLOCK_ENTRY, moving the transition to VM and back, into a block of code that is protected against GC. Before the JRT_BLOCK is called, we stash away the return oop, and after the JRT_BLOCK_END, we restore the top-of-stack oop. In the path when InterpreterRuntime::post_method_exit is called when throwing an exception, we don't have the same problem of retaining an oop result, and hence the JRT_BLOCK/JRT_BLOCK_END section is not performed in this case; the logic is the same as before for this path. > > This is a JVMTI bug that has probably been around for a long time. It crashes with all GCs, but was discovered recently after concurrent stack processing, as StefanK has been running better GC stressing code in JVMTI, and the bug reproduced more easily with concurrent stack processing, as the timings were a bit different. The following reproducer failed pretty much 100% of the time: > while true; do make test JTREG="RETAIN=all" TEST=test/hotspot/jtreg/vmTestbase/nsk/jdi/MethodExitEvent/returnValue/returnValue003/returnValue003.java TEST_OPTS_JAVA_OPTIONS="-XX:+UseZGC -Xmx2g -XX:ZCollectionInterval=0.0001 -XX:ZFragmentationLimit=0.01 -XX:+VerifyOops -XX:+ZVerifyViews -Xint" ; done > > With my fix I can run this repeatedly without any more failures. I have also sanity checked the patch by running tier 1-5, so that it does not introduces any new issues on its own. I have also used Stefan's nice external GC stressing with jcmd technique that was used to trigger crashes with other GCs, to make sure said crashes no longer reproduce either. This pull request has now been integrated. Changeset: 3a02578b Author: Erik ?sterlund URL: https://git.openjdk.java.net/jdk/commit/3a02578b Stats: 58 lines in 3 files changed: 42 ins; 12 del; 4 mod 8255452: Doing GC during JVMTI MethodExit event posting breaks return oop Reviewed-by: coleenp, dlong, rrich, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/930 From dcubed at openjdk.java.net Thu Nov 5 16:59:09 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 5 Nov 2020 16:59:09 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM Message-ID: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Changes from @fisk and @dcubed-ojdk to: - simplify ObjectMonitor list management - get rid of Type-Stable Memory (TSM) This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) - a few minor regressions (<= -0.24%) - Volano is 6.8% better Eric C. has also running promotion perf runs on these bits and says "the results look fine". ------------- Commit messages: - 8253064.v00.part2 - 8253064.v00.part1 Changes: https://git.openjdk.java.net/jdk/pull/642/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8253064 Stats: 2510 lines in 25 files changed: 604 ins; 1721 del; 185 mod Patch: https://git.openjdk.java.net/jdk/pull/642.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/642/head:pull/642 PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Thu Nov 5 16:59:09 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 5 Nov 2020 16:59:09 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Tue, 13 Oct 2020 20:31:44 GMT, Daniel D. Daugherty wrote: > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also running promotion perf runs on these bits and says "the results look fine". Self review done. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Thu Nov 5 16:59:10 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 5 Nov 2020 16:59:10 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: <6eZv40x8Nr--K-r6YCSihm0z_vX_KkXOIskxyYNgo70=.dbc7a86a-46a0-42e7-b9ae-da8802113105@github.com> On Mon, 2 Nov 2020 22:13:30 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also running promotion perf runs on these bits and says "the results look fine". > > Self review done. ### Gory details about these changes from @fisk and @dcubed-ojdk: ### Simplify `ObjectMonitor` List Management: - delete per-thread in-use and free-lists. - delete global free-list and global wait-list; there is a still a global in-use list; after `ObjectMonitor`s on the global in-use list are deflated, they are unlinked and added to a function local `GrowableArray` called `delete_list`; we do a handshake/safepoint with all JavaThreads and that makes all the `ObjectMonitor`s on the `delete_list` safe for deletion; lastly, we delete all the `ObjectMonitor`s on `delete_list`. - move async deflation work from the `ServiceThread` to a dedicated `MonitorDeflationThread`; this prevents `ObjectMonitor` inflation storms from delaying the work done by the `ServiceThread` for other subsystems; this means the `ServiceThread` no longer wakes up every `GuaranteedSafepointInterval` to check for work. - the `AllocationState` enum is dropped along with the `_allocation_state` field and associated getters and setters; the simpler list management no longer requires the allocation state to be tracked. - the safepoint cleanup phase no longer requests async monitor deflation; there is no longer a safepoint cleanup task for monitor deflation, but there is still an auditing/logging hook for debugging purposes. - delete ObjectSynchronizer functions associated with more complicated list management: `deflate_global_idle_monitors()`, `deflate_per_thread_idle_monitors()`, `deflate_common_idle_monitors()`, `om_flush()`, `prepend_list_to_common()`, `prepend_list_to_global_free_list()`, `prepend_list_to_global_wait_list()`, `prepend_list_to_global_in_use_list()`, `prepend_to_common()`, `prepend_to_om_free_list()`, `prepend_to_om_in_use_list()`, `take_from_start_of_common()`, `take_from_start_of_global_free_list()`, `take_from_start_of_om_free_list()` - delete the spin-lock functions needed by the more complicated list management. - delete a number of audit/debug/logging related functions needed by the more complicated list management. - restore the barrier related code that needed relocation due to om_flush()'s access of the weak obj reference; now that om_flush() is gone, the barrier related code can go back to its more natural place. ### Get Rid of Type-Stable Memory (TSM): - `ObjectMonitor` now subclasses `CHeapObj`. - the `ObjectMonitor` constructor and destructor are now more normal C++! - delete `ObjectMonitor` functions associated with TSM: `clear()`, `clear_common()`, `object_addr()`, `Recycle()`, and `set_object()`. - delete the version of `set_owner_from()` that support two possible old values since it is no longer needed; we are not recycling deflated `ObjectMonitor`s anymore so there's no longer a possibility of a `NULL` `_owner` value or a `DEFLATER_MARKER` value on the same code path. - delete ObjectSynchronizer functions associated with TSM: `om_alloc()`, `om_release()`, `prepend_block_to_lists()` - simplify ObjectSynchronizer functions related to TSM: `deflate_idle_monitors()`, `deflate_monitor_list()`, `inflate()` ### Change A Displaced Header is Always at Offset 0 - Change `markWord::displaced_mark_helper()` and `markWord::set_displaced_mark_helper()` to no longer assume that the displaced header in a `BasicLock` or `ObjectMonitor` is at offset 0. - ObjectMonitor::header_addr() no longer requires the offset to be zero. ### New Diagnostic Options - `AvgMonitorsPerThreadEstimate` - Used to estimate a variable ceiling based on number of threads for use with `MonitorUsedDeflationThreshold`; default is 1024, 0 is off, range is 0..max_jint. The current count of inflated `ObjectMonitor`s and the ceiling are used to determine whether the in-use ratio is higher than `MonitorUsedDeflationThreshold` (default 90). - `MonitorDeflationMax` - The maximum number of `ObjectMonitor`s to deflate, unlink and delete at one time; default is 1 million; range is 1024..max_jint. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Thu Nov 5 16:59:10 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 5 Nov 2020 16:59:10 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <6eZv40x8Nr--K-r6YCSihm0z_vX_KkXOIskxyYNgo70=.dbc7a86a-46a0-42e7-b9ae-da8802113105@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <6eZv40x8Nr--K-r6YCSihm0z_vX_KkXOIskxyYNgo70=.dbc7a86a-46a0-42e7-b9ae-da8802113105@github.com> Message-ID: On Mon, 2 Nov 2020 22:13:41 GMT, Daniel D. Daugherty wrote: >> Self review done. > > ### Gory details about these changes from @fisk and @dcubed-ojdk: > > ### Simplify `ObjectMonitor` List Management: > > - delete per-thread in-use and free-lists. > - delete global free-list and global wait-list; there is a still a global in-use list; after `ObjectMonitor`s on the global in-use list are deflated, they are unlinked and added to a function local `GrowableArray` called `delete_list`; we do a handshake/safepoint with all JavaThreads and that makes all the `ObjectMonitor`s on the `delete_list` safe for deletion; lastly, we delete all the `ObjectMonitor`s on `delete_list`. > - move async deflation work from the `ServiceThread` to a dedicated `MonitorDeflationThread`; this prevents `ObjectMonitor` inflation storms from delaying the work done by the `ServiceThread` for other subsystems; this means the `ServiceThread` no longer wakes up every `GuaranteedSafepointInterval` to check for work. > - the `AllocationState` enum is dropped along with the `_allocation_state` field and associated getters and setters; the simpler list management no longer requires the allocation state to be tracked. > - the safepoint cleanup phase no longer requests async monitor deflation; there is no longer a safepoint cleanup task for monitor deflation, but there is still an auditing/logging hook for debugging purposes. > - delete ObjectSynchronizer functions associated with more complicated list management: `deflate_global_idle_monitors()`, `deflate_per_thread_idle_monitors()`, `deflate_common_idle_monitors()`, `om_flush()`, `prepend_list_to_common()`, `prepend_list_to_global_free_list()`, `prepend_list_to_global_wait_list()`, `prepend_list_to_global_in_use_list()`, `prepend_to_common()`, `prepend_to_om_free_list()`, `prepend_to_om_in_use_list()`, `take_from_start_of_common()`, `take_from_start_of_global_free_list()`, `take_from_start_of_om_free_list()` > - delete the spin-lock functions needed by the more complicated list management. > - delete a number of audit/debug/logging related functions needed by the more complicated list management. > - restore the barrier related code that needed relocation due to om_flush()'s access of the weak obj reference; now that om_flush() is gone, the barrier related code can go back to its more natural place. > > ### Get Rid of Type-Stable Memory (TSM): > > - `ObjectMonitor` now subclasses `CHeapObj`. > - the `ObjectMonitor` constructor and destructor are now more normal C++! > - delete `ObjectMonitor` functions associated with TSM: `clear()`, `clear_common()`, `object_addr()`, `Recycle()`, and `set_object()`. > - delete the version of `set_owner_from()` that support two possible old values since it is no longer needed; we are not recycling deflated `ObjectMonitor`s anymore so there's no longer a possibility of a `NULL` `_owner` value or a `DEFLATER_MARKER` value on the same code path. > - delete ObjectSynchronizer functions associated with TSM: `om_alloc()`, `om_release()`, `prepend_block_to_lists()` > - simplify ObjectSynchronizer functions related to TSM: `deflate_idle_monitors()`, `deflate_monitor_list()`, `inflate()` > > ### Change A Displaced Header is Always at Offset 0 > > - Change `markWord::displaced_mark_helper()` and `markWord::set_displaced_mark_helper()` to no longer assume that the displaced header in a `BasicLock` or `ObjectMonitor` is at offset 0. > - ObjectMonitor::header_addr() no longer requires the offset to be zero. > > ### New Diagnostic Options > > - `AvgMonitorsPerThreadEstimate` - Used to estimate a variable ceiling based on number of threads for use with `MonitorUsedDeflationThreshold`; default is 1024, 0 is off, range is 0..max_jint. The current count of inflated `ObjectMonitor`s and the ceiling are used to determine whether the in-use ratio is higher than `MonitorUsedDeflationThreshold` (default 90). > - `MonitorDeflationMax` - The maximum number of `ObjectMonitor`s to deflate, unlink and delete at one time; default is 1 million; range is 1024..max_jint. Rebased the project to jdk-16+23. Local macOS and Linux-X64 builds and KitchensinkSanity runs pass. Kicking off a new round of Mach5 testing... @coleenp, @dholmes-ora, @fisk, and @robehn - as usual for ObjectMonitor stuff, your reviews would be greatly appreciated. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From cjplummer at openjdk.java.net Thu Nov 5 17:29:58 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 5 Nov 2020 17:29:58 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 15:52:59 GMT, Claes Redestad wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > While you might want to wait for someone to suggest a better alternative, this LGTM. As you say the x86 failures on GH look related to the setup of the build image and thus unrelated to this patch. > While it's possible that these warnings on 32-bit builds is just a matter of the compiler being wrong, it could also be that it can't determine all arguments are null-terminated. Specifying max string length in format specifiers seem a benign and cheap safeguard. I don't think this is the case. If you assume the arguments are not null terminated, then there is no limit to how long the string could be, where-as the error messages are very specific with the (incorrectly) calculated range of potential overflow. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From redestad at openjdk.java.net Thu Nov 5 17:59:55 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Thu, 5 Nov 2020 17:59:55 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 17:26:50 GMT, Chris Plummer wrote: > I don't think this is the case. If you assume the arguments are not null terminated, then there is no limit to how long the string could be, where-as the error messages are very specific with the (incorrectly) calculated range of potential overflow. I must be missing what you're suggesting here? ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From stuefe at openjdk.java.net Thu Nov 5 18:25:57 2020 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Thu, 5 Nov 2020 18:25:57 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 17:57:29 GMT, Claes Redestad wrote: >>> While it's possible that these warnings on 32-bit builds is just a matter of the compiler being wrong, it could also be that it can't determine all arguments are null-terminated. Specifying max string length in format specifiers seem a benign and cheap safeguard. >> >> I don't think this is the case. If you assume the arguments are not null terminated, then there is no limit to how long the string could be, where-as the error messages are very specific with the (incorrectly) calculated range of potential overflow. > >> I don't think this is the case. If you assume the arguments are not null terminated, then there is no limit to how long the string could be, where-as the error messages are very specific with the (incorrectly) calculated range of potential overflow. > > I must be missing what you're suggesting here? Hi, I think the compiler is correct. The 3 in "%.3d" of the second argument is precision, not scale. Which is the *minimum* numbers of digits to be printed. So we get: (void)snprintf(tbuf, ltbuf, "%s.%.3d %s", timestamp_date_time, (int)(millisecs), timestamp_timezone); with timestamp_date_time = DT_SIZE = 20 and timestamp_timezone = TZ_SIZE = 56 and the second argument prints at least three digits but possibly more if the argument is longer. MAX_INT is 2147483647. So length could be 10. 56 + 20 + 10 = 86. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From stuefe at openjdk.java.net Thu Nov 5 18:39:00 2020 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Thu, 5 Nov 2020 18:39:00 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 18:22:56 GMT, Thomas Stuefe wrote: >>> I don't think this is the case. If you assume the arguments are not null terminated, then there is no limit to how long the string could be, where-as the error messages are very specific with the (incorrectly) calculated range of potential overflow. >> >> I must be missing what you're suggesting here? > > Hi, > > I think the compiler is correct. The 3 in "%.3d" of the second argument is precision, not scale. Which is the *minimum* numbers of digits to be printed. > > So we get: > > (void)snprintf(tbuf, ltbuf, > "%s.%.3d %s", timestamp_date_time, > (int)(millisecs), timestamp_timezone); > > with timestamp_date_time = DT_SIZE = 20 and timestamp_timezone = TZ_SIZE = 56 > > and the second argument prints at least three digits but possibly more if the argument is longer. MAX_INT is 2147483647. So length could be 10. > > 56 + 20 + 10 = 86. ... so the problem would be that the compiler does not believe us that millisecs will be always <1000. And there is no way to truncate output for numerical format specifiers. One solution could be to first print the millisecs to a buffer large enough to hold MAX_INT. And then print that buffer as a string, which can be truncated with precision. char tmp[10 + 1]; snprintf(tmp, sizeof(tmp), "%d", millisecs); snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); That may be enough to shut the compiler up. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From redestad at openjdk.java.net Thu Nov 5 19:43:57 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Thu, 5 Nov 2020 19:43:57 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 18:35:52 GMT, Thomas Stuefe wrote: > ... so the problem would be that the compiler does not believe us that millisecs will be always <1000. And there is no way to truncate output for numerical format specifiers. Interesting, and a reasonable explanation. Odd this doesn't get caught by the 64-bit build > > One solution could be to first print the millisecs to a buffer large enough to hold MAX_INT. And then print that buffer as a string, which can be truncated with precision. > > ``` > char tmp[10 + 1]; > snprintf(tmp, sizeof(tmp), "%d", millisecs); > snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); > ``` > > That may be enough to shut the compiler up. It'd be interesting to see if statically limiting the value printed to be < 1000 would also convince the compiler, e.g. `snprintf(tbuf, ltbuf, "%s.%.3d %s", timestamp_date_time, millisecs % 1000, timestamp_timezone);` (I'm not sure a perfect solution is worth our while here, but seeing we've gotten this far down the rabbit hole..) ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Thu Nov 5 20:12:06 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 20:12:06 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v2] In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. > > thanks, > Coleen Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Use Thomas's fix instead. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1067/files - new: https://git.openjdk.java.net/jdk/pull/1067/files/cfaee8cb..9d01a505 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=00-01 Stats: 7 lines in 1 file changed: 7 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1067.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1067/head:pull/1067 PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Thu Nov 5 20:12:07 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 20:12:07 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v2] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 19:41:25 GMT, Claes Redestad wrote: >> ... so the problem would be that the compiler does not believe us that millisecs will be always <1000. And there is no way to truncate output for numerical format specifiers. >> >> One solution could be to first print the millisecs to a buffer large enough to hold MAX_INT. And then print that buffer as a string, which can be truncated with precision. >> >> char tmp[10 + 1]; >> snprintf(tmp, sizeof(tmp), "%d", millisecs); >> snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); >> >> That may be enough to shut the compiler up. > >> ... so the problem would be that the compiler does not believe us that millisecs will be always <1000. And there is no way to truncate output for numerical format specifiers. > > Interesting, and a reasonable explanation. Odd this doesn't get caught by the 64-bit build > >> >> One solution could be to first print the millisecs to a buffer large enough to hold MAX_INT. And then print that buffer as a string, which can be truncated with precision. >> >> ``` >> char tmp[10 + 1]; >> snprintf(tmp, sizeof(tmp), "%d", millisecs); >> snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); >> ``` >> >> That may be enough to shut the compiler up. > > It'd be interesting to see if statically limiting the value printed to be < 1000 would also convince the compiler, e.g. > `snprintf(tbuf, ltbuf, "%s.%.3d %s", timestamp_date_time, millisecs % 1000, timestamp_timezone);` > > (I'm not sure a perfect solution is worth our while here, but seeing we've gotten this far down the rabbit hole..) I didn't see your comment @cl4es and I tested out Thomas's solution and that compiles and seems to make sense. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From cjplummer at openjdk.java.net Thu Nov 5 20:26:54 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 5 Nov 2020 20:26:54 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v2] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 18:35:52 GMT, Thomas Stuefe wrote: >> Hi, >> >> I think the compiler is correct. The 3 in "%.3d" of the second argument is precision, not scale. Which is the *minimum* numbers of digits to be printed. >> >> So we get: >> >> (void)snprintf(tbuf, ltbuf, >> "%s.%.3d %s", timestamp_date_time, >> (int)(millisecs), timestamp_timezone); >> >> with timestamp_date_time = DT_SIZE = 20 and timestamp_timezone = TZ_SIZE = 56 >> >> and the second argument prints at least three digits but possibly more if the argument is longer. MAX_INT is 2147483647. So length could be 10. >> >> 56 + 20 + 10 = 86. > > ... so the problem would be that the compiler does not believe us that millisecs will be always <1000. And there is no way to truncate output for numerical format specifiers. > > One solution could be to first print the millisecs to a buffer large enough to hold MAX_INT. And then print that buffer as a string, which can be truncated with precision. > > char tmp[10 + 1]; > snprintf(tmp, sizeof(tmp), "%d", millisecs); > snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); > > That may be enough to shut the compiler up. What @tstuefe says makes sense and is the reason why the error messages are specific about how much the value might overflow. So really the following calculation isn't right in the theoretical case, but we know it is in practice: 47 #define MAXLEN_MS 5 // ".mmm " Since @coleenp fix seems to be working, this would be further proof that the issue lies here. I'm ok with the latest version of the fix, although it might be simpler to just update MAXLEN_MS to 10 ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Thu Nov 5 20:33:08 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 20:33:08 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. > > thanks, > Coleen Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Use Thomas's fix instead. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1067/files - new: https://git.openjdk.java.net/jdk/pull/1067/files/9d01a505..24939e26 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=01-02 Stats: 10 lines in 1 file changed: 0 ins; 5 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/1067.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1067/head:pull/1067 PR: https://git.openjdk.java.net/jdk/pull/1067 From cjplummer at openjdk.java.net Thu Nov 5 20:33:09 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 5 Nov 2020 20:33:09 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v2] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 20:12:06 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Use Thomas's fix instead. Marked as reviewed by cjplummer (Reviewer). src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 85: > 83: "%.19s.%.3d %.50s", timestamp_date_time, > 84: (int)(millisecs), timestamp_timezone); > 85: #endif Don't forget to remove the old code. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From redestad at openjdk.java.net Thu Nov 5 20:33:10 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Thu, 5 Nov 2020 20:33:10 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> Message-ID: On Thu, 5 Nov 2020 20:30:05 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Use Thomas's fix instead. Marked as reviewed by redestad (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Thu Nov 5 20:36:59 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 20:36:59 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v2] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 20:27:11 GMT, Chris Plummer wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Use Thomas's fix instead. > > src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 85: > >> 83: "%.19s.%.3d %.50s", timestamp_date_time, >> 84: (int)(millisecs), timestamp_timezone); >> 85: #endif > > Don't forget to remove the old code. Yes, got it. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From cjplummer at openjdk.java.net Thu Nov 5 21:10:03 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 5 Nov 2020 21:10:03 GMT Subject: RFR: 8255706: The JDWP debug agent unecessarily checks for JVMTI_ERROR_INTERRUPT after calling RawMonitorEnter In-Reply-To: References: Message-ID: <_FqKnKQZKDgcYou0hHxL7K_jwiaE9vEqdl-ueSMVcwU=.d574dd62-c7ad-45b8-a83e-b215d3665267@github.com> On Tue, 3 Nov 2020 07:20:34 GMT, Alan Bateman wrote: >> Remove code that retries if RawMonitorEnter is interrupted since that can't happen: >> >> https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#RawMonitorEnter > > Marked as reviewed by alanb (Reviewer). Ping! I could use one more review. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1022 From dholmes at openjdk.java.net Thu Nov 5 22:15:02 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 5 Nov 2020 22:15:02 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> Message-ID: On Thu, 5 Nov 2020 20:33:08 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Use Thomas's fix instead. src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 82: > 80: "%Z", localtime(&t)); > 81: // Truncate milliseconds in buffer large enough to hold the > 82: // value which is always < 1000 I initially missed the %d to %s change. Can we augment the comment to say: // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") Also I'm not clear whether this will result in extra spaces when the ms value is < 100 ? ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From redestad at openjdk.java.net Thu Nov 5 22:25:02 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Thu, 5 Nov 2020 22:25:02 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> Message-ID: On Thu, 5 Nov 2020 22:11:44 GMT, David Holmes wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Use Thomas's fix instead. > > src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 82: > >> 80: "%Z", localtime(&t)); >> 81: // Truncate milliseconds in buffer large enough to hold the >> 82: // value which is always < 1000 > > I initially missed the %d to %s change. Can we augment the comment to say: > > // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") > > Also I'm not clear whether this will result in extra spaces when the ms value is < 100 ? Right, `snprintf(tmp, sizeof(tmp), "%d", millisecs);` needs to be `snprintf(tmp, sizeof(tmp), "%.3d", millisecs);` to pad it out correctly. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From dholmes at openjdk.java.net Thu Nov 5 22:27:57 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 5 Nov 2020 22:27:57 GMT Subject: RFR: 8255706: The JDWP debug agent unecessarily checks for JVMTI_ERROR_INTERRUPT after calling RawMonitorEnter In-Reply-To: References: Message-ID: <3mxrufx6T8dcFXVjWy3LsJJJVvdzHX1FL3RVCxGekYk=.ab915cf8-6a0e-4c24-a83b-028e5f3fdb6f@github.com> On Mon, 2 Nov 2020 22:11:48 GMT, Chris Plummer wrote: > Remove code that retries if RawMonitorEnter is interrupted since that can't happen: > > https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#RawMonitorEnter LGTM! Thanks, David ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1022 From sspitsyn at openjdk.java.net Thu Nov 5 23:02:56 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 5 Nov 2020 23:02:56 GMT Subject: RFR: 8255706: The JDWP debug agent unecessarily checks for JVMTI_ERROR_INTERRUPT after calling RawMonitorEnter In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 22:11:48 GMT, Chris Plummer wrote: > Remove code that retries if RawMonitorEnter is interrupted since that can't happen: > > https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#RawMonitorEnter LGTM++ ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1022 From cjplummer at openjdk.java.net Thu Nov 5 23:20:56 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 5 Nov 2020 23:20:56 GMT Subject: Integrated: 8255706: The JDWP debug agent unecessarily checks for JVMTI_ERROR_INTERRUPT after calling RawMonitorEnter In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 22:11:48 GMT, Chris Plummer wrote: > Remove code that retries if RawMonitorEnter is interrupted since that can't happen: > > https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#RawMonitorEnter This pull request has now been integrated. Changeset: e42c1340 Author: Chris Plummer URL: https://git.openjdk.java.net/jdk/commit/e42c1340 Stats: 10 lines in 1 file changed: 0 ins; 7 del; 3 mod 8255706: The JDWP debug agent unecessarily checks for JVMTI_ERROR_INTERRUPT after calling RawMonitorEnter Reviewed-by: alanb, dholmes, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/1022 From amenkov at openjdk.java.net Fri Nov 6 00:31:56 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 6 Nov 2020 00:31:56 GMT Subject: RFR: 8254864: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java timed out In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 21:44:09 GMT, Chris Plummer wrote: > > > > Had to implement runtime check (instead of using @requires) because resexhausted001 is also called by resexhausted004. > > So does that mean you do or don't want resexhausted004 disabled on windows? If yes, there's no reason why you can't use @requires in the TestDescriptions.java for each of these tests (although I have no objecting to the runtime check). If no then you have a bug because you are disabling resexhausted004 on windows, and should use @requires in TestDescriptions.java for resexhausted001 instead. resexhausted004 calls resexhausted001-003 (their run method) in random order (to ensure we can get several ResourceExhausted events) I don't want to disable resexhausted004 on Windows completely, just skip resexhausted001 ------------- PR: https://git.openjdk.java.net/jdk/pull/1046 From coleenp at openjdk.java.net Fri Nov 6 00:57:58 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 00:57:58 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> Message-ID: On Thu, 5 Nov 2020 22:22:24 GMT, Claes Redestad wrote: >> src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 82: >> >>> 80: "%Z", localtime(&t)); >>> 81: // Truncate milliseconds in buffer large enough to hold the >>> 82: // value which is always < 1000 >> >> I initially missed the %d to %s change. Can we augment the comment to say: >> >> // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") >> >> Also I'm not clear whether this will result in extra spaces when the ms value is < 100 ? > > Right, `snprintf(tmp, sizeof(tmp), "%d", millisecs);` needs to be > `snprintf(tmp, sizeof(tmp), "%.3d", millisecs);` to pad it out correctly. timestamp 05.11.2020 19:54:13.100 EST This is what I get (in a small test program) for 100 ms with the existing code. Is this not right? ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 01:04:09 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 01:04:09 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v4] In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. > > thanks, > Coleen Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Adjust millisecond format. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1067/files - new: https://git.openjdk.java.net/jdk/pull/1067/files/24939e26..245c7f7f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=02-03 Stats: 4 lines in 1 file changed: 1 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1067.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1067/head:pull/1067 PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 01:04:09 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 01:04:09 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> Message-ID: On Fri, 6 Nov 2020 00:55:22 GMT, Coleen Phillimore wrote: >> Right, `snprintf(tmp, sizeof(tmp), "%d", millisecs);` needs to be >> `snprintf(tmp, sizeof(tmp), "%.3d", millisecs);` to pad it out correctly. > > timestamp 05.11.2020 19:54:13.100 EST > > This is what I get (in a small test program) for 100 ms with the existing code. Is this not right? char tmp[10 + 1]; snprintf(tmp, sizeof(tmp), "%.3d", millisecs); snprintf(tbuf, ltbuf, "%s.%s %s", timestamp_date_time, tmp, timestamp_timezone); This also gets the same thing. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From cjplummer at openjdk.java.net Fri Nov 6 01:04:09 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 6 Nov 2020 01:04:09 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> Message-ID: <-15R3IgetUfb1LOP5zEiG6K1dTsdGPhbUj6A0GttbyA=.362211b5-4282-4f69-8ec5-00b18fe69d02@github.com> On Fri, 6 Nov 2020 00:58:01 GMT, Coleen Phillimore wrote: >> timestamp 05.11.2020 19:54:13.100 EST >> >> This is what I get (in a small test program) for 100 ms with the existing code. Is this not right? > > char tmp[10 + 1]; > snprintf(tmp, sizeof(tmp), "%.3d", millisecs); > snprintf(tbuf, ltbuf, "%s.%s %s", timestamp_date_time, tmp, timestamp_timezone); > This also gets the same thing. The concern is when it is less than 100ms. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From cjplummer at openjdk.java.net Fri Nov 6 01:08:56 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 6 Nov 2020 01:08:56 GMT Subject: RFR: 8254864: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java timed out In-Reply-To: References: Message-ID: <2ISAYrBJBlik-8iFI9HELbdt9uL02tRTPkd2U2cVfg8=.a40e7f35-3879-405e-afe6-62365e226da2@github.com> On Tue, 3 Nov 2020 23:16:50 GMT, Alex Menkov wrote: > - Fixed synchronization logic in resexhausted001; > - On Windows JVM cannot produce OOM caused by thread creation failure. Massive thread creation causes big system (not VM) memory consumption, as a result test host dramatically slows down up to hang. Specifying small heap size we can get RESOURCE_EXHAUSTED_JAVA_HEAP, but this is not the goal of resexhausted001. > So I disabled resexhausted001 on Windows. > Had to implement runtime check (instead of using @requires) because resexhausted001 is also called by resexhausted004. > - Additionally resexhausted001-resexhausted003 test were improved to verify JVMTI generates ResourceExhausted event with correct flag. Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1046 From dholmes at openjdk.java.net Fri Nov 6 01:23:01 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Fri, 6 Nov 2020 01:23:01 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v4] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Fri, 6 Nov 2020 01:04:09 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Adjust millisecond format. Marked as reviewed by dholmes (Reviewer). src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 83: > 81: > 82: // Truncate milliseconds in buffer large enough to hold the > 83: // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") With the updated code the comment is no longer correct - change %.3s to %.3d src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 85: > 83: // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") > 84: char tmp[10 + 1]; > 85: snprintf(tmp, sizeof(tmp), "%.3d", millisecs); Yes this works! ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From dholmes at openjdk.java.net Fri Nov 6 02:44:02 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Fri, 6 Nov 2020 02:44:02 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> On Tue, 13 Oct 2020 20:31:44 GMT, Daniel D. Daugherty wrote: > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also done promotion perf runs on these bits and says "the results look fine". Hi Dan, Overall this looks great. Comparing old and new code is complex but the new code on its own is generally much simpler/clearer (not all though :) ). I have a few nits, comments and queries below. Thanks, David src/hotspot/share/runtime/monitorDeflationThread.hpp line 44: > 42: > 43: // Hide this thread from external view. > 44: bool is_hidden_from_external_view() const { return true; } Style nit: a single space after "const" suffices src/hotspot/share/runtime/objectMonitor.cpp line 380: > 378: if (event.should_commit()) { > 379: event.set_monitorClass(object()->klass()); > 380: event.set_address((uintptr_t)this); This looks wrong - the event should refer to the Object whose monitor we have entered, not the OM itself. src/hotspot/share/runtime/objectMonitor.cpp line 1472: > 1470: event->set_monitorClass(monitor->object()->klass()); > 1471: event->set_timeout(timeout); > 1472: event->set_address((uintptr_t)monitor); Again the event should refer to the Object, not the OM. src/hotspot/share/runtime/objectMonitor.hpp line 145: > 143: // have busy multi-threaded access. _header and _object are set at initial > 144: // inflation. The _object does not change, so it is a good choice to share > 145: // its the cache line with _header. Typo: its the src/hotspot/share/runtime/synchronizer.cpp line 60: > 58: > 59: void MonitorList::add(ObjectMonitor* m) { > 60: for (;;) { Style nit: a do/while loop would like nicer here. src/hotspot/share/runtime/synchronizer.cpp line 94: > 92: // Find next live ObjectMonitor. > 93: ObjectMonitor* next = m; > 94: while (next != NULL && next->is_being_async_deflated()) { Nit: This loop seems odd. Given we know m is_being_async_deflated, this should either be a do/while loop, or else we should initialize: ObjectMonitor* next = m->next_om(); and dispense with the awkwardly named next_next. src/hotspot/share/runtime/synchronizer.cpp line 126: > 124: } > 125: > 126: if (self->is_Java_thread() && Non-JavaThreads may have a monitor list ??? src/hotspot/share/runtime/synchronizer.cpp line 136: > 134: > 135: // Honor block request. > 136: ThreadBlockInVM tbivm(self->as_Java_thread()); ThreadBlockInVM is generally used to wrap blocking code, not to cause the current thread to block (which it will do as a side-effect if a safepoint/handshake is requested). Surely this should just be call to `process_if_requested` (or the new `process_if_requested_with_exit_check`)? src/hotspot/share/runtime/synchronizer.cpp line 1425: > 1423: } > 1424: > 1425: if (self->is_Java_thread() && Again unclear how a non-JavaThread is doing this? Isn't this always done by the MonitorDeflation thread?? src/hotspot/share/runtime/synchronizer.cpp line 1435: > 1433: > 1434: // Honor block request. > 1435: ThreadBlockInVM tbivm(self->as_Java_thread()); Same comment as previous use of TBIVM. (It's hard to see in the PR UI how they two blocks relate.) src/hotspot/share/runtime/synchronizer.cpp line 1460: > 1458: // This function is called by the MonitorDeflationThread to deflate > 1459: // ObjectMonitors. It is also called via do_final_audit_and_print_stats() > 1460: // by the VMThread. Ah! I think this addresses my previous comments about a non-JavaThread doing this. src/hotspot/share/runtime/synchronizer.cpp line 1501: > 1499: if (ls != NULL) { > 1500: timer.stop(); > 1501: ls->print_cr("before handshaking: unlinked_count=" SIZE_FORMAT ", in_use_list stats: ceiling=" SIZE_FORMAT ", count=" SIZE_FORMAT ", max=" SIZE_FORMAT, Style nit: line too long src/hotspot/share/runtime/synchronizer.cpp line 1520: > 1518: // deflated in this cycle. > 1519: size_t deleted_count = 0; > 1520: for (ObjectMonitor* monitor: delete_list) { I didn't realize C++ has a "foreach" loop construct! Is this in our allowed C++ usage? src/hotspot/share/runtime/synchronizer.cpp line 1533: > 1531: > 1532: // Honor block request. > 1533: ThreadBlockInVM tbivm(self->as_Java_thread()); Ditto previous comments on use of TBIVM. src/hotspot/share/runtime/synchronizer.cpp line 1712: > 1710: // Check the in_use_list; log the results of the checks. > 1711: void ObjectSynchronizer::chk_in_use_list(outputStream* out, int *error_cnt_p) { > 1712: size_t l_in_use_count = _in_use_list.count(); Style nit: what is this `l_` prefix? Is that a one or a small L? why do we want want/need it? (Applies elsewhere too) src/hotspot/share/runtime/synchronizer.cpp line 1748: > 1746: int* error_cnt_p) { > 1747: if (n->owner_is_DEFLATER_MARKER()) { > 1748: // This should not happen, but if it does, it is not fatal. Deflating an in-use monitor is not fatal? Please explain how things would recover. src/hotspot/share/runtime/synchronizer.hpp line 173: > 171: > 172: static MonitorList _in_use_list; > 173: static jint _in_use_list_ceiling; Can you add some commentary on what this ceiling is as I could not understand its role just by looking at the code. Thanks. src/hotspot/share/runtime/synchronizer.cpp line 221: > 219: > 220: MonitorList ObjectSynchronizer::_in_use_list; > 221: // Start the ceiling with one thread: This relates to me not understanding what this ceiling is (as commented elsewhere) but why does this say "start with one thread" when the value of AvgMonitorsPerThreadEstimate defaults to 1024 ?? ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From stuefe at openjdk.java.net Fri Nov 6 06:47:56 2020 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Fri, 6 Nov 2020 06:47:56 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v4] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Fri, 6 Nov 2020 01:04:09 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Adjust millisecond format. Some things are still unclear about this. My first calculation may have been a bit off too. The gcc error was this: `log_messages.c:81:11: note: 'snprintf' output between 6 and 86 bytes into a destination of size 81` The maximum ("and 86 bytes") is not clear to me now: First string: a buffer of DT_SIZE = 20 Third string: a buffer of TZ_SIZE = 57 I assume the compiler assumes that those strings are zero terminated. So first string 19 chars max, third string 56 chars max. Then, we have the %.3d - the max len of this is not 10, as I assumed, but 11: if argument = INT_MIN = "-2147483647" = 11 chars. The format string itself brings two characters. So we have: 19 + 56 + 11 + 2 = 88. Including terminating zero, we get 89. So the compiler should complain about 89 characters put into the 81 character buffer, not 86. src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 84: > 82: // Truncate milliseconds in buffer large enough to hold the > 83: // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") > 84: char tmp[10 + 1]; I was wrong yesterday. Max len of %d would be 11 chars (if INT_MIN). Can you make this buffer 11 chars please? This error would have had no practical consequence: tmp[] would be filled to the brim, leaving out the terminating zero, and since we then print with %.3s, this would have had no negative effect) ------------- Changes requested by stuefe (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 12:00:57 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 12:00:57 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: <-15R3IgetUfb1LOP5zEiG6K1dTsdGPhbUj6A0GttbyA=.362211b5-4282-4f69-8ec5-00b18fe69d02@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> <-15R3IgetUfb1LOP5zEiG6K1dTsdGPhbUj6A0GttbyA=.362211b5-4282-4f69-8ec5-00b18fe69d02@github.com> Message-ID: On Fri, 6 Nov 2020 01:01:40 GMT, Chris Plummer wrote: >> char tmp[10 + 1]; >> snprintf(tmp, sizeof(tmp), "%.3d", millisecs); >> snprintf(tbuf, ltbuf, "%s.%s %s", timestamp_date_time, tmp, timestamp_timezone); >> This also gets the same thing. > > The concern is when it is less than 100ms. unsigned millisecs[] = { 2, 20, 200, 1000 }; get_time_stamp(millisecs[i], buf, sizeof(buf)); gets: timestamp 06.11.2020 06:56:08.002 EST timestamp 06.11.2020 06:56:08.020 EST timestamp 06.11.2020 06:56:08.200 EST timestamp 06.11.2020 06:56:08.1000 EST with char tmp[10 + 1]; snprintf(tmp, sizeof(tmp), "%.3d", millisecs); snprintf(tbuf, ltbuf, "%s.%s %s", timestamp_date_time, tmp, timestamp_timezone); Is this what you want? ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 12:07:59 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 12:07:59 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v4] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: <12EsJ86wgR31ObAdbZYv2HWNkwb5W07YQvtQXPVec-8=.3f6c040d-b822-4c32-8f60-7bb4f8ac011a@github.com> On Fri, 6 Nov 2020 06:39:04 GMT, Thomas Stuefe wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Adjust millisecond format. > > src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 84: > >> 82: // Truncate milliseconds in buffer large enough to hold the >> 83: // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") >> 84: char tmp[10 + 1]; > > I was wrong yesterday. Max len of %d would be 11 chars (if INT_MIN). Can you make this buffer 11 chars please? > > (This error would have had no practical consequence: tmp[] would be filled to the brim, leaving out the terminating zero, and since we then print with %.3s, this would have had no negative effect. But its still better to plan for \0 explicitly) > INT_MIN = "-2147483647" = 11 If millisecs is unsigned do we not have to account for the minus sign? Why can't tmp just be 20 and we don't have to count characters since snprintf will null terminate it? ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 12:08:00 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 12:08:00 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v4] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: <3PMQI4eOI42lIvznuA3HxaI9xDpLYLsrrWfeg5kwy8w=.06454ad4-3c44-477a-9b3f-09ea68ca1ebd@github.com> On Fri, 6 Nov 2020 01:20:01 GMT, David Holmes wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Adjust millisecond format. > > src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 83: > >> 81: >> 82: // Truncate milliseconds in buffer large enough to hold the >> 83: // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") > > With the updated code the comment is no longer correct - change %.3s to %.3d Ok. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From stuefe at openjdk.java.net Fri Nov 6 12:08:00 2020 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Fri, 6 Nov 2020 12:08:00 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v4] In-Reply-To: <12EsJ86wgR31ObAdbZYv2HWNkwb5W07YQvtQXPVec-8=.3f6c040d-b822-4c32-8f60-7bb4f8ac011a@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <12EsJ86wgR31ObAdbZYv2HWNkwb5W07YQvtQXPVec-8=.3f6c040d-b822-4c32-8f60-7bb4f8ac011a@github.com> Message-ID: <8XKuXsVC5qwa3IysDxcFjurvuo0aRx4fkmROgWCEc6I=.494f965f-d16b-46bb-a22e-5862c0667a23@github.com> On Fri, 6 Nov 2020 12:02:28 GMT, Coleen Phillimore wrote: >> src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 84: >> >>> 82: // Truncate milliseconds in buffer large enough to hold the >>> 83: // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") >>> 84: char tmp[10 + 1]; >> >> I was wrong yesterday. Max len of %d would be 11 chars (if INT_MIN). Can you make this buffer 11 chars please? >> >> (This error would have had no practical consequence: tmp[] would be filled to the brim, leaving out the terminating zero, and since we then print with %.3s, this would have had no negative effect. But its still better to plan for \0 explicitly) > >> INT_MIN = "-2147483647" = 11 > If millisecs is unsigned do we not have to account for the minus sign? > > Why can't tmp just be 20 and we don't have to count characters since snprintf will null terminate it? Sure, thats fine too! ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 12:16:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 12:16:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v7] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: - Merge branch 'master' into jvmti-table - Add back WeakProcessorPhases::Phase enum. - Serguei 1. - Code review comments from Kim and Albert. - Merge branch 'master' into jvmti-table - Merge branch 'master' into jvmti-table - More review comments from Stefan and ErikO - Code review comments from StefanK. - 8212879: Make JVMTI TagMap table not hash on oop address ------------- Changes: https://git.openjdk.java.net/jdk/pull/967/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=06 Stats: 1748 lines in 41 files changed: 628 ins; 1018 del; 102 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From lzang at openjdk.java.net Fri Nov 6 12:59:05 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Fri, 6 Nov 2020 12:59:05 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap Message-ID: The implementation of jmap tool depends on the implementation of object iteration by different GC heap. This patch extend the BasicJMapTest to cover differet GC Heap. ------------- Commit messages: - 8255982: Extend BasicJMapTest to test with different GC Heap Changes: https://git.openjdk.java.net/jdk/pull/1094/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255982 Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1094.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1094/head:pull/1094 PR: https://git.openjdk.java.net/jdk/pull/1094 From david.holmes at oracle.com Fri Nov 6 13:01:10 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Nov 2020 23:01:10 +1000 Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v4] In-Reply-To: <8XKuXsVC5qwa3IysDxcFjurvuo0aRx4fkmROgWCEc6I=.494f965f-d16b-46bb-a22e-5862c0667a23@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <12EsJ86wgR31ObAdbZYv2HWNkwb5W07YQvtQXPVec-8=.3f6c040d-b822-4c32-8f60-7bb4f8ac011a@github.com> <8XKuXsVC5qwa3IysDxcFjurvuo0aRx4fkmROgWCEc6I=.494f965f-d16b-46bb-a22e-5862c0667a23@github.com> Message-ID: On 6/11/2020 10:08 pm, Thomas Stuefe wrote: > On Fri, 6 Nov 2020 12:02:28 GMT, Coleen Phillimore wrote: > >>> src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 84: >>> >>>> 82: // Truncate milliseconds in buffer large enough to hold the >>>> 83: // value which is always < 1000 (and so a maximum of 3 digits for "%.3s") >>>> 84: char tmp[10 + 1]; >>> >>> I was wrong yesterday. Max len of %d would be 11 chars (if INT_MIN). Can you make this buffer 11 chars please? >>> >>> (This error would have had no practical consequence: tmp[] would be filled to the brim, leaving out the terminating zero, and since we then print with %.3s, this would have had no negative effect. But its still better to plan for \0 explicitly) >> >>> INT_MIN = "-2147483647" = 11 >> If millisecs is unsigned do we not have to account for the minus sign? >> >> Why can't tmp just be 20 and we don't have to count characters since snprintf will null terminate it? > > Sure, thats fine too! Don't we need the value to be smaller so that gcc "calculates" the total to be within the size of the buffer? David ----- > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/1067 > From thomas.stuefe at gmail.com Fri Nov 6 13:06:22 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 6 Nov 2020 14:06:22 +0100 Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v4] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <12EsJ86wgR31ObAdbZYv2HWNkwb5W07YQvtQXPVec-8=.3f6c040d-b822-4c32-8f60-7bb4f8ac011a@github.com> <8XKuXsVC5qwa3IysDxcFjurvuo0aRx4fkmROgWCEc6I=.494f965f-d16b-46bb-a22e-5862c0667a23@github.com> Message-ID: On Fri, Nov 6, 2020 at 2:01 PM David Holmes wrote: > On 6/11/2020 10:08 pm, Thomas Stuefe wrote: > > On Fri, 6 Nov 2020 12:02:28 GMT, Coleen Phillimore > wrote: > > > >>> src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 84: > >>> > >>>> 82: // Truncate milliseconds in buffer large enough to hold the > >>>> 83: // value which is always < 1000 (and so a maximum of 3 digits > for "%.3s") > >>>> 84: char tmp[10 + 1]; > >>> > >>> I was wrong yesterday. Max len of %d would be 11 chars (if INT_MIN). > Can you make this buffer 11 chars please? > >>> > >>> (This error would have had no practical consequence: tmp[] would be > filled to the brim, leaving out the terminating zero, and since we then > print with %.3s, this would have had no negative effect. But its still > better to plan for \0 explicitly) > >> > >>> INT_MIN = "-2147483647" = 11 > >> If millisecs is unsigned do we not have to account for the minus sign? > >> > >> Why can't tmp just be 20 and we don't have to count characters since > snprintf will null terminate it? > > > > Sure, thats fine too! > > Don't we need the value to be smaller so that gcc "calculates" the total > to be within the size of the buffer? > > No, since we print it with "%.3s", so limited to 3 chars. ..Thomas > David > ----- > > > ------------- > > > > PR: https://git.openjdk.java.net/jdk/pull/1067 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From redestad at openjdk.java.net Fri Nov 6 13:20:01 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Fri, 6 Nov 2020 13:20:01 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> <-15R3IgetUfb1LOP5zEiG6K1dTsdGPhbUj6A0GttbyA=.362211b5-4282-4f69-8ec5-00b18fe69d02@github.com> Message-ID: On Fri, 6 Nov 2020 11:58:04 GMT, Coleen Phillimore wrote: >> The concern is when it is less than 100ms. > > unsigned millisecs[] = { 2, 20, 200, 1000 }; > get_time_stamp(millisecs[i], buf, sizeof(buf)); > > gets: > > timestamp 06.11.2020 06:56:08.002 EST > timestamp 06.11.2020 06:56:08.020 EST > timestamp 06.11.2020 06:56:08.200 EST > timestamp 06.11.2020 06:56:08.1000 EST > > with > > char tmp[10 + 1]; > snprintf(tmp, sizeof(tmp), "%.3d", millisecs); > snprintf(tbuf, ltbuf, "%s.%s %s", timestamp_date_time, tmp, timestamp_timezone); > > Is this what you want? I think you need .3 in both places, otherwise I expect the warning will still be there? (We don't need to worry about values of millisecs larger than 999): char tmp[11 + 1]; snprintf(tmp, sizeof(tmp), "%.3d", millisecs); snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 13:28:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 13:28:12 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v5] In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: <0bnw5j0fjNPshSTnu610ex_CKQCiDJHajkcxM0pElpM=.98b0b0e1-86f7-4452-a0c5-31502babd0c4@github.com> > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. > > thanks, > Coleen Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: More. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1067/files - new: https://git.openjdk.java.net/jdk/pull/1067/files/245c7f7f..1de96d43 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=03-04 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1067.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1067/head:pull/1067 PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 13:42:56 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 13:42:56 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v3] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <-QIXSepx_VoFk-MYJnMIQbcu0t2DG7TV2YOpdg3B6Yk=.a069d511-7f24-43ae-a44c-9173854030d7@github.com> <-15R3IgetUfb1LOP5zEiG6K1dTsdGPhbUj6A0GttbyA=.362211b5-4282-4f69-8ec5-00b18fe69d02@github.com> Message-ID: On Fri, 6 Nov 2020 13:16:54 GMT, Claes Redestad wrote: >> unsigned millisecs[] = { 2, 20, 200, 1000 }; >> get_time_stamp(millisecs[i], buf, sizeof(buf)); >> >> gets: >> >> timestamp 06.11.2020 06:56:08.002 EST >> timestamp 06.11.2020 06:56:08.020 EST >> timestamp 06.11.2020 06:56:08.200 EST >> timestamp 06.11.2020 06:56:08.1000 EST >> >> with >> >> char tmp[10 + 1]; >> snprintf(tmp, sizeof(tmp), "%.3d", millisecs); >> snprintf(tbuf, ltbuf, "%s.%s %s", timestamp_date_time, tmp, timestamp_timezone); >> >> Is this what you want? > > I think you need .3 in both places, otherwise I expect the warning will still be there? (We don't need to worry about values of millisecs larger than 999): > > char tmp[11 + 1]; > snprintf(tmp, sizeof(tmp), "%.3d", millisecs); > snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); You're right. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From redestad at openjdk.java.net Fri Nov 6 13:49:58 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Fri, 6 Nov 2020 13:49:58 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v5] In-Reply-To: <0bnw5j0fjNPshSTnu610ex_CKQCiDJHajkcxM0pElpM=.98b0b0e1-86f7-4452-a0c5-31502babd0c4@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <0bnw5j0fjNPshSTnu610ex_CKQCiDJHajkcxM0pElpM=.98b0b0e1-86f7-4452-a0c5-31502babd0c4@github.com> Message-ID: On Fri, 6 Nov 2020 13:28:12 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > More. Marked as reviewed by redestad (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From stuefe at openjdk.java.net Fri Nov 6 14:06:59 2020 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Fri, 6 Nov 2020 14:06:59 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v5] In-Reply-To: <0bnw5j0fjNPshSTnu610ex_CKQCiDJHajkcxM0pElpM=.98b0b0e1-86f7-4452-a0c5-31502babd0c4@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <0bnw5j0fjNPshSTnu610ex_CKQCiDJHajkcxM0pElpM=.98b0b0e1-86f7-4452-a0c5-31502babd0c4@github.com> Message-ID: <7O8lATQxVufpxHp1dOmkvRstWVcpPCU5PeLEHT6fX04=.8c655d8f-a212-48e6-bf67-96631f2ea619@github.com> On Fri, 6 Nov 2020 13:28:12 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > More. src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 86: > 84: char tmp[20]; > 85: snprintf(tmp, sizeof(tmp), "%.3d", millisecs); > 86: snprintf(tbuf, ltbuf, "%s.%s %s", timestamp_date_time, tmp, timestamp_timezone); Sorry, Coleen, we still miss the precision limit in the second printf. char tmp[20]; snprintf(tmp, sizeof(tmp), "%.3d", millisecs); snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); ^^^^ The point was to first print the digit with enough space to hold its stringified value (20 is more than enough). We need to print that one with .3d to get zero-padding to three digits if the value is <100. Then to print that stringified version with ".3s" since, in contrast to %d, with %s the precision actually works as a limiter. Thanks for your patience. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 14:30:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 14:30:14 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v6] In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. > > thanks, > Coleen Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Now this builds. I hope it's perfect. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1067/files - new: https://git.openjdk.java.net/jdk/pull/1067/files/1de96d43..930ca55e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1067&range=04-05 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1067.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1067/head:pull/1067 PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 14:30:16 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 14:30:16 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v5] In-Reply-To: <7O8lATQxVufpxHp1dOmkvRstWVcpPCU5PeLEHT6fX04=.8c655d8f-a212-48e6-bf67-96631f2ea619@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> <0bnw5j0fjNPshSTnu610ex_CKQCiDJHajkcxM0pElpM=.98b0b0e1-86f7-4452-a0c5-31502babd0c4@github.com> <7O8lATQxVufpxHp1dOmkvRstWVcpPCU5PeLEHT6fX04=.8c655d8f-a212-48e6-bf67-96631f2ea619@github.com> Message-ID: On Fri, 6 Nov 2020 14:03:10 GMT, Thomas Stuefe wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> More. > > src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c line 86: > >> 84: char tmp[20]; >> 85: snprintf(tmp, sizeof(tmp), "%.3d", millisecs); >> 86: snprintf(tbuf, ltbuf, "%s.%s %s", timestamp_date_time, tmp, timestamp_timezone); > > Sorry, Coleen, we still miss the precision limit in the second printf. > > char tmp[20]; > snprintf(tmp, sizeof(tmp), "%.3d", millisecs); > snprintf(tbuf, ltbuf, "%s.%.3s %s", timestamp_date_time, tmp, timestamp_timezone); > ^^^^ > > The point was to first print the digit with enough space to hold its stringified value (20 is more than enough). We need to print that one with .3d to get zero-padding to three digits if the value is <100. > > Then to print that stringified version with ".3s" since, in contrast to %d, with %s the precision actually works as a limiter. > > Thanks for your patience. Thanks for all your help and quick responses. I made that change, see new commit. I should realize that there are no simple changes! ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From stuefe at openjdk.java.net Fri Nov 6 14:34:59 2020 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Fri, 6 Nov 2020 14:34:59 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v6] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Fri, 6 Nov 2020 14:30:14 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Now this builds. I hope it's perfect. Looks fine to me now. Thank you! ------------- Marked as reviewed by stuefe (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1067 From redestad at openjdk.java.net Fri Nov 6 14:37:57 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Fri, 6 Nov 2020 14:37:57 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v6] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Fri, 6 Nov 2020 14:30:14 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Now this builds. I hope it's perfect. Perfect! ------------- Marked as reviewed by redestad (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 16:02:57 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 16:02:57 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v6] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Fri, 6 Nov 2020 14:35:34 GMT, Claes Redestad wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Now this builds. I hope it's perfect. > > Perfect! Thanks Claes and Thomas for all your help and the code reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From cjplummer at openjdk.java.net Fri Nov 6 17:46:57 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 6 Nov 2020 17:46:57 GMT Subject: RFR: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c [v6] In-Reply-To: References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Fri, 6 Nov 2020 14:30:14 GMT, Coleen Phillimore wrote: >> Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. >> >> thanks, >> Coleen > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Now this builds. I hope it's perfect. Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From coleenp at openjdk.java.net Fri Nov 6 19:06:58 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 19:06:58 GMT Subject: Integrated: 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c In-Reply-To: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> References: <6E43PFJF-NovGd6WS3qOqq3DSykqxZKDSJHTWU03-oI=.61849c92-9bcb-485f-9fa0-706a286514f2@github.com> Message-ID: On Thu, 5 Nov 2020 00:37:15 GMT, Coleen Phillimore wrote: > Apply patch suggested by @cl4es in the bug report. Passes linux-x86-open,linux-x64-open,linux-s390x-open,linux-arm32-debug,linux-ppc64le-debug builds with this patch, and tier1. > > thanks, > Coleen This pull request has now been integrated. Changeset: 0b7fba75 Author: Coleen Phillimore URL: https://git.openjdk.java.net/jdk/commit/0b7fba75 Stats: 7 lines in 1 file changed: 3 ins; 0 del; 4 mod 8254270: linux 32 bit build doesn't compile libjdwp/log_messages.c Reviewed-by: redestad, cjplummer, dholmes, stuefe ------------- PR: https://git.openjdk.java.net/jdk/pull/1067 From amenkov at openjdk.java.net Fri Nov 6 22:01:56 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 6 Nov 2020 22:01:56 GMT Subject: Integrated: 8254864: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java timed out In-Reply-To: References: Message-ID: <7XhlBGYxiyzpGeuNTdSdur7fg6cZI1NyJaeuQxlpn_w=.41e5dc89-eefb-492f-b688-3177e81cc6a6@github.com> On Tue, 3 Nov 2020 23:16:50 GMT, Alex Menkov wrote: > - Fixed synchronization logic in resexhausted001; > - On Windows JVM cannot produce OOM caused by thread creation failure. Massive thread creation causes big system (not VM) memory consumption, as a result test host dramatically slows down up to hang. Specifying small heap size we can get RESOURCE_EXHAUSTED_JAVA_HEAP, but this is not the goal of resexhausted001. > So I disabled resexhausted001 on Windows. > Had to implement runtime check (instead of using @requires) because resexhausted001 is also called by resexhausted004. > - Additionally resexhausted001-resexhausted003 test were improved to verify JVMTI generates ResourceExhausted event with correct flag. This pull request has now been integrated. Changeset: a9dff942 Author: Alex Menkov URL: https://git.openjdk.java.net/jdk/commit/a9dff942 Stats: 47 lines in 6 files changed: 26 ins; 1 del; 20 mod 8254864: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java timed out Reviewed-by: sspitsyn, cjplummer ------------- PR: https://git.openjdk.java.net/jdk/pull/1046 From github.com+654217+bastie at openjdk.java.net Sat Nov 7 10:50:00 2020 From: github.com+654217+bastie at openjdk.java.net (Sebastian Ritter) Date: Sat, 7 Nov 2020 10:50:00 GMT Subject: RFR: 8066622 8066637: remove deprecated using java.io.File.toURL() in internal classes In-Reply-To: References: Message-ID: On Sat, 7 Nov 2020 07:55:03 GMT, Sebastian Ritter wrote: > In result of Javadoc to do not use java.io.File.toURL() > Change use java.io.File.toURL().toURI() instead. 8066622 8066637: only for java.io.File.toURL ------------- PR: https://git.openjdk.java.net/jdk/pull/1108 From github.com+654217+bastie at openjdk.java.net Sat Nov 7 10:50:00 2020 From: github.com+654217+bastie at openjdk.java.net (Sebastian Ritter) Date: Sat, 7 Nov 2020 10:50:00 GMT Subject: RFR: 8066622 8066637: remove deprecated using java.io.File.toURL() in internal classes Message-ID: In result of Javadoc to do not use java.io.File.toURL() Change use java.io.File.toURL().toURI() instead. ------------- Commit messages: - in result of Javadoc to java.io.File.toURL do use internal java.io.File.toURL().toURI instead Changes: https://git.openjdk.java.net/jdk/pull/1108/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1108&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8066622 Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1108.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1108/head:pull/1108 PR: https://git.openjdk.java.net/jdk/pull/1108 From dcubed at openjdk.java.net Sat Nov 7 16:57:55 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 16:57:55 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 01:54:52 GMT, David Holmes wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > src/hotspot/share/runtime/monitorDeflationThread.hpp line 44: > >> 42: >> 43: // Hide this thread from external view. >> 44: bool is_hidden_from_external_view() const { return true; } > > Style nit: a single space after "const" suffices Fixed. I copied that from the serviceThread.hpp file, but, since I haven't otherwise touched that file in this changeset, I'm going to leave serviceThread.hpp alone. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Sat Nov 7 17:04:58 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 17:04:58 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 01:57:59 GMT, David Holmes wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > src/hotspot/share/runtime/objectMonitor.cpp line 380: > >> 378: if (event.should_commit()) { >> 379: event.set_monitorClass(object()->klass()); >> 380: event.set_address((uintptr_t)this); > > This looks wrong - the event should refer to the Object whose monitor we have entered, not the OM itself. I noticed that in my preliminary review of Erik's changes. He checked with the JFR guys and they said it just needed to be an address and does not have to refer to the Object. @fisk - can you think of a comment we should add here? > src/hotspot/share/runtime/objectMonitor.cpp line 1472: > >> 1470: event->set_monitorClass(monitor->object()->klass()); >> 1471: event->set_timeout(timeout); >> 1472: event->set_address((uintptr_t)monitor); > > Again the event should refer to the Object, not the OM. I noticed that in my preliminary review of Erik's changes. He checked with the JFR guys and they said it just needed to be an address and does not have to refer to the Object. @fisk - can you think of a comment we should add here? ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Sat Nov 7 17:16:58 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 17:16:58 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 02:00:07 GMT, David Holmes wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > src/hotspot/share/runtime/objectMonitor.hpp line 145: > >> 143: // have busy multi-threaded access. _header and _object are set at initial >> 144: // inflation. The _object does not change, so it is a good choice to share >> 145: // its the cache line with _header. > > Typo: its the Nice catch. Fixed. > src/hotspot/share/runtime/synchronizer.cpp line 60: > >> 58: >> 59: void MonitorList::add(ObjectMonitor* m) { >> 60: for (;;) { > > Style nit: a do/while loop would like nicer here. Fixed: do { ObjectMonitor* head = Atomic::load(&_head); m->set_next_om(head); } while (Atomic::cmpxchg(&_head, head, m) != head); > src/hotspot/share/runtime/synchronizer.cpp line 94: > >> 92: // Find next live ObjectMonitor. >> 93: ObjectMonitor* next = m; >> 94: while (next != NULL && next->is_being_async_deflated()) { > > Nit: This loop seems odd. Given we know m is_being_async_deflated, this should either be a do/while loop, or else we should initialize: > > ObjectMonitor* next = m->next_om(); > > and dispense with the awkwardly named next_next. @fisk - I'm leaving this one for you for now. > src/hotspot/share/runtime/synchronizer.cpp line 126: > >> 124: } >> 125: >> 126: if (self->is_Java_thread() && > > Non-JavaThreads may have a monitor list ??? This function, MonitorList::unlink_deflated(), may be called by either the MonitorDeflationThread or the VMThread. The VMThread does not need to block for the safepoint which is what the `if (self->is_Java_thread()` prevents. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Sat Nov 7 17:21:59 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 17:21:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 02:14:56 GMT, David Holmes wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > src/hotspot/share/runtime/synchronizer.cpp line 136: > >> 134: >> 135: // Honor block request. >> 136: ThreadBlockInVM tbivm(self->as_Java_thread()); > > ThreadBlockInVM is generally used to wrap blocking code, not to cause the current thread to block (which it will do as a side-effect if a safepoint/handshake is requested). Surely this should just be call to `process_if_requested` (or the new `process_if_requested_with_exit_check`)? This kind of use of ThreadBlockInVM predates this changeset so while the location is new, then code style is old, very old... I'll hold off changing this for now. > src/hotspot/share/runtime/synchronizer.cpp line 1425: > >> 1423: } >> 1424: >> 1425: if (self->is_Java_thread() && > > Again unclear how a non-JavaThread is doing this? Isn't this always done by the MonitorDeflation thread?? This function, ObjectSynchronizer::deflate_monitor_list(), may be called by either the MonitorDeflationThread or the VMThread. The VMThread does not need to block for the safepoint which is what the if (self->is_Java_thread() prevents. > src/hotspot/share/runtime/synchronizer.cpp line 1435: > >> 1433: >> 1434: // Honor block request. >> 1435: ThreadBlockInVM tbivm(self->as_Java_thread()); > > Same comment as previous use of TBIVM. (It's hard to see in the PR UI how they two blocks relate.) This kind of use of ThreadBlockInVM predates this changeset so while the location is new, then code style is old, very old... I'll hold off changing this for now. > src/hotspot/share/runtime/synchronizer.cpp line 1460: > >> 1458: // This function is called by the MonitorDeflationThread to deflate >> 1459: // ObjectMonitors. It is also called via do_final_audit_and_print_stats() >> 1460: // by the VMThread. > > Ah! I think this addresses my previous comments about a non-JavaThread doing this. Yup... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Sat Nov 7 17:27:58 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 17:27:58 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 02:25:23 GMT, David Holmes wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > src/hotspot/share/runtime/synchronizer.cpp line 1501: > >> 1499: if (ls != NULL) { >> 1500: timer.stop(); >> 1501: ls->print_cr("before handshaking: unlinked_count=" SIZE_FORMAT ", in_use_list stats: ceiling=" SIZE_FORMAT ", count=" SIZE_FORMAT ", max=" SIZE_FORMAT, > > Style nit: line too long Yeah... I keep going back and forth for long logging lines... I reformatted that one and another one a few lines farther down. > src/hotspot/share/runtime/synchronizer.cpp line 1520: > >> 1518: // deflated in this cycle. >> 1519: size_t deleted_count = 0; >> 1520: for (ObjectMonitor* monitor: delete_list) { > > I didn't realize C++ has a "foreach" loop construct! Is this in our allowed C++ usage? @fisk - this one is for you... :-) > src/hotspot/share/runtime/synchronizer.cpp line 1533: > >> 1531: >> 1532: // Honor block request. >> 1533: ThreadBlockInVM tbivm(self->as_Java_thread()); > > Ditto previous comments on use of TBIVM. This kind of use of ThreadBlockInVM predates this changeset so while the location is new, then code style is old, very old... I'll hold off changing this for now. > src/hotspot/share/runtime/synchronizer.cpp line 1712: > >> 1710: // Check the in_use_list; log the results of the checks. >> 1711: void ObjectSynchronizer::chk_in_use_list(outputStream* out, int *error_cnt_p) { >> 1712: size_t l_in_use_count = _in_use_list.count(); > > Style nit: what is this `l_` prefix? Is that a one or a small L? why do we want want/need it? (Applies elsewhere too) The "l_" prefix is used for a local copy of a value where we want to make sure that we use a consistent value for the check and for the resulting audit/logging message. This is not a new thing with this changeset and that style was used in previous versions of the ObjectMonitor audit/logging code. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Sat Nov 7 17:40:55 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 17:40:55 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 02:31:12 GMT, David Holmes wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > src/hotspot/share/runtime/synchronizer.cpp line 1748: > >> 1746: int* error_cnt_p) { >> 1747: if (n->owner_is_DEFLATER_MARKER()) { >> 1748: // This should not happen, but if it does, it is not fatal. > > Deflating an in-use monitor is not fatal? Please explain how things would recover. When the MonitorDeflationThread is doing its final async deflation cycle before VM shutdown, it is possible for it to finish a deflation pass on the in-use list and then block for the final safepoint before unlinking the deflated ObjectMonitors. If the VMThread happens to be doing an audit_and_print_stats() call when this happens (see the end of SafepointSynchronize::do_cleanup_tasks()), then we don't want that audit to FAIL. As for recovery, if the VMThread is doing a final audit, then when it sees an already deflated ObjectMonitor on the in-use list, it will take care of the unlinking and deleting... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Sat Nov 7 18:01:57 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 18:01:57 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 02:33:37 GMT, David Holmes wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > src/hotspot/share/runtime/synchronizer.hpp line 173: > >> 171: >> 172: static MonitorList _in_use_list; >> 173: static jint _in_use_list_ceiling; > > Can you add some commentary on what this ceiling is as I could not understand its role just by looking at the code. Thanks. How about this: static MonitorList _in_use_list; // The ratio of the current _in_use_list count to the ceiling is used // to determine if we are above MonitorUsedDeflationThreshold and need // to do an async monitor deflation cycle. The ceiling is increased by // AvgMonitorsPerThreadEstimate when a thread is added to the system // and is decreased by AvgMonitorsPerThreadEstimate when a thread is // removed from the system. // Note: If the _in_use_list max exceeds the ceiling, then // monitors_used_above_threshold() will use the in_use_list max instead // of the thread count derived ceiling because we have used more // ObjectMonitors than the estimated average. static jint _in_use_list_ceiling; > src/hotspot/share/runtime/synchronizer.cpp line 221: > >> 219: >> 220: MonitorList ObjectSynchronizer::_in_use_list; >> 221: // Start the ceiling with one thread: > > This relates to me not understanding what this ceiling is (as commented elsewhere) but why does this say "start with one thread" when the value of AvgMonitorsPerThreadEstimate defaults to 1024 ?? The estimate is that a single thread will generate at most 1024 inflated ObjectMonitors on average. I changed the comment like this: // Start the ceiling with the estimate for one thread: jint ObjectSynchronizer::_in_use_list_ceiling = AvgMonitorsPerThreadEstimate; Does that help? ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Sat Nov 7 18:05:00 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 18:05:00 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 02:40:44 GMT, David Holmes wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Hi Dan, > > Overall this looks great. Comparing old and new code is complex but the new code on its own is generally much simpler/clearer (not all though :) ). > > I have a few nits, comments and queries below. > > Thanks, > David @dholmes-ora - Thanks for the review! Hmm... I'm not sure why the GitHub UI send out my replies one-at-a-time. Perhaps I should have replied from the "files" view instead of the main PR view? ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Sat Nov 7 18:20:17 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 7 Nov 2020 18:20:17 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also done promotion perf runs on these bits and says "the results look fine". Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: Resolve most of dholmes-ora comments. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/642/files - new: https://git.openjdk.java.net/jdk/pull/642/files/b7d0c1e9..6c2db34a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=00-01 Stats: 28 lines in 4 files changed: 15 ins; 2 del; 11 mod Patch: https://git.openjdk.java.net/jdk/pull/642.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/642/head:pull/642 PR: https://git.openjdk.java.net/jdk/pull/642 From prr at openjdk.java.net Sun Nov 8 17:00:55 2020 From: prr at openjdk.java.net (Phil Race) Date: Sun, 8 Nov 2020 17:00:55 GMT Subject: RFR: 8066622 8066637: remove deprecated using java.io.File.toURL() in internal classes In-Reply-To: References: Message-ID: On Sat, 7 Nov 2020 07:55:03 GMT, Sebastian Ritter wrote: > In result of Javadoc to do not use java.io.File.toURL() > Change use java.io.File.toURL().toURI() instead. You reference a desktop bug that discusses many, many deprecations and skara has identified https://bugs.openjdk.java.net/browse/JDK-8066622 as the issue being fixed. Yet you propose to fix precisely one of these. But when this is integrated that bug will be closed out as resolved. I think you need a new bug about JUST the changes you are making. So I don't think you should use that bug id any where in this PR. And I'd like to hear whether you actually *tested* splashscreen with your change ? I see no sign that you did. ------------- Changes requested by prr (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1108 From dholmes at openjdk.java.net Sun Nov 8 21:45:56 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Sun, 8 Nov 2020 21:45:56 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Sat, 7 Nov 2020 17:55:34 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/synchronizer.hpp line 173: >> >>> 171: >>> 172: static MonitorList _in_use_list; >>> 173: static jint _in_use_list_ceiling; >> >> Can you add some commentary on what this ceiling is as I could not understand its role just by looking at the code. Thanks. > > How about this: > static MonitorList _in_use_list; > // The ratio of the current _in_use_list count to the ceiling is used > // to determine if we are above MonitorUsedDeflationThreshold and need > // to do an async monitor deflation cycle. The ceiling is increased by > // AvgMonitorsPerThreadEstimate when a thread is added to the system > // and is decreased by AvgMonitorsPerThreadEstimate when a thread is > // removed from the system. > // Note: If the _in_use_list max exceeds the ceiling, then > // monitors_used_above_threshold() will use the in_use_list max instead > // of the thread count derived ceiling because we have used more > // ObjectMonitors than the estimated average. > static jint _in_use_list_ceiling; Thanks for the comment. So instead of checking the threshhold on each OM allocation we use this averaging technique to estimate the number of monitors in use? Can you explain how this came about rather than the simple/obvious check at allocation time. Thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From david.holmes at oracle.com Sun Nov 8 22:15:33 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 9 Nov 2020 08:15:33 +1000 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: <2c5e0f26-ef31-2ca4-d51f-f197efdfbcb2@oracle.com> Hi Dan, On 8/11/2020 3:40 am, Daniel D.Daugherty wrote: > On Fri, 6 Nov 2020 02:31:12 GMT, David Holmes wrote: > >>> Changes from @fisk and @dcubed-ojdk to: >>> >>> - simplify ObjectMonitor list management >>> - get rid of Type-Stable Memory (TSM) >>> >>> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >>> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >>> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >>> - a few minor regressions (<= -0.24%) >>> - Volano is 6.8% better >>> >>> Eric C. has also done promotion perf runs on these bits and says "the results look fine". >> >> src/hotspot/share/runtime/synchronizer.cpp line 1748: >> >>> 1746: int* error_cnt_p) { >>> 1747: if (n->owner_is_DEFLATER_MARKER()) { >>> 1748: // This should not happen, but if it does, it is not fatal. >> >> Deflating an in-use monitor is not fatal? Please explain how things would recover. > > When the MonitorDeflationThread is doing its final async deflation cycle before > VM shutdown, it is possible for it to finish a deflation pass on the in-use list and > then block for the final safepoint before unlinking the deflated ObjectMonitors. > > If the VMThread happens to be doing an audit_and_print_stats() call when this > happens (see the end of SafepointSynchronize::do_cleanup_tasks()), then we > don't want that audit to FAIL. > > As for recovery, if the VMThread is doing a final audit, then when it sees an > already deflated ObjectMonitor on the in-use list, it will take care of the unlinking > and deleting... So it isn't really an "in-use" monitor, it is a monitor that is still in the in-use-list - is that right? Thanks, David ----- > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/642 > From dcubed at openjdk.java.net Mon Nov 9 03:21:56 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 9 Nov 2020 03:21:56 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Sat, 7 Nov 2020 18:01:57 GMT, Daniel D. Daugherty wrote: >> Hi Dan, >> >> Overall this looks great. Comparing old and new code is complex but the new code on its own is generally much simpler/clearer (not all though :) ). >> >> I have a few nits, comments and queries below. >> >> Thanks, >> David > > @dholmes-ora - Thanks for the review! Hmm... I'm not sure why the GitHub UI > send out my replies one-at-a-time. Perhaps I should have replied from the > "files" view instead of the main PR view? It is a deflated monitor that is still on the in-use list. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Mon Nov 9 03:50:56 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 9 Nov 2020 03:50:56 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Sun, 8 Nov 2020 21:43:00 GMT, David Holmes wrote: >> How about this: >> static MonitorList _in_use_list; >> // The ratio of the current _in_use_list count to the ceiling is used >> // to determine if we are above MonitorUsedDeflationThreshold and need >> // to do an async monitor deflation cycle. The ceiling is increased by >> // AvgMonitorsPerThreadEstimate when a thread is added to the system >> // and is decreased by AvgMonitorsPerThreadEstimate when a thread is >> // removed from the system. >> // Note: If the _in_use_list max exceeds the ceiling, then >> // monitors_used_above_threshold() will use the in_use_list max instead >> // of the thread count derived ceiling because we have used more >> // ObjectMonitors than the estimated average. >> static jint _in_use_list_ceiling; > > Thanks for the comment. So instead of checking the threshhold on each OM allocation we use this averaging technique to estimate the number of monitors in use? Can you explain how this came about rather than the simple/obvious check at allocation time. Thanks. I'm not sure I understand your question, but let me that a stab at it anyway... We used to compare the sum of the in-use counts from all the in-use lists with the total population of ObjectMonitors. If that ratio was higher than MonitorUsedDeflationThreshold, then we would do an async deflation cycle. Since we got rid of TSM, we no longer had a population of already allocated ObjectMonitors, we had a max value instead. However, when the VMs use of ObjectMonitors is first spinning up, the max value is typically very close to the in-use count so we would always be asking for an async-deflation during that spinning up phase. I created the idea of a ceiling value that is tied to thread count and the AvgMonitorsPerThreadEstimate to replace the population value that we used to have. By comparing the in-use count against the ceiling value, we no longer exceed the MonitorUsedDeflationThreshold when the VMs use of ObjectMonitors is first spinning up so we no longer do async deflations continuously during that phase. If the max value exceeds the ceiling value, then we're using a LOT of ObjectMonitors and, in that case, we compare the in-use count against the max to determine if we're exceeding the MonitorUsedDeflationThreshold. Does this help? ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From david.holmes at oracle.com Mon Nov 9 06:06:34 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 9 Nov 2020 16:06:34 +1000 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: Hi Dan, On 9/11/2020 1:50 pm, Daniel D.Daugherty wrote: > On Sun, 8 Nov 2020 21:43:00 GMT, David Holmes wrote: > >>> How about this: >>> static MonitorList _in_use_list; >>> // The ratio of the current _in_use_list count to the ceiling is used >>> // to determine if we are above MonitorUsedDeflationThreshold and need >>> // to do an async monitor deflation cycle. The ceiling is increased by >>> // AvgMonitorsPerThreadEstimate when a thread is added to the system >>> // and is decreased by AvgMonitorsPerThreadEstimate when a thread is >>> // removed from the system. >>> // Note: If the _in_use_list max exceeds the ceiling, then >>> // monitors_used_above_threshold() will use the in_use_list max instead >>> // of the thread count derived ceiling because we have used more >>> // ObjectMonitors than the estimated average. >>> static jint _in_use_list_ceiling; >> >> Thanks for the comment. So instead of checking the threshhold on each OM allocation we use this averaging technique to estimate the number of monitors in use? Can you explain how this came about rather than the simple/obvious check at allocation time. Thanks. > > I'm not sure I understand your question, but let me that a stab at it anyway... > > We used to compare the sum of the in-use counts from all the in-use lists > with the total population of ObjectMonitors. If that ratio was higher than > MonitorUsedDeflationThreshold, then we would do an async deflation cycle. > Since we got rid of TSM, we no longer had a population of already allocated > ObjectMonitors, we had a max value instead. However, when the VMs use > of ObjectMonitors is first spinning up, the max value is typically very close > to the in-use count so we would always be asking for an async-deflation > during that spinning up phase. > > I created the idea of a ceiling value that is tied to thread count and the > AvgMonitorsPerThreadEstimate to replace the population value that we > used to have. By comparing the in-use count against the ceiling value, we > no longer exceed the MonitorUsedDeflationThreshold when the VMs use > of ObjectMonitors is first spinning up so we no longer do async deflations > continuously during that phase. If the max value exceeds the ceiling value, > then we're using a LOT of ObjectMonitors and, in that case, we compare > the in-use count against the max to determine if we're exceeding the > MonitorUsedDeflationThreshold. > > Does this help? It helps but I'm still wrestling with what MonitorUsedDeflationThreshold actually means now. So the existing MonitorUsedDeflationThreshold is used as a measure of the proportion of monitors actually in-use compared to the number of monitors pre-allocated. If an inflation request requires a new block to be allocated and we're above MonitorUsedDeflationThreshold % then a request for async deflation occurs (when we actually check). The new code, IIUC, says, lets assume we expect AvgMonitorsPerThreadEstimate monitors-per-thread. If the number of monitors in-use is > MonitorUsedDeflationThreshold % of (AvgMonitorsPerThreadEstimate * number_of_threads), then we request async deflation. So ... obviously we need some kind of watermark based system for requesting deflation otherwise there will be far too many deflation requests. And we also don't want to have check for exceeding the threshold on every monitor allocation. So the deflation thread will wakeup periodically and check if the threshold is exceeded. Okay ... so then it comes down to deciding whether AvgMonitorsPerThreadEstimate is the best way to establish the watermark and what the default value should be. This doesn't seem like something that an application developer could reasonably try to estimate so it is just going to be a tuning knob they adjust somewhat arbitrarily. I assume the 1024 default came from tuning something? Have you looked at the affect on memory use these changes have (ie peak RSS use)? Did your performance measurements look at using different values? (I can imagine that with enough memory we can effectively disable deflation and so potentially increase performance. OTOH maybe deflation is so infrequent it is a non-issue.) I have to confess that I never really thought about the old set of heuristics for this, but the fact we're changing the heuristics does raise a concern about what impact applications may see. BTW MonitorUsedDeflationThreshold should really be diagnostic not experimental, as real applications may need to tune it (and people often don't want to use experimental flags in production as a matter of policy). Thanks, David ----- > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/642 > From eosterlund at openjdk.java.net Mon Nov 9 08:38:01 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 9 Nov 2020 08:38:01 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Sat, 7 Nov 2020 17:01:42 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/objectMonitor.cpp line 380: >> >>> 378: if (event.should_commit()) { >>> 379: event.set_monitorClass(object()->klass()); >>> 380: event.set_address((uintptr_t)this); >> >> This looks wrong - the event should refer to the Object whose monitor we have entered, not the OM itself. > > I noticed that in my preliminary review of Erik's changes. He checked > with the JFR guys and they said it just needed to be an address and > does not have to refer to the Object. > > @fisk - can you think of a comment we should add here? We could write something along the lines of "An address that is 'unique enough', such that events close in time and with the same address are likely (but not guaranteed) to belong to the same object". This uniqueness property has always been more of a heuristic thing than anything else, as deflation shuffles the addresses around. Taking the this pointer vs an offset into the this pointer does however serve the exact same purpose; there was never any correlation to the contents of the object field. >> src/hotspot/share/runtime/objectMonitor.cpp line 1472: >> >>> 1470: event->set_monitorClass(monitor->object()->klass()); >>> 1471: event->set_timeout(timeout); >>> 1472: event->set_address((uintptr_t)monitor); >> >> Again the event should refer to the Object, not the OM. > > I noticed that in my preliminary review of Erik's changes. He checked > with the JFR guys and they said it just needed to be an address and > does not have to refer to the Object. > > @fisk - can you think of a comment we should add here? I wrote one in the section above, hope it is useful. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From eosterlund at openjdk.java.net Mon Nov 9 08:49:00 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 9 Nov 2020 08:49:00 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Sat, 7 Nov 2020 17:11:42 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/synchronizer.cpp line 94: >> >>> 92: // Find next live ObjectMonitor. >>> 93: ObjectMonitor* next = m; >>> 94: while (next != NULL && next->is_being_async_deflated()) { >> >> Nit: This loop seems odd. Given we know m is_being_async_deflated, this should either be a do/while loop, or else we should initialize: >> >> ObjectMonitor* next = m->next_om(); >> >> and dispense with the awkwardly named next_next. > > @fisk - I'm leaving this one for you for now. Changing it to a do/while loop makes sense. The while condition is always true the first iteration, so doing a while or do/while loop is equivalent. If you find the do/while loop easier to read, then that sounds good to me. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From eosterlund at openjdk.java.net Mon Nov 9 09:05:59 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 9 Nov 2020 09:05:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Sat, 7 Nov 2020 17:22:21 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/synchronizer.cpp line 1520: >> >>> 1518: // deflated in this cycle. >>> 1519: size_t deleted_count = 0; >>> 1520: for (ObjectMonitor* monitor: delete_list) { >> >> I didn't realize C++ has a "foreach" loop construct! Is this in our allowed C++ usage? > > @fisk - this one is for you... :-) Yeah this is one of the new cool features we can use. I thought it is allowed, because it is neither in the excluded nor undecided list of features in our doc/hotspot-style.md file. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From fparain at openjdk.java.net Mon Nov 9 15:48:01 2020 From: fparain at openjdk.java.net (Frederic Parain) Date: Mon, 9 Nov 2020 15:48:01 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo Message-ID: Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. Thank you, Fred ------------- Commit messages: - Cleanup fieldInfo structure Changes: https://git.openjdk.java.net/jdk/pull/1130/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1130&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256052 Stats: 121 lines in 5 files changed: 6 ins; 98 del; 17 mod Patch: https://git.openjdk.java.net/jdk/pull/1130.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1130/head:pull/1130 PR: https://git.openjdk.java.net/jdk/pull/1130 From redestad at openjdk.java.net Mon Nov 9 16:38:57 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Mon, 9 Nov 2020 16:38:57 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 15:43:09 GMT, Frederic Parain wrote: > Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. > > Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. > > Thank you, > > Fred Nice cleanup! src/hotspot/share/classfile/classFileParser.cpp line 1708: > 1706: > 1707: // Remember how many oops we encountered and compute allocation type > 1708: const FieldAllocationType atype = fac->update(is_static, type); The returned `FieldAllocationType` is never used at either call-site, so maybe the `update` method can be simplified, too? (It seems all `update` does is increment a per-type counter, so the name is a bit surprising) src/hotspot/share/runtime/vmStructs.cpp line 2261: > 2259: declare_preprocessor_constant("FIELDINFO_TAG_SIZE", FIELDINFO_TAG_SIZE) \ > 2260: declare_preprocessor_constant("FIELDINFO_TAG_OFFSET", FIELDINFO_TAG_OFFSET) \ > 2261: declare_preprocessor_constant("FIELDINFO_TAG_CONTENDED", FIELDINFO_TAG_CONTENDED) \ Not sure it's necessary to add this with no usage? ------------- Marked as reviewed by redestad (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1130 From shade at openjdk.java.net Mon Nov 9 19:06:00 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 9 Nov 2020 19:06:00 GMT Subject: RFR: 8253525: Implement getInstanceSize/sizeOf intrinsics [v5] In-Reply-To: <_TFbHUvH8zI29hGvpE3TGGNLGgi7PhP7rfed23up13U=.5a19aaed-cc1e-4d18-997d-f37616323e61@github.com> References: <_TFbHUvH8zI29hGvpE3TGGNLGgi7PhP7rfed23up13U=.5a19aaed-cc1e-4d18-997d-f37616323e61@github.com> Message-ID: On Wed, 21 Oct 2020 17:37:20 GMT, Aleksey Shipilev wrote: >> Good. > > Thanks for review, @kvn! I would also like a review from someone from serviceability. Friendly reminder. ------------- PR: https://git.openjdk.java.net/jdk/pull/650 From fparain at openjdk.java.net Mon Nov 9 19:06:12 2020 From: fparain at openjdk.java.net (Frederic Parain) Date: Mon, 9 Nov 2020 19:06:12 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v2] In-Reply-To: References: Message-ID: > Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. > > Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. > > Thank you, > > Fred Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: More cleanup ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1130/files - new: https://git.openjdk.java.net/jdk/pull/1130/files/c048832e..b822ba0e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1130&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1130&range=00-01 Stats: 6 lines in 1 file changed: 0 ins; 1 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/1130.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1130/head:pull/1130 PR: https://git.openjdk.java.net/jdk/pull/1130 From fparain at openjdk.java.net Mon Nov 9 19:58:11 2020 From: fparain at openjdk.java.net (Frederic Parain) Date: Mon, 9 Nov 2020 19:58:11 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v3] In-Reply-To: References: Message-ID: <0TM274lRiS3ITNLU8TU7z5IMIhTqJNifNA6DjsqOH5s=.4311e578-e29e-4469-b734-ac666e299f8b@github.com> > Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. > > Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. > > Thank you, > > Fred Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: Remove unused symbol from vmStruct ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1130/files - new: https://git.openjdk.java.net/jdk/pull/1130/files/b822ba0e..b4b24792 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1130&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1130&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1130.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1130/head:pull/1130 PR: https://git.openjdk.java.net/jdk/pull/1130 From lfoltan at openjdk.java.net Mon Nov 9 19:58:12 2020 From: lfoltan at openjdk.java.net (Lois Foltan) Date: Mon, 9 Nov 2020 19:58:12 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v3] In-Reply-To: <0TM274lRiS3ITNLU8TU7z5IMIhTqJNifNA6DjsqOH5s=.4311e578-e29e-4469-b734-ac666e299f8b@github.com> References: <0TM274lRiS3ITNLU8TU7z5IMIhTqJNifNA6DjsqOH5s=.4311e578-e29e-4469-b734-ac666e299f8b@github.com> Message-ID: On Mon, 9 Nov 2020 19:54:58 GMT, Frederic Parain wrote: >> Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. >> >> Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. >> >> Thank you, >> >> Fred > > Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: > > Remove unused symbol from vmStruct Marked as reviewed by lfoltan (Reviewer). Looks good Fred. ------------- PR: https://git.openjdk.java.net/jdk/pull/1130Marked as reviewed by lfoltan (Reviewer). From lfoltan at openjdk.java.net Mon Nov 9 19:58:14 2020 From: lfoltan at openjdk.java.net (Lois Foltan) Date: Mon, 9 Nov 2020 19:58:14 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v3] In-Reply-To: References: <0TM274lRiS3ITNLU8TU7z5IMIhTqJNifNA6DjsqOH5s=.4311e578-e29e-4469-b734-ac666e299f8b@github.com> Message-ID: On Mon, 9 Nov 2020 19:54:09 GMT, Lois Foltan wrote: >> Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove unused symbol from vmStruct > > Marked as reviewed by lfoltan (Reviewer). Looks good. ------------- PR: https://git.openjdk.java.net/jdk/pull/1130 From fparain at openjdk.java.net Mon Nov 9 19:58:13 2020 From: fparain at openjdk.java.net (Frederic Parain) Date: Mon, 9 Nov 2020 19:58:13 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v3] In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 16:36:07 GMT, Claes Redestad wrote: >> Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove unused symbol from vmStruct > > Nice cleanup! Hi Claes, Thank you for your review, the new version should address the points you raised. Fred > src/hotspot/share/classfile/classFileParser.cpp line 1708: > >> 1706: >> 1707: // Remember how many oops we encountered and compute allocation type >> 1708: const FieldAllocationType atype = fac->update(is_static, type); > > The returned `FieldAllocationType` is never used at either call-site, so maybe the `update` method can be simplified, too? (It seems all `update` does is increment a per-type counter, so the name is a bit surprising) Right, I removed the value returned by the `update()` method. > src/hotspot/share/runtime/vmStructs.cpp line 2261: > >> 2259: declare_preprocessor_constant("FIELDINFO_TAG_SIZE", FIELDINFO_TAG_SIZE) \ >> 2260: declare_preprocessor_constant("FIELDINFO_TAG_OFFSET", FIELDINFO_TAG_OFFSET) \ >> 2261: declare_preprocessor_constant("FIELDINFO_TAG_CONTENDED", FIELDINFO_TAG_CONTENDED) \ > > Not sure it's necessary to add this with no usage? Not necessary, removed. ------------- PR: https://git.openjdk.java.net/jdk/pull/1130 From hseigel at openjdk.java.net Mon Nov 9 19:58:15 2020 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 9 Nov 2020 19:58:15 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v3] In-Reply-To: <0TM274lRiS3ITNLU8TU7z5IMIhTqJNifNA6DjsqOH5s=.4311e578-e29e-4469-b734-ac666e299f8b@github.com> References: <0TM274lRiS3ITNLU8TU7z5IMIhTqJNifNA6DjsqOH5s=.4311e578-e29e-4469-b734-ac666e299f8b@github.com> Message-ID: On Mon, 9 Nov 2020 19:54:58 GMT, Frederic Parain wrote: >> Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. >> >> Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. >> >> Thank you, >> >> Fred > > Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: > > Remove unused symbol from vmStruct src/hotspot/share/oops/fieldInfo.hpp line 61: > 59: // [------------------offset----------------]01 - real field offset > 60: > 61: // Bit O indicates if the packed field contains an offset (O=1) or not (O=1) Hi Fred, should this comment say "... or not (0=0) ? ------------- PR: https://git.openjdk.java.net/jdk/pull/1130 From fparain at openjdk.java.net Mon Nov 9 20:05:10 2020 From: fparain at openjdk.java.net (Frederic Parain) Date: Mon, 9 Nov 2020 20:05:10 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v3] In-Reply-To: References: <0TM274lRiS3ITNLU8TU7z5IMIhTqJNifNA6DjsqOH5s=.4311e578-e29e-4469-b734-ac666e299f8b@github.com> Message-ID: On Mon, 9 Nov 2020 19:53:32 GMT, Harold Seigel wrote: >> Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove unused symbol from vmStruct > > src/hotspot/share/oops/fieldInfo.hpp line 61: > >> 59: // [------------------offset----------------]01 - real field offset >> 60: >> 61: // Bit O indicates if the packed field contains an offset (O=1) or not (O=1) > > Hi Fred, should this comment say "... or not (0=0) ? You're right. I've fixed the comment (and the line below which had the same issue). Fred ------------- PR: https://git.openjdk.java.net/jdk/pull/1130 From fparain at openjdk.java.net Mon Nov 9 20:05:09 2020 From: fparain at openjdk.java.net (Frederic Parain) Date: Mon, 9 Nov 2020 20:05:09 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v4] In-Reply-To: References: Message-ID: > Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. > > Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. > > Thank you, > > Fred Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: Fix comment ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1130/files - new: https://git.openjdk.java.net/jdk/pull/1130/files/b4b24792..18ad6490 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1130&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1130&range=02-03 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1130.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1130/head:pull/1130 PR: https://git.openjdk.java.net/jdk/pull/1130 From daniel.daugherty at oracle.com Mon Nov 9 20:16:58 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 9 Nov 2020 15:16:58 -0500 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: Hi David, I'm going to try replying to this review comment via email to see how that works out... On 11/9/20 1:06 AM, David Holmes wrote: > Hi Dan, > > On 9/11/2020 1:50 pm, Daniel D.Daugherty wrote: >> On Sun, 8 Nov 2020 21:43:00 GMT, David Holmes >> wrote: >> >>>> How about this: >>>> ?? static MonitorList?? _in_use_list; >>>> ?? // The ratio of the current _in_use_list count to the ceiling is >>>> used >>>> ?? // to determine if we are above MonitorUsedDeflationThreshold >>>> and need >>>> ?? // to do an async monitor deflation cycle. The ceiling is >>>> increased by >>>> ?? // AvgMonitorsPerThreadEstimate when a thread is added to the >>>> system >>>> ?? // and is decreased by AvgMonitorsPerThreadEstimate when a >>>> thread is >>>> ?? // removed from the system. >>>> ?? // Note: If the _in_use_list max exceeds the ceiling, then >>>> ?? // monitors_used_above_threshold() will use the in_use_list max >>>> instead >>>> ?? // of the thread count derived ceiling because we have used more >>>> ?? // ObjectMonitors than the estimated average. >>>> ?? static jint????????? _in_use_list_ceiling; >>> >>> Thanks for the comment. So instead of checking the threshhold on >>> each OM allocation we use this averaging technique to estimate the >>> number of monitors in use? Can you explain how this came about >>> rather than the simple/obvious check at allocation time. Thanks. >> >> I'm not sure I understand your question, but let me that a stab at it >> anyway... >> >> We used to compare the sum of the in-use counts from all the in-use >> lists >> with the total population of ObjectMonitors. If that ratio was higher >> than >> MonitorUsedDeflationThreshold, then we would do an async deflation >> cycle. >> Since we got rid of TSM, we no longer had a population of already >> allocated >> ObjectMonitors, we had a max value instead. However, when the VMs use >> of ObjectMonitors is first spinning up, the max value is typically >> very close >> to the in-use count so we would always be asking for an async-deflation >> during that spinning up phase. >> >> I created the idea of a ceiling value that is tied to thread count >> and the >> AvgMonitorsPerThreadEstimate to replace the population value that we >> used to have. By comparing the in-use count against the ceiling >> value, we >> no longer exceed the MonitorUsedDeflationThreshold when the VMs use >> of ObjectMonitors is first spinning up so we no longer do async >> deflations >> continuously during that phase. If the max value exceeds the ceiling >> value, >> then we're using a LOT of ObjectMonitors and, in that case, we compare >> the in-use count against the max to determine if we're exceeding the >> MonitorUsedDeflationThreshold. >> >> Does this help? > > It helps but I'm still wrestling with what > MonitorUsedDeflationThreshold actually means now. > > So the existing MonitorUsedDeflationThreshold is used as a measure of > the proportion of monitors actually in-use compared to the number of > monitors pre-allocated. Slight correction: not a measure of proportion, but a threshold on the proportion. > If an inflation request requires a new block to be allocated and we're > above MonitorUsedDeflationThreshold % then a request for async > deflation occurs (when we actually check). Not quite. We did have code in om_alloc() that could cause a deflation cycle to be invoked, but that code was not enabled by default and was removed. See the fix for: ??? JDK-8230940 Obsolete MonitorBound ??? https://bugs.openjdk.java.net/browse/JDK-8230940 which was pushed in jdk-15-B22. In the current baseline, ObjectSynchronizer::is_async_deflation_needed() is used to ask if we need to perform an async deflation cycle. That function calls monitors_used_above_threshold() which is where the logic that uses the MonitorUsedDeflationThreshold value comes into play. > The new code, IIUC, says, lets assume we expect > AvgMonitorsPerThreadEstimate monitors-per-thread. If the number of > monitors in-use is > MonitorUsedDeflationThreshold % of > (AvgMonitorsPerThreadEstimate * number_of_threads), then we request > async deflation. You understand correctly. > So ... obviously we need some kind of watermark based system for > requesting deflation otherwise there will be far too many deflation > requests. Yes. That's what I saw before I added AvgMonitorsPerThreadEstimate and the ceiling. Although, I think I like the phrase "watermark" better! Should I rename this new ceiling concept to watermark?? > And we also don't want to have check for exceeding the threshold on > every monitor allocation. So the deflation thread will wakeup > periodically and check if the threshold is exceeded. Exactly. GuaranteedSafepointInterval (default 1sec) controls how often the MonitorDeflationThread will wake up on its own to check for work. AsyncDeflationInterval (default 250ms) controls at most how often the MonitorDeflationThread will actually do the work. However, if an async deflation cycle is directly requested, then it will be honored regardless of the limits. At this point, only the WhiteBox support uses that feature. Although we do have an RFE from you to restore that feature to System.gc(): ? JDK-8249638 Re-instate idle monitor deflation as part of System.gc() ? https://bugs.openjdk.java.net/browse/JDK-8249638 > Okay ... so then it comes down to deciding whether > AvgMonitorsPerThreadEstimate is the best way to establish the > watermark and what the default value should be. This doesn't seem like > something that an application developer could reasonably try to > estimate so it is just going to be a tuning knob they adjust somewhat > arbitrarily. I actually don't expect anyone to care about AvgMonitorsPerThreadEstimate, but I've created it just-in-case. > I assume the 1024 default came from tuning something? The default 1024 value is the closest I can come to the baseline VM behavior where we allowed a per-thread free-list to allocate at most 1024 ObjectMonitors in one attempt. Since the per-thread free list is the fastest code path for an ObjectMonitor allocation in the baseline VM, I thought using that value made for a reasonable default for the AvgMonitorsPerThreadEstimate. However, we don't pre-allocate in the new code so a thread doesn't actually have (at most) 1024 ObjectMonitors waiting on the per-thread free list for fast allocation. > Have you looked at the affect on memory use these changes have (ie > peak RSS use)? Actually what I've been looking at is population values in the baseline VM and the max in the new code with Kitchensink8H runs. I don't have the absolutely latest values, but, IIRC, the new code has a much smaller max value than the baseline population value. Obviously this just tells me the raw numbers of ObjectMonitors and doesn't include any "hidden" overhead that would be captured by RSS values, but I think it gives me a reasonable feel for memory utilization. > Did your performance measurements look at using different values? No. All my performance runs used default values. I tend to focus on out-of-the-box settings unless I'm looking to justify changing some default value. > (I can imagine that with enough memory we can effectively disable > deflation and so potentially increase performance. OTOH maybe > deflation is so infrequent it is a non-issue.) > > I have to confess that I never really thought about the old set of > heuristics for this, but the fact we're changing the heuristics does > raise a concern about what impact applications may see. None of the performance testing that we've done so far has raised any concerns about the new heuristics. The new ObjectMonitor inflation mechanism is much, much faster than the old mechanism. During my Inflate2 stress testing, the baseline VM would peak at a population of about 12 million at 4.5 hours into an 8 hour run with release bits. With the new code, Inflate2 would reach a max of 400+ million at about an hour with no signs that was the actual peak when I stopped the run. > BTW MonitorUsedDeflationThreshold should really be diagnostic not > experimental, as real applications may need to tune it (and people > often don't want to use experimental flags in production as a matter > of policy). MonitorUsedDeflationThreshold wasn't added with this project. It was added by Robbin using this bug ID: ??? JDK-8181859 Monitor deflation is not checked in cleanup path ??? https://bugs.openjdk.java.net/browse/JDK-8181859 way back in jdk-10-B21... I don't know the reason that Robbin created the option as experimental rather than diagnostic, but I can investigate. Thanks again for the review! At this point, I don't see anything that I plan to change in response to this set of comments. I do have a query up above about renaming the ceiling concept to watermark. Please let me know what you think. Dan > > Thanks, > David > ----- > >> ------------- >> >> PR: https://git.openjdk.java.net/jdk/pull/642 >> From hseigel at openjdk.java.net Mon Nov 9 20:26:57 2020 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 9 Nov 2020 20:26:57 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v4] In-Reply-To: References: Message-ID: <-Wc3FeXsLIx0x0U3gt0oSW6qiTGSRuMKVr4qJ3aTTXE=.7ed06bf5-f795-44ca-99ca-848d199a2733@github.com> On Mon, 9 Nov 2020 20:05:09 GMT, Frederic Parain wrote: >> Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. >> >> Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. >> >> Thank you, >> >> Fred > > Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: > > Fix comment Looks good! ------------- Marked as reviewed by hseigel (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1130 From dcubed at openjdk.java.net Mon Nov 9 20:39:59 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 9 Nov 2020 20:39:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: <-VFMKh1P4U674scgRXbBch7b4eE91O4ZelExZOeXByw=.e4acb6ea-a3c6-4ac0-a6d2-fe2a09cc178b@github.com> On Mon, 9 Nov 2020 08:34:56 GMT, Erik ?sterlund wrote: >> I noticed that in my preliminary review of Erik's changes. He checked >> with the JFR guys and they said it just needed to be an address and >> does not have to refer to the Object. >> >> @fisk - can you think of a comment we should add here? > > We could write something along the lines of "An address that is 'unique enough', such that events close in time and with the same address are likely (but not guaranteed) to belong to the same object". This uniqueness property has always been more of a heuristic thing than anything else, as deflation shuffles the addresses around. Taking the this pointer vs an offset into the this pointer does however serve the exact same purpose; there was never any correlation to the contents of the object field. Thanks @fisk! I've added a slightly edited version of the comment: // Set an address that is 'unique enough', such that events close in // time and with the same address are likely (but not guaranteed) to // belong to the same object. @dholmes-ora - does this work for you? >> I noticed that in my preliminary review of Erik's changes. He checked >> with the JFR guys and they said it just needed to be an address and >> does not have to refer to the Object. >> >> @fisk - can you think of a comment we should add here? > > I wrote one in the section above, hope it is useful. Thanks @fisk. I copied the same comment here since it is about 1000 lines away from the other comment. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Mon Nov 9 20:43:06 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 9 Nov 2020 20:43:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: On Thu, 5 Nov 2020 12:42:40 GMT, Coleen Phillimore wrote: >> Thank you for the update, Coleen! >> I leave it for you to decide to refactor the gc_notification or not. >> Thanks, >> Serguei > > Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change. This change also passes tier 7,8 testing. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From dcubed at openjdk.java.net Mon Nov 9 20:44:56 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 9 Nov 2020 20:44:56 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: <9K1VRMjKJ4TIYx2Dc2Cn98vmFTIAaZ-l903nmvcl3rI=.c91f9d0b-e385-4c6c-977e-5555175a3783@github.com> On Mon, 9 Nov 2020 08:45:49 GMT, Erik ?sterlund wrote: >> @fisk - I'm leaving this one for you for now. > > Changing it to a do/while loop makes sense. The while condition is always true the first iteration, so doing a while or do/while loop is equivalent. If you find the do/while loop easier to read, then that sounds good to me. Okay. I've changed it to a do-while loop. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Mon Nov 9 20:51:17 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 9 Nov 2020 20:51:17 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Mon, 9 Nov 2020 03:19:17 GMT, Daniel D. Daugherty wrote: >> @dholmes-ora - Thanks for the review! Hmm... I'm not sure why the GitHub UI >> send out my replies one-at-a-time. Perhaps I should have replied from the >> "files" view instead of the main PR view? > > It is a deflated monitor that is still on the in-use list. I've made all the changes based on @fisk's replies to @dholmes-ora comments. @dholmes-ora - I think I've addressed all of your comments. Please let me know if I've missing something. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Mon Nov 9 20:51:17 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 9 Nov 2020 20:51:17 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also done promotion perf runs on these bits and says "the results look fine". Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: Resolve more @dholmes-ora comments with help from @fisk. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/642/files - new: https://git.openjdk.java.net/jdk/pull/642/files/6c2db34a..2b668f08 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=01-02 Stats: 8 lines in 2 files changed: 6 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/642.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/642/head:pull/642 PR: https://git.openjdk.java.net/jdk/pull/642 From david.holmes at oracle.com Mon Nov 9 23:07:21 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 10 Nov 2020 09:07:21 +1000 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On 9/11/2020 7:05 pm, Erik ?sterlund wrote: > On Sat, 7 Nov 2020 17:22:21 GMT, Daniel D. Daugherty wrote: > >>> src/hotspot/share/runtime/synchronizer.cpp line 1520: >>> >>>> 1518: // deflated in this cycle. >>>> 1519: size_t deleted_count = 0; >>>> 1520: for (ObjectMonitor* monitor: delete_list) { >>> >>> I didn't realize C++ has a "foreach" loop construct! Is this in our allowed C++ usage? >> >> @fisk - this one is for you... :-) > > Yeah this is one of the new cool features we can use. I thought it is allowed, because it is neither in the excluded nor undecided list of features in our doc/hotspot-style.md file. But also not in the allowed list yet, so I'm checking on this - ref: https://bugs.openjdk.java.net/browse/JDK-8254733 Cheers, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/642 > From david.holmes at oracle.com Mon Nov 9 23:09:59 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 10 Nov 2020 09:09:59 +1000 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: <265f6954-5a6d-1535-2e3f-48f56cb7e927@oracle.com> Hi Dan, I'm going to top-post just to keep this short :) Thanks for all the details below on the performance aspects and use of the heuristics - all good. Regarding "ceiling" versus "watermark" ... a "high watermark" is a ceiling so its really a matter of personal preference what terminology you want to use. Thanks again, David On 10/11/2020 6:16 am, Daniel D. Daugherty wrote: > Hi David, > > I'm going to try replying to this review comment via email to see how that > works out... > > On 11/9/20 1:06 AM, David Holmes wrote: >> Hi Dan, >> >> On 9/11/2020 1:50 pm, Daniel D.Daugherty wrote: >>> On Sun, 8 Nov 2020 21:43:00 GMT, David Holmes >>> wrote: >>> >>>>> How about this: >>>>> ?? static MonitorList?? _in_use_list; >>>>> ?? // The ratio of the current _in_use_list count to the ceiling is >>>>> used >>>>> ?? // to determine if we are above MonitorUsedDeflationThreshold >>>>> and need >>>>> ?? // to do an async monitor deflation cycle. The ceiling is >>>>> increased by >>>>> ?? // AvgMonitorsPerThreadEstimate when a thread is added to the >>>>> system >>>>> ?? // and is decreased by AvgMonitorsPerThreadEstimate when a >>>>> thread is >>>>> ?? // removed from the system. >>>>> ?? // Note: If the _in_use_list max exceeds the ceiling, then >>>>> ?? // monitors_used_above_threshold() will use the in_use_list max >>>>> instead >>>>> ?? // of the thread count derived ceiling because we have used more >>>>> ?? // ObjectMonitors than the estimated average. >>>>> ?? static jint????????? _in_use_list_ceiling; >>>> >>>> Thanks for the comment. So instead of checking the threshhold on >>>> each OM allocation we use this averaging technique to estimate the >>>> number of monitors in use? Can you explain how this came about >>>> rather than the simple/obvious check at allocation time. Thanks. >>> >>> I'm not sure I understand your question, but let me that a stab at it >>> anyway... >>> >>> We used to compare the sum of the in-use counts from all the in-use >>> lists >>> with the total population of ObjectMonitors. If that ratio was higher >>> than >>> MonitorUsedDeflationThreshold, then we would do an async deflation >>> cycle. >>> Since we got rid of TSM, we no longer had a population of already >>> allocated >>> ObjectMonitors, we had a max value instead. However, when the VMs use >>> of ObjectMonitors is first spinning up, the max value is typically >>> very close >>> to the in-use count so we would always be asking for an async-deflation >>> during that spinning up phase. >>> >>> I created the idea of a ceiling value that is tied to thread count >>> and the >>> AvgMonitorsPerThreadEstimate to replace the population value that we >>> used to have. By comparing the in-use count against the ceiling >>> value, we >>> no longer exceed the MonitorUsedDeflationThreshold when the VMs use >>> of ObjectMonitors is first spinning up so we no longer do async >>> deflations >>> continuously during that phase. If the max value exceeds the ceiling >>> value, >>> then we're using a LOT of ObjectMonitors and, in that case, we compare >>> the in-use count against the max to determine if we're exceeding the >>> MonitorUsedDeflationThreshold. >>> >>> Does this help? >> >> It helps but I'm still wrestling with what >> MonitorUsedDeflationThreshold actually means now. >> >> So the existing MonitorUsedDeflationThreshold is used as a measure of >> the proportion of monitors actually in-use compared to the number of >> monitors pre-allocated. > > Slight correction: not a measure of proportion, but a threshold on the > proportion. > > >> If an inflation request requires a new block to be allocated and we're >> above MonitorUsedDeflationThreshold % then a request for async >> deflation occurs (when we actually check). > > Not quite. We did have code in om_alloc() that could cause a deflation > cycle to be invoked, but that code was not enabled by default and was > removed. See the fix for: > > ??? JDK-8230940 Obsolete MonitorBound > ??? https://bugs.openjdk.java.net/browse/JDK-8230940 > > which was pushed in jdk-15-B22. > > In the current baseline, ObjectSynchronizer::is_async_deflation_needed() > is used to ask if we need to perform an async deflation cycle. That > function calls monitors_used_above_threshold() which is where the logic > that uses the MonitorUsedDeflationThreshold value comes into play. > > >> The new code, IIUC, says, lets assume we expect >> AvgMonitorsPerThreadEstimate monitors-per-thread. If the number of >> monitors in-use is > MonitorUsedDeflationThreshold % of >> (AvgMonitorsPerThreadEstimate * number_of_threads), then we request >> async deflation. > > You understand correctly. > > >> So ... obviously we need some kind of watermark based system for >> requesting deflation otherwise there will be far too many deflation >> requests. > > Yes. That's what I saw before I added AvgMonitorsPerThreadEstimate and > the ceiling. Although, I think I like the phrase "watermark" better! > > Should I rename this new ceiling concept to watermark?? > > >> And we also don't want to have check for exceeding the threshold on >> every monitor allocation. So the deflation thread will wakeup >> periodically and check if the threshold is exceeded. > > Exactly. GuaranteedSafepointInterval (default 1sec) controls how often > the MonitorDeflationThread will wake up on its own to check for work. > AsyncDeflationInterval (default 250ms) controls at most how often > the MonitorDeflationThread will actually do the work. > > However, if an async deflation cycle is directly requested, then it > will be honored regardless of the limits. At this point, only the > WhiteBox support uses that feature. Although we do have an RFE from > you to restore that feature to System.gc(): > > ? JDK-8249638 Re-instate idle monitor deflation as part of System.gc() > ? https://bugs.openjdk.java.net/browse/JDK-8249638 > > >> Okay ... so then it comes down to deciding whether >> AvgMonitorsPerThreadEstimate is the best way to establish the >> watermark and what the default value should be. This doesn't seem like >> something that an application developer could reasonably try to >> estimate so it is just going to be a tuning knob they adjust somewhat >> arbitrarily. > > I actually don't expect anyone to care about AvgMonitorsPerThreadEstimate, > but I've created it just-in-case. > > >> I assume the 1024 default came from tuning something? > > The default 1024 value is the closest I can come to the baseline VM > behavior where we allowed a per-thread free-list to allocate at most > 1024 ObjectMonitors in one attempt. Since the per-thread free list is > the fastest code path for an ObjectMonitor allocation in the baseline > VM, I thought using that value made for a reasonable default for the > AvgMonitorsPerThreadEstimate. > > However, we don't pre-allocate in the new code so a thread doesn't > actually have (at most) 1024 ObjectMonitors waiting on the per-thread > free list for fast allocation. > > >> Have you looked at the affect on memory use these changes have (ie >> peak RSS use)? > > Actually what I've been looking at is population values in the baseline > VM and the max in the new code with Kitchensink8H runs. I don't have > the absolutely latest values, but, IIRC, the new code has a much smaller > max value than the baseline population value. > > Obviously this just tells me the raw numbers of ObjectMonitors and doesn't > include any "hidden" overhead that would be captured by RSS values, but I > think it gives me a reasonable feel for memory utilization. > > >> Did your performance measurements look at using different values? > > No. All my performance runs used default values. I tend to focus on > out-of-the-box settings unless I'm looking to justify changing some > default value. > > >> (I can imagine that with enough memory we can effectively disable >> deflation and so potentially increase performance. OTOH maybe >> deflation is so infrequent it is a non-issue.) >> >> I have to confess that I never really thought about the old set of >> heuristics for this, but the fact we're changing the heuristics does >> raise a concern about what impact applications may see. > > None of the performance testing that we've done so far has raised any > concerns about the new heuristics. > > The new ObjectMonitor inflation mechanism is much, much faster than > the old mechanism. During my Inflate2 stress testing, the baseline VM > would peak at a population of about 12 million at 4.5 hours into an 8 > hour run with release bits. With the new code, Inflate2 would reach a > max of 400+ million at about an hour with no signs that was the actual > peak when I stopped the run. > > >> BTW MonitorUsedDeflationThreshold should really be diagnostic not >> experimental, as real applications may need to tune it (and people >> often don't want to use experimental flags in production as a matter >> of policy). > > MonitorUsedDeflationThreshold wasn't added with this project. It was > added by Robbin using this bug ID: > > ??? JDK-8181859 Monitor deflation is not checked in cleanup path > ??? https://bugs.openjdk.java.net/browse/JDK-8181859 > > way back in jdk-10-B21... I don't know the reason that Robbin created > the option as experimental rather than diagnostic, but I can investigate. > > Thanks again for the review! > > > At this point, I don't see anything that I plan to change in response > to this set of comments. I do have a query up above about renaming the > ceiling concept to watermark. Please let me know what you think. > > Dan > >> >> Thanks, >> David >> ----- >> >>> ------------- >>> >>> PR: https://git.openjdk.java.net/jdk/pull/642 >>> > From dcubed at openjdk.java.net Mon Nov 9 23:09:59 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 9 Nov 2020 23:09:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Mon, 9 Nov 2020 09:03:37 GMT, Erik ?sterlund wrote: >> @fisk - this one is for you... :-) > > Yeah this is one of the new cool features we can use. I thought it is allowed, because it is neither in the excluded nor undecided list of features in our doc/hotspot-style.md file. https://bugs.openjdk.java.net/browse/JDK-8254733 ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Mon Nov 9 23:18:03 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 9 Nov 2020 23:18:03 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Fri, 6 Nov 2020 02:40:44 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> Resolve more @dholmes-ora comments with help from @fisk. > > Hi Dan, > > Overall this looks great. Comparing old and new code is complex but the new code on its own is generally much simpler/clearer (not all though :) ). > > I have a few nits, comments and queries below. > > Thanks, > David @dholmes-ora - I'll stick with ceiling for now. If you're satisfied with the changeset, please mark this PR as approved. @fisk - Please mark this PR as approved if you're happy with the current version. @robehn and @coleenp - It would be good to hear from one or both of you... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From david.holmes at oracle.com Mon Nov 9 23:18:38 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 10 Nov 2020 09:18:38 +1000 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v2] In-Reply-To: <-VFMKh1P4U674scgRXbBch7b4eE91O4ZelExZOeXByw=.e4acb6ea-a3c6-4ac0-a6d2-fe2a09cc178b@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> <-VFMKh1P4U674scgRXbBch7b4eE91O4ZelExZOeXByw=.e4acb6ea-a3c6-4ac0-a6d2-fe2a09cc178b@github.com> Message-ID: <6dcb9110-fb3e-11c6-6685-6415650025c8@oracle.com> On 10/11/2020 6:39 am, Daniel D.Daugherty wrote: > On Mon, 9 Nov 2020 08:34:56 GMT, Erik ?sterlund wrote: > >>> I noticed that in my preliminary review of Erik's changes. He checked >>> with the JFR guys and they said it just needed to be an address and >>> does not have to refer to the Object. >>> >>> @fisk - can you think of a comment we should add here? >> >> We could write something along the lines of "An address that is 'unique enough', such that events close in time and with the same address are likely (but not guaranteed) to belong to the same object". This uniqueness property has always been more of a heuristic thing than anything else, as deflation shuffles the addresses around. Taking the this pointer vs an offset into the this pointer does however serve the exact same purpose; there was never any correlation to the contents of the object field. > > Thanks @fisk! I've added a slightly edited version of the comment: > // Set an address that is 'unique enough', such that events close in > // time and with the same address are likely (but not guaranteed) to > // belong to the same object. > @dholmes-ora - does this work for you? Yes that is fine - thanks. David ----- From dholmes at openjdk.java.net Mon Nov 9 23:25:00 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Mon, 9 Nov 2020 23:25:00 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Mon, 9 Nov 2020 20:51:17 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > Resolve more @dholmes-ora comments with help from @fisk. src/hotspot/share/runtime/synchronizer.cpp line 89: > 87: ObjectMonitor* head = Atomic::load_acquire(&_head); > 88: ObjectMonitor* m = head; > 89: do { This wasn't the loop I was referring to. It is the while loop below this at line 93. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From daniel.daugherty at oracle.com Mon Nov 9 23:29:03 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 9 Nov 2020 18:29:03 -0500 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On 11/9/20 6:25 PM, David Holmes wrote: > On Mon, 9 Nov 2020 20:51:17 GMT, Daniel D. Daugherty wrote: > >>> Changes from @fisk and @dcubed-ojdk to: >>> >>> - simplify ObjectMonitor list management >>> - get rid of Type-Stable Memory (TSM) >>> >>> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >>> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >>> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >>> - a few minor regressions (<= -0.24%) >>> - Volano is 6.8% better >>> >>> Eric C. has also done promotion perf runs on these bits and says "the results look fine". >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> Resolve more @dholmes-ora comments with help from @fisk. > src/hotspot/share/runtime/synchronizer.cpp line 89: > >> 87: ObjectMonitor* head = Atomic::load_acquire(&_head); >> 88: ObjectMonitor* m = head; >> 89: do { > This wasn't the loop I was referring to. It is the while loop below this at line 93. The PR is showing comments from you for both while loops. And at this point, both have been tweaked... Dan > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/642 From dholmes at openjdk.java.net Tue Nov 10 01:05:59 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 10 Nov 2020 01:05:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: <9K1VRMjKJ4TIYx2Dc2Cn98vmFTIAaZ-l903nmvcl3rI=.c91f9d0b-e385-4c6c-977e-5555175a3783@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> <9K1VRMjKJ4TIYx2Dc2Cn98vmFTIAaZ-l903nmvcl3rI=.c91f9d0b-e385-4c6c-977e-5555175a3783@github.com> Message-ID: On Mon, 9 Nov 2020 20:42:19 GMT, Daniel D. Daugherty wrote: >> Changing it to a do/while loop makes sense. The while condition is always true the first iteration, so doing a while or do/while loop is equivalent. If you find the do/while loop easier to read, then that sounds good to me. > > Okay. I've changed it to a do-while loop. The loop that was being discussed here is the one on line 93 of the original changeset: `while (next != NULL && next->is_being_async_deflated()) {` I made no comment on the outer while loop. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 02:17:57 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 02:17:57 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> <9K1VRMjKJ4TIYx2Dc2Cn98vmFTIAaZ-l903nmvcl3rI=.c91f9d0b-e385-4c6c-977e-5555175a3783@github.com> Message-ID: On Tue, 10 Nov 2020 01:03:00 GMT, David Holmes wrote: >> Okay. I've changed it to a do-while loop. > > The loop that was being discussed here is the one on line 93 of the original changeset: > `while (next != NULL && next->is_being_async_deflated()) {` > I made no comment on the outer while loop. You're correct. I got confused by the line renumbering due to other changes. Fortunately, the outer while loop can also correctly and easily be changed into a do-while loop so I'll keep that change. I'm looking at the correct inner while loop now. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 02:35:18 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 02:35:18 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> <9K1VRMjKJ4TIYx2Dc2Cn98vmFTIAaZ-l903nmvcl3rI=.c91f9d0b-e385-4c6c-977e-5555175a3783@github.com> Message-ID: On Tue, 10 Nov 2020 02:15:19 GMT, Daniel D. Daugherty wrote: >> The loop that was being discussed here is the one on line 93 of the original changeset: >> `while (next != NULL && next->is_being_async_deflated()) {` >> I made no comment on the outer while loop. > > You're correct. I got confused by the line renumbering due to other changes. > Fortunately, the outer while loop can also correctly and easily be changed > into a do-while loop so I'll keep that change. I'm looking at the correct inner > while loop now. The inner while loop is now converted into a do-while loop. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 02:35:17 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 02:35:17 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also done promotion perf runs on these bits and says "the results look fine". Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/642/files - new: https://git.openjdk.java.net/jdk/pull/642/files/2b668f08..aec90b9a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=02-03 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/642.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/642/head:pull/642 PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 02:35:19 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 02:35:19 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v3] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: <9lQzjB5MoTaMBnO_24juUkl6I3s8DONmuSt5CiKCyP8=.8dd7b150-6953-430d-bf56-91b738af7832@github.com> On Mon, 9 Nov 2020 23:21:48 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> Resolve more @dholmes-ora comments with help from @fisk. > > src/hotspot/share/runtime/synchronizer.cpp line 89: > >> 87: ObjectMonitor* head = Atomic::load_acquire(&_head); >> 88: ObjectMonitor* m = head; >> 89: do { > > This wasn't the loop I was referring to. It is the while loop below this at line 93. The inner while loop is now converted into a do-while loop. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From david.holmes at oracle.com Tue Nov 10 03:30:13 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 10 Nov 2020 13:30:13 +1000 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> <9K1VRMjKJ4TIYx2Dc2Cn98vmFTIAaZ-l903nmvcl3rI=.c91f9d0b-e385-4c6c-977e-5555175a3783@github.com> Message-ID: <081aacad-8ab4-a928-5d43-3584777bc7cf@oracle.com> On 10/11/2020 12:35 pm, Daniel D.Daugherty wrote: > On Tue, 10 Nov 2020 02:15:19 GMT, Daniel D. Daugherty wrote: > >>> The loop that was being discussed here is the one on line 93 of the original changeset: >>> `while (next != NULL && next->is_being_async_deflated()) {` >>> I made no comment on the outer while loop. >> >> You're correct. I got confused by the line renumbering due to other changes. >> Fortunately, the outer while loop can also correctly and easily be changed >> into a do-while loop so I'll keep that change. I'm looking at the correct inner >> while loop now. > > The inner while loop is now converted into a do-while loop. There was a little more to it than just that, but nevermind I think I see why next_next is needed, so lets move on. Thanks, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/642 > From dholmes at openjdk.java.net Tue Nov 10 03:32:01 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 10 Nov 2020 03:32:01 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 02:35:17 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). Marked as reviewed by dholmes (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From sspitsyn at openjdk.java.net Tue Nov 10 11:05:02 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Tue, 10 Nov 2020 11:05:02 GMT Subject: RFR: 8253525: Implement getInstanceSize/sizeOf intrinsics [v5] In-Reply-To: References: <_TFbHUvH8zI29hGvpE3TGGNLGgi7PhP7rfed23up13U=.5a19aaed-cc1e-4d18-997d-f37616323e61@github.com> Message-ID: On Mon, 9 Nov 2020 19:02:41 GMT, Aleksey Shipilev wrote: >> Thanks for review, @kvn! I would also like a review from someone from serviceability. > > Friendly reminder. Hi Aleksey, I've not looked at the compiler generated code. The fix looks okay to me. I have a question and a couple of minor suggestions on new test. Q: Why the value of ITERS is that big? What is the need to have this number of iterations? Also, I do not like one-letter identifiers, especially if they are not local. Could you, please, replace identifiers R and A with some short versions that give a hint. Something like REFSIZE and ALIGNMENT would be good enough. Also, what tests did you run to make sure no regression is introduced? Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/650 From github.com+654217+bastie at openjdk.java.net Tue Nov 10 11:47:55 2020 From: github.com+654217+bastie at openjdk.java.net (Sebastian Ritter) Date: Tue, 10 Nov 2020 11:47:55 GMT Subject: RFR: 8066622 8066637: remove deprecated using java.io.File.toURL() in internal classes In-Reply-To: References: Message-ID: On Sun, 8 Nov 2020 16:58:24 GMT, Phil Race wrote: > You reference a desktop bug that discusses many, many deprecations ... Yep. In my opinion this is a bot problem here and need other place to discuss. > Yet you propose to fix precisely one of these. @prrace: Not really. The way to work with problems differ. Both bugs - maybe these are more quality change requests than bugs - are bullet lists and are created on build process. The different is, I work on single quality tests and remove deprecated code from source base. So in my clean Java build the method java.io.File.toUrl() does not(!) exist. Because java.io.File.toURL() doesn't exist, also two patches are needed for JDK build process (internal I use diff-patch). > And I'd like to hear whether you actually _tested_ splashscreen with your change ? I see no sign that you did. I'm not sure, what you want to listen. I work with a clean Java build with patch to remove java.io.File.toURL() and change the collateral damages. ------------- PR: https://git.openjdk.java.net/jdk/pull/1108 From shade at openjdk.java.net Tue Nov 10 12:04:21 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 10 Nov 2020 12:04:21 GMT Subject: RFR: 8253525: Implement getInstanceSize/sizeOf intrinsics [v5] In-Reply-To: References: <_TFbHUvH8zI29hGvpE3TGGNLGgi7PhP7rfed23up13U=.5a19aaed-cc1e-4d18-997d-f37616323e61@github.com> Message-ID: On Tue, 10 Nov 2020 11:01:54 GMT, Serguei Spitsyn wrote: > I have a question and a couple of minor suggestions on new test. > Q: Why the value of ITERS is that big? What is the need to have this number of iterations? The test verifies the answer does not change if we hit JIT compilers in that code. Since we are doing C1/C2 intrinsics, we need to execute the loops more than compilation-threshold times. Since the current threshold is about 100K, doing 1M iterations seems good: it is well past the compilation threshold times, and there is time to re-enter the newly compiled method. The test run time is still sane, 1 minute on my Linux x86_64 fastdebug. I can do 200K iterations and -Xbatch instead, though, see new change. This drops the test run time to 30 seconds. > Also, I do not like one-letter identifiers, especially if they are not local. > Could you, please, replace identifiers R and A with some short versions that give a hint. > Something like REFSIZE and ALIGNMENT would be good enough. Renamed to `REF_SIZE` and `OBJ_ALIGN` instead. > Also, what tests did you run to make sure no regression is introduced? Old code calls into `oop::size()` to get the object size. That method decodes the object's layout helper. So when we replace it with intrinsic, we now have to test the different shapes of the layout helper and varying conditions for that decoding. So the new test tries to cover the comprehensive matrix: - the usual object shapes: objects, primitive arrays, object arrays; - different compressed oops modes that affect reference sizes; - different object alignment modes that affect object sizes; - different compilation modes: interpreter, C1, C2; - special paths like carrying special bits in layout helper for allocation slow-paths; I know that test is sensitive to compiler intrinsics bugs, as I used these tests to develop the intrinsics. If you inject simple off-by-one bugs into current C1/C2 intrinsics, new test catches that. The additional safety comes from the somewhat loose API requirement: it is specified to return the guess, and that guess might as well be wrong (not overly wrong though, for a quality implementation). ------------- PR: https://git.openjdk.java.net/jdk/pull/650 From shade at openjdk.java.net Tue Nov 10 12:04:21 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 10 Nov 2020 12:04:21 GMT Subject: RFR: 8253525: Implement getInstanceSize/sizeOf intrinsics [v6] In-Reply-To: References: Message-ID: > This is fork off the SizeOf JEP, JDK-8249196. There is already the entry point in JDK that can use the intrinsic like this: `Instrumentation.getInstanceSize`. Therefore, we can implement the C1/C2 intrinsic now, hook it up to `Instrumentation`, and let the tools use that fast path today. > > With this patch, JOL is able to be close to `deepSizeOf` implementation from SizeOf JEP. > > Example performance improvements for sizing up a custom linked list: > > Benchmark (size) Mode Cnt Score Error Units > > # Default > LinkedChainBench.linkedChain 1 avgt 5 705.835 ? 8.051 ns/op > LinkedChainBench.linkedChain 10 avgt 5 3148.874 ? 37.856 ns/op > LinkedChainBench.linkedChain 100 avgt 5 28693.256 ? 142.254 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 290161.590 ? 4594.631 ns/op > > # Instrumentation attached, no intrinsics > LinkedChainBench.linkedChain 1 avgt 5 159.659 ? 19.238 ns/op > LinkedChainBench.linkedChain 10 avgt 5 717.659 ? 22.540 ns/op > LinkedChainBench.linkedChain 100 avgt 5 7739.394 ? 111.683 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 80724.238 ? 2887.794 ns/op > > # Instrumentation attached, new intrinsics > LinkedChainBench.linkedChain 1 avgt 5 95.254 ? 0.808 ns/op > LinkedChainBench.linkedChain 10 avgt 5 261.564 ? 8.524 ns/op > LinkedChainBench.linkedChain 100 avgt 5 3367.192 ? 21.128 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 34148.851 ? 373.080 ns/op Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: - Trim down the iteration count, and use -Xbatch to wait for compilation - Use proper constant names in the test - Merge branch 'master' into JDK-8253525-sizeof-intrinsics - The new intrinsic-related test - Revert the change to test - Merge branch 'master' into JDK-8253525-sizeof-intrinsics - Add new intrinsics to toBeInvestigated list in CheckGraalIntrinsics.java - 8253525: Implement getInstanceSize/sizeOf intrinsics ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/650/files - new: https://git.openjdk.java.net/jdk/pull/650/files/482c2f24..1b7290a3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=650&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=650&range=04-05 Stats: 138167 lines in 1930 files changed: 82950 ins; 42900 del; 12317 mod Patch: https://git.openjdk.java.net/jdk/pull/650.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/650/head:pull/650 PR: https://git.openjdk.java.net/jdk/pull/650 From coleenp at openjdk.java.net Tue Nov 10 13:22:09 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 13:22:09 GMT Subject: RFR: 8138588: VerifyMergedCPBytecodes option cleanup needed Message-ID: This option has been removed in favor of always verifying the bytecodes in debug mode. Tested with tier1-3. ------------- Commit messages: - Fix test using option. - 8138588: VerifyMergedCPBytecodes option cleanup needed Changes: https://git.openjdk.java.net/jdk/pull/1137/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1137&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8138588 Stats: 9 lines in 4 files changed: 3 ins; 5 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1137.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1137/head:pull/1137 PR: https://git.openjdk.java.net/jdk/pull/1137 From hseigel at openjdk.java.net Tue Nov 10 13:54:56 2020 From: hseigel at openjdk.java.net (Harold Seigel) Date: Tue, 10 Nov 2020 13:54:56 GMT Subject: RFR: 8138588: VerifyMergedCPBytecodes option cleanup needed In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 23:46:04 GMT, Coleen Phillimore wrote: > This option has been removed in favor of always verifying the bytecodes in debug mode. Tested with tier1-3. Looks good! Thanks, Harold ------------- Marked as reviewed by hseigel (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1137 From dcubed at openjdk.java.net Tue Nov 10 14:04:03 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 14:04:03 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 03:28:52 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). > > Marked as reviewed by dholmes (Reviewer). Thanks for closing the loop on this one. Yes, next_next is needed and it's a carry-over name from the baseline code. And again, thanks for the thorough review. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Tue Nov 10 14:18:00 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 14:18:00 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Sat, 7 Nov 2020 17:17:01 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/synchronizer.cpp line 136: >> >>> 134: >>> 135: // Honor block request. >>> 136: ThreadBlockInVM tbivm(self->as_Java_thread()); >> >> ThreadBlockInVM is generally used to wrap blocking code, not to cause the current thread to block (which it will do as a side-effect if a safepoint/handshake is requested). Surely this should just be call to `process_if_requested` (or the new `process_if_requested_with_exit_check`)? > > This kind of use of ThreadBlockInVM predates this changeset so while > the location is new, then code style is old, very old... I'll hold off changing > this for now. I'd rather see ThreadBlockInVM as the convention of allowing the thread to block if a safepoint is requested. The calls like process_if_requested are becoming alphabet soup and keep changing, so having TBIVM is better in my opinion. That said, this is a strange usage. This code appears three times. It should be a function like allow_safepoint_block(LogStream* ls, timer), with some comment above. Then it's clear that it's checking for a safepoint in a loop that could take a long time and the logging is ancillary. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From eosterlund at openjdk.java.net Tue Nov 10 14:33:58 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 10 Nov 2020 14:33:58 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 02:35:17 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). Looks very good! Thanks for picking this up and taking it all the way! ------------- Marked as reviewed by eosterlund (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 14:45:01 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 14:45:01 GMT Subject: RFR: 8138588: VerifyMergedCPBytecodes option cleanup needed In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 23:46:04 GMT, Coleen Phillimore wrote: > This option has been removed in favor of always verifying the bytecodes in debug mode. Tested with tier1-3. Thumbs up! Please make sure this is okay with someone on the Serviceability team. src/hotspot/share/runtime/globals.hpp line 875: > 873: "Force ldc -> ldc_w rewrite during RedefineClasses") \ > 874: \ > 875: product(bool, AllowRedefinitionToAddDeleteMethods, false, \ Well... it's definitely sometime after Mustang/1.6.0... :-) ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1137 From dcubed at openjdk.java.net Tue Nov 10 14:55:59 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 14:55:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 14:31:14 GMT, Erik ?sterlund wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). > > Looks very good! Thanks for picking this up and taking it all the way! @fisk - Thanks for the sanity check review. And thanks for prototyping this work and showing that this crazy idea could work! :-) ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 15:06:01 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 15:06:01 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: On Tue, 10 Nov 2020 14:15:02 GMT, Coleen Phillimore wrote: >> This kind of use of ThreadBlockInVM predates this changeset so while >> the location is new, then code style is old, very old... I'll hold off changing >> this for now. > > I'd rather see ThreadBlockInVM as the convention of allowing the thread to block if a safepoint is requested. The calls like process_if_requested are becoming alphabet soup and keep changing, so having TBIVM is better in my opinion. > That said, this is a strange usage. This code appears three times. It should be a function like allow_safepoint_block(LogStream* ls, timer), with some comment above. Then it's clear that it's checking for a safepoint in a loop that could take a long time and the logging is ancillary. I previously looked at refactoring the three locations where `ThreadBlockInVM` is used. The problem with the refactoring is that the log messages and the parameters have some differences and some commonalities. Each of these logging sites is trying to communicate some local context that is unique to that call site along with some global context that might have changed from call site to call site. I'll take another look at refactoring shortly and will let you know what I come up with. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 17:39:25 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 17:39:25 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also done promotion perf runs on these bits and says "the results look fine". Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: coleenp CR - refactor common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/642/files - new: https://git.openjdk.java.net/jdk/pull/642/files/aec90b9a..15ad3526 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=03-04 Stats: 80 lines in 2 files changed: 30 ins; 37 del; 13 mod Patch: https://git.openjdk.java.net/jdk/pull/642.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/642/head:pull/642 PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 17:45:58 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 17:45:58 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 14:53:34 GMT, Daniel D. Daugherty wrote: >> Looks very good! Thanks for picking this up and taking it all the way! > > @fisk - Thanks for the sanity check review. And thanks for prototyping > this work and showing that this crazy idea could work! :-) @coleenp - I refactored common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). Please let me know if you're okay with it. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From shade at openjdk.java.net Tue Nov 10 17:52:55 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 10 Nov 2020 17:52:55 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap In-Reply-To: References: Message-ID: <0_3pc_EJdHcDUEV8wcb14AL0Cj6DC_ezPVF1qQocVzc=.66bc32bb-b062-4011-8667-d35e903fdc3c@github.com> On Fri, 6 Nov 2020 12:54:28 GMT, Lin Zang wrote: > The implementation of jmap tool depends on the implementation of object iteration by different GC heap. > This patch extend the BasicJMapTest to cover differet GC Heap. I believe this would fail when some GCs are not available. For example, in Minimal/Zero only Serial and Parallel are available. ZGC and Shenandoah are not available on all platforms. Plus, specifying another GC with `TEST_VM_OPTS` would probably fail with "multiple GCs selected". You need to split the tests like this, and protect each config with `@requires`: /* * @test * @summary Unit test for jmap utility * @key intermittent * @requires vm.gc.Parallel * @library /test/lib * @build jdk.test.lib.hprof.* * @build jdk.test.lib.hprof.model.* * @build jdk.test.lib.hprof.parser.* * @build jdk.test.lib.hprof.util.* * @run main/othervm/timeout=240 -XX:+UseParallelGC BasicJMapTest */ /* * @test * @summary Unit test for jmap utility * @key intermittent * @requires vm.gc.G1 * @library /test/lib * @build jdk.test.lib.hprof.* * @build jdk.test.lib.hprof.model.* * @build jdk.test.lib.hprof.parser.* * @build jdk.test.lib.hprof.util.* * @run main/othervm/timeout=240 -XX:+UseG1GC BasicJMapTest */ ... Maybe there is a way to clean up multiple `@build` tags to make the test config less verbose. ------------- Changes requested by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1094 From dcubed at openjdk.java.net Tue Nov 10 18:14:57 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 18:14:57 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <0fYueANbbMJQZaXMqos-O_pCe-OEf5fgMHWEnOqJr3M=.8884e351-76f5-45d9-b21a-c6160396f116@github.com> Message-ID: <3kTvc5Z5p0PiaJgA0Aj7NoaSVWX2vc1ii8g_Y3NBdf8=.72a0228f-73b1-4a89-9b2b-313f017e43e5@github.com> On Tue, 10 Nov 2020 15:02:31 GMT, Daniel D. Daugherty wrote: >> I'd rather see ThreadBlockInVM as the convention of allowing the thread to block if a safepoint is requested. The calls like process_if_requested are becoming alphabet soup and keep changing, so having TBIVM is better in my opinion. >> That said, this is a strange usage. This code appears three times. It should be a function like allow_safepoint_block(LogStream* ls, timer), with some comment above. Then it's clear that it's checking for a safepoint in a loop that could take a long time and the logging is ancillary. > > I previously looked at refactoring the three locations where > `ThreadBlockInVM` is used. The problem with the refactoring > is that the log messages and the parameters have some > differences and some commonalities. Each of these logging > sites is trying to communicate some local context that is > unique to that call site along with some global context that > might have changed from call site to call site. > > I'll take another look at refactoring shortly and will let you > know what I come up with. I refactored common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). Please let me know if you're okay with it. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From fparain at openjdk.java.net Tue Nov 10 19:19:59 2020 From: fparain at openjdk.java.net (Frederic Parain) Date: Tue, 10 Nov 2020 19:19:59 GMT Subject: RFR: 8256052: Remove unused allocation type from fieldInfo [v4] In-Reply-To: <-Wc3FeXsLIx0x0U3gt0oSW6qiTGSRuMKVr4qJ3aTTXE=.7ed06bf5-f795-44ca-99ca-848d199a2733@github.com> References: <-Wc3FeXsLIx0x0U3gt0oSW6qiTGSRuMKVr4qJ3aTTXE=.7ed06bf5-f795-44ca-99ca-848d199a2733@github.com> Message-ID: On Mon, 9 Nov 2020 20:23:55 GMT, Harold Seigel wrote: >> Frederic Parain has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix comment > > Looks good! Claes, Lois, Harold, Thank you for your reviews and feedback. Fred ------------- PR: https://git.openjdk.java.net/jdk/pull/1130 From fparain at openjdk.java.net Tue Nov 10 19:20:01 2020 From: fparain at openjdk.java.net (Frederic Parain) Date: Tue, 10 Nov 2020 19:20:01 GMT Subject: Integrated: 8256052: Remove unused allocation type from fieldInfo In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 15:43:09 GMT, Frederic Parain wrote: > Please review this small cleanup code, removing the now unused allocation type from the fieldInfo structure. > > Tested with Mach5, tiers 1 to 3 and locally by running test/hotspot/jtreg/serviceability/sa tests. > > Thank you, > > Fred This pull request has now been integrated. Changeset: bd3e65b5 Author: Frederic Parain URL: https://git.openjdk.java.net/jdk/commit/bd3e65b5 Stats: 126 lines in 5 files changed: 5 ins; 99 del; 22 mod 8256052: Remove unused allocation type from fieldInfo Reviewed-by: redestad, lfoltan, hseigel ------------- PR: https://git.openjdk.java.net/jdk/pull/1130 From sspitsyn at openjdk.java.net Tue Nov 10 20:11:58 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Tue, 10 Nov 2020 20:11:58 GMT Subject: RFR: 8138588: VerifyMergedCPBytecodes option cleanup needed In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 23:46:04 GMT, Coleen Phillimore wrote: > This option has been removed in favor of always verifying the bytecodes in debug mode. Tested with tier1-3. Hi Coleen, This looks good to me. Thank you for taking care about this flag! Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1137 From sspitsyn at openjdk.java.net Tue Nov 10 20:17:08 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Tue, 10 Nov 2020 20:17:08 GMT Subject: RFR: 8253525: Implement getInstanceSize/sizeOf intrinsics [v6] In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 12:04:21 GMT, Aleksey Shipilev wrote: >> This is fork off the SizeOf JEP, JDK-8249196. There is already the entry point in JDK that can use the intrinsic like this: `Instrumentation.getInstanceSize`. Therefore, we can implement the C1/C2 intrinsic now, hook it up to `Instrumentation`, and let the tools use that fast path today. >> >> With this patch, JOL is able to be close to `deepSizeOf` implementation from SizeOf JEP. >> >> Example performance improvements for sizing up a custom linked list: >> >> Benchmark (size) Mode Cnt Score Error Units >> >> # Default >> LinkedChainBench.linkedChain 1 avgt 5 705.835 ? 8.051 ns/op >> LinkedChainBench.linkedChain 10 avgt 5 3148.874 ? 37.856 ns/op >> LinkedChainBench.linkedChain 100 avgt 5 28693.256 ? 142.254 ns/op >> LinkedChainBench.linkedChain 1000 avgt 5 290161.590 ? 4594.631 ns/op >> >> # Instrumentation attached, no intrinsics >> LinkedChainBench.linkedChain 1 avgt 5 159.659 ? 19.238 ns/op >> LinkedChainBench.linkedChain 10 avgt 5 717.659 ? 22.540 ns/op >> LinkedChainBench.linkedChain 100 avgt 5 7739.394 ? 111.683 ns/op >> LinkedChainBench.linkedChain 1000 avgt 5 80724.238 ? 2887.794 ns/op >> >> # Instrumentation attached, new intrinsics >> LinkedChainBench.linkedChain 1 avgt 5 95.254 ? 0.808 ns/op >> LinkedChainBench.linkedChain 10 avgt 5 261.564 ? 8.524 ns/op >> LinkedChainBench.linkedChain 100 avgt 5 3367.192 ? 21.128 ns/op >> LinkedChainBench.linkedChain 1000 avgt 5 34148.851 ? 373.080 ns/op > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: > > - Trim down the iteration count, and use -Xbatch to wait for compilation > - Use proper constant names in the test > - Merge branch 'master' into JDK-8253525-sizeof-intrinsics > - The new intrinsic-related test > - Revert the change to test > - Merge branch 'master' into JDK-8253525-sizeof-intrinsics > - Add new intrinsics to toBeInvestigated list in CheckGraalIntrinsics.java > - 8253525: Implement getInstanceSize/sizeOf intrinsics Aleksey, Thank you for the update! It looks good to me. One more nit, I forgot to list in my previous comment, is uneeded '()' around comparisons: `+ static final int REF_SIZE = ((compressedOops == null) || (compressedOops == true)) ? 4 : 8;` Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/650 From rehn at openjdk.java.net Tue Nov 10 20:37:01 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 10 Nov 2020 20:37:01 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Tue, 10 Nov 2020 17:39:25 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > coleenp CR - refactor common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). Hi, thanks for fixing. I had some comments nothing so approving. src/hotspot/share/runtime/monitorDeflationThread.cpp line 92: > 90: // We wait for GuaranteedSafepointInterval so that > 91: // is_async_deflation_needed() is checked at the same interval. > 92: ml.wait(GuaranteedSafepointInterval); I don't like that we add a new global monitor just for the whitebox API. Without WB poking this could just a plain sleep. If we must have this new global monitor there seems be no reason for this to be no safepoint ? So ThreadBlockInVM would not be needed if we did safepoint checks instead? I would either skip WB notification and run the test with a very low GuaranteedSafepointInterval and just set flag and wait a second. Or if this is really important use a local semaphore and skip the boolean, each increase on sema would result in a deflation pass. src/hotspot/share/runtime/synchronizer.cpp line 1419: > 1417: ", count=" SIZE_FORMAT ", max=" SIZE_FORMAT, op_name, > 1418: in_use_list_ceiling(), _in_use_list.count(), _in_use_list.max()); > 1419: timer_p->start(); ThreadBlockInVM have a miss-feature: it safepoint polls on front-edge and back-edge. It should only have that poll on backedge, once that is fixed, this will do the wrong thing. Also you may safepoint on both front-edge and back-edge now, so the timer would show the wrong thing then. So to get the timer correct you should use: SafepointMechanism::process_if_requested(thread); src/hotspot/share/runtime/synchronizer.cpp line 1532: > 1530: // A JavaThread must check for a safepoint/handshake and honor it. > 1531: chk_for_block_req(self->as_Java_thread(), "deletion", "deleted_count", > 1532: deleted_count, ls, &timer); If you release oopStorage when deflating you can do this entire loop while blocked instead, which will be faster. src/hotspot/share/runtime/objectMonitor.hpp line 148: > 146: DEFINE_PAD_MINUS_SIZE(0, OM_CACHE_LINE_SIZE, sizeof(volatile markWord) + > 147: sizeof(WeakHandle)); > 148: // Used by async deflation as a marker in the _owner field: I have test with and without padding, I saw no difference. ------------- Marked as reviewed by rehn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Tue Nov 10 20:41:58 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 20:41:58 GMT Subject: Integrated: 8138588: VerifyMergedCPBytecodes option cleanup needed In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 23:46:04 GMT, Coleen Phillimore wrote: > This option has been removed in favor of always verifying the bytecodes in debug mode. Tested with tier1-3. This pull request has now been integrated. Changeset: 7d4e86be Author: Coleen Phillimore URL: https://git.openjdk.java.net/jdk/commit/7d4e86be Stats: 9 lines in 4 files changed: 3 ins; 5 del; 1 mod 8138588: VerifyMergedCPBytecodes option cleanup needed Reviewed-by: hseigel, dcubed, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/1137 From coleenp at openjdk.java.net Tue Nov 10 20:41:56 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 20:41:56 GMT Subject: RFR: 8138588: VerifyMergedCPBytecodes option cleanup needed In-Reply-To: References: Message-ID: <5acS29YfcnHztBpSs1ft6gDgM18W_4kgcJDYsipR-68=.e781cccf-d7f9-48cd-8335-276113122492@github.com> On Tue, 10 Nov 2020 20:09:06 GMT, Serguei Spitsyn wrote: >> This option has been removed in favor of always verifying the bytecodes in debug mode. Tested with tier1-3. > > Hi Coleen, > > This looks good to me. > Thank you for taking care about this flag! > > Thanks, > Serguei Thanks Dan, Harold and Serguei. ------------- PR: https://git.openjdk.java.net/jdk/pull/1137 From dcubed at openjdk.java.net Tue Nov 10 20:48:00 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 20:48:00 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> On Tue, 10 Nov 2020 19:41:23 GMT, Robbin Ehn wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> coleenp CR - refactor common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). > > src/hotspot/share/runtime/monitorDeflationThread.cpp line 92: > >> 90: // We wait for GuaranteedSafepointInterval so that >> 91: // is_async_deflation_needed() is checked at the same interval. >> 92: ml.wait(GuaranteedSafepointInterval); > > I don't like that we add a new global monitor just for the whitebox API. > Without WB poking this could just a plain sleep. > > If we must have this new global monitor there seems be no reason for this to be no safepoint ? > So ThreadBlockInVM would not be needed if we did safepoint checks instead? > > I would either skip WB notification and run the test with a very low GuaranteedSafepointInterval and just set flag and wait a second. > Or if this is really important use a local semaphore and skip the boolean, each increase on sema would result in a deflation pass. We may still decide to do this fix (even though the _object field is now weak): JDK-8249638 Re-instate idle monitor deflation as part of System.gc() https://bugs.openjdk.java.net/browse/JDK-8249638 and if we do, then we'll still need the request mechanism. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 20:59:58 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 20:59:58 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Tue, 10 Nov 2020 20:34:12 GMT, Robbin Ehn wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> coleenp CR - refactor common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). > > Hi, thanks for fixing. > > I had some comments nothing major so approving. @robehn - Thanks for the review!! And thanks for approving. > src/hotspot/share/runtime/synchronizer.cpp line 1419: > >> 1417: ", count=" SIZE_FORMAT ", max=" SIZE_FORMAT, op_name, >> 1418: in_use_list_ceiling(), _in_use_list.count(), _in_use_list.max()); >> 1419: timer_p->start(); > > ThreadBlockInVM have a miss-feature: it safepoint polls on front-edge and back-edge. > It should only have that poll on backedge, once that is fixed, this will do the wrong thing. > Also you may safepoint on both front-edge and back-edge now, so the timer would show the wrong thing then. > > So to get the timer correct you should use: > SafepointMechanism::process_if_requested(thread); The baseline code (ObjectSynchronizer::deflate_common_idle_monitors()) uses ThreadBlockInVM currently. I don't want to change that as part of this work. If we want to generally change uses of ThreadBlockInVM to something else, then we should do that with a dedicated bug. > src/hotspot/share/runtime/synchronizer.cpp line 1532: > >> 1530: // A JavaThread must check for a safepoint/handshake and honor it. >> 1531: chk_for_block_req(self->as_Java_thread(), "deletion", "deleted_count", >> 1532: deleted_count, ls, &timer); > > If you release oopStorage when deflating you can do this entire loop while blocked instead, which will be faster. > > (From what I remember you can actually release during a safepoint, but that is not future-prof as I understood it then. So I skipped going into that rabbit hole this time also.) @fisk said we should not release the oopStorage during a safepoint because that's not safe or will not be safe. I can't remember which. > src/hotspot/share/runtime/objectMonitor.hpp line 148: > >> 146: DEFINE_PAD_MINUS_SIZE(0, OM_CACHE_LINE_SIZE, sizeof(volatile markWord) + >> 147: sizeof(WeakHandle)); >> 148: // Used by async deflation as a marker in the _owner field: > > I have test with and without padding, I saw no difference. We've removed enough padding with this work already. If we want to do more padding removal, then we need to use a different RFE. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From rehn at openjdk.java.net Tue Nov 10 21:00:00 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 10 Nov 2020 21:00:00 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> Message-ID: On Tue, 10 Nov 2020 20:45:18 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/monitorDeflationThread.cpp line 92: >> >>> 90: // We wait for GuaranteedSafepointInterval so that >>> 91: // is_async_deflation_needed() is checked at the same interval. >>> 92: ml.wait(GuaranteedSafepointInterval); >> >> I don't like that we add a new global monitor just for the whitebox API. >> Without WB poking this could just a plain sleep. >> >> If we must have this new global monitor there seems be no reason for this to be no safepoint ? >> So ThreadBlockInVM would not be needed if we did safepoint checks instead? >> >> I would either skip WB notification and run the test with a very low GuaranteedSafepointInterval and just set flag and wait a second. >> Or if this is really important use a local semaphore and skip the boolean, each increase on sema would result in a deflation pass. > > We may still decide to do this fix (even though the _object field is now weak): > > JDK-8249638 Re-instate idle monitor deflation as part of System.gc() > https://bugs.openjdk.java.net/browse/JDK-8249638 > > and if we do, then we'll still need the request mechanism. So why not use a local semaphore and wait with safepoint check instead? >> src/hotspot/share/runtime/synchronizer.cpp line 1419: >> >>> 1417: ", count=" SIZE_FORMAT ", max=" SIZE_FORMAT, op_name, >>> 1418: in_use_list_ceiling(), _in_use_list.count(), _in_use_list.max()); >>> 1419: timer_p->start(); >> >> ThreadBlockInVM have a miss-feature: it safepoint polls on front-edge and back-edge. >> It should only have that poll on backedge, once that is fixed, this will do the wrong thing. >> Also you may safepoint on both front-edge and back-edge now, so the timer would show the wrong thing then. >> >> So to get the timer correct you should use: >> SafepointMechanism::process_if_requested(thread); > > The baseline code (ObjectSynchronizer::deflate_common_idle_monitors()) > uses ThreadBlockInVM currently. I don't want to change that as part of > this work. If we want to generally change uses of ThreadBlockInVM to > something else, then we should do that with a dedicated bug. Currently there is no issue with ThreadBlockInVM since there is no code inside those scopes. This adds code there which assumes the timer will be 'resumed', and logs "resume" when it actually could be going to a safepoint. >> src/hotspot/share/runtime/synchronizer.cpp line 1532: >> >>> 1530: // A JavaThread must check for a safepoint/handshake and honor it. >>> 1531: chk_for_block_req(self->as_Java_thread(), "deletion", "deleted_count", >>> 1532: deleted_count, ls, &timer); >> >> If you release oopStorage when deflating you can do this entire loop while blocked instead, which will be faster. >> >> (From what I remember you can actually release during a safepoint, but that is not future-prof as I understood it then. So I skipped going into that rabbit hole this time also.) > > @fisk said we should not release the oopStorage during a safepoint > because that's not safe or will not be safe. I can't remember which. Yes that's why I said you can release it during deflation instead. (not saying you should do this in this feeature change-set) >> src/hotspot/share/runtime/objectMonitor.hpp line 148: >> >>> 146: DEFINE_PAD_MINUS_SIZE(0, OM_CACHE_LINE_SIZE, sizeof(volatile markWord) + >>> 147: sizeof(WeakHandle)); >>> 148: // Used by async deflation as a marker in the _owner field: >> >> I have test with and without padding, I saw no difference. > > We've removed enough padding with this work already. If we > want to do more padding removal, then we need to use a > different RFE. Sure, this was more a FYI. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Tue Nov 10 21:07:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 21:07:03 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 02:35:17 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). Changes requested by coleenp (Reviewer). src/hotspot/share/oops/markWord.cpp line 63: > 61: fatal("bad header=" INTPTR_FORMAT, value()); > 62: } > 63: This makes it so much clearer where the displaced markWord is. src/hotspot/share/runtime/globals.hpp line 750: > 748: product(intx, MonitorUsedDeflationThreshold, 90, EXPERIMENTAL, \ > 749: "Percentage of used monitors before triggering deflation (0 is " \ > 750: "off). The check is performed on GuaranteedSafepointInterval " \ Should there still be experimental options after this change? src/hotspot/share/runtime/monitorDeflationThread.cpp line 85: > 83: // visible to external suspension. > 84: > 85: ThreadBlockInVM tbivm(jt); Does this have to be a JavaThread? Could it be a non-java thread since deflating monitors doesn't have to call any Java code? You'd have to lock down the Monitor list maybe, but couldn't this be a NamedThread? This isn't a request to change it right now. src/hotspot/share/runtime/synchronizer.cpp line 1641: > 1639: > 1640: // Do the final audit and print of ObjectMonitor stats; must be done > 1641: // by the VMThread (at VM exit time). Can you take (at VM exit time) out of parenthesis? it made me wonder when else is this called. src/hotspot/share/runtime/objectMonitor.cpp line 509: > 507: // > 508: bool ObjectMonitor::deflate_monitor() { > 509: if (is_busy()) { is_busy should be checked != 0 since it doesn't return a bool. src/hotspot/share/runtime/objectMonitor.cpp line 540: > 538: if (try_set_owner_from(NULL, DEFLATER_MARKER) != NULL) { > 539: // The owner field is no longer NULL so we lost the race since the > 540: // ObjectMonitor is now busy. So here would contentions be > 0? Can it be asserted? Doesn't need to be, the comment really helps to understand why the cas failed. src/hotspot/share/runtime/objectMonitor.cpp line 551: > 549: if (try_set_owner_from(DEFLATER_MARKER, NULL) != DEFLATER_MARKER) { > 550: // Deferred decrement for the JT EnterI() that cancelled the async deflation. > 551: add_to_contentions(-1); contentions is essentially a refcount, isn't it. Can you fix the comment to include this at line 360 since that's not the only purpose of this count. // Keep track of contention for JVM/TI and M&M queries. add_to_contentions(1); src/hotspot/share/runtime/synchronizer.hpp line 61: > 59: bool has_next() const { return _current != NULL; } > 60: ObjectMonitor* next(); > 61: }; Can MonitorList be defined in the .cpp file? I don't see anything outside of synchronizer.cpp that refers to it. src/hotspot/share/runtime/synchronizer.hpp line 28: > 26: #define SHARE_RUNTIME_SYNCHRONIZER_HPP > 27: > 28: #include "logging/logStream.hpp" If you need to put MonitorList in the header file, use a forward declaration for LogStream instead of #including logstream.hpp. src/hotspot/share/runtime/objectMonitor.hpp line 171: > 169: volatile int _SpinDuration; > 170: > 171: jint _contentions; // Number of active contentions in enter(). It is used by is_busy() Future RFE - can we replace jint with int32_t or even int or some C++ types. We're trying not to have Java types leak into runtime code since this doesn't directly interface with Java. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 21:07:03 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 21:07:03 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> Message-ID: On Tue, 10 Nov 2020 20:52:15 GMT, Robbin Ehn wrote: >> We may still decide to do this fix (even though the _object field is now weak): >> >> JDK-8249638 Re-instate idle monitor deflation as part of System.gc() >> https://bugs.openjdk.java.net/browse/JDK-8249638 >> >> and if we do, then we'll still need the request mechanism. > > So why not use a local semaphore and wait with safepoint check instead? Sorry my preference is for Monitors instead of semaphores. Let's take that discussion off this PR and you can explain why you dislike the Monitor so much and think the local semaphore is the way to go. >> The baseline code (ObjectSynchronizer::deflate_common_idle_monitors()) >> uses ThreadBlockInVM currently. I don't want to change that as part of >> this work. If we want to generally change uses of ThreadBlockInVM to >> something else, then we should do that with a dedicated bug. > > Currently there is no issue with ThreadBlockInVM since there is no code inside those scopes. > This adds code there which assumes the timer will be 'resumed', and logs "resume" when it actually could be going to a safepoint. So if I narrow the scope around the ThreadBlockInVM, then it would be fine? { // Honor block request. ThreadBlockInVM tbivm(self); } I can make that change before I integrate... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 21:12:02 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 21:12:02 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> Message-ID: <_fjxyjNA-4UWOBnqEj6DVaS4BZEVbTUazpMJRolWcxg=.00e91263-7f75-4fb3-9350-36580395b1a6@github.com> On Tue, 10 Nov 2020 20:55:17 GMT, Robbin Ehn wrote: >> @fisk said we should not release the oopStorage during a safepoint >> because that's not safe or will not be safe. I can't remember which. > > Yes that's why I said you can release it during deflation instead. > > (not saying you should do this in this feeature change-set) What does this "you can do this entire loop while blocked instead" mean? Releasing during deflation kind of messes with the life-cycle I was trying to enforce since deletion is the nature end-of-life for these... But to think about that I need to know what you mean by "you can do this entire loop while blocked instead"... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 21:12:04 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 21:12:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 16:41:49 GMT, Coleen Phillimore wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). > > src/hotspot/share/oops/markWord.cpp line 63: > >> 61: fatal("bad header=" INTPTR_FORMAT, value()); >> 62: } >> 63: > > This makes it so much clearer where the displaced markWord is. Yup. That was the point. Get rid of the magic offset zero coding... > src/hotspot/share/runtime/globals.hpp line 750: > >> 748: product(intx, MonitorUsedDeflationThreshold, 90, EXPERIMENTAL, \ >> 749: "Percentage of used monitors before triggering deflation (0 is " \ >> 750: "off). The check is performed on GuaranteedSafepointInterval " \ > > Should there still be experimental options after this change? Robbin added MonitorUsedDeflationThreshold as an experimental option back in JDK10. See the longer reply to David's comment. I don't plan to change that option with this changeset. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From rehn at openjdk.java.net Tue Nov 10 21:21:04 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 10 Nov 2020 21:21:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> Message-ID: <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> On Tue, 10 Nov 2020 21:01:21 GMT, Daniel D. Daugherty wrote: >> So why not use a local semaphore and wait with safepoint check instead? > > Sorry my preference is for Monitors instead of semaphores. Let's > take that discussion off this PR and you can explain why you dislike > the Monitor so much and think the local semaphore is the way to go. Yes >> Currently there is no issue with ThreadBlockInVM since there is no code inside those scopes. >> This adds code there which assumes the timer will be 'resumed', and logs "resume" when it actually could be going to a safepoint. > > So if I narrow the scope around the ThreadBlockInVM, then it would be fine? > > { > // Honor block request. > ThreadBlockInVM tbivm(self); > } > > I can make that change before I integrate... Yes that avoids it! >> Yes that's why I said you can release it during deflation instead. >> >> (not saying you should do this in this feeature change-set) > > What does this "you can do this entire loop while blocked instead" mean? > > Releasing during deflation kind of messes with the life-cycle I was trying > to enforce since deletion is the nature end-of-life for these... But to think > about that I need to know what you mean by "you can do this entire loop > while blocked instead"... If you only need to free CHeap memory, you can do: size_t deleted_count = 0; ThreadBlockInVM tbivm(self); for (ObjectMonitor* monitor: delete_list) { delete monitor; deleted_count++; } } ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 21:21:04 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 21:21:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: <-NIFifwEMk1oYQugLdKvFNgAxmeyEsf05sku_ccaNW0=.e54c02a9-acd4-449e-8fae-05da29aad1a9@github.com> On Tue, 10 Nov 2020 16:46:10 GMT, Coleen Phillimore wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). > > src/hotspot/share/runtime/monitorDeflationThread.cpp line 85: > >> 83: // visible to external suspension. >> 84: >> 85: ThreadBlockInVM tbivm(jt); > > Does this have to be a JavaThread? Could it be a non-java thread since deflating monitors doesn't have to call any Java code? You'd have to lock down the Monitor list maybe, but couldn't this be a NamedThread? This isn't a request to change it right now. Ummm... we use a JavaThread because we have to stop async deflation during safepoints so that we're not messing with Object headers during GC. Yes, it's possible to use a non-JavaThread because this is "just software", but I don't want to try to figure out those races... > src/hotspot/share/runtime/synchronizer.cpp line 1641: > >> 1639: >> 1640: // Do the final audit and print of ObjectMonitor stats; must be done >> 1641: // by the VMThread (at VM exit time). > > Can you take (at VM exit time) out of parenthesis? it made me wonder when else is this called. Okay. > src/hotspot/share/runtime/objectMonitor.cpp line 509: > >> 507: // >> 508: bool ObjectMonitor::deflate_monitor() { >> 509: if (is_busy()) { > > is_busy should be checked != 0 since it doesn't return a bool. Nice catch! That has been there for many, many years... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 21:25:04 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 21:25:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 17:05:16 GMT, Coleen Phillimore wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). > > src/hotspot/share/runtime/objectMonitor.cpp line 540: > >> 538: if (try_set_owner_from(NULL, DEFLATER_MARKER) != NULL) { >> 539: // The owner field is no longer NULL so we lost the race since the >> 540: // ObjectMonitor is now busy. > > So here would contentions be > 0? Can it be asserted? Doesn't need to be, the comment really helps to understand why the cas failed. No we can't assert that (contentions > 0). The ownership might have been taken by a fast path thread so it grabbed ownership without having to update contentions and wait. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 21:30:05 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 21:30:05 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: <_8BktJ0NHE_i7LQ7_X8JQyyfriKl8_-PA7hOfdBrC3U=.ac425b59-0e89-47f2-9fa7-e461e55689a2@github.com> On Tue, 10 Nov 2020 17:32:56 GMT, Coleen Phillimore wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). > > src/hotspot/share/runtime/objectMonitor.cpp line 551: > >> 549: if (try_set_owner_from(DEFLATER_MARKER, NULL) != DEFLATER_MARKER) { >> 550: // Deferred decrement for the JT EnterI() that cancelled the async deflation. >> 551: add_to_contentions(-1); > > contentions is essentially a refcount, isn't it. Can you fix the comment to include this at line 360 since that's not the only purpose of this count. > > // Keep track of contention for JVM/TI and M&M queries. > add_to_contentions(1); No it is not a ref_count. We got rid of the accurate ref_count field because maintaining it was too slow. contentions just tells you how many threads have blocked on the slow-path trying to enter the monitor. The fast-paths never touch contentions. The comment on line 360 is accurate. > src/hotspot/share/runtime/synchronizer.hpp line 61: > >> 59: bool has_next() const { return _current != NULL; } >> 60: ObjectMonitor* next(); >> 61: }; > > Can MonitorList be defined in the .cpp file? I don't see anything outside of synchronizer.cpp that refers to it. I can see if that will work. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From hseigel at openjdk.java.net Tue Nov 10 21:30:04 2020 From: hseigel at openjdk.java.net (Harold Seigel) Date: Tue, 10 Nov 2020 21:30:04 GMT Subject: RFR: 8255787: Tag container tests that use cGroups with cgroups keyword Message-ID: Please review this small change to add a cgroups keyword to tests that use cgroups. The fix was tested by running Mach5 container tests. ------------- Commit messages: - 8255787: Tag container tests that use cGroups with cgroups keyword Changes: https://git.openjdk.java.net/jdk/pull/1148/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1148&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255787 Stats: 20 lines in 15 files changed: 15 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/1148.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1148/head:pull/1148 PR: https://git.openjdk.java.net/jdk/pull/1148 From dcubed at openjdk.java.net Tue Nov 10 21:51:03 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 21:51:03 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> Message-ID: <7rQQpVtTPecL-Vt4JJ0CR3U_1523KksQwFAsn9QpJK8=.5e872ffe-d1e4-40e9-9dd4-595712452e5c@github.com> On Tue, 10 Nov 2020 21:16:33 GMT, Robbin Ehn wrote: >> So if I narrow the scope around the ThreadBlockInVM, then it would be fine? >> >> { >> // Honor block request. >> ThreadBlockInVM tbivm(self); >> } >> >> I can make that change before I integrate... > > Yes that avoids it! Done. I also did the one in ObjectSynchronizer::request_deflate_idle_monitors(). >> What does this "you can do this entire loop while blocked instead" mean? >> >> Releasing during deflation kind of messes with the life-cycle I was trying >> to enforce since deletion is the nature end-of-life for these... But to think >> about that I need to know what you mean by "you can do this entire loop >> while blocked instead"... > > If you only need to free CHeap memory, you can do: > size_t deleted_count = 0; > ThreadBlockInVM tbivm(self); > for (ObjectMonitor* monitor: delete_list) { > delete monitor; > deleted_count++; > } > } Ahhh... but that only works if we release the oopStorage when we deflate. Okay. I grok it now, but don't want to do that in this changeset. I would want a complete stress test cycle for that kind of a change. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 21:51:02 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 21:51:02 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: <3oT-RoU0dfm8DXWOSdkbjLG_xJnVsniox-6kpJ_h2cA=.15fff0b3-c8b1-4cc9-9113-8f764c20fa5e@github.com> On Tue, 10 Nov 2020 20:57:48 GMT, Coleen Phillimore wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). > > src/hotspot/share/runtime/synchronizer.hpp line 28: > >> 26: #define SHARE_RUNTIME_SYNCHRONIZER_HPP >> 27: >> 28: #include "logging/logStream.hpp" > > If you need to put MonitorList in the header file, use a forward declaration for LogStream instead of #including logstream.hpp. I can see if that will work. > src/hotspot/share/runtime/objectMonitor.hpp line 171: > >> 169: volatile int _SpinDuration; >> 170: >> 171: jint _contentions; // Number of active contentions in enter(). It is used by is_busy() > > Future RFE - can we replace jint with int32_t or even int or some C++ types. We're trying not to have Java types leak into runtime code since this doesn't directly interface with Java. It can be a future RFE, but it won't be at the top of my list of things to do. There may already be an RFE for that. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Tue Nov 10 22:25:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 22:25:02 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: <7rQQpVtTPecL-Vt4JJ0CR3U_1523KksQwFAsn9QpJK8=.5e872ffe-d1e4-40e9-9dd4-595712452e5c@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> <7rQQpVtTPecL-Vt4JJ0CR3U_1523KksQwFAsn9QpJK8=.5e872ffe-d1e4-40e9-9dd4-595712452e5c@github.com> Message-ID: <_EeIXcUypiIUutwEAJRwUR49nfILH3PNmz75RLBmy0M=.dca51ee5-c4b5-4bcf-874f-3c3919f332ee@github.com> On Tue, 10 Nov 2020 21:35:43 GMT, Daniel D. Daugherty wrote: >> Yes that avoids it! > > Done. I also did the one in ObjectSynchronizer::request_deflate_idle_monitors(). I like it but can you do one thing for me? Can you s/chk/check/ in the name? ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Tue Nov 10 22:25:04 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 22:25:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <-NIFifwEMk1oYQugLdKvFNgAxmeyEsf05sku_ccaNW0=.e54c02a9-acd4-449e-8fae-05da29aad1a9@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> <-NIFifwEMk1oYQugLdKvFNgAxmeyEsf05sku_ccaNW0=.e54c02a9-acd4-449e-8fae-05da29aad1a9@github.com> Message-ID: On Tue, 10 Nov 2020 21:12:25 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/monitorDeflationThread.cpp line 85: >> >>> 83: // visible to external suspension. >>> 84: >>> 85: ThreadBlockInVM tbivm(jt); >> >> Does this have to be a JavaThread? Could it be a non-java thread since deflating monitors doesn't have to call any Java code? You'd have to lock down the Monitor list maybe, but couldn't this be a NamedThread? This isn't a request to change it right now. > > Ummm... we use a JavaThread because we have to stop async deflation > during safepoints so that we're not messing with Object headers during GC. > Yes, it's possible to use a non-JavaThread because this is "just software", > but I don't want to try to figure out those races... Ok, I see why it has to be a JavaThread. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Tue Nov 10 22:31:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 22:31:01 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <_8BktJ0NHE_i7LQ7_X8JQyyfriKl8_-PA7hOfdBrC3U=.ac425b59-0e89-47f2-9fa7-e461e55689a2@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> <_8BktJ0NHE_i7LQ7_X8JQyyfriKl8_-PA7hOfdBrC3U=.ac425b59-0e89-47f2-9fa7-e461e55689a2@github.com> Message-ID: <_91UeV37-LcU43acy9VUD7rt3woXrc-jdz-ILC7ichs=.57f26cbb-1ac0-4c9e-ba2e-9b620d60b2c9@github.com> On Tue, 10 Nov 2020 21:25:51 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/objectMonitor.cpp line 551: >> >>> 549: if (try_set_owner_from(DEFLATER_MARKER, NULL) != DEFLATER_MARKER) { >>> 550: // Deferred decrement for the JT EnterI() that cancelled the async deflation. >>> 551: add_to_contentions(-1); >> >> contentions is essentially a refcount, isn't it. Can you fix the comment to include this at line 360 since that's not the only purpose of this count. >> >> // Keep track of contention for JVM/TI and M&M queries. >> add_to_contentions(1); > > No it is not a ref_count. We got rid of the accurate ref_count field > because maintaining it was too slow. > > contentions just tells you how many threads have blocked on the > slow-path trying to enter the monitor. The fast-paths never touch > contentions. The comment on line 360 is accurate. But contentions is used for more than informing JVMTI, it's used to test whether the monitor is_busy on the slow path. That's why I wanted the comment to say something like your last sentence, since I spent time trying to understand why the various calls to add_to_contentions(-1) in deflate_monitor earlier. >> src/hotspot/share/runtime/objectMonitor.hpp line 171: >> >>> 169: volatile int _SpinDuration; >>> 170: >>> 171: jint _contentions; // Number of active contentions in enter(). It is used by is_busy() >> >> Future RFE - can we replace jint with int32_t or even int or some C++ types. We're trying not to have Java types leak into runtime code since this doesn't directly interface with Java. > > It can be a future RFE, but it won't be at the top of my list of > things to do. There may already be an RFE for that. No, I assume it's not high priority. I'll file an RFE because someday I want these to be cleaned up as a personal nit. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 22:35:04 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 22:35:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: <_EeIXcUypiIUutwEAJRwUR49nfILH3PNmz75RLBmy0M=.dca51ee5-c4b5-4bcf-874f-3c3919f332ee@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> <7rQQpVtTPecL-Vt4JJ0CR3U_1523KksQwFAsn9QpJK8=.5e872ffe-d1e4-40e9-9dd4-595712452e5c@github.com> <_EeIXcUypiIUutwEAJRwUR49nfILH3PNmz75RLBmy0M=.dca51ee5-c4b5-4bcf-874f-3c3919f332ee@github.com> Message-ID: On Tue, 10 Nov 2020 22:08:10 GMT, Coleen Phillimore wrote: >> Done. I also did the one in ObjectSynchronizer::request_deflate_idle_monitors(). > > I like it but can you do one thing for me? Can you s/chk/check/ in the name? I'd rather not. It is a function with six freaking parameters so I went with names as short as I could... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 22:56:00 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 22:56:00 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <_91UeV37-LcU43acy9VUD7rt3woXrc-jdz-ILC7ichs=.57f26cbb-1ac0-4c9e-ba2e-9b620d60b2c9@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> <_8BktJ0NHE_i7LQ7_X8JQyyfriKl8_-PA7hOfdBrC3U=.ac425b59-0e89-47f2-9fa7-e461e55689a2@github.com> <_91UeV37-LcU43acy9VUD7rt3woXrc-jdz-ILC7ichs=.57f26cbb-1ac0-4c9e-ba2e-9b620d60b2c9@github.com> Message-ID: <3sH90dwOxC6MSBbtSZw8XNQpIvDyhsciRtPsrHJ8UPQ=.88463ff9-ca47-4dc1-8334-f436cd16276a@github.com> On Tue, 10 Nov 2020 22:27:06 GMT, Coleen Phillimore wrote: >> No it is not a ref_count. We got rid of the accurate ref_count field >> because maintaining it was too slow. >> >> contentions just tells you how many threads have blocked on the >> slow-path trying to enter the monitor. The fast-paths never touch >> contentions. The comment on line 360 is accurate. > > But contentions is used for more than informing JVMTI, it's used to test whether the monitor is_busy on the slow path. That's why I wanted the comment to say something like your last sentence, since I spent time trying to understand why the various calls to add_to_contentions(-1) in deflate_monitor earlier. Ahhh... I think I understand your confusion. This line: L550: // Deferred decrement for the JT EnterI() that cancelled the async deflation. L551: add_to_contentions(-1); doesn't match up with this line: L361: add_to_contentions(1); It matches up with one of these: if (try_set_owner_from(DEFLATER_MARKER, Self) == DEFLATER_MARKER) { // Cancelled the in-progress async deflation by changing owner from // DEFLATER_MARKER to Self. As part of the contended enter protocol, // contentions was incremented to a positive value before EnterI() // was called and that prevents the deflater thread from winning the // last part of the 2-part async deflation protocol. After EnterI() // returns to enter(), contentions is decremented because the caller // now owns the monitor. We bump contentions an extra time here to // prevent the deflater thread from winning the last part of the // 2-part async deflation protocol after the regular decrement // occurs in enter(). The deflater thread will decrement contentions // after it recognizes that the async deflation was cancelled. add_to_contentions(1); The long comments are in the two places where we temporarily increment contentions to stop the race with the deflater thread and the shorter comment, e.g., L550, are for where we undo the temporary increment. The primary purpose of the contentions field is for JVM/TI and M&M queries. We just (temporarily) steal it for async deflation purposes... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Tue Nov 10 23:18:59 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 10 Nov 2020 23:18:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <3sH90dwOxC6MSBbtSZw8XNQpIvDyhsciRtPsrHJ8UPQ=.88463ff9-ca47-4dc1-8334-f436cd16276a@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> <_8BktJ0NHE_i7LQ7_X8JQyyfriKl8_-PA7hOfdBrC3U=.ac425b59-0e89-47f2-9fa7-e461e55689a2@github.com> <_91UeV37-LcU43acy9VUD7rt3woXrc-jdz-ILC7ichs=.57f26cbb-1ac0-4c9e-ba2e-9b620d60b2c9@github.com> <3sH90dwOxC6MSBbtSZw8XNQpIvDyhsciRtPsrHJ8UPQ=.88463ff9-ca47-4dc1-8334-f436cd16276a@github.com> Message-ID: On Tue, 10 Nov 2020 22:53:10 GMT, Daniel D. Daugherty wrote: >> But contentions is used for more than informing JVMTI, it's used to test whether the monitor is_busy on the slow path. That's why I wanted the comment to say something like your last sentence, since I spent time trying to understand why the various calls to add_to_contentions(-1) in deflate_monitor earlier. > > Ahhh... I think I understand your confusion. This line: > > L550: // Deferred decrement for the JT EnterI() that cancelled the async deflation. > L551: add_to_contentions(-1); > > doesn't match up with this line: > > L361: add_to_contentions(1); > > It matches up with one of these: > if (try_set_owner_from(DEFLATER_MARKER, Self) == DEFLATER_MARKER) { > // Cancelled the in-progress async deflation by changing owner from > // DEFLATER_MARKER to Self. As part of the contended enter protocol, > // contentions was incremented to a positive value before EnterI() > // was called and that prevents the deflater thread from winning the > // last part of the 2-part async deflation protocol. After EnterI() > // returns to enter(), contentions is decremented because the caller > // now owns the monitor. We bump contentions an extra time here to > // prevent the deflater thread from winning the last part of the > // 2-part async deflation protocol after the regular decrement > // occurs in enter(). The deflater thread will decrement contentions > // after it recognizes that the async deflation was cancelled. > add_to_contentions(1); > > The long comments are in the two places where we temporarily > increment contentions to stop the race with the deflater thread > and the shorter comment, e.g., L550, are for where we undo the > temporary increment. > > The primary purpose of the contentions field is for JVM/TI and > M&M queries. We just (temporarily) steal it for async deflation > purposes... Well since it controls async deflation, it should probably get a mention since this comment on its own is not true: // Keep track of contention for JVM/TI and M&M queries and control async deflation. The field _contentions has a good comment. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 23:24:24 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 23:24:24 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also done promotion perf runs on these bits and says "the results look fine". Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: resolve more robehn and coleenp comments. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/642/files - new: https://git.openjdk.java.net/jdk/pull/642/files/15ad3526..61d36884 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=04-05 Stats: 92 lines in 3 files changed: 46 ins; 38 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/642.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/642/head:pull/642 PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 23:24:25 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 23:24:25 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Tue, 10 Nov 2020 20:34:12 GMT, Robbin Ehn wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> coleenp CR - refactor common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). > > Hi, thanks for fixing. > > I had some comments nothing major so approving. @robehn and @coleenp - resolved more comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 23:33:59 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 23:33:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> <_8BktJ0NHE_i7LQ7_X8JQyyfriKl8_-PA7hOfdBrC3U=.ac425b59-0e89-47f2-9fa7-e461e55689a2@github.com> <_91UeV37-LcU43acy9VUD7rt3woXrc-jdz-ILC7ichs=.57f26cbb-1ac0-4c9e-ba2e-9b620d60b2c9@github.com> <3sH90dwOxC6MSBbtSZw8XNQpIvDyhsciRtPsrHJ8UPQ=.88463ff9-ca47-4dc1-8334-f436cd16276a@github.com> Message-ID: On Tue, 10 Nov 2020 23:15:15 GMT, Coleen Phillimore wrote: >> Ahhh... I think I understand your confusion. This line: >> >> L550: // Deferred decrement for the JT EnterI() that cancelled the async deflation. >> L551: add_to_contentions(-1); >> >> doesn't match up with this line: >> >> L361: add_to_contentions(1); >> >> It matches up with one of these: >> if (try_set_owner_from(DEFLATER_MARKER, Self) == DEFLATER_MARKER) { >> // Cancelled the in-progress async deflation by changing owner from >> // DEFLATER_MARKER to Self. As part of the contended enter protocol, >> // contentions was incremented to a positive value before EnterI() >> // was called and that prevents the deflater thread from winning the >> // last part of the 2-part async deflation protocol. After EnterI() >> // returns to enter(), contentions is decremented because the caller >> // now owns the monitor. We bump contentions an extra time here to >> // prevent the deflater thread from winning the last part of the >> // 2-part async deflation protocol after the regular decrement >> // occurs in enter(). The deflater thread will decrement contentions >> // after it recognizes that the async deflation was cancelled. >> add_to_contentions(1); >> >> The long comments are in the two places where we temporarily >> increment contentions to stop the race with the deflater thread >> and the shorter comment, e.g., L550, are for where we undo the >> temporary increment. >> >> The primary purpose of the contentions field is for JVM/TI and >> M&M queries. We just (temporarily) steal it for async deflation >> purposes... > > Well since it controls async deflation, it should probably get a mention since this comment on its own is not true: > > // Keep track of contention for JVM/TI and M&M queries and control async deflation. > > The field _contentions has a good comment. The comment for **that** location is correct. That's the location where we keep track of contended monitors for JVM/TI and M&M. The comments at the other locations are also correct (very verbose for two of them, but correct). ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Tue Nov 10 23:33:59 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 10 Nov 2020 23:33:59 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> <_8BktJ0NHE_i7LQ7_X8JQyyfriKl8_-PA7hOfdBrC3U=.ac425b59-0e89-47f2-9fa7-e461e55689a2@github.com> <_91UeV37-LcU43acy9VUD7rt3woXrc-jdz-ILC7ichs=.57f26cbb-1ac0-4c9e-ba2e-9b620d60b2c9@github.com> <3sH90dwOxC6MSBbtSZw8XNQpIvDyhsciRtPsrHJ8UPQ=.88463ff9-ca47-4dc1-8334-f436cd16276a@github.com> Message-ID: On Tue, 10 Nov 2020 23:27:52 GMT, Daniel D. Daugherty wrote: >> Well since it controls async deflation, it should probably get a mention since this comment on its own is not true: >> >> // Keep track of contention for JVM/TI and M&M queries and control async deflation. >> >> The field _contentions has a good comment. > > The comment for **that** location is correct. That's the location > where we keep track of contended monitors for JVM/TI and M&M. > The comments at the other locations are also correct (very verbose > for two of them, but correct). Look at the entire context of that location: // Keep track of contention for JVM/TI and M&M queries. add_to_contentions(1); if (is_being_async_deflated()) { // Async deflation is in progress and our contentions increment // above lost the race to async deflation. Undo the work and // force the caller to retry. const oop l_object = object(); if (l_object != NULL) { // Attempt to restore the header/dmw to the object's header so that // we only retry once if the deflater thread happens to be slow. install_displaced_markword_in_object(l_object); } Self->_Stalled = 0; add_to_contentions(-1); return false; } We do the increment for JVM/TI and M&M purposes and then we check to see if async deflation beat us... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From sspitsyn at openjdk.java.net Wed Nov 11 00:30:54 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 11 Nov 2020 00:30:54 GMT Subject: RFR: 8255787: Tag container tests that use cGroups with cgroups keyword In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 21:24:25 GMT, Harold Seigel wrote: > Please review this small change to add a cgroups keyword to tests that use cgroups. The fix was tested by running Mach5 container tests. Hi Harold, The fix looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1148 From dholmes at openjdk.java.net Wed Nov 11 05:15:02 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 11 Nov 2020 05:15:02 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> On Tue, 10 Nov 2020 23:24:24 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > resolve more robehn and coleenp comments. One change requested in relation to use of jint instead of size_t. One code simplification suggestion. Thanks, David src/hotspot/share/runtime/synchronizer.cpp line 153: > 151: if (self->is_Java_thread()) { > 152: // A JavaThread must check for a safepoint/handshake and honor it. > 153: ObjectSynchronizer::chk_for_block_req(self->as_Java_thread(), "unlinking", I won't disapprove but this is a case where refactoring is IMO worse than code duplication. Logging parameters should not be a part of this API IMO. src/hotspot/share/runtime/synchronizer.cpp line 1228: > 1226: os::naked_short_sleep(999); // sleep for almost 1 second > 1227: } else { > 1228: os::naked_short_sleep(999); // sleep for almost 1 second So this block can now just be: > if (self->is_Java_thread()) { > ThreadBlockInVM tbivm(self->as_Java_thread()); > } > os::naked_short_sleep(999); // sleep for almost 1 second src/hotspot/share/runtime/synchronizer.cpp line 246: > 244: // > 245: // Start the ceiling with the estimate for one thread: > 246: jint _in_use_list_ceiling = AvgMonitorsPerThreadEstimate; Why is this a jint when you use size_t for its accessor and all the other sizes that you compare with the ceiling are also size_t? I'm not sure size_t is right to use in these cases (do we really expect different maximums on 32-bit versus 64-bit?) but it should be all or none IMO. ------------- Changes requested by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/642 From rehn at openjdk.java.net Wed Nov 11 08:29:01 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 11 Nov 2020 08:29:01 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Tue, 10 Nov 2020 23:24:24 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > resolve more robehn and coleenp comments. src/hotspot/share/runtime/synchronizer.cpp line 1226: > 1224: ThreadBlockInVM tbivm(self->as_Java_thread()); > 1225: } > 1226: os::naked_short_sleep(999); // sleep for almost 1 second Hi Dan, you need to be blocked while sleeping, otherwise you are blocking safepoints for "almost 1 second". So previous code was correct. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From lzang at openjdk.java.net Wed Nov 11 09:09:15 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 11 Nov 2020 09:09:15 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v2] In-Reply-To: References: Message-ID: > The implementation of jmap tool depends on the implementation of object iteration by different GC heap. > This patch extend the BasicJMapTest to cover differet GC Heap. Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Merge branch 'master' into jmap-test - 8255982: Extend BasicJMapTest to test with different GC Heap The implementation of jmap tool depends on the implementation of object iteration by different GC heap. This patch extend the BasicJMapTest to cover differet GC Heap. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1094/files - new: https://git.openjdk.java.net/jdk/pull/1094/files/f712e6f6..499e75a9 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=00-01 Stats: 47904 lines in 413 files changed: 26824 ins; 13713 del; 7367 mod Patch: https://git.openjdk.java.net/jdk/pull/1094.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1094/head:pull/1094 PR: https://git.openjdk.java.net/jdk/pull/1094 From shade at openjdk.java.net Wed Nov 11 09:39:15 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 09:39:15 GMT Subject: RFR: 8253525: Implement getInstanceSize/sizeOf intrinsics [v7] In-Reply-To: References: Message-ID: > This is fork off the SizeOf JEP, JDK-8249196. There is already the entry point in JDK that can use the intrinsic like this: `Instrumentation.getInstanceSize`. Therefore, we can implement the C1/C2 intrinsic now, hook it up to `Instrumentation`, and let the tools use that fast path today. > > With this patch, JOL is able to be close to `deepSizeOf` implementation from SizeOf JEP. > > Example performance improvements for sizing up a custom linked list: > > Benchmark (size) Mode Cnt Score Error Units > > # Default > LinkedChainBench.linkedChain 1 avgt 5 705.835 ? 8.051 ns/op > LinkedChainBench.linkedChain 10 avgt 5 3148.874 ? 37.856 ns/op > LinkedChainBench.linkedChain 100 avgt 5 28693.256 ? 142.254 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 290161.590 ? 4594.631 ns/op > > # Instrumentation attached, no intrinsics > LinkedChainBench.linkedChain 1 avgt 5 159.659 ? 19.238 ns/op > LinkedChainBench.linkedChain 10 avgt 5 717.659 ? 22.540 ns/op > LinkedChainBench.linkedChain 100 avgt 5 7739.394 ? 111.683 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 80724.238 ? 2887.794 ns/op > > # Instrumentation attached, new intrinsics > LinkedChainBench.linkedChain 1 avgt 5 95.254 ? 0.808 ns/op > LinkedChainBench.linkedChain 10 avgt 5 261.564 ? 8.524 ns/op > LinkedChainBench.linkedChain 100 avgt 5 3367.192 ? 21.128 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 34148.851 ? 373.080 ns/op Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Drop parentheses around comparisons ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/650/files - new: https://git.openjdk.java.net/jdk/pull/650/files/1b7290a3..89881e9f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=650&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=650&range=05-06 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/650.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/650/head:pull/650 PR: https://git.openjdk.java.net/jdk/pull/650 From lzang at openjdk.java.net Wed Nov 11 10:48:12 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 11 Nov 2020 10:48:12 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v3] In-Reply-To: References: Message-ID: > The implementation of jmap tool depends on the implementation of object iteration by different GC heap. > This patch extend the BasicJMapTest to cover differet GC Heap. Lin Zang has updated the pull request incrementally with one additional commit since the last revision: Refine the test configuration. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1094/files - new: https://git.openjdk.java.net/jdk/pull/1094/files/499e75a9..02b37920 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=01-02 Stats: 49 lines in 2 files changed: 43 ins; 4 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1094.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1094/head:pull/1094 PR: https://git.openjdk.java.net/jdk/pull/1094 From lzang at openjdk.java.net Wed Nov 11 10:56:57 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 11 Nov 2020 10:56:57 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v3] In-Reply-To: <0_3pc_EJdHcDUEV8wcb14AL0Cj6DC_ezPVF1qQocVzc=.66bc32bb-b062-4011-8667-d35e903fdc3c@github.com> References: <0_3pc_EJdHcDUEV8wcb14AL0Cj6DC_ezPVF1qQocVzc=.66bc32bb-b062-4011-8667-d35e903fdc3c@github.com> Message-ID: On Tue, 10 Nov 2020 17:50:26 GMT, Aleksey Shipilev wrote: >> Lin Zang has updated the pull request incrementally with one additional commit since the last revision: >> >> Refine the test configuration. > > I believe this would fail when some GCs are not available. For example, in Minimal/Zero only Serial and Parallel are available. ZGC and Shenandoah are not available on all platforms. Plus, specifying another GC with `TEST_VM_OPTS` would probably fail with "multiple GCs selected". > > You need to split the tests like this, and protect each config with `@requires`: > > /* > * @test > * @summary Unit test for jmap utility > * @key intermittent > * @requires vm.gc.Parallel > * @library /test/lib > * @build jdk.test.lib.hprof.* > * @build jdk.test.lib.hprof.model.* > * @build jdk.test.lib.hprof.parser.* > * @build jdk.test.lib.hprof.util.* > * @run main/othervm/timeout=240 -XX:+UseParallelGC BasicJMapTest > */ > > /* > * @test > * @summary Unit test for jmap utility > * @key intermittent > * @requires vm.gc.G1 > * @library /test/lib > * @build jdk.test.lib.hprof.* > * @build jdk.test.lib.hprof.model.* > * @build jdk.test.lib.hprof.parser.* > * @build jdk.test.lib.hprof.util.* > * @run main/othervm/timeout=240 -XX:+UseG1GC BasicJMapTest > */ > ... > > Maybe there is a way to clean up multiple `@build` tags to make the test config less verbose. Hi @shipilev, Thanks for reviewing, I have updated the refine commit. One problem is about epsilonGC, this test case keep fail with OOME even with -Xmx set to 4GB. I think it may be better to set up a new issue to investigate whether the OOME is expected, as it may take more efforts that is not quite related with testing jmap. so I didn't include it for this test in this PR. What do you think? Thanks! Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/1094 From shade at openjdk.java.net Wed Nov 11 11:15:56 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 11:15:56 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v3] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 10:48:12 GMT, Lin Zang wrote: >> The implementation of jmap tool depends on the implementation of object iteration by different GC heap. >> This patch extend the BasicJMapTest to cover differet GC Heap. > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > Refine the test configuration. I think it is fine to omit Epsilon from this testing. There is another trick you can use: `@test id=serial` would name the tests appropriately, not just "id#": Running test 'jtreg:test/jdk/sun/tools/jmap/BasicJMapTest.java' Passed: sun/tools/jmap/BasicJMapTest.java#parallel Passed: sun/tools/jmap/BasicJMapTest.java#serial Passed: sun/tools/jmap/BasicJMapTest.java#z Passed: sun/tools/jmap/BasicJMapTest.java#g1 Passed: sun/tools/jmap/BasicJMapTest.java#shenandoah Test results: passed: 5 I also tested this patch with x86_32 that implicitly disables Z, and it passes. Also passes with TEST_VM_OPTS="-XX:+UseSerialGC", with only one config automatically selected. @iignatev might want to ack this, as our resident test overseer. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1094 From lzang at openjdk.java.net Wed Nov 11 11:24:08 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 11 Nov 2020 11:24:08 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v4] In-Reply-To: References: Message-ID: > The implementation of jmap tool depends on the implementation of object iteration by different GC heap. > This patch extend the BasicJMapTest to cover differet GC Heap. Lin Zang has updated the pull request incrementally with one additional commit since the last revision: Add test id ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1094/files - new: https://git.openjdk.java.net/jdk/pull/1094/files/02b37920..3d9f6e5d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=02-03 Stats: 5 lines in 1 file changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/1094.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1094/head:pull/1094 PR: https://git.openjdk.java.net/jdk/pull/1094 From lzang at openjdk.java.net Wed Nov 11 11:24:08 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 11 Nov 2020 11:24:08 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v3] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 11:13:29 GMT, Aleksey Shipilev wrote: >> Lin Zang has updated the pull request incrementally with one additional commit since the last revision: >> >> Refine the test configuration. > > I think it is fine to omit Epsilon from this testing. There is another trick you can use: `@test id=serial` would name the tests appropriately, not just "id#": > > Running test 'jtreg:test/jdk/sun/tools/jmap/BasicJMapTest.java' > Passed: sun/tools/jmap/BasicJMapTest.java#parallel > Passed: sun/tools/jmap/BasicJMapTest.java#serial > Passed: sun/tools/jmap/BasicJMapTest.java#z > Passed: sun/tools/jmap/BasicJMapTest.java#g1 > Passed: sun/tools/jmap/BasicJMapTest.java#shenandoah > Test results: passed: 5 > > I also tested this patch with x86_32 that implicitly disables Z, and it passes. Also passes with TEST_VM_OPTS="-XX:+UseSerialGC", with only one config automatically selected. > > @iignatev might want to ack this, as our resident test overseer. @shipilev Thanks for guidance! Added the id tag. -Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/1094 From tschatzl at openjdk.java.net Wed Nov 11 13:17:03 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 11 Nov 2020 13:17:03 GMT Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality Message-ID: Hi all, can I get reviews for this change that removes the "Allocation of old generation of Java heap on alternate memory devices" functionality introduced with JDK 12 with [JDK-8202286](https://bugs.openjdk.java.net/browse/JDK-8202286) due to being - not used by anyone - not maintained by anyone, i.e. several bugs open for a long time and bit rotting - requiring some workarounds for new feature development wrt to heap management All flags covered by this feature were experimental flags, so there are no additional procedural issues to take. I tried to remove all but a few minor cleanups that I thought useful, but of course this is very subjective. Testing: hs-tier1-5 ------------- Commit messages: - Initial import Changes: https://git.openjdk.java.net/jdk/pull/1162/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1162&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256181 Stats: 2309 lines in 49 files changed: 4 ins; 2177 del; 128 mod Patch: https://git.openjdk.java.net/jdk/pull/1162.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1162/head:pull/1162 PR: https://git.openjdk.java.net/jdk/pull/1162 From coleenp at openjdk.java.net Wed Nov 11 13:39:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 11 Nov 2020 13:39:02 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Tue, 10 Nov 2020 23:24:24 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > resolve more robehn and coleenp comments. Looks good! I don't have more comments blocking approval. ------------- Changes requested by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Wed Nov 11 13:53:08 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 11 Nov 2020 13:53:08 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: On Wed, 11 Nov 2020 05:11:10 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> resolve more robehn and coleenp comments. > > src/hotspot/share/runtime/synchronizer.cpp line 246: > >> 244: // >> 245: // Start the ceiling with the estimate for one thread: >> 246: jint _in_use_list_ceiling = AvgMonitorsPerThreadEstimate; > > Why is this a jint when you use size_t for its accessor and all the other sizes that you compare with the ceiling are also size_t? > I'm not sure size_t is right to use in these cases (do we really expect different maximums on 32-bit versus 64-bit?) but it should be all or none IMO. Our int types are really confused. AvgMonitorsPerThreadEstimate is defined as an intx which is intptr_t and the range of it is 0..max_jint which is 0 .. 0x7fffffff . jint is long on windows (the problematic type) and int on unix. Since this is a new declaration, it probably should be something other than jint but what? At any rate, it should be declared as 'static'. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From coleenp at openjdk.java.net Wed Nov 11 13:59:04 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 11 Nov 2020 13:59:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Tue, 10 Nov 2020 23:24:24 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > resolve more robehn and coleenp comments. Marked as reviewed by coleenp (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From thomas.stuefe at gmail.com Wed Nov 11 14:06:52 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 11 Nov 2020 15:06:52 +0100 Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality In-Reply-To: References: Message-ID: Hi Thomas, I think this makes sense. Just a question, how are your thoughts about JEP316? Do you consider AllocateHeapAt similarly unused? Cheers, Thomas On Wed, Nov 11, 2020 at 2:17 PM Thomas Schatzl wrote: > Hi all, > > can I get reviews for this change that removes the "Allocation of old > generation of Java heap on alternate memory devices" functionality > introduced with JDK 12 with [JDK-8202286]( > https://bugs.openjdk.java.net/browse/JDK-8202286) due to being > > - not used by anyone > - not maintained by anyone, i.e. several bugs open for a long time and bit > rotting > - requiring some workarounds for new feature development wrt to heap > management > > All flags covered by this feature were experimental flags, so there are no > additional procedural issues to take. > > I tried to remove all but a few minor cleanups that I thought useful, but > of course this is very subjective. > > Testing: hs-tier1-5 > > ------------- > > Commit messages: > - Initial import > > Changes: https://git.openjdk.java.net/jdk/pull/1162/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1162&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8256181 > Stats: 2309 lines in 49 files changed: 4 ins; 2177 del; 128 mod > Patch: https://git.openjdk.java.net/jdk/pull/1162.diff > Fetch: git fetch https://git.openjdk.java.net/jdk > pull/1162/head:pull/1162 > > PR: https://git.openjdk.java.net/jdk/pull/1162 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcubed at openjdk.java.net Wed Nov 11 15:09:04 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 15:09:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: <9FnB_Mqa-xxw6ckDqqHgwo8RrtDOxwE5ji25mSZnCBc=.5664b114-621e-4c44-be28-47cdf45d855d@github.com> On Wed, 11 Nov 2020 04:50:52 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> resolve more robehn and coleenp comments. > > src/hotspot/share/runtime/synchronizer.cpp line 153: > >> 151: if (self->is_Java_thread()) { >> 152: // A JavaThread must check for a safepoint/handshake and honor it. >> 153: ObjectSynchronizer::chk_for_block_req(self->as_Java_thread(), "unlinking", > > I won't disapprove but this is a case where refactoring is IMO worse than code duplication. Logging parameters should not be a part of this API IMO. At this point, the refactoring has grown on me. I like the fact that it reduces the "noise" in those three functions. > src/hotspot/share/runtime/synchronizer.cpp line 1228: > >> 1226: os::naked_short_sleep(999); // sleep for almost 1 second >> 1227: } else { >> 1228: os::naked_short_sleep(999); // sleep for almost 1 second > > So this block can now just be: > >> if (self->is_Java_thread()) { >> ThreadBlockInVM tbivm(self->as_Java_thread()); >> } >> os::naked_short_sleep(999); // sleep for almost 1 second As @robehn pointed out, I screwed up that change because I lost the block over duration of the sleep. I've reverted that change to the original. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From iignatyev at openjdk.java.net Wed Nov 11 15:13:57 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Wed, 11 Nov 2020 15:13:57 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v4] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 11:24:08 GMT, Lin Zang wrote: >> The implementation of jmap tool depends on the implementation of object iteration by different GC heap. >> This patch extend the BasicJMapTest to cover differet GC Heap. > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > Add test id Hi @linzang , it all looks good to me, I'd, however, keep `@build` tags as they were before, this is one of a few difference b/w jdk and hotspot test suites ( in this particular case, it actually matters as discrepancies can lead to sporadic failures). in jdk test-suite, people use a sufficient set of `@build` tag. -- Igor ------------- Changes requested by iignatyev (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1094 From dcubed at openjdk.java.net Wed Nov 11 15:16:03 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 15:16:03 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: On Wed, 11 Nov 2020 13:50:08 GMT, Coleen Phillimore wrote: >> src/hotspot/share/runtime/synchronizer.cpp line 246: >> >>> 244: // >>> 245: // Start the ceiling with the estimate for one thread: >>> 246: jint _in_use_list_ceiling = AvgMonitorsPerThreadEstimate; >> >> Why is this a jint when you use size_t for its accessor and all the other sizes that you compare with the ceiling are also size_t? >> I'm not sure size_t is right to use in these cases (do we really expect different maximums on 32-bit versus 64-bit?) but it should be all or none IMO. > > Our int types are really confused. AvgMonitorsPerThreadEstimate is defined as an intx which is intptr_t and the range of it is 0..max_jint which is 0 .. 0x7fffffff . jint is long on windows (the problematic type) and int on unix. Since this is a new declaration, it probably should be something other than jint but what? > At any rate, it should be declared as 'static'. `_in_use_list_ceiling` is a jint **because** we've specified the range as `0..max_jint` and I wanted some sanity to that variable's type. If I change `_in_use_list_ceiling` to `size_t`, then I get a compile time error probably because `AvgMonitorsPerThreadEstimate` is an `intx` which (I think) is my only choice for a command line option. @fisk will have to chime in with the background on why he picked `size_t`. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Wed Nov 11 15:16:04 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 15:16:04 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: <8BzDrxXiWh2l4ZOq6k0D5-cU76gtjFXa9S1esX4Kkqg=.a42d08ea-8662-461a-a5e1-0335ef4ecd83@github.com> On Wed, 11 Nov 2020 08:26:14 GMT, Robbin Ehn wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> resolve more robehn and coleenp comments. > > src/hotspot/share/runtime/synchronizer.cpp line 1226: > >> 1224: ThreadBlockInVM tbivm(self->as_Java_thread()); >> 1225: } >> 1226: os::naked_short_sleep(999); // sleep for almost 1 second > > Hi Dan, you need to be blocked while sleeping, otherwise you are blocking safepoints for "almost 1 second". > So previous code was correct. Good catch! Yes, I should not have written that change so late at night. I know better. I have reverted it. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Wed Nov 11 15:19:03 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 15:19:03 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: On Wed, 11 Nov 2020 05:12:21 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> resolve more robehn and coleenp comments. > > One change requested in relation to use of jint instead of size_t. > One code simplification suggestion. > Thanks, > David @dholmes-ora, @robehn and @coleenp - Thanks for chiming in on the review again. For the first time, I have a real conflict in a file so I'm updating my repo to the latest and greatest to see how that works. Then I'll make the minor tweaks that I mumbled about above. I did a Mach5 Tier1,2,3,4,5,6,7 last night and got through it with a single Tier7 unrelated and known failure. Of course, the baseline for last night was jdk-16+23 so... ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From ayang at openjdk.java.net Wed Nov 11 15:21:56 2020 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Wed, 11 Nov 2020 15:21:56 GMT Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality In-Reply-To: References: Message-ID: <0nEbOq5anud0koNcq_5EV57wUd8wc8qHUCjI1GP2L0I=.ae3defc8-6c6b-40dd-894b-e5a4dde4d4bb@github.com> On Wed, 11 Nov 2020 11:11:25 GMT, Thomas Schatzl wrote: > Hi all, > > can I get reviews for this change that removes the "Allocation of old generation of Java heap on alternate memory devices" functionality introduced with JDK 12 with [JDK-8202286](https://bugs.openjdk.java.net/browse/JDK-8202286) due to being > > - not used by anyone > - not maintained by anyone, i.e. several bugs open for a long time and bit rotting > - requiring some workarounds for new feature development wrt to heap management > > All flags covered by this feature were experimental flags, so there are no additional procedural issues to take. > > I tried to remove all but a few minor cleanups that I thought useful, but of course this is very subjective. > > Testing: hs-tier1-5 A general comment for future PRs: I think it's best to isolate mechanical changes into their own commits; e.g. `HeapRegionManager* _hrm;` -> `HeapRegionManager _hrm;`. Otherwise, a real change, buried in the immense size of diff, might slip through. test/hotspot/jtreg/TEST.ROOT line 78: > 76: vm.musl \ > 77: docker.support \ > 78: test.vm.gc.nvdimm \ `test/jtreg-ext/requires/VMProps.java` still references `test.vm.gc.nvdimm`, btw. ------------- Marked as reviewed by ayang (Author). PR: https://git.openjdk.java.net/jdk/pull/1162 From dcubed at openjdk.java.net Wed Nov 11 15:26:02 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 15:26:02 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: On Wed, 11 Nov 2020 13:50:08 GMT, Coleen Phillimore wrote: >> src/hotspot/share/runtime/synchronizer.cpp line 246: >> >>> 244: // >>> 245: // Start the ceiling with the estimate for one thread: >>> 246: jint _in_use_list_ceiling = AvgMonitorsPerThreadEstimate; >> >> Why is this a jint when you use size_t for its accessor and all the other sizes that you compare with the ceiling are also size_t? >> I'm not sure size_t is right to use in these cases (do we really expect different maximums on 32-bit versus 64-bit?) but it should be all or none IMO. > > Our int types are really confused. AvgMonitorsPerThreadEstimate is defined as an intx which is intptr_t and the range of it is 0..max_jint which is 0 .. 0x7fffffff . jint is long on windows (the problematic type) and int on unix. Since this is a new declaration, it probably should be something other than jint but what? > At any rate, it should be declared as 'static'. @coleenp - Nice catch on the missing 'static'. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From tschatzl at openjdk.java.net Wed Nov 11 15:38:10 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 11 Nov 2020 15:38:10 GMT Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality [v2] In-Reply-To: References: Message-ID: > Hi all, > > can I get reviews for this change that removes the "Allocation of old generation of Java heap on alternate memory devices" functionality introduced with JDK 12 with [JDK-8202286](https://bugs.openjdk.java.net/browse/JDK-8202286) due to being > > - not used by anyone > - not maintained by anyone, i.e. several bugs open for a long time and bit rotting > - requiring some workarounds for new feature development wrt to heap management > > All flags covered by this feature were experimental flags, so there are no additional procedural issues to take. > > I tried to remove all but a few minor cleanups that I thought useful, but of course this is very subjective. > > Testing: hs-tier1-5 Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: ayang review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1162/files - new: https://git.openjdk.java.net/jdk/pull/1162/files/222739bc..9e849b60 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1162&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1162&range=00-01 Stats: 6 lines in 1 file changed: 0 ins; 6 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1162.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1162/head:pull/1162 PR: https://git.openjdk.java.net/jdk/pull/1162 From lzang at openjdk.java.net Wed Nov 11 15:45:13 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 11 Nov 2020 15:45:13 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v5] In-Reply-To: References: Message-ID: <6GuMUg9fijsnxqSWZ6jakR-Fv5Eac2DRgqU64svKPzI=.f12729d8-b28f-4457-9e96-aa4c5cb09b61@github.com> > The implementation of jmap tool depends on the implementation of object iteration by different GC heap. > This patch extend the BasicJMapTest to cover differet GC Heap. Lin Zang has updated the pull request incrementally with one additional commit since the last revision: recover build tag configuration ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1094/files - new: https://git.openjdk.java.net/jdk/pull/1094/files/3d9f6e5d..548b48ed Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1094&range=03-04 Stats: 15 lines in 1 file changed: 15 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1094.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1094/head:pull/1094 PR: https://git.openjdk.java.net/jdk/pull/1094 From iignatyev at openjdk.java.net Wed Nov 11 15:45:14 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Wed, 11 Nov 2020 15:45:14 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v5] In-Reply-To: <6GuMUg9fijsnxqSWZ6jakR-Fv5Eac2DRgqU64svKPzI=.f12729d8-b28f-4457-9e96-aa4c5cb09b61@github.com> References: <6GuMUg9fijsnxqSWZ6jakR-Fv5Eac2DRgqU64svKPzI=.f12729d8-b28f-4457-9e96-aa4c5cb09b61@github.com> Message-ID: <7CZKkBof8HnFlzzmW9XPbRUAcPDUkd5UOJm-7aq1zP0=.b75ed177-8444-45d0-b67d-1e45c3baca3f@github.com> On Wed, 11 Nov 2020 15:42:36 GMT, Lin Zang wrote: >> The implementation of jmap tool depends on the implementation of object iteration by different GC heap. >> This patch extend the BasicJMapTest to cover differet GC Heap. > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > recover build tag configuration Marked as reviewed by iignatyev (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1094 From lzang at openjdk.java.net Wed Nov 11 15:45:14 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 11 Nov 2020 15:45:14 GMT Subject: RFR: 8255982: Extend BasicJMapTest to test with different GC Heap [v4] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 15:10:54 GMT, Igor Ignatyev wrote: >> Lin Zang has updated the pull request incrementally with one additional commit since the last revision: >> >> Add test id > > Hi @linzang , > > it all looks good to me, I'd, however, keep `@build` tags as they were before, this is one of a few difference b/w jdk and hotspot test suites ( in this particular case, it actually matters as discrepancies can lead to sporadic failures). in jdk test-suite, people use a sufficient set of `@build` tag. > > -- Igor Hi @iignatev, Thanks for your explanation! recovered the @build tag. -Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/1094 From tschatzl at openjdk.java.net Wed Nov 11 15:47:59 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 11 Nov 2020 15:47:59 GMT Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality [v2] In-Reply-To: <0nEbOq5anud0koNcq_5EV57wUd8wc8qHUCjI1GP2L0I=.ae3defc8-6c6b-40dd-894b-e5a4dde4d4bb@github.com> References: <0nEbOq5anud0koNcq_5EV57wUd8wc8qHUCjI1GP2L0I=.ae3defc8-6c6b-40dd-894b-e5a4dde4d4bb@github.com> Message-ID: <_bANQzB5ETr8QdwEiSRFKzMLwWE_IV-Pwpf0mSjtjmM=.574e4b29-db0b-4eaa-8065-a2c713c8c3c2@github.com> On Wed, 11 Nov 2020 15:09:44 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review > > test/hotspot/jtreg/TEST.ROOT line 78: > >> 76: vm.musl \ >> 77: docker.support \ >> 78: test.vm.gc.nvdimm \ > > `test/jtreg-ext/requires/VMProps.java` still references `test.vm.gc.nvdimm`, btw. The original change introduced the `_hrm` -> `*_hrm` change, so this is a straightforward reversal, no optimization :) There has been one more removal possible by the `test.vm.gc.nvdimm` line in VMProps.java. Thanks for your review! ------------- PR: https://git.openjdk.java.net/jdk/pull/1162 From dcubed at openjdk.java.net Wed Nov 11 16:01:15 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 16:01:15 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v7] In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also done promotion perf runs on these bits and says "the results look fine". Daniel D. Daugherty has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: - @dholmes-ora, @robehn and @coleenp CR - a few more minor tweaks. - Merge branch 'master' into JDK-8253064 - resolve more robehn and coleenp comments. - coleenp CR - refactor common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). - dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). - Resolve more @dholmes-ora comments with help from @fisk. - Resolve most of dholmes-ora comments. - 8253064.v00.part2 - 8253064.v00.part1 ------------- Changes: https://git.openjdk.java.net/jdk/pull/642/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=642&range=06 Stats: 2506 lines in 25 files changed: 602 ins; 1697 del; 207 mod Patch: https://git.openjdk.java.net/jdk/pull/642.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/642/head:pull/642 PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Wed Nov 11 16:24:06 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 16:24:06 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v7] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Wed, 11 Nov 2020 16:18:41 GMT, Robbin Ehn wrote: >> Daniel D. Daugherty has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: >> >> - @dholmes-ora, @robehn and @coleenp CR - a few more minor tweaks. >> - Merge branch 'master' into JDK-8253064 >> - resolve more robehn and coleenp comments. >> - coleenp CR - refactor common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). >> - dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). >> - Resolve more @dholmes-ora comments with help from @fisk. >> - Resolve most of dholmes-ora comments. >> - 8253064.v00.part2 >> - 8253064.v00.part1 > > Thanks! Rebuild on my MBP13 and KitchensinkSanity look good. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From rehn at openjdk.java.net Wed Nov 11 16:24:06 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 11 Nov 2020 16:24:06 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v7] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Wed, 11 Nov 2020 16:01:15 GMT, Daniel D. Daugherty wrote: >> Changes from @fisk and @dcubed-ojdk to: >> >> - simplify ObjectMonitor list management >> - get rid of Type-Stable Memory (TSM) >> >> This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. >> Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, >> SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) >> - a few minor regressions (<= -0.24%) >> - Volano is 6.8% better >> >> Eric C. has also done promotion perf runs on these bits and says "the results look fine". > > Daniel D. Daugherty has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: > > - @dholmes-ora, @robehn and @coleenp CR - a few more minor tweaks. > - Merge branch 'master' into JDK-8253064 > - resolve more robehn and coleenp comments. > - coleenp CR - refactor common ThreadBlockInVM code block into ObjectSynchronizer::chk_for_block_req(). > - dholmes-ora - convert inner while loop to do-while loop in unlink_deflated(). > - Resolve more @dholmes-ora comments with help from @fisk. > - Resolve most of dholmes-ora comments. > - 8253064.v00.part2 > - 8253064.v00.part1 Thanks! ------------- Marked as reviewed by rehn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Wed Nov 11 16:24:08 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 16:24:08 GMT Subject: Integrated: 8253064: monitor list simplifications and getting rid of TSM In-Reply-To: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> Message-ID: On Tue, 13 Oct 2020 20:31:44 GMT, Daniel D. Daugherty wrote: > Changes from @fisk and @dcubed-ojdk to: > > - simplify ObjectMonitor list management > - get rid of Type-Stable Memory (TSM) > > This change has been tested with Mach5 Tier[1-3],4,5,6,7,8; no new regressions. > Aurora Perf runs have also been done (DaCapo-h2, Quick Startup/Footprint, > SPECjbb2015-Tuned-G1, SPECjbb2015-Tuned-ParGC, Volano) > - a few minor regressions (<= -0.24%) > - Volano is 6.8% better > > Eric C. has also done promotion perf runs on these bits and says "the results look fine". This pull request has now been integrated. Changeset: 2e19026d Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/2e19026d Stats: 2506 lines in 25 files changed: 602 ins; 1697 del; 207 mod 8253064: monitor list simplifications and getting rid of TSM Co-authored-by: Erik ?sterlund Reviewed-by: eosterlund, rehn, coleenp ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From eosterlund at openjdk.java.net Wed Nov 11 16:24:07 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Wed, 11 Nov 2020 16:24:07 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: On Wed, 11 Nov 2020 15:23:15 GMT, Daniel D. Daugherty wrote: >> Our int types are really confused. AvgMonitorsPerThreadEstimate is defined as an intx which is intptr_t and the range of it is 0..max_jint which is 0 .. 0x7fffffff . jint is long on windows (the problematic type) and int on unix. Since this is a new declaration, it probably should be something other than jint but what? >> At any rate, it should be declared as 'static'. > > @coleenp - Nice catch on the missing 'static'. I typically use size_t for entities that can scale with the size of the machine's memory, so I don't have to think about whether there are enough bits. Could AvgMonitorsPerThreadEstimate be uintx instead of intx? And then maybe we don't need to declare a range, as the natural range of the uintx seems perfectly valid. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Wed Nov 11 16:39:02 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 16:39:02 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: <4xLwmW1Wya1aTmUbGPONAA7V-ScyRsdUK467gNvdmCQ=.5c2329ea-4b45-4732-8abd-e94d1f89ad5f@github.com> On Wed, 11 Nov 2020 16:16:19 GMT, Erik ?sterlund wrote: >> @coleenp - Nice catch on the missing 'static'. > > I typically use size_t for entities that can scale with the size of the machine's memory, so I don't have to think about whether there are enough bits. Could AvgMonitorsPerThreadEstimate be uintx instead of intx? And then maybe we don't need to declare a range, as the natural range of the uintx seems perfectly valid. I'm pretty sure I copied the decl for AvgMonitorsPerThreadEstimate from some other already existing option. That's SOP for me anyway... If we make any more changes here it will have to be in a follow up. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Wed Nov 11 16:39:01 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 16:39:01 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: On Wed, 11 Nov 2020 05:12:21 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> resolve more robehn and coleenp comments. > > One change requested in relation to use of jint instead of size_t. > One code simplification suggestion. > Thanks, > David This PR is now integrated! @dholmes-ora - for some reason you are not listed as a reviewer and I'm not sure why. I'm sorry I didn't notice that you were dropped off before I pulled the trigger. ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From iklam at openjdk.java.net Wed Nov 11 17:41:59 2020 From: iklam at openjdk.java.net (Ioi Lam) Date: Wed, 11 Nov 2020 17:41:59 GMT Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality [v2] In-Reply-To: <0nEbOq5anud0koNcq_5EV57wUd8wc8qHUCjI1GP2L0I=.ae3defc8-6c6b-40dd-894b-e5a4dde4d4bb@github.com> References: <0nEbOq5anud0koNcq_5EV57wUd8wc8qHUCjI1GP2L0I=.ae3defc8-6c6b-40dd-894b-e5a4dde4d4bb@github.com> Message-ID: On Wed, 11 Nov 2020 15:19:05 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review > > A general comment for future PRs: I think it's best to isolate mechanical changes into their own commits; e.g. `HeapRegionManager* _hrm;` -> `HeapRegionManager _hrm;`. Otherwise, a real change, buried in the immense size of diff, might slip through. This is a good fix. It removes the dependency on the problematic API `ReservedSpace::first_part(..., split=true)`, which cannot be implemented correctly on Windows (see [JDK-8256079](https://bugs.openjdk.java.net/browse/JDK-8256079)). This API is used only by CDS (to be removed in [JDK-8255917](https://bugs.openjdk.java.net/browse/JDK-8255917)) and heterogenous heap. So hopefully we can remove this problematic API soon. ------------- PR: https://git.openjdk.java.net/jdk/pull/1162 From iignatyev at openjdk.java.net Wed Nov 11 17:45:58 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Wed, 11 Nov 2020 17:45:58 GMT Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality [v2] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 15:38:10 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I get reviews for this change that removes the "Allocation of old generation of Java heap on alternate memory devices" functionality introduced with JDK 12 with [JDK-8202286](https://bugs.openjdk.java.net/browse/JDK-8202286) due to being >> >> - not used by anyone >> - not maintained by anyone, i.e. several bugs open for a long time and bit rotting >> - requiring some workarounds for new feature development wrt to heap management >> >> All flags covered by this feature were experimental flags, so there are no additional procedural issues to take. >> >> I tried to remove all but a few minor cleanups that I thought useful, but of course this is very subjective. >> >> Testing: hs-tier1-5 > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > ayang review Marked as reviewed by iignatyev (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1162 From shade at openjdk.java.net Wed Nov 11 20:04:00 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 20:04:00 GMT Subject: RFR: 8253525: Implement getInstanceSize/sizeOf intrinsics [v6] In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 20:13:41 GMT, Serguei Spitsyn wrote: > One more nit, I forgot to list in my previous comment, is uneeded '()' around comparisons: > `+ static final int REF_SIZE = ((compressedOops == null) || (compressedOops == true)) ? 4 : 8;` Right. Fixed that. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/650 From rehn at openjdk.java.net Wed Nov 11 21:20:04 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 11 Nov 2020 21:20:04 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" Message-ID: As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. Resulting in an unexpected result to GetCurrentContendedMonitor(). Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. Instead I here just use sleep + volatile for the barriers. I cannot reproduce with these changes. Chewing through T6 as most issues have been seen there. ------------- Commit messages: - Volatile and sleep Changes: https://git.openjdk.java.net/jdk/pull/1177/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1177&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8244679 Stats: 39 lines in 2 files changed: 25 ins; 5 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/1177.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1177/head:pull/1177 PR: https://git.openjdk.java.net/jdk/pull/1177 From dholmes at openjdk.java.net Wed Nov 11 22:06:53 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 11 Nov 2020 22:06:53 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 20:33:15 GMT, Robbin Ehn wrote: > As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. > Resulting in an unexpected result to GetCurrentContendedMonitor(). > Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. > Instead I here just use sleep + volatile for the barriers. > > I cannot reproduce with these changes. > > Chewing through T6 as most issues have been seen there. Hi Robbin, It isn't at all clear to me exactly what class initialization is happening, when, and how the contention is arising. I see in the bug report a stack dump for wicket.unlock and cannot see how unlock can trigger class initialization when all classes would have been needed by the lock. That aside avoiding the class loading/initialization seems more of a workaround. Ideally the test would be more discriminating about what monitors it checks for; or more accurately ensures that the code has reached the right place before doing the check. Thanks, David ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From david.holmes at oracle.com Wed Nov 11 22:15:41 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 12 Nov 2020 08:15:41 +1000 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> Message-ID: <68912b7c-a600-fbea-6c0b-b4350ec8cc5c@oracle.com> On 12/11/2020 2:39 am, Daniel D.Daugherty wrote: > On Wed, 11 Nov 2020 05:12:21 GMT, David Holmes wrote: > >>> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >>> >>> resolve more robehn and coleenp comments. >> >> One change requested in relation to use of jint instead of size_t. >> One code simplification suggestion. >> Thanks, >> David > > This PR is now integrated! @dholmes-ora - for some reason you are not listed > as a reviewer and I'm not sure why. I'm sorry I didn't notice that you were dropped > off before I pulled the trigger. I'm not listed as a Reviewer because I did not mark this as approved and had an outstanding "change requested" status. That's just the way things work. I still think the jint/size_t situation is an unnecessary mess, but appreciate the extra comments you added. Thanks, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/642 > From david.holmes at oracle.com Wed Nov 11 22:17:48 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 12 Nov 2020 08:17:48 +1000 Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: <7rQQpVtTPecL-Vt4JJ0CR3U_1523KksQwFAsn9QpJK8=.5e872ffe-d1e4-40e9-9dd4-595712452e5c@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> <7rQQpVtTPecL-Vt4JJ0CR3U_1523KksQwFAsn9QpJK8=.5e872ffe-d1e4-40e9-9dd4-595712452e5c@github.com> Message-ID: <900e4b3d-47d1-eecd-53f3-15ff16da3741@oracle.com> On 11/11/2020 7:51 am, Daniel D.Daugherty wrote: > On Tue, 10 Nov 2020 21:16:33 GMT, Robbin Ehn wrote: > >>> So if I narrow the scope around the ThreadBlockInVM, then it would be fine? >>> >>> { >>> // Honor block request. >>> ThreadBlockInVM tbivm(self); >>> } >>> >>> I can make that change before I integrate... >> >> Yes that avoids it! > > Done. I also did the one in ObjectSynchronizer::request_deflate_idle_monitors(). Just to be crystal clear, the change in request_deflate_idle_monitors() was not needed as there is no logging code in the scope, and changing it was wrong as it put the sleep outside the scope of the TBIVM. Hence it was reverted and all is well. Thanks, David From pchilanomate at openjdk.java.net Wed Nov 11 22:26:54 2020 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Wed, 11 Nov 2020 22:26:54 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: <_-GUMtFBAMIFb1RQkHIj9GNafYZpSX-4w_IqKkLWdng=.225b5639-4baa-4e54-9a24-d4dfcd5d286f@github.com> On Wed, 11 Nov 2020 20:33:15 GMT, Robbin Ehn wrote: > As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. > Resulting in an unexpected result to GetCurrentContendedMonitor(). > Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. > Instead I here just use sleep + volatile for the barriers. > > I cannot reproduce with these changes. > > Chewing through T6 as most issues have been seen there. LGTM. Thanks for finding the issue! Patricio ------------- Marked as reviewed by pchilanomate (Committer). PR: https://git.openjdk.java.net/jdk/pull/1177 From pchilanomate at openjdk.java.net Wed Nov 11 22:29:58 2020 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Wed, 11 Nov 2020 22:29:58 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: <6Uj6LtroTGcZbJz6qLLF87ohUbpymP8B9_w6DRuR0pc=.36134043-4387-4863-b843-e49c7262f5c7@github.com> On Wed, 11 Nov 2020 20:33:15 GMT, Robbin Ehn wrote: > As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. > Resulting in an unexpected result to GetCurrentContendedMonitor(). > Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. > Instead I here just use sleep + volatile for the barriers. > > I cannot reproduce with these changes. > > Chewing through T6 as most issues have been seen there. test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 67: > 65: public static int run(String argv[], PrintStream ref) { > 66: out = ref; > 67: doSleep(); // If it would do any class loading, do it now. I think now the spawned thread should not try to resolve any new methods after setting the boolean so we shouldn't have the same ObjectLocker issue. ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From pchilanomate at openjdk.java.net Wed Nov 11 22:36:55 2020 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Wed, 11 Nov 2020 22:36:55 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 20:33:15 GMT, Robbin Ehn wrote: > As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. > Resulting in an unexpected result to GetCurrentContendedMonitor(). > Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. > Instead I here just use sleep + volatile for the barriers. > > I cannot reproduce with these changes. > > Chewing through T6 as most issues have been seen there. test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon002.java line 61: > 59: > 60: public static int run(String argv[], PrintStream ref) { > 61: doSleep(); // If it would do any class loading, do it now. I think now the spawned thread should not try to resolve any new methods after setting the boolean so we shouldn't have the same ObjectLocker issue. ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From dcubed at openjdk.java.net Wed Nov 11 22:57:59 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 22:57:59 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 22:33:57 GMT, Patricio Chilano Mateo wrote: >> As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. >> Resulting in an unexpected result to GetCurrentContendedMonitor(). >> Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. >> Instead I here just use sleep + volatile for the barriers. >> >> I cannot reproduce with these changes. >> >> Chewing through T6 as most issues have been seen there. > > test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon002.java line 61: > >> 59: >> 60: public static int run(String argv[], PrintStream ref) { >> 61: doSleep(); // If it would do any class loading, do it now. > > I think now the spawned thread should not try to resolve any new methods after setting the boolean so we shouldn't have the same ObjectLocker issue. Perhaps: // If we need to load any classes do execute doSleep(), do it now. ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From dcubed at openjdk.java.net Wed Nov 11 22:57:57 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 11 Nov 2020 22:57:57 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 20:33:15 GMT, Robbin Ehn wrote: > As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. > Resulting in an unexpected result to GetCurrentContendedMonitor(). > Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. > Instead I here just use sleep + volatile for the barriers. > > I cannot reproduce with these changes. > > Chewing through T6 as most issues have been seen there. I like the fix. Taking Wicket out of play will help with class loading issues in this test. You don't really need to deal with unexpected locking events from the ObjectLocker usage by class loading. test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 67: > 65: public static int run(String argv[], PrintStream ref) { > 66: out = ref; > 67: doSleep(); // If it would do any class loading, do it now. Perhaps: // If we need to load any classes do execute doSleep(), do it now. test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 79: > 77: contmon001a thr = new contmon001a(); > 78: startingBarrier = new Wicket(); > 79: waitingBarrier = new Wicket(); If you want to continue to use Wicket, then "all" you have to do is execute one back-and-forth of the Wicket before executing the test portion itself. That should get all the Wicket classes loaded before you really need them. Unless, of course, there's some class that isn't loaded unless there's an error or something... However, by using volatile booleans instead, you get out of the game of trying to figure out how much of the Wicket code needs to be exercised to get everything you might need loaded. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1177 From david.holmes at oracle.com Wed Nov 11 23:09:18 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 12 Nov 2020 09:09:18 +1000 Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: <23613b5d-e043-cedf-95b2-98f7153cc307@oracle.com> On 12/11/2020 8:57 am, Daniel D.Daugherty wrote: > On Wed, 11 Nov 2020 20:33:15 GMT, Robbin Ehn wrote: > >> As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. >> Resulting in an unexpected result to GetCurrentContendedMonitor(). >> Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. >> Instead I here just use sleep + volatile for the barriers. >> >> I cannot reproduce with these changes. >> >> Chewing through T6 as most issues have been seen there. > > I like the fix. Taking Wicket out of play will help with class loading > issues in this test. You don't really need to deal with unexpected > locking events from the ObjectLocker usage by class loading. The whole point of using Wicket in these tests is to try and correctly synchronize the threads involved so that we only inspect things when the target thread is in the right state. The serviceability folk have done a lot of test stabilisation work in that area. This change seems to be taking things in the wrong direction to me. But this is a serviceability test issue and it is up to the serviceability folk to determine how they'd like this test fixed. > test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 67: > >> 65: public static int run(String argv[], PrintStream ref) { >> 66: out = ref; >> 67: doSleep(); // If it would do any class loading, do it now. > > Perhaps: > // If we need to load any classes do execute doSleep(), do it now. Typo: do execute -> to execute Cheers, David ----- > test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 79: > >> 77: contmon001a thr = new contmon001a(); >> 78: startingBarrier = new Wicket(); >> 79: waitingBarrier = new Wicket(); > > If you want to continue to use Wicket, then "all" you have to do is > execute one back-and-forth of the Wicket before executing the > test portion itself. That should get all the Wicket classes loaded > before you really need them. Unless, of course, there's some class > that isn't loaded unless there's an error or something... > > However, by using volatile booleans instead, you get out of the > game of trying to figure out how much of the Wicket code needs > to be exercised to get everything you might need loaded. > > ------------- > > Marked as reviewed by dcubed (Reviewer). > > PR: https://git.openjdk.java.net/jdk/pull/1177 > From iklam at openjdk.java.net Thu Nov 12 00:34:55 2020 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 12 Nov 2020 00:34:55 GMT Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality [v2] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 15:38:10 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I get reviews for this change that removes the "Allocation of old generation of Java heap on alternate memory devices" functionality introduced with JDK 12 with [JDK-8202286](https://bugs.openjdk.java.net/browse/JDK-8202286) due to being >> >> - not used by anyone >> - not maintained by anyone, i.e. several bugs open for a long time and bit rotting >> - requiring some workarounds for new feature development wrt to heap management >> >> All flags covered by this feature were experimental flags, so there are no additional procedural issues to take. >> >> I tried to remove all but a few minor cleanups that I thought useful, but of course this is very subjective. >> >> Testing: hs-tier1-5 > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > ayang review LGTM ------------- Marked as reviewed by iklam (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1162 From ysuenaga at openjdk.java.net Thu Nov 12 01:36:56 2020 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Thu, 12 Nov 2020 01:36:56 GMT Subject: RFR: 8252657: JVMTI agent is not unloaded when Agent_OnAttach is failed In-Reply-To: <-Xrp6c94000-jE1p6NvzjsxFUW5ILrH_F1eT1i7esw8=.9d609f81-1b61-4ebf-9afd-73b834c1b18c@github.com> References: <1H1wUQdxCLU2qddqEIYSx2iOhIKL3b5etUmjsS6NBlU=.0bf1fe0c-8dcf-4ca0-bd57-b8794d5f2810@github.com> <80LJDTCsT_y-KlThryd5Bxu5RRyrjmKfs5p9vJUn61E=.68b594a0-fe58-4f4d-a49c-eec2e90f9373@github.com> <-Xrp6c94000-jE1p6NvzjsxFUW5ILrH_F1eT1i7esw8=.9d609f81-1b61-4ebf-9afd-73b834c1b18c@github.com> Message-ID: On Mon, 19 Oct 2020 01:32:45 GMT, Yasumasa Suenaga wrote: >>> * Q1: Is it necessary to call the Agent_OnUnload()? >> >> [JVMTI spec of Agent_OnUnload()](https://docs.oracle.com/en/java/javase/15/docs/specs/jvmti.html#onunload) says this function will be called when the agent library will be unloaded by platform specific mechanism. OTOH it also says `Agent_OnUnload()` will be called both at VM termination and **by other reasons**. >> The spec don't say for the case if `Agent_OnAttach()` would be failed. IMHO `Agent_OnUnload()` should be called because this PR would unload library if `Agent_OnAttach()` failed. >> >>> * Q2: Would it be a JVMTI spec violation to call the Agent_OnAttach() multiple times? (It seems to be the case to me.) >> >> `Agent_OnAttach()` should be called only once per attach request, but VM should accept multiple attach request for same agent library. >> >> For example, we can add multiple `-agentlib` and `-agentpath` request as below. JVMTI agent might change behavior due to arguments or configuration file. >> >> -agentlib:test=profile=A -agentlib:test=profile=B -agentpath:/path/to/libtest=profile=C >> >> Agent developers should have responsibility for the behavior when more than one agent is loaded at a time. >> >>> * Q3: What has to be done for statically linked agent? >> >> JVMTI spec says "unless it is statically linked into the executable", so I think we can ignore about Agent_OnUnload_L() in this PR. >> >>> * Q4: Should the agent be correctly loadable in the first place? What were the reasons its loading to fail? >> >> Agent (`Agent_OnAttach()`) might fail due to error in agent logic. For example, some agents load configuration file at initialization. If the user gives wrong value, it will fail. >> >>> Yes, at least, a CSR is needed for this. >> >> I will file CSR for this PR after this discussion. > > If we can change the spec that agent library would not be unloaded when `Agent_OnAttach()` failed, we can change like [webrev.00](https://cr.openjdk.java.net/~ysuenaga/JDK-8252657/webrev.00/). It is simple, and similar behavior with `Agent_OnLoad()`. It might be prefer for JVMTI agent developers. In case of `Agent_OnLoad()`, if it is failed (it returns other than zero), JVM is aborted and `Agent_OnUnload()` is not called. I think it is compliant with [JVMTI spec of Agent_OnUnload()](https://docs.oracle.com/en/java/javase/15/docs/specs/jvmti.html#onunload) which says uncontrolled shutdown (aborting JVM) is an exception to this rule. I will add CSR for this fix, but I want to discuss what we should do before. I like that `Agent_OnUnload()` wouldn't be called when `Agent_OnAttach()` is failed if we can change the spec - it is consistent and friendly with `Agent_OnUnload()`. ------------- PR: https://git.openjdk.java.net/jdk/pull/19 From lzang at openjdk.java.net Thu Nov 12 02:15:53 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Thu, 12 Nov 2020 02:15:53 GMT Subject: Integrated: 8255982: Extend BasicJMapTest to test with different GC Heap In-Reply-To: References: Message-ID: On Fri, 6 Nov 2020 12:54:28 GMT, Lin Zang wrote: > The implementation of jmap tool depends on the implementation of object iteration by different GC heap. > This patch extend the BasicJMapTest to cover differet GC Heap. This pull request has now been integrated. Changeset: 14e25e20 Author: Lin Zang Committer: Igor Ignatyev URL: https://git.openjdk.java.net/jdk/commit/14e25e20 Stats: 62 lines in 2 files changed: 58 ins; 0 del; 4 mod 8255982: Extend BasicJMapTest to test with different GC Heap Reviewed-by: shade, iignatyev ------------- PR: https://git.openjdk.java.net/jdk/pull/1094 From rehn at openjdk.java.net Thu Nov 12 07:52:55 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 12 Nov 2020 07:52:55 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: <32XslD8U3ZZosl9IOGJ_pAcIqNL7IhoRk2AzLqqzh-M=.2718521a-66ec-495b-bf20-02c1c0f7f69a@github.com> On Wed, 11 Nov 2020 22:55:16 GMT, Daniel D. Daugherty wrote: >> As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. >> Resulting in an unexpected result to GetCurrentContendedMonitor(). >> Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. >> Instead I here just use sleep + volatile for the barriers. >> >> I cannot reproduce with these changes. >> >> Chewing through T6 as most issues have been seen there. > > I like the fix. Taking Wicket out of play will help with class loading > issues in this test. You don't really need to deal with unexpected > locking events from the ObjectLocker usage by class loading. > Hi Robbin, > > It isn't at all clear to me exactly what class initialization is happening, when, and how the contention is arising. I see in the bug report a stack dump for wicket.unlock and cannot see how unlock can trigger class initialization when all classes would have been needed by the lock. Hey, David. Different paths uses different classes. In this case it's the AbstractQueuedSynchronizer that uses LockSupport if a thread have already been put to sleep. If we call unlock() before the other thread hit the parker in waitFor(), LockSupport is not used, and thus not loaded. Without any other synchronization we do not know when it will loaded LockSupport. Having a second synchronization we do not need the Wicket, hence may change removes the Wicket. As I said, we need to exercise all code paths (since any difference in timing may result in a different path), to do that reliable, we must use a second primitive. But it very to hard to guarantee that we actually manage that. Implementing some kind of warm-up in a static initializer in the Wicket might be doable. > That aside avoiding the class loading/initialization seems more of a workaround. Ideally the test would be more discriminating about what monitors it checks for; or more accurately ensures that the code has reached the right place before doing the check. Avoiding loading class and "accurately ensures that the code has reached the right place" is the same thing here, since unlock may do class loading thus making sure we do not call unlock() at the same time as we do GetCurrentContendedMonitor solves this. I looked at other primitives (which are not using synchronized), I saw none that would be any better than the Wicket (class loading-wise). In most use-cases of the Wicket there is no apparent reason why a j.u.c.Semaphore could not be used instead, maybe all. So I'm not even sure the Wicket is still something worth pursuing. Thanks, Robbin > > Thanks, > David ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Thu Nov 12 07:55:54 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 12 Nov 2020 07:55:54 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 22:43:49 GMT, Daniel D. Daugherty wrote: >> As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. >> Resulting in an unexpected result to GetCurrentContendedMonitor(). >> Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. >> Instead I here just use sleep + volatile for the barriers. >> >> I cannot reproduce with these changes. >> >> Chewing through T6 as most issues have been seen there. > > test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 67: > >> 65: public static int run(String argv[], PrintStream ref) { >> 66: out = ref; >> 67: doSleep(); // If it would do any class loading, do it now. > > Perhaps: > // If we need to load any classes do execute doSleep(), do it now. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Thu Nov 12 07:55:56 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 12 Nov 2020 07:55:56 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 22:53:08 GMT, Daniel D. Daugherty wrote: >> test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon002.java line 61: >> >>> 59: >>> 60: public static int run(String argv[], PrintStream ref) { >>> 61: doSleep(); // If it would do any class loading, do it now. >> >> I think now the spawned thread should not try to resolve any new methods after setting the boolean so we shouldn't have the same ObjectLocker issue. > > Perhaps: > // If we need to load any classes do execute doSleep(), do it now. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Thu Nov 12 08:04:55 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 12 Nov 2020 08:04:55 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: <_-GUMtFBAMIFb1RQkHIj9GNafYZpSX-4w_IqKkLWdng=.225b5639-4baa-4e54-9a24-d4dfcd5d286f@github.com> References: <_-GUMtFBAMIFb1RQkHIj9GNafYZpSX-4w_IqKkLWdng=.225b5639-4baa-4e54-9a24-d4dfcd5d286f@github.com> Message-ID: On Wed, 11 Nov 2020 22:24:11 GMT, Patricio Chilano Mateo wrote: > LGTM. Thanks for finding the issue! > > Patricio Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Thu Nov 12 08:04:56 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 12 Nov 2020 08:04:56 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: <_-GUMtFBAMIFb1RQkHIj9GNafYZpSX-4w_IqKkLWdng=.225b5639-4baa-4e54-9a24-d4dfcd5d286f@github.com> Message-ID: On Thu, 12 Nov 2020 08:00:32 GMT, Robbin Ehn wrote: >> LGTM. Thanks for finding the issue! >> >> Patricio > >> LGTM. Thanks for finding the issue! >> >> Patricio > > Thanks! > Typo: do execute -> to execute Thanks, fixed! ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Thu Nov 12 08:04:57 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 12 Nov 2020 08:04:57 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: <6GNJmL3G8relvmQZUk9Higx4IH81e-h58XcDxf5HPSs=.45f93631-2ef3-407a-9be7-43798bc2a9e7@github.com> On Wed, 11 Nov 2020 22:48:38 GMT, Daniel D. Daugherty wrote: >> As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. >> Resulting in an unexpected result to GetCurrentContendedMonitor(). >> Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. >> Instead I here just use sleep + volatile for the barriers. >> >> I cannot reproduce with these changes. >> >> Chewing through T6 as most issues have been seen there. > > test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 79: > >> 77: contmon001a thr = new contmon001a(); >> 78: startingBarrier = new Wicket(); >> 79: waitingBarrier = new Wicket(); > > If you want to continue to use Wicket, then "all" you have to do is > execute one back-and-forth of the Wicket before executing the > test portion itself. That should get all the Wicket classes loaded > before you really need them. Unless, of course, there's some class > that isn't loaded unless there's an error or something... > > However, by using volatile booleans instead, you get out of the > game of trying to figure out how much of the Wicket code needs > to be exercised to get everything you might need loaded. We may load different classes depending on far in method waitFor we have reach before the unlock is called, on both sides. So unlocked or waitFor may see a new class deeding on if are we added to queue, have we called the parker, etc.. Even if that is easy to figure out, if someone changes the Wicket or e.g. AbstractQueuedSynchronizer, the test may fail again. ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Thu Nov 12 08:11:10 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 12 Nov 2020 08:11:10 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: References: Message-ID: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> > As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. > Resulting in an unexpected result to GetCurrentContendedMonitor(). > Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. > Instead I here just use sleep + volatile for the barriers. > > I cannot reproduce with these changes. > > Chewing through T6 as most issues have been seen there - passed. Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: Fixed comment ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1177/files - new: https://git.openjdk.java.net/jdk/pull/1177/files/8103a641..32879e2d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1177&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1177&range=00-01 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1177.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1177/head:pull/1177 PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Thu Nov 12 08:11:11 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 12 Nov 2020 08:11:11 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 22:55:16 GMT, Daniel D. Daugherty wrote: > I like the fix. Taking Wicket out of play will help with class loading > issues in this test. You don't really need to deal with unexpected > locking events from the ObjectLocker usage by class loading. Great, that was my feeling also. @sspitsyn What do you think about this ? Thanks, Robbin ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From tschatzl at openjdk.java.net Thu Nov 12 09:06:06 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 12 Nov 2020 09:06:06 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 11:39:59 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? > > Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. > > With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. > > This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. > > Testing: tier1-5, one or two 6-8 runs > > The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements > > Thanks, > Thomas Change the handling of Open Archive areas, instead of assuming that everything in there is live always, a root containing references to all live root objects is provided. Adapt G1 to handle Open Archive regions as any other old region apart from never compacting or evacuating them. ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From tschatzl at openjdk.java.net Thu Nov 12 09:06:05 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 12 Nov 2020 09:06:05 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions Message-ID: Hi all, can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. Testing: tier1-5, one or two 6-8 runs The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements Thanks, Thomas ------------- Commit messages: - Initial import Changes: https://git.openjdk.java.net/jdk/pull/1163/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1163&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8253081 Stats: 657 lines in 32 files changed: 467 ins; 83 del; 107 mod Patch: https://git.openjdk.java.net/jdk/pull/1163.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1163/head:pull/1163 PR: https://git.openjdk.java.net/jdk/pull/1163 From iklam at openjdk.java.net Thu Nov 12 09:06:06 2020 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 12 Nov 2020 09:06:06 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 12:31:52 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? >> >> Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. >> >> With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. >> >> This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. >> >> Testing: tier1-5, one or two 6-8 runs >> >> The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements >> >> Thanks, >> Thomas > > Change the handling of Open Archive areas, instead of assuming that everything in there is live always, a root containing references to all live root objects is provided. Adapt G1 to handle Open Archive regions as any other old region apart from never compacting or evacuating them. The CDS changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From hseigel at openjdk.java.net Thu Nov 12 13:26:54 2020 From: hseigel at openjdk.java.net (Harold Seigel) Date: Thu, 12 Nov 2020 13:26:54 GMT Subject: Integrated: 8255787: Tag container tests that use cGroups with cgroups keyword In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 21:24:25 GMT, Harold Seigel wrote: > Please review this small change to add a cgroups keyword to tests that use cgroups. The fix was tested by running Mach5 container tests. This pull request has now been integrated. Changeset: 4df8abc2 Author: Harold Seigel URL: https://git.openjdk.java.net/jdk/commit/4df8abc2 Stats: 20 lines in 15 files changed: 15 ins; 0 del; 5 mod 8255787: Tag container tests that use cGroups with cgroups keyword Reviewed-by: sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/1148 From hseigel at openjdk.java.net Thu Nov 12 13:30:57 2020 From: hseigel at openjdk.java.net (Harold Seigel) Date: Thu, 12 Nov 2020 13:30:57 GMT Subject: RFR: 8255787: Tag container tests that use cGroups with cgroups keyword In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 00:27:59 GMT, Serguei Spitsyn wrote: >> Please review this small change to add a cgroups keyword to tests that use cgroups. The fix was tested by running Mach5 container tests. > > Hi Harold, > > The fix looks good. > > Thanks, > Serguei Thanks Serguei! Harold ------------- PR: https://git.openjdk.java.net/jdk/pull/1148 From tschatzl at openjdk.java.net Thu Nov 12 14:09:00 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 12 Nov 2020 14:09:00 GMT Subject: RFR: 8256181: Remove Allocation of old generation on alternate memory devices functionality [v2] In-Reply-To: <0nEbOq5anud0koNcq_5EV57wUd8wc8qHUCjI1GP2L0I=.ae3defc8-6c6b-40dd-894b-e5a4dde4d4bb@github.com> References: <0nEbOq5anud0koNcq_5EV57wUd8wc8qHUCjI1GP2L0I=.ae3defc8-6c6b-40dd-894b-e5a4dde4d4bb@github.com> Message-ID: On Wed, 11 Nov 2020 15:19:05 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review > > A general comment for future PRs: I think it's best to isolate mechanical changes into their own commits; e.g. `HeapRegionManager* _hrm;` -> `HeapRegionManager _hrm;`. Otherwise, a real change, buried in the immense size of diff, might slip through. Thanks @albertnetymk @iklam @iignatev for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/1162 From tschatzl at openjdk.java.net Thu Nov 12 14:09:01 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 12 Nov 2020 14:09:01 GMT Subject: Integrated: 8256181: Remove Allocation of old generation on alternate memory devices functionality In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 11:11:25 GMT, Thomas Schatzl wrote: > Hi all, > > can I get reviews for this change that removes the "Allocation of old generation of Java heap on alternate memory devices" functionality introduced with JDK 12 with [JDK-8202286](https://bugs.openjdk.java.net/browse/JDK-8202286) due to being > > - not used by anyone > - not maintained by anyone, i.e. several bugs open for a long time and bit rotting > - requiring some workarounds for new feature development wrt to heap management > > All flags covered by this feature were experimental flags, so there are no additional procedural issues to take. > > I tried to remove all but a few minor cleanups that I thought useful, but of course this is very subjective. > > Testing: hs-tier1-5 This pull request has now been integrated. Changeset: bd8693a0 Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/bd8693a0 Stats: 2315 lines in 50 files changed: 4 ins; 2183 del; 128 mod 8256181: Remove Allocation of old generation on alternate memory devices functionality Reviewed-by: ayang, iignatyev, iklam ------------- PR: https://git.openjdk.java.net/jdk/pull/1162 From coleenp at openjdk.java.net Thu Nov 12 16:23:23 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 12 Nov 2020 16:23:23 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v8] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Add logging to event posting in case of pauses. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/6dec83d8..0487b84c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=06-07 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From dcubed at openjdk.java.net Thu Nov 12 20:45:03 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 12 Nov 2020 20:45:03 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> Message-ID: On Thu, 12 Nov 2020 08:11:10 GMT, Robbin Ehn wrote: >> As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. >> Resulting in an unexpected result to GetCurrentContendedMonitor(). >> Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. >> Instead I here just use sleep + volatile for the barriers. >> >> I cannot reproduce with these changes. >> >> Chewing through T6 as most issues have been seen there - passed. > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed comment Still good. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1177 From dcubed at openjdk.java.net Thu Nov 12 21:06:01 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 12 Nov 2020 21:06:01 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> Message-ID: On Tue, 10 Nov 2020 21:16:20 GMT, Robbin Ehn wrote: >> Sorry my preference is for Monitors instead of semaphores. Let's >> take that discussion off this PR and you can explain why you dislike >> the Monitor so much and think the local semaphore is the way to go. > > Yes Filed the following new RFE: JDK-8256241 replace MonitorDeflation_lock with a semaphore https://bugs.openjdk.java.net/browse/JDK-8256241 ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Thu Nov 12 21:14:07 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 12 Nov 2020 21:14:07 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: <7rQQpVtTPecL-Vt4JJ0CR3U_1523KksQwFAsn9QpJK8=.5e872ffe-d1e4-40e9-9dd4-595712452e5c@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> <6na55oAsYSTSML28qV7bzizCY1BqkmpkNsc1eHEZZ4U=.c58c86f2-b090-4c42-a73e-f36ff2572963@github.com> <7rQQpVtTPecL-Vt4JJ0CR3U_1523KksQwFAsn9QpJK8=.5e872ffe-d1e4-40e9-9dd4-595712452e5c@github.com> Message-ID: On Tue, 10 Nov 2020 21:37:40 GMT, Daniel D. Daugherty wrote: >> If you only need to free CHeap memory, you can do: >> size_t deleted_count = 0; >> ThreadBlockInVM tbivm(self); >> for (ObjectMonitor* monitor: delete_list) { >> delete monitor; >> deleted_count++; >> } >> } > > Ahhh... but that only works if we release the oopStorage when > we deflate. Okay. I grok it now, but don't want to do that in this > changeset. I would want a complete stress test cycle for that > kind of a change. Filed this new RFE: JDK-8256302 releasing oopStorage when deflating allows for faster deleting https://bugs.openjdk.java.net/browse/JDK-8256302 ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Thu Nov 12 21:20:00 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 12 Nov 2020 21:20:00 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v5] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <05PVdNFn9-u7KHSDPXNSr62kvfFQHKCoyT6Z1TIAC2s=.40e0c36c-e7be-48e4-8352-dbc5ebae25b0@github.com> Message-ID: On Tue, 10 Nov 2020 20:57:00 GMT, Robbin Ehn wrote: >> We've removed enough padding with this work already. If we >> want to do more padding removal, then we need to use a >> different RFE. > > Sure, this was more a FYI. Filed this new RFE: JDK-8256303 revisit ObjectMonitor padding between fields https://bugs.openjdk.java.net/browse/JDK-8256303 ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Thu Nov 12 21:25:05 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 12 Nov 2020 21:25:05 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> Message-ID: On Tue, 10 Nov 2020 21:08:53 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/globals.hpp line 750: >> >>> 748: product(intx, MonitorUsedDeflationThreshold, 90, EXPERIMENTAL, \ >>> 749: "Percentage of used monitors before triggering deflation (0 is " \ >>> 750: "off). The check is performed on GuaranteedSafepointInterval " \ >> >> Should there still be experimental options after this change? > > Robbin added MonitorUsedDeflationThreshold as an experimental > option back in JDK10. See the longer reply to David's comment. > I don't plan to change that option with this changeset. Filed the following new RFE: JDK-8256304 should MonitorUsedDeflationThreshold be experimental or diagnostic https://bugs.openjdk.java.net/browse/JDK-8256304 >> src/hotspot/share/runtime/objectMonitor.cpp line 509: >> >>> 507: // >>> 508: bool ObjectMonitor::deflate_monitor() { >>> 509: if (is_busy()) { >> >> is_busy should be checked != 0 since it doesn't return a bool. > > Nice catch! That has been there for many, many years... Filed the following new RFE: JDK-8256301 ObjectMonitor::is_busy() should return bool https://bugs.openjdk.java.net/browse/JDK-8256301 ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Thu Nov 12 21:47:06 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 12 Nov 2020 21:47:06 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v6] In-Reply-To: <4xLwmW1Wya1aTmUbGPONAA7V-ScyRsdUK467gNvdmCQ=.5c2329ea-4b45-4732-8abd-e94d1f89ad5f@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <4oqXqtvTejqhnmPrBcVFQ_2y8F6rTuYmgRmaaV2kk0U=.a260df3f-a1b4-4efa-b3eb-7e5558e84327@github.com> <4xLwmW1Wya1aTmUbGPONAA7V-ScyRsdUK467gNvdmCQ=.5c2329ea-4b45-4732-8abd-e94d1f89ad5f@github.com> Message-ID: On Wed, 11 Nov 2020 16:35:50 GMT, Daniel D. Daugherty wrote: >> I typically use size_t for entities that can scale with the size of the machine's memory, so I don't have to think about whether there are enough bits. Could AvgMonitorsPerThreadEstimate be uintx instead of intx? And then maybe we don't need to declare a range, as the natural range of the uintx seems perfectly valid. > > I'm pretty sure I copied the decl for AvgMonitorsPerThreadEstimate > from some other already existing option. That's SOP for me anyway... > If we make any more changes here it will have to be in a follow up. Filed the following new RFE: JDK-8256307 cleanup AvgMonitorsPerThreadEstimate and _in_use_list_ceiling types https://bugs.openjdk.java.net/browse/JDK-8256307 ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dcubed at openjdk.java.net Thu Nov 12 21:47:05 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 12 Nov 2020 21:47:05 GMT Subject: RFR: 8253064: monitor list simplifications and getting rid of TSM [v4] In-Reply-To: <_91UeV37-LcU43acy9VUD7rt3woXrc-jdz-ILC7ichs=.57f26cbb-1ac0-4c9e-ba2e-9b620d60b2c9@github.com> References: <1ejVYakgL_jBrD0qpBMBNGJE3E2sLxKvt0ccOs7aKiA=.eadc65bc-3964-4917-b2c2-d18cf4925ac3@github.com> <3hoc3iLbt_d7hTF-3GIbPTSE7QaJhapR5EjsCH8nRkA=.996c5e96-f760-489c-93ae-b6631dc9446d@github.com> <_8BktJ0NHE_i7LQ7_X8JQyyfriKl8_-PA7hOfdBrC3U=.ac425b59-0e89-47f2-9fa7-e461e55689a2@github.com> <_91UeV37-LcU43acy9VUD7rt3woXrc-jdz-ILC7ichs=.57f26cbb-1ac0-4c9e-ba2e-9b620d60b2c9@github.com> Message-ID: On Tue, 10 Nov 2020 22:28:12 GMT, Coleen Phillimore wrote: >> It can be a future RFE, but it won't be at the top of my list of >> things to do. There may already be an RFE for that. > > No, I assume it's not high priority. I'll file an RFE because someday I want these to be cleaned up as a personal nit. Filed the following new RFE: JDK-8256306 ObjectMonitor::_contentions field should not be 'jint' https://bugs.openjdk.java.net/browse/JDK-8256306 ------------- PR: https://git.openjdk.java.net/jdk/pull/642 From dholmes at openjdk.java.net Fri Nov 13 02:34:56 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Fri, 13 Nov 2020 02:34:56 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> Message-ID: On Thu, 12 Nov 2020 20:42:39 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed comment > > Still good. I have updated the bug report. I am not convinced the underlying issue has been identified. ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From sspitsyn at openjdk.java.net Fri Nov 13 05:30:59 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 13 Nov 2020 05:30:59 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> Message-ID: <1T0ATf4jsgUOfg6nKvleLCuOAq9oufjNV6DelCfmGnc=.5148edeb-c4a0-47ea-9804-c553a6ed7d7c@github.com> On Fri, 13 Nov 2020 02:32:14 GMT, David Holmes wrote: >> Still good. > > I have updated the bug report. I am not convinced the underlying issue has been identified. > @sspitsyn What do you think about this ? Hi Robbin, I'm not sure, a conclusion about the failure mode and root cause is correct. Please, see my comment in the bug report. At least, one of the failure modes does not match the explanation and fix. In fact, I'm puzzled with this issue and how these failures could happen. Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From shade at openjdk.java.net Fri Nov 13 08:23:03 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 08:23:03 GMT Subject: Integrated: 8253525: Implement getInstanceSize/sizeOf intrinsics In-Reply-To: References: Message-ID: On Wed, 14 Oct 2020 10:11:23 GMT, Aleksey Shipilev wrote: > This is fork off the SizeOf JEP, JDK-8249196. There is already the entry point in JDK that can use the intrinsic like this: `Instrumentation.getInstanceSize`. Therefore, we can implement the C1/C2 intrinsic now, hook it up to `Instrumentation`, and let the tools use that fast path today. > > With this patch, JOL is able to be close to `deepSizeOf` implementation from SizeOf JEP. > > Example performance improvements for sizing up a custom linked list: > > Benchmark (size) Mode Cnt Score Error Units > > # Default > LinkedChainBench.linkedChain 1 avgt 5 705.835 ? 8.051 ns/op > LinkedChainBench.linkedChain 10 avgt 5 3148.874 ? 37.856 ns/op > LinkedChainBench.linkedChain 100 avgt 5 28693.256 ? 142.254 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 290161.590 ? 4594.631 ns/op > > # Instrumentation attached, no intrinsics > LinkedChainBench.linkedChain 1 avgt 5 159.659 ? 19.238 ns/op > LinkedChainBench.linkedChain 10 avgt 5 717.659 ? 22.540 ns/op > LinkedChainBench.linkedChain 100 avgt 5 7739.394 ? 111.683 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 80724.238 ? 2887.794 ns/op > > # Instrumentation attached, new intrinsics > LinkedChainBench.linkedChain 1 avgt 5 95.254 ? 0.808 ns/op > LinkedChainBench.linkedChain 10 avgt 5 261.564 ? 8.524 ns/op > LinkedChainBench.linkedChain 100 avgt 5 3367.192 ? 21.128 ns/op > LinkedChainBench.linkedChain 1000 avgt 5 34148.851 ? 373.080 ns/op This pull request has now been integrated. Changeset: b4d01867 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/b4d01867 Stats: 655 lines in 12 files changed: 655 ins; 0 del; 0 mod 8253525: Implement getInstanceSize/sizeOf intrinsics Reviewed-by: kvn, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/650 From shade at openjdk.java.net Fri Nov 13 08:23:01 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 08:23:01 GMT Subject: RFR: 8253525: Implement getInstanceSize/sizeOf intrinsics [v5] In-Reply-To: References: Message-ID: On Wed, 21 Oct 2020 17:33:27 GMT, Vladimir Kozlov wrote: >> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: >> >> - The new intrinsic-related test >> - Revert the change to test >> - Merge branch 'master' into JDK-8253525-sizeof-intrinsics >> - Add new intrinsics to toBeInvestigated list in CheckGraalIntrinsics.java >> - 8253525: Implement getInstanceSize/sizeOf intrinsics > > Good. Thanks @vnkozlov and @sspitsyn! ------------- PR: https://git.openjdk.java.net/jdk/pull/650 From stefank at openjdk.java.net Fri Nov 13 14:18:12 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Fri, 13 Nov 2020 14:18:12 GMT Subject: RFR: 8256337: ap01t001.cpp, 67: Received unexpected number of ObjectFree events: 7 Message-ID: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> The ap01t001 test creates six extra instances of the tested class, let them die, and then checks that it gets exactly six ObjectFree callbacks. The problem is that this is verified in the VMDeath callback and at that point the instance has gone out-of-scope and and a seventh ObjectFree event has been triggered. My proposed fix is to ensure that the test instance is kept alive. ------------- Commit messages: - 8256337: ap01t001.cpp, 67: Received unexpected number of ObjectFree events: 7 Changes: https://git.openjdk.java.net/jdk/pull/1204/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1204&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256337 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1204.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1204/head:pull/1204 PR: https://git.openjdk.java.net/jdk/pull/1204 From stefank at openjdk.java.net Fri Nov 13 14:18:12 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Fri, 13 Nov 2020 14:18:12 GMT Subject: RFR: 8256337: ap01t001.cpp, 67: Received unexpected number of ObjectFree events: 7 In-Reply-To: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> References: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> Message-ID: On Fri, 13 Nov 2020 14:11:23 GMT, Stefan Karlsson wrote: > The ap01t001 test creates six extra instances of the tested class, let them die, and then checks that it gets exactly six ObjectFree callbacks. The problem is that this is verified in the VMDeath callback and at that point the instance has gone out-of-scope and and a seventh ObjectFree event has been triggered. > > My proposed fix is to ensure that the test instance is kept alive. I've tested this by running the following reproducer that used to trigger this bug: `while true; do make -C ../build/fastdebug jdk test-only TEST=test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP01/ap01t001/TestDescription.java JTREG="JAVA_OPTIONS= -XX:+UseZGC -Xmx2g -XX:ZCollectionInterval=0.01"; done` ------------- PR: https://git.openjdk.java.net/jdk/pull/1204 From coleenp at openjdk.java.net Fri Nov 13 14:38:58 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 13 Nov 2020 14:38:58 GMT Subject: RFR: 8256337: ap01t001.cpp, 67: Received unexpected number of ObjectFree events: 7 In-Reply-To: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> References: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> Message-ID: <-EkGKsgKr_YhOvzonEuoZKQQKdakFxI8kMoQ75uwCWE=.637c895f-3496-42b7-ade8-4e9dd1f3524c@github.com> On Fri, 13 Nov 2020 14:11:23 GMT, Stefan Karlsson wrote: > The ap01t001 test creates six extra instances of the tested class, let them die, and then checks that it gets exactly six ObjectFree callbacks. The problem is that this is verified in the VMDeath callback and at that point the instance has gone out-of-scope and and a seventh ObjectFree event has been triggered. > > My proposed fix is to ensure that the test instance is kept alive. Looks good and trivial. ------------- Marked as reviewed by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1204 From rehn at openjdk.java.net Fri Nov 13 17:04:04 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Fri, 13 Nov 2020 17:04:04 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: <1T0ATf4jsgUOfg6nKvleLCuOAq9oufjNV6DelCfmGnc=.5148edeb-c4a0-47ea-9804-c553a6ed7d7c@github.com> References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> <1T0ATf4jsgUOfg6nKvleLCuOAq9oufjNV6DelCfmGnc=.5148edeb-c4a0-47ea-9804-c553a6ed7d7c@github.com> Message-ID: On Fri, 13 Nov 2020 05:28:40 GMT, Serguei Spitsyn wrote: >> I have updated the bug report. I am not convinced the underlying issue has been identified. > >> @sspitsyn What do you think about this ? > > Hi Robbin, > I'm not sure, a conclusion about the failure mode and root cause is correct. > Please, see my comment in the bug report. > At least, one of the failure modes does not match the explanation and fix. > In fact, I'm puzzled with this issue and how these failures could happen. > Thanks, > Serguei I updated the bug, with the info. We cannot call code which 'randomly' load classes :) ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From sspitsyn at openjdk.java.net Fri Nov 13 19:29:00 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 13 Nov 2020 19:29:00 GMT Subject: RFR: 8256337: ap01t001.cpp, 67: Received unexpected number of ObjectFree events: 7 In-Reply-To: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> References: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> Message-ID: On Fri, 13 Nov 2020 14:11:23 GMT, Stefan Karlsson wrote: > The ap01t001 test creates six extra instances of the tested class, let them die, and then checks that it gets exactly six ObjectFree callbacks. The problem is that this is verified in the VMDeath callback and at that point the instance has gone out-of-scope and and a seventh ObjectFree event has been triggered. > > My proposed fix is to ensure that the test instance is kept alive. Hi Stefan, It looks good to me. Thank you for fixing it! Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1204 From github.com+1926185+robberphex at openjdk.java.net Sat Nov 14 16:06:01 2020 From: github.com+1926185+robberphex at openjdk.java.net (Robert LU) Date: Sat, 14 Nov 2020 16:06:01 GMT Subject: RFR: 8253280: Use class name as class loading lock Message-ID: <5DnzvOcYJwLegT48SxwqBUqvg8HnihAgaqUpdraS3CU=.516c26fb-e88f-4b7f-9090-26f770e54f05@github.com> When many thread try to load same class, the thread will stuck on `ClassLoader.loadClass`. At current jdk, the stacktrace by example program is: "Thread-1" prio=5 Id=13 BLOCKED on java.lang.String at 724af044 owned by "Thread-0" Id=12 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:646) - blocked on java.lang.String at 724af044 val="java.lang.String" at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:634) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) at app//Main2.test(Main2.java:19) at app//Main2$$Lambda$37/0x00000001000c2a20.run(Unknown Source) at java.base/java.lang.Thread.run(Thread.java:831) There is no way to get which class stuck the thread. **After this patch, the stacktrace will be**: "Thread-2" prio=5 Id=13 BLOCKED on java.lang.String at 724af044 owned by "Thread-3" Id=14 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:612) - blocked on java.lang.String at 724af044 val="java.lang.String" at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:600) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) at app//Main2.test(Main2.java:18) at app//Main2$$Lambda$38/0x0000000100097440.run(Unknown Source) at java.base/java.lang.Thread.run(Thread.java:832) That is, user will know which class stuck the thread, in this example, the class is `java.lang.String`. It's helpful for troubleshooting. The example program: Before patch:
Main.java // Main.java import java.io.PrintStream; import java.lang.management.*; public final class Main { private synchronized static void test1() { while (true) { try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } } private static void test() { while (true) { try { Main.class.getClassLoader().loadClass("java.lang.String"); } catch (ClassNotFoundException e) { } } } public static void main(String[] args) throws InterruptedException, NoSuchFieldException, IllegalAccessException { new Thread(Main::test).start(); new Thread(Main::test).start(); new Thread(Main::test).start(); new Thread(Main::test).start(); new Thread(Main::test).start(); while (true) { Thread.sleep(1000); ThreadMXBean bean = ManagementFactory.getThreadMXBean(); ThreadInfo[] infos = bean.dumpAllThreads(true, true); for (ThreadInfo info : infos) { System.out.println(printThreadInfo(info)); } } } private static String printThreadInfo(ThreadInfo threadInfo) { StringBuilder sb = new StringBuilder(""" + threadInfo.getThreadName() + """ + (threadInfo.isDaemon() ? " daemon" : "") + " prio=" + threadInfo.getPriority() + " Id=" + threadInfo.getThreadId() + " " + threadInfo.getThreadState()); if (threadInfo.getLockName() != null) { sb.append(" on " + threadInfo.getLockName()); } if (threadInfo.getLockOwnerName() != null) { sb.append(" owned by "" + threadInfo.getLockOwnerName() + "" Id=" + threadInfo.getLockOwnerId()); } if (threadInfo.isSuspended()) { sb.append(" (suspended)"); } if (threadInfo.isInNative()) { sb.append(" (in native)"); } sb.append('\n'); int i = 0; StackTraceElement[] stackTrace = threadInfo.getStackTrace(); for (; i < stackTrace.length; i++) { StackTraceElement ste = stackTrace[i]; sb.append("\tat " + ste.toString()); sb.append('\n'); if (i == 0 && threadInfo.getLockInfo() != null) { Thread.State ts = threadInfo.getThreadState(); switch (ts) { case BLOCKED: sb.append("\t- blocked on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; case WAITING: sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; case TIMED_WAITING: sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; default: } } for (MonitorInfo mi : threadInfo.getLockedMonitors()) { if (mi.getLockedStackDepth() == i) { sb.append("\t- locked " + printLockInfo(mi)); sb.append('\n'); } } } if (i < stackTrace.length) { sb.append("\t..."); sb.append('\n'); } LockInfo[] locks = threadInfo.getLockedSynchronizers(); if (locks.length > 0) { sb.append("\n\tNumber of locked synchronizers = " + locks.length); sb.append('\n'); for (LockInfo li : locks) { sb.append("\t- " + printLockInfo(li)); sb.append('\n'); } } sb.append('\n'); return sb.toString(); } private static String printLockInfo(LockInfo li) { String res = li.getClassName() + '@' + Integer.toHexString(li.getIdentityHashCode()); return res; } }
After patch:
Main2.java // Main2.java import java.io.PrintStream; import java.lang.management.*; public final class Main2 { private synchronized static void test1() { while (true) { try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } } private static void test() { while (true) { try { Main2.class.getClassLoader().loadClass("java.lang.String"); } catch (ClassNotFoundException e) { } } } public static void main(String[] args) throws InterruptedException, NoSuchFieldException, IllegalAccessException { new Thread(Main2::test).start(); new Thread(Main2::test).start(); new Thread(Main2::test).start(); new Thread(Main2::test).start(); new Thread(Main2::test).start(); while (true) { Thread.sleep(1000); ThreadMXBean bean = ManagementFactory.getThreadMXBean(); ThreadInfo[] infos = bean.dumpAllThreads(true, true); for (ThreadInfo info : infos) { System.out.println(printThreadInfo(info)); } } } private static String printThreadInfo(ThreadInfo threadInfo) { StringBuilder sb = new StringBuilder(""" + threadInfo.getThreadName() + """ + (threadInfo.isDaemon() ? " daemon" : "") + " prio=" + threadInfo.getPriority() + " Id=" + threadInfo.getThreadId() + " " + threadInfo.getThreadState()); if (threadInfo.getLockName() != null) { sb.append(" on " + threadInfo.getLockName()); } if (threadInfo.getLockOwnerName() != null) { sb.append(" owned by "" + threadInfo.getLockOwnerName() + "" Id=" + threadInfo.getLockOwnerId()); } if (threadInfo.isSuspended()) { sb.append(" (suspended)"); } if (threadInfo.isInNative()) { sb.append(" (in native)"); } sb.append('\n'); int i = 0; StackTraceElement[] stackTrace = threadInfo.getStackTrace(); for (; i < stackTrace.length; i++) { StackTraceElement ste = stackTrace[i]; sb.append("\tat " + ste.toString()); sb.append('\n'); if (i == 0 && threadInfo.getLockInfo() != null) { Thread.State ts = threadInfo.getThreadState(); switch (ts) { case BLOCKED: sb.append("\t- blocked on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; case WAITING: sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; case TIMED_WAITING: sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; default: } } for (MonitorInfo mi : threadInfo.getLockedMonitors()) { if (mi.getLockedStackDepth() == i) { sb.append("\t- locked " + printLockInfo(mi)); sb.append('\n'); } } } if (i < stackTrace.length) { sb.append("\t..."); sb.append('\n'); } LockInfo[] locks = threadInfo.getLockedSynchronizers(); if (locks.length > 0) { sb.append("\n\tNumber of locked synchronizers = " + locks.length); sb.append('\n'); for (LockInfo li : locks) { sb.append("\t- " + printLockInfo(li)); sb.append('\n'); } } sb.append('\n'); return sb.toString(); } private static String printLockInfo(LockInfo li) { String res = li.getClassName() + '@' + Integer.toHexString(li.getIdentityHashCode()); // There is no getLock method in current jdk if (li.getStringValue() != null) { return res + " val="" + li.getStringValue() + """; } return res; } }
------------- Commit messages: - 8253280: Use class name as class loading lock Changes: https://git.openjdk.java.net/jdk/pull/104/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=104&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8253280 Stats: 73 lines in 5 files changed: 72 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/104.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/104/head:pull/104 PR: https://git.openjdk.java.net/jdk/pull/104 From mchung at openjdk.java.net Sat Nov 14 16:06:02 2020 From: mchung at openjdk.java.net (Mandy Chung) Date: Sat, 14 Nov 2020 16:06:02 GMT Subject: RFR: 8253280: Use class name as class loading lock In-Reply-To: <5DnzvOcYJwLegT48SxwqBUqvg8HnihAgaqUpdraS3CU=.516c26fb-e88f-4b7f-9090-26f770e54f05@github.com> References: <5DnzvOcYJwLegT48SxwqBUqvg8HnihAgaqUpdraS3CU=.516c26fb-e88f-4b7f-9090-26f770e54f05@github.com> Message-ID: On Thu, 10 Sep 2020 05:25:43 GMT, Robert LU wrote: > When many thread try to load same class, the thread will stuck on `ClassLoader.loadClass`. > At current jdk, the stacktrace by example program is: > "Thread-1" prio=5 Id=13 BLOCKED on java.lang.String at 724af044 owned by "Thread-0" Id=12 > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:646) > - blocked on java.lang.String at 724af044 val="java.lang.String" > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:634) > at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) > at app//Main2.test(Main2.java:19) > at app//Main2$$Lambda$37/0x00000001000c2a20.run(Unknown Source) > at java.base/java.lang.Thread.run(Thread.java:831) > There is no way to get which class stuck the thread. > > **After this patch, the stacktrace will be**: > "Thread-2" prio=5 Id=13 BLOCKED on java.lang.String at 724af044 owned by "Thread-3" Id=14 > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:612) > - blocked on java.lang.String at 724af044 val="java.lang.String" > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:600) > at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) > at app//Main2.test(Main2.java:18) > at app//Main2$$Lambda$38/0x0000000100097440.run(Unknown Source) > at java.base/java.lang.Thread.run(Thread.java:832) > That is, user will know which class stuck the thread, in this example, the class is `java.lang.String`. It's helpful for troubleshooting. > > The example program: > Before patch: >
> Main.java > > // Main.java > import java.io.PrintStream; > import java.lang.management.*; > > public final class Main { > private synchronized static void test1() { > while (true) { > try { > Thread.sleep(1000); > } catch (InterruptedException e) { > e.printStackTrace(); > } > } > } > > private static void test() { > while (true) { > try { > Main.class.getClassLoader().loadClass("java.lang.String"); > } catch (ClassNotFoundException e) { > } > } > } > > public static void main(String[] args) throws InterruptedException, NoSuchFieldException, IllegalAccessException { > new Thread(Main::test).start(); > new Thread(Main::test).start(); > new Thread(Main::test).start(); > new Thread(Main::test).start(); > new Thread(Main::test).start(); > > while (true) { > Thread.sleep(1000); > ThreadMXBean bean = ManagementFactory.getThreadMXBean(); > ThreadInfo[] infos = bean.dumpAllThreads(true, true); > for (ThreadInfo info : infos) { > System.out.println(printThreadInfo(info)); > } > } > } > > private static String printThreadInfo(ThreadInfo threadInfo) { > StringBuilder sb = new StringBuilder(""" + threadInfo.getThreadName() + """ + > (threadInfo.isDaemon() ? " daemon" : "") + > " prio=" + threadInfo.getPriority() + > " Id=" + threadInfo.getThreadId() + " " + > threadInfo.getThreadState()); > if (threadInfo.getLockName() != null) { > sb.append(" on " + threadInfo.getLockName()); > } > if (threadInfo.getLockOwnerName() != null) { > sb.append(" owned by "" + threadInfo.getLockOwnerName() + > "" Id=" + threadInfo.getLockOwnerId()); > } > if (threadInfo.isSuspended()) { > sb.append(" (suspended)"); > } > if (threadInfo.isInNative()) { > sb.append(" (in native)"); > } > sb.append('\n'); > int i = 0; > StackTraceElement[] stackTrace = threadInfo.getStackTrace(); > for (; i < stackTrace.length; i++) { > StackTraceElement ste = stackTrace[i]; > sb.append("\tat " + ste.toString()); > sb.append('\n'); > if (i == 0 && threadInfo.getLockInfo() != null) { > Thread.State ts = threadInfo.getThreadState(); > switch (ts) { > case BLOCKED: > sb.append("\t- blocked on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case WAITING: > sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case TIMED_WAITING: > sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > default: > } > } > > for (MonitorInfo mi : threadInfo.getLockedMonitors()) { > if (mi.getLockedStackDepth() == i) { > sb.append("\t- locked " + printLockInfo(mi)); > sb.append('\n'); > } > } > } > if (i < stackTrace.length) { > sb.append("\t..."); > sb.append('\n'); > } > > LockInfo[] locks = threadInfo.getLockedSynchronizers(); > if (locks.length > 0) { > sb.append("\n\tNumber of locked synchronizers = " + locks.length); > sb.append('\n'); > for (LockInfo li : locks) { > sb.append("\t- " + printLockInfo(li)); > sb.append('\n'); > } > } > sb.append('\n'); > return sb.toString(); > } > > private static String printLockInfo(LockInfo li) { > String res = li.getClassName() + '@' + Integer.toHexString(li.getIdentityHashCode()); > return res; > } > } >
> After patch: >
> Main2.java > > // Main2.java > import java.io.PrintStream; > import java.lang.management.*; > > public final class Main2 { > private synchronized static void test1() { > while (true) { > try { > Thread.sleep(1000); > } catch (InterruptedException e) { > e.printStackTrace(); > } > } > } > > private static void test() { > while (true) { > try { > Main2.class.getClassLoader().loadClass("java.lang.String"); > } catch (ClassNotFoundException e) { > } > } > } > > public static void main(String[] args) throws InterruptedException, NoSuchFieldException, IllegalAccessException { > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > > while (true) { > Thread.sleep(1000); > ThreadMXBean bean = ManagementFactory.getThreadMXBean(); > ThreadInfo[] infos = bean.dumpAllThreads(true, true); > for (ThreadInfo info : infos) { > System.out.println(printThreadInfo(info)); > } > } > } > > private static String printThreadInfo(ThreadInfo threadInfo) { > StringBuilder sb = new StringBuilder(""" + threadInfo.getThreadName() + """ + > (threadInfo.isDaemon() ? " daemon" : "") + > " prio=" + threadInfo.getPriority() + > " Id=" + threadInfo.getThreadId() + " " + > threadInfo.getThreadState()); > if (threadInfo.getLockName() != null) { > sb.append(" on " + threadInfo.getLockName()); > } > if (threadInfo.getLockOwnerName() != null) { > sb.append(" owned by "" + threadInfo.getLockOwnerName() + > "" Id=" + threadInfo.getLockOwnerId()); > } > if (threadInfo.isSuspended()) { > sb.append(" (suspended)"); > } > if (threadInfo.isInNative()) { > sb.append(" (in native)"); > } > sb.append('\n'); > int i = 0; > StackTraceElement[] stackTrace = threadInfo.getStackTrace(); > for (; i < stackTrace.length; i++) { > StackTraceElement ste = stackTrace[i]; > sb.append("\tat " + ste.toString()); > sb.append('\n'); > if (i == 0 && threadInfo.getLockInfo() != null) { > Thread.State ts = threadInfo.getThreadState(); > switch (ts) { > case BLOCKED: > sb.append("\t- blocked on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case WAITING: > sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case TIMED_WAITING: > sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > default: > } > } > > for (MonitorInfo mi : threadInfo.getLockedMonitors()) { > if (mi.getLockedStackDepth() == i) { > sb.append("\t- locked " + printLockInfo(mi)); > sb.append('\n'); > } > } > } > if (i < stackTrace.length) { > sb.append("\t..."); > sb.append('\n'); > } > > LockInfo[] locks = threadInfo.getLockedSynchronizers(); > if (locks.length > 0) { > sb.append("\n\tNumber of locked synchronizers = " + locks.length); > sb.append('\n'); > for (LockInfo li : locks) { > sb.append("\t- " + printLockInfo(li)); > sb.append('\n'); > } > } > sb.append('\n'); > return sb.toString(); > } > > private static String printLockInfo(LockInfo li) { > String res = li.getClassName() + '@' + Integer.toHexString(li.getIdentityHashCode()); > // There is no getLock method in current jdk > if (li.getStringValue() != null) { > return res + " val="" + li.getStringValue() + """; > } > return res; > } > } >
This patch proposes to add two public APIs `java.lang.management.LockInfo::getLock` and the new `java.lang.management.MonitorInfo` constructor taking the lock object. `java.lang.management.MonitorInfo` and `LockInfo` by design do not expose the lock object for remote monitoring reason (may not be serializable) and for security reason. The API instead provides the class name and identity hash code of the lock object serving. as an unique ID. I read JDK-8253280 as improving the diagnosability of ClassLoader-specific lock returned from `ClassLoader::getClassLoadingLock` for example improving the VM thread dump by control-break to special case the lock object for class loading operation. `LockInfo` is not specific for ClassLoader and so it does not seem the appropriate place to embed this information. For example the class loading lock is a new ClassLoader$Lock class and VM will perhaps call Lock::toString (or print a specific field) in the thread dump output. The attached program uses `java.lang.management.ThreadMXBean` to get a thread dump and prints the output. ------------- PR: https://git.openjdk.java.net/jdk/pull/104 From robberphex at gmail.com Sat Nov 14 16:26:01 2020 From: robberphex at gmail.com (Robert Lu) Date: Sun, 15 Nov 2020 00:26:01 +0800 Subject: RFR: 8253280: Use class name as class loading lock Message-ID: When many thread try to load same class, the thread will stuck on ClassLoader.loadClass. At current jdk, the stacktrace by example program is: "Thread-1" prio=5 Id=12 BLOCKED on java.lang.Object at 2e817b38 owned by "Thread-0" Id=11 at java.base at 15/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:616) - blocked on java.lang.Object at 2e817b38 at java.base at 15/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:604) at java.base at 15/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:168) at java.base at 15/java.lang.ClassLoader.loadClass(ClassLoader.java:522) at app//Main.test(Main.java:19) at app//Main$$Lambda$2/0x0000000800b8c468.run(Unknown Source) at java.base at 15/java.lang.Thread.run(Thread.java:832) There is no way to get which class stuck the thread. *After this patch, the stacktrace will be*: "Thread-1" prio=5 Id=13 BLOCKED on java.lang.String at 724af044 owned by "Thread-0" Id=12 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:646) - blocked on java.lang.String at 724af044 val="java.lang.String" at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:634) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) at app//Main2.test(Main2.java:19) at app//Main2$$Lambda$37/0x00000001000c2a20.run(Unknown Source) at java.base/java.lang.Thread.run(Thread.java:831) That is, user will know which class stuck the thread, in this example, the class is java.lang.String. It's helpful for troubleshooting. The example program: // Main2.javaimport java.io.PrintStream;import java.lang.management.*; public final class Main2 { private synchronized static void test1() { while (true) { try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } } private static void test() { while (true) { try { Main2.class.getClassLoader().loadClass("java.lang.String"); } catch (ClassNotFoundException e) { } } } public static void main(String[] args) throws InterruptedException, NoSuchFieldException, IllegalAccessException { new Thread(Main2::test).start(); new Thread(Main2::test).start(); new Thread(Main2::test).start(); new Thread(Main2::test).start(); new Thread(Main2::test).start(); while (true) { Thread.sleep(1000); ThreadMXBean bean = ManagementFactory.getThreadMXBean(); ThreadInfo[] infos = bean.dumpAllThreads(true, true); for (ThreadInfo info : infos) { System.out.println(printThreadInfo(info)); } } } private static String printThreadInfo(ThreadInfo threadInfo) { StringBuilder sb = new StringBuilder("\"" + threadInfo.getThreadName() + "\"" + (threadInfo.isDaemon() ? " daemon" : "") + " prio=" + threadInfo.getPriority() + " Id=" + threadInfo.getThreadId() + " " + threadInfo.getThreadState()); if (threadInfo.getLockName() != null) { sb.append(" on " + threadInfo.getLockName()); } if (threadInfo.getLockOwnerName() != null) { sb.append(" owned by \"" + threadInfo.getLockOwnerName() + "\" Id=" + threadInfo.getLockOwnerId()); } if (threadInfo.isSuspended()) { sb.append(" (suspended)"); } if (threadInfo.isInNative()) { sb.append(" (in native)"); } sb.append('\n'); int i = 0; StackTraceElement[] stackTrace = threadInfo.getStackTrace(); for (; i < stackTrace.length; i++) { StackTraceElement ste = stackTrace[i]; sb.append("\tat " + ste.toString()); sb.append('\n'); if (i == 0 && threadInfo.getLockInfo() != null) { Thread.State ts = threadInfo.getThreadState(); switch (ts) { case BLOCKED: sb.append("\t- blocked on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; case WAITING: sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; case TIMED_WAITING: sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); sb.append('\n'); break; default: } } for (MonitorInfo mi : threadInfo.getLockedMonitors()) { if (mi.getLockedStackDepth() == i) { sb.append("\t- locked " + printLockInfo(mi)); sb.append('\n'); } } } if (i < stackTrace.length) { sb.append("\t..."); sb.append('\n'); } LockInfo[] locks = threadInfo.getLockedSynchronizers(); if (locks.length > 0) { sb.append("\n\tNumber of locked synchronizers = " + locks.length); sb.append('\n'); for (LockInfo li : locks) { sb.append("\t- " + printLockInfo(li)); sb.append('\n'); } } sb.append('\n'); return sb.toString(); } private static String printLockInfo(LockInfo li) { String res = li.getClassName() + '@' + Integer.toHexString(li.getIdentityHashCode()); // There is no getLock method in current jdk if (li.getStringValue() != null) { return res + " val=\"" + li.getStringValue() + "\""; } return res; } } ---------- Commit messages: - 8253280: Use class name as class loading lock Changes: https://github.com/openjdk/jdk/pull/104/files Webrev: https://openjdk.github.io/cr/?repo=jdk&pr=104&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8253280 Stats: 74 lines changed; 73 ins; 1 del Patch: https://git.openjdk.java.net/jdk/pull/104.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/104/head:pull/104 PR: https://git.openjdk.java.net/jdk/pull/104 -- Robert Lu -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sun Nov 15 07:45:59 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 15 Nov 2020 07:45:59 +0000 Subject: RFR: 8253280: Use class name as class loading lock In-Reply-To: References: Message-ID: Robert, I think you need to start a discussion on serviceability-dev about the diagnostic challenges in this area before proposing API changes to java.lang.management that have wider implications and potential interop issues. It might be that your starting point is the thread dump instead. -Alan On 14/11/2020 16:26, Robert Lu wrote: > When many thread try to load same class, the thread will stuck on > ClassLoader.loadClass. > > At current jdk, the stacktrace by example program is: > > "Thread-1" prio=5 Id=12 BLOCKED on java.lang.Object at 2e817b38 owned by > "Thread-0" Id=11 > at java.base at 15/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:616) > - blocked on java.lang.Object at 2e817b38 > at java.base at 15/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:604) > at java.base at 15/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:168) > at java.base at 15/java.lang.ClassLoader.loadClass(ClassLoader.java:522) > at app//Main.test(Main.java:19) > at app//Main$$Lambda$2/0x0000000800b8c468.run(Unknown Source) > at java.base at 15/java.lang.Thread.run(Thread.java:832) > > There is no way to get which class stuck the thread. > > *After this patch, the stacktrace will be*: > > "Thread-1" prio=5 Id=13 BLOCKED on java.lang.String at 724af044 owned by > "Thread-0" Id=12 > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:646) > - blocked on java.lang.String at 724af044 val="java.lang.String" > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:634) > at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) > at app//Main2.test(Main2.java:19) > at app//Main2$$Lambda$37/0x00000001000c2a20.run(Unknown Source) > at java.base/java.lang.Thread.run(Thread.java:831) > > That is, user will know which class stuck the thread, in this example, the > class is java.lang.String. It's helpful for troubleshooting. > The example program: > > // Main2.javaimport java.io.PrintStream;import java.lang.management.*; > public final class Main2 { > private synchronized static void test1() { > while (true) { > try { > Thread.sleep(1000); > } catch (InterruptedException e) { > e.printStackTrace(); > } > } > } > > private static void test() { > while (true) { > try { > Main2.class.getClassLoader().loadClass("java.lang.String"); > } catch (ClassNotFoundException e) { > } > } > } > > public static void main(String[] args) throws > InterruptedException, NoSuchFieldException, IllegalAccessException { > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > > while (true) { > Thread.sleep(1000); > ThreadMXBean bean = ManagementFactory.getThreadMXBean(); > ThreadInfo[] infos = bean.dumpAllThreads(true, true); > for (ThreadInfo info : infos) { > System.out.println(printThreadInfo(info)); > } > } > } > > private static String printThreadInfo(ThreadInfo threadInfo) { > StringBuilder sb = new StringBuilder("\"" + > threadInfo.getThreadName() + "\"" + > (threadInfo.isDaemon() ? " daemon" : "") + > " prio=" + threadInfo.getPriority() + > " Id=" + threadInfo.getThreadId() + " " + > threadInfo.getThreadState()); > if (threadInfo.getLockName() != null) { > sb.append(" on " + threadInfo.getLockName()); > } > if (threadInfo.getLockOwnerName() != null) { > sb.append(" owned by \"" + threadInfo.getLockOwnerName() + > "\" Id=" + threadInfo.getLockOwnerId()); > } > if (threadInfo.isSuspended()) { > sb.append(" (suspended)"); > } > if (threadInfo.isInNative()) { > sb.append(" (in native)"); > } > sb.append('\n'); > int i = 0; > StackTraceElement[] stackTrace = threadInfo.getStackTrace(); > for (; i < stackTrace.length; i++) { > StackTraceElement ste = stackTrace[i]; > sb.append("\tat " + ste.toString()); > sb.append('\n'); > if (i == 0 && threadInfo.getLockInfo() != null) { > Thread.State ts = threadInfo.getThreadState(); > switch (ts) { > case BLOCKED: > sb.append("\t- blocked on " + > printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case WAITING: > sb.append("\t- waiting on " + > printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case TIMED_WAITING: > sb.append("\t- waiting on " + > printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > default: > } > } > > for (MonitorInfo mi : threadInfo.getLockedMonitors()) { > if (mi.getLockedStackDepth() == i) { > sb.append("\t- locked " + printLockInfo(mi)); > sb.append('\n'); > } > } > } > if (i < stackTrace.length) { > sb.append("\t..."); > sb.append('\n'); > } > > LockInfo[] locks = threadInfo.getLockedSynchronizers(); > if (locks.length > 0) { > sb.append("\n\tNumber of locked synchronizers = " + locks.length); > sb.append('\n'); > for (LockInfo li : locks) { > sb.append("\t- " + printLockInfo(li)); > sb.append('\n'); > } > } > sb.append('\n'); > return sb.toString(); > } > > private static String printLockInfo(LockInfo li) { > String res = li.getClassName() + '@' + > Integer.toHexString(li.getIdentityHashCode()); > // There is no getLock method in current jdk > if (li.getStringValue() != null) { > return res + " val=\"" + li.getStringValue() + "\""; > } > return res; > } > } > > > ---------- > > Commit messages: > - 8253280: Use class name as class loading lock > > Changes: https://github.com/openjdk/jdk/pull/104/files > Webrev: https://openjdk.github.io/cr/?repo=jdk&pr=104&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8253280 > Stats: 74 lines changed; 73 ins; 1 del > Patch: https://git.openjdk.java.net/jdk/pull/104.diff > Fetch: git fetch https://git.openjdk.java.net/jdk pull/104/head:pull/104 > > PR: https://git.openjdk.java.net/jdk/pull/104 > > From dholmes at openjdk.java.net Mon Nov 16 04:53:01 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Mon, 16 Nov 2020 04:53:01 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> Message-ID: On Thu, 12 Nov 2020 08:11:10 GMT, Robbin Ehn wrote: >> As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. >> Resulting in an unexpected result to GetCurrentContendedMonitor(). >> Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. >> Instead I here just use sleep + volatile for the barriers. >> >> I cannot reproduce with these changes. >> >> Chewing through T6 as most issues have been seen there - passed. > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed comment Hi Robbin, Please see the bug report for more discussion. Bottom line: I now agree this is the right kind of fix for this situation. I could nit pick on the variable naming but lets just get this done. Thanks for your patience on this. It is important to fully understand how these situations can arise. David test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 67: > 65: public static int run(String argv[], PrintStream ref) { > 66: out = ref; > 67: doSleep(); // If we need to load any classes to execute doSleep(), do it now. Well intentioned but not really useful. The classes used on the normal execution path are already loaded during VM initialization. The exceptional paths can still lead to class loading/linking/synchronization, so this preemptive call doesn't help that case anyway. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1177 From david.holmes at oracle.com Mon Nov 16 05:55:59 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Nov 2020 15:55:59 +1000 Subject: RFR: 8253280: Use class name as class loading lock In-Reply-To: <5DnzvOcYJwLegT48SxwqBUqvg8HnihAgaqUpdraS3CU=.516c26fb-e88f-4b7f-9090-26f770e54f05@github.com> References: <5DnzvOcYJwLegT48SxwqBUqvg8HnihAgaqUpdraS3CU=.516c26fb-e88f-4b7f-9090-26f770e54f05@github.com> Message-ID: Hi Robert, I have to agree with Mandy and Alan here. What you propose, in terms of getting more visibility into which class the classloader is trying to load, is quite reasonable. But the mechanism is quite problematic for a number of reasons as Mandy mentioned. In particular platform code needs to avoid calling toString() on user-defined objects as it can do anything. Also note that java.lang.String is not a good example here as you should never be loading it (or any core class) explicitly via a user-defined classloader. On 15/11/2020 2:06 am, Robert LU wrote: > When many thread try to load same class, the thread will stuck on `ClassLoader.loadClass`. > At current jdk, the stacktrace by example program is: > "Thread-1" prio=5 Id=13 BLOCKED on java.lang.String at 724af044 owned by "Thread-0" Id=12 > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:646) > - blocked on java.lang.String at 724af044 val="java.lang.String" This seems to be the after case, not before. Cheers, David ----- > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:634) > at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) > at app//Main2.test(Main2.java:19) > at app//Main2$$Lambda$37/0x00000001000c2a20.run(Unknown Source) > at java.base/java.lang.Thread.run(Thread.java:831) > There is no way to get which class stuck the thread. > > **After this patch, the stacktrace will be**: > "Thread-2" prio=5 Id=13 BLOCKED on java.lang.String at 724af044 owned by "Thread-3" Id=14 > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:612) > - blocked on java.lang.String at 724af044 val="java.lang.String" > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:600) > at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) > at app//Main2.test(Main2.java:18) > at app//Main2$$Lambda$38/0x0000000100097440.run(Unknown Source) > at java.base/java.lang.Thread.run(Thread.java:832) > That is, user will know which class stuck the thread, in this example, the class is `java.lang.String`. It's helpful for troubleshooting. > > The example program: > Before patch: >
> Main.java > > // Main.java > import java.io.PrintStream; > import java.lang.management.*; > > public final class Main { > private synchronized static void test1() { > while (true) { > try { > Thread.sleep(1000); > } catch (InterruptedException e) { > e.printStackTrace(); > } > } > } > > private static void test() { > while (true) { > try { > Main.class.getClassLoader().loadClass("java.lang.String"); > } catch (ClassNotFoundException e) { > } > } > } > > public static void main(String[] args) throws InterruptedException, NoSuchFieldException, IllegalAccessException { > new Thread(Main::test).start(); > new Thread(Main::test).start(); > new Thread(Main::test).start(); > new Thread(Main::test).start(); > new Thread(Main::test).start(); > > while (true) { > Thread.sleep(1000); > ThreadMXBean bean = ManagementFactory.getThreadMXBean(); > ThreadInfo[] infos = bean.dumpAllThreads(true, true); > for (ThreadInfo info : infos) { > System.out.println(printThreadInfo(info)); > } > } > } > > private static String printThreadInfo(ThreadInfo threadInfo) { > StringBuilder sb = new StringBuilder(""" + threadInfo.getThreadName() + """ + > (threadInfo.isDaemon() ? " daemon" : "") + > " prio=" + threadInfo.getPriority() + > " Id=" + threadInfo.getThreadId() + " " + > threadInfo.getThreadState()); > if (threadInfo.getLockName() != null) { > sb.append(" on " + threadInfo.getLockName()); > } > if (threadInfo.getLockOwnerName() != null) { > sb.append(" owned by "" + threadInfo.getLockOwnerName() + > "" Id=" + threadInfo.getLockOwnerId()); > } > if (threadInfo.isSuspended()) { > sb.append(" (suspended)"); > } > if (threadInfo.isInNative()) { > sb.append(" (in native)"); > } > sb.append('\n'); > int i = 0; > StackTraceElement[] stackTrace = threadInfo.getStackTrace(); > for (; i < stackTrace.length; i++) { > StackTraceElement ste = stackTrace[i]; > sb.append("\tat " + ste.toString()); > sb.append('\n'); > if (i == 0 && threadInfo.getLockInfo() != null) { > Thread.State ts = threadInfo.getThreadState(); > switch (ts) { > case BLOCKED: > sb.append("\t- blocked on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case WAITING: > sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case TIMED_WAITING: > sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > default: > } > } > > for (MonitorInfo mi : threadInfo.getLockedMonitors()) { > if (mi.getLockedStackDepth() == i) { > sb.append("\t- locked " + printLockInfo(mi)); > sb.append('\n'); > } > } > } > if (i < stackTrace.length) { > sb.append("\t..."); > sb.append('\n'); > } > > LockInfo[] locks = threadInfo.getLockedSynchronizers(); > if (locks.length > 0) { > sb.append("\n\tNumber of locked synchronizers = " + locks.length); > sb.append('\n'); > for (LockInfo li : locks) { > sb.append("\t- " + printLockInfo(li)); > sb.append('\n'); > } > } > sb.append('\n'); > return sb.toString(); > } > > private static String printLockInfo(LockInfo li) { > String res = li.getClassName() + '@' + Integer.toHexString(li.getIdentityHashCode()); > return res; > } > } >
> After patch: >
> Main2.java > > // Main2.java > import java.io.PrintStream; > import java.lang.management.*; > > public final class Main2 { > private synchronized static void test1() { > while (true) { > try { > Thread.sleep(1000); > } catch (InterruptedException e) { > e.printStackTrace(); > } > } > } > > private static void test() { > while (true) { > try { > Main2.class.getClassLoader().loadClass("java.lang.String"); > } catch (ClassNotFoundException e) { > } > } > } > > public static void main(String[] args) throws InterruptedException, NoSuchFieldException, IllegalAccessException { > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > new Thread(Main2::test).start(); > > while (true) { > Thread.sleep(1000); > ThreadMXBean bean = ManagementFactory.getThreadMXBean(); > ThreadInfo[] infos = bean.dumpAllThreads(true, true); > for (ThreadInfo info : infos) { > System.out.println(printThreadInfo(info)); > } > } > } > > private static String printThreadInfo(ThreadInfo threadInfo) { > StringBuilder sb = new StringBuilder(""" + threadInfo.getThreadName() + """ + > (threadInfo.isDaemon() ? " daemon" : "") + > " prio=" + threadInfo.getPriority() + > " Id=" + threadInfo.getThreadId() + " " + > threadInfo.getThreadState()); > if (threadInfo.getLockName() != null) { > sb.append(" on " + threadInfo.getLockName()); > } > if (threadInfo.getLockOwnerName() != null) { > sb.append(" owned by "" + threadInfo.getLockOwnerName() + > "" Id=" + threadInfo.getLockOwnerId()); > } > if (threadInfo.isSuspended()) { > sb.append(" (suspended)"); > } > if (threadInfo.isInNative()) { > sb.append(" (in native)"); > } > sb.append('\n'); > int i = 0; > StackTraceElement[] stackTrace = threadInfo.getStackTrace(); > for (; i < stackTrace.length; i++) { > StackTraceElement ste = stackTrace[i]; > sb.append("\tat " + ste.toString()); > sb.append('\n'); > if (i == 0 && threadInfo.getLockInfo() != null) { > Thread.State ts = threadInfo.getThreadState(); > switch (ts) { > case BLOCKED: > sb.append("\t- blocked on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case WAITING: > sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > case TIMED_WAITING: > sb.append("\t- waiting on " + printLockInfo(threadInfo.getLockInfo())); > sb.append('\n'); > break; > default: > } > } > > for (MonitorInfo mi : threadInfo.getLockedMonitors()) { > if (mi.getLockedStackDepth() == i) { > sb.append("\t- locked " + printLockInfo(mi)); > sb.append('\n'); > } > } > } > if (i < stackTrace.length) { > sb.append("\t..."); > sb.append('\n'); > } > > LockInfo[] locks = threadInfo.getLockedSynchronizers(); > if (locks.length > 0) { > sb.append("\n\tNumber of locked synchronizers = " + locks.length); > sb.append('\n'); > for (LockInfo li : locks) { > sb.append("\t- " + printLockInfo(li)); > sb.append('\n'); > } > } > sb.append('\n'); > return sb.toString(); > } > > private static String printLockInfo(LockInfo li) { > String res = li.getClassName() + '@' + Integer.toHexString(li.getIdentityHashCode()); > // There is no getLock method in current jdk > if (li.getStringValue() != null) { > return res + " val="" + li.getStringValue() + """; > } > return res; > } > } >
> > ------------- > > Commit messages: > - 8253280: Use class name as class loading lock > > Changes: https://git.openjdk.java.net/jdk/pull/104/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=104&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8253280 > Stats: 73 lines in 5 files changed: 72 ins; 0 del; 1 mod > Patch: https://git.openjdk.java.net/jdk/pull/104.diff > Fetch: git fetch https://git.openjdk.java.net/jdk pull/104/head:pull/104 > > PR: https://git.openjdk.java.net/jdk/pull/104 > From rehn at openjdk.java.net Mon Nov 16 07:57:55 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 16 Nov 2020 07:57:55 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> Message-ID: On Mon, 16 Nov 2020 04:50:13 GMT, David Holmes wrote: > Hi Robbin, > > Please see the bug report for more discussion. > > Bottom line: I now agree this is the right kind of fix for this situation. I could nit pick on the variable naming but lets just get this done. > > Thanks for your patience on this. It is important to fully understand how these situations can arise. > > David Thanks, great! I will wait for at least acknowledge from @sspitsyn. > test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 67: > >> 65: public static int run(String argv[], PrintStream ref) { >> 66: out = ref; >> 67: doSleep(); // If we need to load any classes to execute doSleep(), do it now. > > Well intentioned but not really useful. The classes used on the normal execution path are already loaded during VM initialization. The exceptional paths can still lead to class loading/linking/synchronization, so this preemptive call doesn't help that case anyway. I can remove the 'dummy' calls in each test? ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From david.holmes at oracle.com Mon Nov 16 08:01:11 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Nov 2020 18:01:11 +1000 Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> Message-ID: <7b6fe9e6-0c6b-c95f-1701-4a8ef7bd84ff@oracle.com> On 16/11/2020 5:57 pm, Robbin Ehn wrote: > On Mon, 16 Nov 2020 04:50:13 GMT, David Holmes wrote: >> test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001.java line 67: >> >>> 65: public static int run(String argv[], PrintStream ref) { >>> 66: out = ref; >>> 67: doSleep(); // If we need to load any classes to execute doSleep(), do it now. >> >> Well intentioned but not really useful. The classes used on the normal execution path are already loaded during VM initialization. The exceptional paths can still lead to class loading/linking/synchronization, so this preemptive call doesn't help that case anyway. > > I can remove the 'dummy' calls in each test? I would, but no big deal either way. David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/1177 > From stefank at openjdk.java.net Mon Nov 16 08:01:58 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Mon, 16 Nov 2020 08:01:58 GMT Subject: Integrated: 8256337: ap01t001.cpp, 67: Received unexpected number of ObjectFree events: 7 In-Reply-To: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> References: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> Message-ID: On Fri, 13 Nov 2020 14:11:23 GMT, Stefan Karlsson wrote: > The ap01t001 test creates six extra instances of the tested class, let them die, and then checks that it gets exactly six ObjectFree callbacks. The problem is that this is verified in the VMDeath callback and at that point the instance has gone out-of-scope and and a seventh ObjectFree event has been triggered. > > My proposed fix is to ensure that the test instance is kept alive. This pull request has now been integrated. Changeset: 6a69e304 Author: Stefan Karlsson URL: https://git.openjdk.java.net/jdk/commit/6a69e304 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod 8256337: ap01t001.cpp, 67: Received unexpected number of ObjectFree events: 7 Reviewed-by: coleenp, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/1204 From stefank at openjdk.java.net Mon Nov 16 08:01:57 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Mon, 16 Nov 2020 08:01:57 GMT Subject: RFR: 8256337: ap01t001.cpp, 67: Received unexpected number of ObjectFree events: 7 In-Reply-To: References: <7thDKusqZ5qwzxJF5J3GpAJZZXNmxeXGFUIHnb9uGYY=.57151319-45fb-48e4-beb7-458fb388093b@github.com> Message-ID: On Fri, 13 Nov 2020 19:25:57 GMT, Serguei Spitsyn wrote: >> The ap01t001 test creates six extra instances of the tested class, let them die, and then checks that it gets exactly six ObjectFree callbacks. The problem is that this is verified in the VMDeath callback and at that point the instance has gone out-of-scope and and a seventh ObjectFree event has been triggered. >> >> My proposed fix is to ensure that the test instance is kept alive. > > Hi Stefan, > It looks good to me. > Thank you for fixing it! > Serguei Thanks for reviewing! ------------- PR: https://git.openjdk.java.net/jdk/pull/1204 From sspitsyn at openjdk.java.net Mon Nov 16 08:29:01 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Mon, 16 Nov 2020 08:29:01 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> Message-ID: On Thu, 12 Nov 2020 08:11:10 GMT, Robbin Ehn wrote: >> As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. >> Resulting in an unexpected result to GetCurrentContendedMonitor(). >> Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. >> Instead I here just use sleep + volatile for the barriers. >> >> I cannot reproduce with these changes. >> >> Chewing through T6 as most issues have been seen there - passed. > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed comment Robbin, thank you for nice analysis in the the bug report. The fix looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Mon Nov 16 08:36:09 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 16 Nov 2020 08:36:09 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v3] In-Reply-To: References: Message-ID: > As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. > Resulting in an unexpected result to GetCurrentContendedMonitor(). > Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. > Instead I here just use sleep + volatile for the barriers. > > I cannot reproduce with these changes. > > Chewing through T6 as most issues have been seen there - passed. Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: Removed dummy sleeps ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1177/files - new: https://git.openjdk.java.net/jdk/pull/1177/files/32879e2d..5fc31ebd Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1177&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1177&range=01-02 Stats: 3 lines in 2 files changed: 0 ins; 3 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1177.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1177/head:pull/1177 PR: https://git.openjdk.java.net/jdk/pull/1177 From rehn at openjdk.java.net Mon Nov 16 08:36:10 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 16 Nov 2020 08:36:10 GMT Subject: RFR: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" [v2] In-Reply-To: References: <2YYGycSVlwAS-fMdQjZ5osxGakdiu7OE8s3cUBuX8gg=.b2b6c8cc-d11a-49e4-8ee0-5b604c30ec37@github.com> Message-ID: On Mon, 16 Nov 2020 08:26:29 GMT, Serguei Spitsyn wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed comment > > Robbin, thank you for nice analysis in the the bug report. > The fix looks good. > Thanks, > Serguei Thanks, all! I push removal of the two dummy doSleep(), I count that as trivial, tested locally only. I'll integrate in a few hours. ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From kbarrett at openjdk.java.net Mon Nov 16 09:38:00 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Mon, 16 Nov 2020 09:38:00 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 11:39:59 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? > > Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. > > With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. > > This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. > > Testing: tier1-5, one or two 6-8 runs > > The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements > > Thanks, > Thomas I've only skimmed the non-GC changes. src/hotspot/share/oops/klass.cpp line 207: > 205: _shared_class_path_index(-1) { > 206: CDS_ONLY(_shared_class_flags = 0); > 207: CDS_JAVA_HEAP_ONLY(_archived_mirror_index = -1); Why are the semi-colons being moved out of the macros here? This isn't needed, and is contrary to usage elsewhere. src/hotspot/share/memory/heapShared.hpp line 70: > 68: GrowableArray* _subgraph_entry_fields; > 69: > 70: // Does this KlassSubGraphInfo belong to the arcived full module graph s/arcived/archived/ src/hotspot/share/memory/heapShared.hpp line 85: > 83: _is_full_module_graph(is_full_module_graph), > 84: _has_non_early_klasses(false) {} > 85: ~KlassSubGraphInfo() { Please add a blank line between the constructor and the destructor. src/hotspot/share/gc/g1/heapRegion.hpp line 181: > 179: // Returns whether the given object is dead based on TAMS and bitmap. > 180: // An object is dead iff a) it was not allocated since the last mark, b) it > 181: // is not marked, and c) it is not in a closed archive region. The first, unchanged, line isn't consistent with the additional comment. I suggest ending it after "dead", and adding "(TAMS)" and "(bitmap)" before the ending commas of the first two alternatives. src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 60: > 58: // Never free closed archive regions. This is also be the only other allowed > 59: // type at this point. > 60: assert(hr->is_closed_archive(), "Only closed archive regions can also be pinned."); I found the assert message here very confusing. It's really that all other pinned region cases have been covered, and closed_archive is the last remaining one. src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 128: > 126: } > 127: > 128: void G1FullGCPrepareTask::G1CalculatePointersClosure::free_pinned_region(HeapRegion* hr) { Should this be called free_archive_region (or free_open_archive_region)? The statistics counter is `_pinned_archive_regions_removed`, so this presumably can't be used for some other kind of pinned region. ------------- Changes requested by kbarrett (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1163 From tschatzl at openjdk.java.net Mon Nov 16 10:33:10 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 16 Nov 2020 10:33:10 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v2] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? > > Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. > > With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. > > This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. > > Testing: tier1-5, one or two 6-8 runs > > The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: kbarrett review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1163/files - new: https://git.openjdk.java.net/jdk/pull/1163/files/c29503b6..a324c9c8 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1163&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1163&range=00-01 Stats: 28 lines in 9 files changed: 4 ins; 0 del; 24 mod Patch: https://git.openjdk.java.net/jdk/pull/1163.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1163/head:pull/1163 PR: https://git.openjdk.java.net/jdk/pull/1163 From tschatzl at openjdk.java.net Mon Nov 16 12:33:10 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 16 Nov 2020 12:33:10 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v3] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? > > Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. > > With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. > > This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. > > Testing: tier1-5, one or two 6-8 runs > > The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements > > Thanks, > Thomas Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: - Merge branch 'master' of https://git.openjdk.java.net/jdk into 8253081-null-narrow-klass-changes2 - kbarrett review - Initial import ------------- Changes: https://git.openjdk.java.net/jdk/pull/1163/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1163&range=02 Stats: 666 lines in 32 files changed: 471 ins; 83 del; 112 mod Patch: https://git.openjdk.java.net/jdk/pull/1163.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1163/head:pull/1163 PR: https://git.openjdk.java.net/jdk/pull/1163 From tschatzl at openjdk.java.net Mon Nov 16 12:40:58 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 16 Nov 2020 12:40:58 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v3] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 09:34:51 GMT, Kim Barrett wrote: >> Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' of https://git.openjdk.java.net/jdk into 8253081-null-narrow-klass-changes2 >> - kbarrett review >> - Initial import > > I've only skimmed the non-GC changes. @kimbarrett : I think the latest update fixes all your concerns. Also had to rebase. Reran tier1. ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From rehn at openjdk.java.net Mon Nov 16 13:10:03 2020 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 16 Nov 2020 13:10:03 GMT Subject: Integrated: 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 20:33:15 GMT, Robbin Ehn wrote: > As the stack trace in the bug shows, we cannot load classes, since we may take a monitor. > Resulting in an unexpected result to GetCurrentContendedMonitor(). > Trying to use some decent primitive, e.g. Wicket/Semaphore/.., without being implementation dependent means we must warm up every possible scenario, since it may use a new class. > Instead I here just use sleep + volatile for the barriers. > > I cannot reproduce with these changes. > > Chewing through T6 as most issues have been seen there - passed. This pull request has now been integrated. Changeset: c5fe2c1f Author: Robbin Ehn URL: https://git.openjdk.java.net/jdk/commit/c5fe2c1f Stats: 36 lines in 2 files changed: 22 ins; 5 del; 9 mod 8244679: JVM/TI GetCurrentContendedMonitor/contmon001 failed due to "(IsSameObject#3) unexpected monitor object: 0x000000562336DBA8" Reviewed-by: pchilanomate, dcubed, dholmes, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/1177 From coleenp at openjdk.java.net Mon Nov 16 17:12:32 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 17:12:32 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v9] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with three additional commits since the last revision: - Add shenandoah set_needs_cleaning but this doesn't work. - fix vmTestbase/nsk/jvmti tests - improve tagmap cleanup and objectfree event posting ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/0487b84c..283696f6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=08 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=07-08 Stats: 223 lines in 16 files changed: 174 ins; 12 del; 37 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 16 23:10:21 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 23:10:21 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v10] In-Reply-To: References: Message-ID: <4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com> > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits: - Reverse remove_dead_entries_locked function names. - Merge branch 'master' into jvmti-table - Add shenandoah set_needs_cleaning but this doesn't work. - fix vmTestbase/nsk/jvmti tests - improve tagmap cleanup and objectfree event posting - Add logging to event posting in case of pauses. - Merge branch 'master' into jvmti-table - Add back WeakProcessorPhases::Phase enum. - Serguei 1. - Code review comments from Kim and Albert. - ... and 5 more: https://git.openjdk.java.net/jdk/compare/0357db35...daaa13fe ------------- Changes: https://git.openjdk.java.net/jdk/pull/967/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=09 Stats: 1884 lines in 49 files changed: 768 ins; 993 del; 123 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 16 23:19:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 23:19:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> Message-ID: On Thu, 5 Nov 2020 14:36:44 GMT, Erik ?sterlund wrote: >> Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times. > > The point of this change was not necessarily to be lazy about updating the tagmap, until someone uses it. The point was to get rid of the last annoying serial GC phase. Doing it all lazily would certainly also achieve that. But it would also lead to situations where no event is ever posted from GC to GC. So you would get the event 20 GCs later, which might come as a surprise. It did come as a surprise to some tests, so it is reasonable to assume it would come as a surprise to users too. And I don't think we want such surprises unless we couldn't deal with them. And we can. Kim's change to post the events from the service thread or before other JVMTI operations removes posting events from the gc_notification, which was the objection. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 16 23:19:13 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 23:19:13 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 20:39:40 GMT, Coleen Phillimore wrote: >> Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change. > > This change also passes tier 7,8 testing. does this work? I've added two commits from @kimbarrett that defer the ObjectFree posting to the service thread or to a place where it could be removed before posting. I also remerged and added the call JvmtiTagMap::set_needs_cleaning() to shenandoah which works after merging the latest code from shenandoah. Testing tiers 1-6 currently. jvmti/jdi tests pass with G1 and ZGC stress options, and JVMTI tests pass with shenandoah. Thanks! Coleen ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 16 23:30:25 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 23:30:25 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Fix minimal build. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/daaa13fe..1940eaf1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=10 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=09-10 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Tue Nov 17 03:52:14 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 17 Nov 2020 03:52:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 23:30:25 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Fix minimal build. Marked as reviewed by kbarrett (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From lzang at openjdk.java.net Tue Nov 17 08:48:12 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Tue, 17 Nov 2020 08:48:12 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump Message-ID: This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. example: jmap -dump:live,gz=1,file=dump.gz ------------- Commit messages: - 8256450: Add gz option to jmap to write a gzipped heap dump Changes: https://git.openjdk.java.net/jdk/pull/1251/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256450 Stats: 23 lines in 2 files changed: 21 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1251.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1251/head:pull/1251 PR: https://git.openjdk.java.net/jdk/pull/1251 From lzang at openjdk.java.net Tue Nov 17 09:18:20 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Tue, 17 Nov 2020 09:18:20 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: References: Message-ID: > This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. > > example: > jmap -dump:live,gz=1,file=dump.gz Lin Zang has updated the pull request incrementally with one additional commit since the last revision: revise the uintx format issue for output message ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1251/files - new: https://git.openjdk.java.net/jdk/pull/1251/files/8b877776..84c9913d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1251.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1251/head:pull/1251 PR: https://git.openjdk.java.net/jdk/pull/1251 From sjohanss at openjdk.java.net Tue Nov 17 10:19:08 2020 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 17 Nov 2020 10:19:08 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v3] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 12:33:10 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? >> >> Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. >> >> With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. >> >> This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. >> >> Testing: tier1-5, one or two 6-8 runs >> >> The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: > > - Merge branch 'master' of https://git.openjdk.java.net/jdk into 8253081-null-narrow-klass-changes2 > - kbarrett review > - Initial import I've been focusing on the GC-parts and it looks good in general just a few comments. src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 4458: > 4456: heap_region_iterate(&cl); > 4457: > 4458: remove_from_old_gen_sets(0, 0, cl.humongous_regions_reclaimed()); Looking at this call and now having three parameters that are "optional" for `remove_from_old_gen_sets()` I wonder if it would be cleaner to have three functions, one for each set. It would increase the number of times we take the look, but we could restructure the code in `G1ReclaimEmptyRegionsTask` to not do the updates in the worker threads and that way only take the lock when there will be no contention. If you fell like this is outside the scope of this change, please file an issue instead. src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1266: > 1264: _g1h->remove_from_old_gen_sets(cl.old_regions_removed(), > 1265: cl.archive_regions_removed(), > 1266: cl.humongous_regions_removed()); If we want to go with the one call per set approach, here we could just atomically add these to a task-counter for each type and then do the calls to update the sets after the task has finished. src/hotspot/share/gc/g1/g1FullGCPrepareTask.hpp line 60: > 58: G1FullGCCompactionPoint* _cp; > 59: uint _humongous_regions_removed; > 60: uint _open_archive_regions_freed; Since we no longer use these counters to update the sets, it is enough to just have one counter to track if any region has been freed. Or use a boolean if you prefer that. ------------- Changes requested by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1163 From tschatzl at openjdk.java.net Tue Nov 17 10:55:19 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 17 Nov 2020 10:55:19 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v4] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? > > Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. > > With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. > > This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. > > Testing: tier1-5, one or two 6-8 runs > > The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: - sjohanss review - Remove code that "activates" dormant objects as now all active objects are reachable via the root object array ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1163/files - new: https://git.openjdk.java.net/jdk/pull/1163/files/099ec2f4..6669b529 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1163&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1163&range=02-03 Stats: 56 lines in 6 files changed: 2 ins; 35 del; 19 mod Patch: https://git.openjdk.java.net/jdk/pull/1163.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1163/head:pull/1163 PR: https://git.openjdk.java.net/jdk/pull/1163 From tschatzl at openjdk.java.net Tue Nov 17 11:01:09 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 17 Nov 2020 11:01:09 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v3] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 10:16:28 GMT, Stefan Johansson wrote: >> Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' of https://git.openjdk.java.net/jdk into 8253081-null-narrow-klass-changes2 >> - kbarrett review >> - Initial import > > I've been focusing on the GC-parts and it looks good in general just a few comments. Ran tier1-4, tier5-gc with the changes from ea78aa1 (contributed by @iklam after some discussion). > src/hotspot/share/gc/g1/g1FullGCPrepareTask.hpp line 60: > >> 58: G1FullGCCompactionPoint* _cp; >> 59: uint _humongous_regions_removed; >> 60: uint _open_archive_regions_freed; > > Since we no longer use these counters to update the sets, it is enough to just have one counter to track if any region has been freed. Or use a boolean if you prefer that. Fixed. > src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 4458: > >> 4456: heap_region_iterate(&cl); >> 4457: >> 4458: remove_from_old_gen_sets(0, 0, cl.humongous_regions_reclaimed()); > > Looking at this call and now having three parameters that are "optional" for `remove_from_old_gen_sets()` I wonder if it would be cleaner to have three functions, one for each set. It would increase the number of times we take the look, but we could restructure the code in `G1ReclaimEmptyRegionsTask` to not do the updates in the worker threads and that way only take the lock when there will be no contention. > > If you fell like this is outside the scope of this change, please file an issue instead. I am not very clear on what the problem there is and how the parameters are optional. I do not completely understand how adding an extra method just for this caller would improve the code significantly. I'll opt to defer this cleanup to a separate CR. ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From sjohanss at openjdk.java.net Tue Nov 17 13:00:05 2020 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 17 Nov 2020 13:00:05 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v4] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 10:55:19 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? >> >> Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. >> >> With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. >> >> This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. >> >> Testing: tier1-5, one or two 6-8 runs >> >> The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - sjohanss review > - Remove code that "activates" dormant objects as now all active objects are reachable via the root object array Marked as reviewed by sjohanss (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From sjohanss at openjdk.java.net Tue Nov 17 13:00:07 2020 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 17 Nov 2020 13:00:07 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v3] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 10:56:44 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 4458: >> >>> 4456: heap_region_iterate(&cl); >>> 4457: >>> 4458: remove_from_old_gen_sets(0, 0, cl.humongous_regions_reclaimed()); >> >> Looking at this call and now having three parameters that are "optional" for `remove_from_old_gen_sets()` I wonder if it would be cleaner to have three functions, one for each set. It would increase the number of times we take the look, but we could restructure the code in `G1ReclaimEmptyRegionsTask` to not do the updates in the worker threads and that way only take the lock when there will be no contention. >> >> If you fell like this is outside the scope of this change, please file an issue instead. > > I am not very clear on what the problem there is and how the parameters are optional. I do not completely understand how adding an extra method just for this caller would improve the code significantly. > > I'll opt to defer this cleanup to a separate CR. Sounds good. ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From kbarrett at openjdk.java.net Tue Nov 17 14:55:07 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 17 Nov 2020 14:55:07 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v4] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 10:55:19 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? >> >> Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. >> >> With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. >> >> This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. >> >> Testing: tier1-5, one or two 6-8 runs >> >> The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - sjohanss review > - Remove code that "activates" dormant objects as now all active objects are reachable via the root object array Marked as reviewed by kbarrett (Reviewer). src/hotspot/share/memory/heapShared.cpp line 660: > 658: VM_Verify verify_op; > 659: VMThread::execute(&verify_op); > 660: if (!FLAG_IS_DEFAULT(VerifyArchivedFields)) { Comment says "command line", so this should be FLAG_IS_CMDLINE rather than !FLAG_IS_DEFAULT. src/hotspot/share/memory/heapShared.cpp line 662: > 660: if (!FLAG_IS_DEFAULT(VerifyArchivedFields)) { > 661: // If this -XX:+VerifyArchivedFields is specified on the command-line, do extra > 662: // checks. s/this// ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From redestad at openjdk.java.net Tue Nov 17 16:14:08 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Tue, 17 Nov 2020 16:14:08 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v4] In-Reply-To: References: Message-ID: <_3YMX9f2F3xUZnge_avARhEOikx-f_hZB2VQ1SKZ4DM=.e2dbb88d-6c69-423e-9099-88ec9ec85827@github.com> On Tue, 17 Nov 2020 10:55:19 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? >> >> Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. >> >> With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. >> >> This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. >> >> Testing: tier1-5, one or two 6-8 runs >> >> The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - sjohanss review > - Remove code that "activates" dormant objects as now all active objects are reachable via the root object array Looks good! I took a sweep through the code and have some nits that you may choose to ignore. src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 4538: > 4536: } else { > 4537: // We ignore free regions, we'll empty the free list afterwards. > 4538: assert(hr->is_free(), Can make this one line src/hotspot/share/memory/heapShared.cpp line 334: > 332: } > 333: > 334: // Returns an objArray that contains all the roots of the archived objects It does..? src/hotspot/share/memory/heapShared.cpp line 413: > 411: int length = _pending_roots != NULL ? _pending_roots->length() : 0; > 412: int size = objArrayOopDesc::object_size(length); > 413: Klass *k = Universe::objectArrayKlassObj(); // already relocated to point to archived klass `Klass* k` ------------- Marked as reviewed by redestad (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1163 From iklam at openjdk.java.net Tue Nov 17 18:09:12 2020 From: iklam at openjdk.java.net (Ioi Lam) Date: Tue, 17 Nov 2020 18:09:12 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v4] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 14:51:23 GMT, Kim Barrett wrote: >> Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: >> >> - sjohanss review >> - Remove code that "activates" dormant objects as now all active objects are reachable via the root object array > > src/hotspot/share/memory/heapShared.cpp line 662: > >> 660: if (!FLAG_IS_DEFAULT(VerifyArchivedFields)) { >> 661: // If this -XX:+VerifyArchivedFields is specified on the command-line, do extra >> 662: // checks. > > s/this// I think we can keep the code and change the comments to the following: // If VerifyArchivedFields has a non-default value (e.g., specified on the command-line), do // more expensive checks. ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From iklam at openjdk.java.net Tue Nov 17 18:13:05 2020 From: iklam at openjdk.java.net (Ioi Lam) Date: Tue, 17 Nov 2020 18:13:05 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v4] In-Reply-To: <_3YMX9f2F3xUZnge_avARhEOikx-f_hZB2VQ1SKZ4DM=.e2dbb88d-6c69-423e-9099-88ec9ec85827@github.com> References: <_3YMX9f2F3xUZnge_avARhEOikx-f_hZB2VQ1SKZ4DM=.e2dbb88d-6c69-423e-9099-88ec9ec85827@github.com> Message-ID: On Tue, 17 Nov 2020 15:55:07 GMT, Claes Redestad wrote: >> Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: >> >> - sjohanss review >> - Remove code that "activates" dormant objects as now all active objects are reachable via the root object array > > src/hotspot/share/memory/heapShared.cpp line 334: > >> 332: } >> 333: >> 334: // Returns an objArray that contains all the roots of the archived objects > > It does..? Oops good catch! This comment should be moved to above `oop HeapShared::get_root(int index, bool clear) {` ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From tschatzl at openjdk.java.net Tue Nov 17 18:20:17 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 17 Nov 2020 18:20:17 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v5] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? > > Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. > > With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. > > This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. > > Testing: tier1-5, one or two 6-8 runs > > The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: kbarrett cl4es review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1163/files - new: https://git.openjdk.java.net/jdk/pull/1163/files/6669b529..d68d7527 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1163&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1163&range=03-04 Stats: 10 lines in 2 files changed: 1 ins; 2 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/1163.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1163/head:pull/1163 PR: https://git.openjdk.java.net/jdk/pull/1163 From amenkov at openjdk.java.net Tue Nov 17 18:44:03 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Tue, 17 Nov 2020 18:44:03 GMT Subject: RFR: 8255934: JConsole 14 and greater fails to connect to older JVM In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 23:00:05 GMT, Alex Menkov wrote: > OperatingSystemMXBean was changed in jdk14 (see JDK-8226575): > New methods getTotalMemorySize and getFreeMemorySize were added, old getTotalPhysicalMemorySize and getFreePhysicalMemorySize were deprecated. > > The fix adds fallbacks for the values (i.e. if new methods fail, jconsole calls old methods) Adding comment to get notification in serviceability mailing list ------------- PR: https://git.openjdk.java.net/jdk/pull/1243 From cjplummer at openjdk.java.net Tue Nov 17 19:00:05 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 17 Nov 2020 19:00:05 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 09:18:20 GMT, Lin Zang wrote: >> This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. >> >> example: >> jmap -dump:live,gz=1,file=dump.gz > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > revise the uintx format issue for output message Are the Attach API changes backwards compatible? You've added a new arg, but that arg could be passed to an older JVM that doesn't support it (I think it just gets ignored), or an older JVM would fail to pass the arg to a new JVM that is expecting it. Also, please see my comments in [JDK-8256451](https://bugs.openjdk.java.net/browse/JDK-8256451) regarding SA heap dumping support, and how it will lack this ability to compress the heap dump. ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From redestad at openjdk.java.net Tue Nov 17 20:25:06 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Tue, 17 Nov 2020 20:25:06 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v5] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 18:20:17 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? >> >> Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. >> >> With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. >> >> This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. >> >> Testing: tier1-5, one or two 6-8 runs >> >> The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > kbarrett cl4es review Marked as reviewed by redestad (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From lzang at openjdk.java.net Wed Nov 18 01:25:11 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 18 Nov 2020 01:25:11 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 18:57:10 GMT, Chris Plummer wrote: >> Lin Zang has updated the pull request incrementally with one additional commit since the last revision: >> >> revise the uintx format issue for output message > > Are the Attach API changes backwards compatible? You've added a new arg, but that arg could be passed to an older JVM that doesn't support it (I think it just gets ignored), or an older JVM would fail to pass the arg to a new JVM that is expecting it. > > Also, please see my comments in [JDK-8256451](https://bugs.openjdk.java.net/browse/JDK-8256451) regarding SA heap dumping support, and how it will lack this ability to compress the heap dump. Hi @plummercj, Thanks for your comments! > Are the Attach API changes backwards compatible? You've added a new arg, but that arg could be passed to an older JVM that doesn't support it (I think it just gets ignored), or an older JVM would fail to pass the arg to a new JVM that is expecting it. Correct, the new "jmap -dump:gz=1 [pid]" could work with old JVM and the "gz" option is ignored. And the old jmap -dump can not accept "gz" option, it fails with error message printed, no matter what jvm version it work with. I also tested that jcmd GC.heap_dump command with old version of jcmd can not accept the option "-gz=[number]". Moreover, This behavior is similar to "parallel=" option recently added to jmap -histo. So IMHO the risk is not high. If the "gz" option is not used, jmap -dump could work normally, with different version of jmap command and JVM. (I only tested with jdk8, jdk11 and latest jdk build) > Also, please see my comments in [JDK-8256451](https://bugs.openjdk.java.net/browse/JDK-8256451) regarding SA heap dumping support, and how it will lack this ability to compress the heap dump. Yes, I am also considering adding compression support to them, but I am not sure whether there should be new CSR issues created separately to tracking them, and whether I should create those issues at present because I am only planing the work. Let's discuss in the CSR. Thanks! BRs, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From cjplummer at openjdk.java.net Wed Nov 18 04:08:03 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 18 Nov 2020 04:08:03 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: References: Message-ID: On Wed, 18 Nov 2020 01:22:08 GMT, Lin Zang wrote: > And the old jmap -dump can not accept "gz" option, it fails with error message printed, no matter what jvm version it work with. I just want to clarify what I was referring to. I was not talking about trying to use gz with the old jmap command (from the command line). I was asking what happens if you use the old jmap command on a newer jvm that does support (and I assume expects) the new "compression level" argument to be passed, but it won't be. We've had this issue before. See [JDK-8219721](https://bugs.openjdk.java.net/browse/JDK-8219721). I haven't had a chance to look at the fix for JDK-8219721 to see if it also solves the issue with this change, or if something similar also needs to be done with this change. ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From lzang at openjdk.java.net Wed Nov 18 04:23:04 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 18 Nov 2020 04:23:04 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: References: Message-ID: On Wed, 18 Nov 2020 04:04:49 GMT, Chris Plummer wrote: >> Hi @plummercj, >> Thanks for your comments! >> >>> Are the Attach API changes backwards compatible? You've added a new arg, but that arg could be passed to an older JVM that doesn't support it (I think it just gets ignored), or an older JVM would fail to pass the arg to a new JVM that is expecting it. >> >> Correct, the new "jmap -dump:gz=1 [pid]" could work with old JVM and the "gz" option is ignored. And the old jmap -dump can not accept "gz" option, it fails with error message printed, no matter what jvm version it work with. I also tested that jcmd GC.heap_dump command with old version of jcmd can not accept the option "-gz=[number]". Moreover, This behavior is similar to "parallel=" option recently added to jmap -histo. So IMHO the risk is not high. >> >> If the "gz" option is not used, jmap -dump could work normally, with different version of jmap command and JVM. (I only tested with jdk8, jdk11 and latest jdk build) >> >>> Also, please see my comments in [JDK-8256451](https://bugs.openjdk.java.net/browse/JDK-8256451) regarding SA heap dumping support, and how it will lack this ability to compress the heap dump. >> >> Yes, I am also considering adding compression support to them, but I am not sure whether there should be new CSR issues created separately to tracking them, and whether I should create those issues at present because I am only planing the work. Let's discuss in the CSR. >> Thanks! >> >> BRs, >> Lin > >> And the old jmap -dump can not accept "gz" option, it fails with error message printed, no matter what jvm version it work with. > > I just want to clarify what I was referring to. I was not talking about trying to use gz with the old jmap command (from the command line). I was asking what happens if you use the old jmap command on a newer jvm that does support (and I assume expects) the new "compression level" argument to be passed, but it won't be. We've had this issue before. See [JDK-8219721](https://bugs.openjdk.java.net/browse/JDK-8219721). I haven't had a chance to look at the fix for JDK-8219721 to see if it also solves the issue with this change, or if something similar also needs to be done with this change. Hi @plummercj , Thanks for your explaination. I get your point. The issue of JDK-8219721 was fixed by reverting the problematic change. And the root cause of that issue is there is fixed limitation of number of arguments that passing to jvm from attacher, I have figured out a way to avoid touching it when adding new options to jcmd-alike tools, However this PR doesn't even exceed the argument limitation, so it doesn't have chance to cause the similar issue. BRs, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From tschatzl at openjdk.java.net Wed Nov 18 08:24:04 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 18 Nov 2020 08:24:04 GMT Subject: Integrated: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 11:39:59 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that changes the way how archive regions are managed in general and specifically by the G1 collector, fixing the crashes caused by adding the module graph into the archive in [JDK-8244778](https://bugs.openjdk.java.net/browse/JDK-8244778)? > > Previously before the JDK-8244778 change, archived objects could always be assumed as live, and so the G1 collector did so, not caring about the archive region's contents at all. With JDK-8244778 however, archived objects could die, and keep stale references to objects outside of the archive regions, which obviously causes crashes when walking these objects. > > With this change, open archive region contents are basically handled as any other objects; to support that, all open archive regions are now reachable via a single object array root. This hopefully also facilitates implementation in other collectors. > > This allows us to remove quite a bit of special handling in G1 too; the only difference is that open archive regions will generally not be collected unless they are completely empty: we do want to profit from the sharing across VMs as much as possible. > > Testing: tier1-5, one or two 6-8 runs > > The appcds changes were done by @iklam. These changes are described in this document: https://wiki.openjdk.java.net/display/HotSpot/CDS+Archived+Heap+Improvements > > Thanks, > Thomas This pull request has now been integrated. Changeset: d3095605 Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/d3095605 Stats: 694 lines in 32 files changed: 464 ins; 110 del; 120 mod 8253081: G1 fails on stale objects in archived module graph in Open Archive regions Change the handling of Open Archive areas, instead of assuming that everything in there is live always, a root containing references to all live root objects is provided. Adapt G1 to handle Open Archive regions as any other old region apart from never compacting or evacuating them. Co-authored-by: Ioi Lam Reviewed-by: kbarrett, sjohanss, redestad ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From tschatzl at openjdk.java.net Wed Nov 18 08:24:02 2020 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 18 Nov 2020 08:24:02 GMT Subject: RFR: 8253081: G1 fails on stale objects in archived module graph in Open Archive regions [v4] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 14:52:35 GMT, Kim Barrett wrote: >> Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: >> >> - sjohanss review >> - Remove code that "activates" dormant objects as now all active objects are reachable via the root object array > > Marked as reviewed by kbarrett (Reviewer). Thanks @kimbarrett @kstefanj @cl4es for the reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/1163 From cjplummer at openjdk.java.net Wed Nov 18 08:45:03 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 18 Nov 2020 08:45:03 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: References: Message-ID: On Wed, 18 Nov 2020 04:19:51 GMT, Lin Zang wrote: >>> And the old jmap -dump can not accept "gz" option, it fails with error message printed, no matter what jvm version it work with. >> >> I just want to clarify what I was referring to. I was not talking about trying to use gz with the old jmap command (from the command line). I was asking what happens if you use the old jmap command on a newer jvm that does support (and I assume expects) the new "compression level" argument to be passed, but it won't be. We've had this issue before. See [JDK-8219721](https://bugs.openjdk.java.net/browse/JDK-8219721). I haven't had a chance to look at the fix for JDK-8219721 to see if it also solves the issue with this change, or if something similar also needs to be done with this change. > > Hi @plummercj , > > Thanks for your explaination. I get your point. > The issue of JDK-8219721 was fixed by reverting the problematic change. And the root cause of that issue is there is fixed limitation of number of arguments that passing to jvm from attacher, I have figured out a way to avoid touching it when adding new options to jcmd-alike tools, However this PR doesn't even exceed the argument limitation, so it doesn't have chance to cause the similar issue. > > BRs, > Lin So I assume then that the `op->arg(2)` reference will return `NULL` when and older `jmap` is used? Is that correct? ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From lzang at openjdk.java.net Wed Nov 18 10:40:04 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 18 Nov 2020 10:40:04 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: References: Message-ID: On Wed, 18 Nov 2020 08:41:59 GMT, Chris Plummer wrote: >> Hi @plummercj , >> >> Thanks for your explaination. I get your point. >> The issue of JDK-8219721 was fixed by reverting the problematic change. And the root cause of that issue is there is fixed limitation of number of arguments that passing to jvm from attacher, I have figured out a way to avoid touching it when adding new options to jcmd-alike tools, However this PR doesn't even exceed the argument limitation, so it doesn't have chance to cause the similar issue. >> >> BRs, >> Lin > > So I assume then that the `op->arg(2)` reference will return `NULL` when an older `jmap` is used? Is that correct? Hi @plummercj, For old jmap, if "gz" option is used in commandline, it prompt usage info indicating that option is illegal. if "gz" option is not used, the op->arg(2) will be an empty string "", which is set from the attacher side in the method execute() of VirtualMachineImpl.java. ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From sspitsyn at openjdk.java.net Wed Nov 18 19:34:12 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 18 Nov 2020 19:34:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: Message-ID: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> On Mon, 16 Nov 2020 23:30:25 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Fix minimal build. Hi Coleen, It looks good to me. Just a couple of nits below. src/hotspot/share/prims/jvmtiTagMap.cpp: There is a double-check for _needs_cleaning, so the one at line 136 can be removed: 136 if (_needs_cleaning && 137 post_events && 138 env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { 139 remove_dead_entries(true /* post_object_free */); 1158 void JvmtiTagMap::remove_dead_entries(bool post_object_free) { 1159 assert(is_locked(), "precondition"); 1160 if (_needs_cleaning) { 1161 log_info(jvmti, table)("TagMap table needs cleaning%s", 1162 (post_object_free ? " and posting" : "")); 1163 hashmap()->remove_dead_entries(env(), post_object_free); 1164 _needs_cleaning = false; 1165 } 1166 } test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); + + /* Flush any pending ObjectFree events, which may set success to 1 */ + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, + JVMTI_EVENT_OBJECT_FREE, + NULL) != JVMTI_ERROR_NONE) { + success = 0; + } + + nsk_aod_agentFinished(jni, agentName, success); } ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From cjplummer at openjdk.java.net Wed Nov 18 19:59:06 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 18 Nov 2020 19:59:06 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: References: Message-ID: <1w4muLr2LKiFFFqaYcc2mMDhWEaCxRri0FBPanxrheU=.53dd5eb6-5350-45da-92ca-86c1bac8be53@github.com> On Wed, 18 Nov 2020 10:37:43 GMT, Lin Zang wrote: >> So I assume then that the `op->arg(2)` reference will return `NULL` when an older `jmap` is used? Is that correct? > > Hi @plummercj, > For old jmap, if "gz" option is used in commandline, it prompt usage info indicating that option is illegal. > if "gz" option is not used, the op->arg(2) will be an empty string "", which is set from the attacher side in the method execute() of VirtualMachineImpl.java. Ok, I see now that 3 args are always written, even if empty, so you at least get a "" if there is no value for the argument position. ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From cjplummer at openjdk.java.net Wed Nov 18 19:59:07 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 18 Nov 2020 19:59:07 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: <1w4muLr2LKiFFFqaYcc2mMDhWEaCxRri0FBPanxrheU=.53dd5eb6-5350-45da-92ca-86c1bac8be53@github.com> References: <1w4muLr2LKiFFFqaYcc2mMDhWEaCxRri0FBPanxrheU=.53dd5eb6-5350-45da-92ca-86c1bac8be53@github.com> Message-ID: <7fDWtu0568LmEeMcAQlUG0QgG5_WwFTIaacvk3maNoQ=.42c4ca1c-5aff-46ab-b3f0-51a35f86821b@github.com> On Wed, 18 Nov 2020 19:53:42 GMT, Chris Plummer wrote: >> Hi @plummercj, >> For old jmap, if "gz" option is used in commandline, it prompt usage info indicating that option is illegal. >> if "gz" option is not used, the op->arg(2) will be an empty string "", which is set from the attacher side in the method execute() of VirtualMachineImpl.java. > > Ok, I see now that 3 args are always written, even if empty, so you at least get a "" if there is no value for the argument position. I think you need to add a test case. You can use the work done for JDK-8237354 dcmd as a template. See http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.3/. I don't know that's it's necessary to add a test case for each GC, but if you do, I suggest putting them all in one file. The use of separate files per test case was not necessary. ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From coleenp at openjdk.java.net Wed Nov 18 23:24:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 18 Nov 2020 23:24:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v10] In-Reply-To: <4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com> References: <4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com> Message-ID: <6MQhIGtA1LWvXvSFSZWRkyU6d9AXf_LVdkE0zAq0ekc=.151e621a-eec4-4366-8ffa-729eb1d5bb63@github.com> On Mon, 16 Nov 2020 23:10:21 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits: > > - Reverse remove_dead_entries_locked function names. > - Merge branch 'master' into jvmti-table > - Add shenandoah set_needs_cleaning but this doesn't work. > - fix vmTestbase/nsk/jvmti tests > - improve tagmap cleanup and objectfree event posting > - Add logging to event posting in case of pauses. > - Merge branch 'master' into jvmti-table > - Add back WeakProcessorPhases::Phase enum. > - Serguei 1. > - Code review comments from Kim and Albert. > - ... and 5 more: https://git.openjdk.java.net/jdk/compare/0357db35...daaa13fe test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp line 72: > 70: Java_nsk_jvmti_AttachOnDemand_attach021_attach021Target_shutdownAgent(JNIEnv * jni, > 71: jclass klass) { > 72: @kimbarrett ? I'm not sure why you made this change. See Serguei's comment. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Wed Nov 18 23:29:10 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 18 Nov 2020 23:29:10 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> Message-ID: On Wed, 18 Nov 2020 19:31:44 GMT, Serguei Spitsyn wrote: > There is a double-check for _needs_cleaning, so the one at line 136 can be removed: I want to leave _needs_cleaning at 136 because even though the boolean is checked twice, it doesn't hurt performance and it has a nice symmetry in that function. I asked Kim about the other change. Thank you for reviewing, Serguei! ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 00:20:24 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 00:20:24 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v12] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits: - Merge branch 'master' into jvmti-table - Fix minimal build. - Reverse remove_dead_entries_locked function names. - Merge branch 'master' into jvmti-table - Add shenandoah set_needs_cleaning but this doesn't work. - fix vmTestbase/nsk/jvmti tests - improve tagmap cleanup and objectfree event posting - Add logging to event posting in case of pauses. - Merge branch 'master' into jvmti-table - Add back WeakProcessorPhases::Phase enum. - ... and 7 more: https://git.openjdk.java.net/jdk/compare/2b155713...9ef44f28 ------------- Changes: https://git.openjdk.java.net/jdk/pull/967/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=11 Stats: 1884 lines in 49 files changed: 768 ins; 992 del; 124 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Thu Nov 19 00:43:09 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Thu, 19 Nov 2020 00:43:09 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> Message-ID: On Wed, 18 Nov 2020 19:31:44 GMT, Serguei Spitsyn wrote: > test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: > > The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: > > ``` > - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); > + > + /* Flush any pending ObjectFree events, which may set success to 1 */ > + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, > + JVMTI_EVENT_OBJECT_FREE, > + NULL) != JVMTI_ERROR_NONE) { > + success = 0; > + } > + > + nsk_aod_agentFinished(jni, agentName, success); > } > ``` This change really is needed. The success variable in the test is a global, initially 0, set to 1 by the ObjectFree handler. In the old code, if the ObjectFree event hasn't been posted yet, we pass the initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where it's a local variable (so unaffected by any changes to the variable in the test), so stays 0 through to the call to nsk_aod_agentFinished. So the test fails. The split in the change causes the updated post-ObjectFree event success value of 1 to be passed to agentFinished. So the test passes. That required some head scratching to find at the time. That's the point of the comment about flushing pending events. Maybe the comment should be improved. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From lzang at openjdk.java.net Thu Nov 19 06:04:14 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Thu, 19 Nov 2020 06:04:14 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v3] In-Reply-To: References: Message-ID: > This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. > > example: > jmap -dump:live,gz=1,file=dump.gz Lin Zang has updated the pull request incrementally with one additional commit since the last revision: Add test cases ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1251/files - new: https://git.openjdk.java.net/jdk/pull/1251/files/84c9913d..1358e45f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=01-02 Stats: 35 lines in 1 file changed: 26 ins; 0 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/1251.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1251/head:pull/1251 PR: https://git.openjdk.java.net/jdk/pull/1251 From lzang at openjdk.java.net Thu Nov 19 06:08:04 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Thu, 19 Nov 2020 06:08:04 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v2] In-Reply-To: <7fDWtu0568LmEeMcAQlUG0QgG5_WwFTIaacvk3maNoQ=.42c4ca1c-5aff-46ab-b3f0-51a35f86821b@github.com> References: <1w4muLr2LKiFFFqaYcc2mMDhWEaCxRri0FBPanxrheU=.53dd5eb6-5350-45da-92ca-86c1bac8be53@github.com> <7fDWtu0568LmEeMcAQlUG0QgG5_WwFTIaacvk3maNoQ=.42c4ca1c-5aff-46ab-b3f0-51a35f86821b@github.com> Message-ID: On Wed, 18 Nov 2020 19:55:48 GMT, Chris Plummer wrote: > I think you need to add a test case. You can use the work done for JDK-8237354 dcmd as a template. See http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.3/. I don't know that's it's necessary to add a test case for each GC, but if you do, I suggest putting them all in one file. The use of separate files per test case was not necessary. I extended the BasicJMapTest.java to test the compressed heap dump. would you like to help review? Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From lzang at openjdk.java.net Thu Nov 19 06:16:17 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Thu, 19 Nov 2020 06:16:17 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v4] In-Reply-To: References: Message-ID: > This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. > > example: > jmap -dump:live,gz=1,file=dump.gz Lin Zang has updated the pull request incrementally with one additional commit since the last revision: fix indentation issue of BasicJmapTest.java ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1251/files - new: https://git.openjdk.java.net/jdk/pull/1251/files/1358e45f..f0bd8d81 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1251.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1251/head:pull/1251 PR: https://git.openjdk.java.net/jdk/pull/1251 From sspitsyn at openjdk.java.net Thu Nov 19 10:13:13 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 19 Nov 2020 10:13:13 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> Message-ID: On Thu, 19 Nov 2020 00:39:52 GMT, Kim Barrett wrote: >> Hi Coleen, >> It looks good to me. >> Just a couple of nits below. >> >> src/hotspot/share/prims/jvmtiTagMap.cpp: >> >> There is a double-check for _needs_cleaning, so the one at line 136 can be removed: >> 136 if (_needs_cleaning && >> 137 post_events && >> 138 env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { >> 139 remove_dead_entries(true /* post_object_free */); >> 1158 void JvmtiTagMap::remove_dead_entries(bool post_object_free) { >> 1159 assert(is_locked(), "precondition"); >> 1160 if (_needs_cleaning) { >> 1161 log_info(jvmti, table)("TagMap table needs cleaning%s", >> 1162 (post_object_free ? " and posting" : "")); >> 1163 hashmap()->remove_dead_entries(env(), post_object_free); >> 1164 _needs_cleaning = false; >> 1165 } >> 1166 } >> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: >> >> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: >> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); >> + >> + /* Flush any pending ObjectFree events, which may set success to 1 */ >> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, >> + JVMTI_EVENT_OBJECT_FREE, >> + NULL) != JVMTI_ERROR_NONE) { >> + success = 0; >> + } >> + >> + nsk_aod_agentFinished(jni, agentName, success); >> } > >> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: >> >> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: >> >> ``` >> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); >> + >> + /* Flush any pending ObjectFree events, which may set success to 1 */ >> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, >> + JVMTI_EVENT_OBJECT_FREE, >> + NULL) != JVMTI_ERROR_NONE) { >> + success = 0; >> + } >> + >> + nsk_aod_agentFinished(jni, agentName, success); >> } >> ``` > > This change really is needed. > > The success variable in the test is a global, initially 0, set to 1 by the > ObjectFree handler. > > In the old code, if the ObjectFree event hasn't been posted yet, we pass the > initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where > it's a local variable (so unaffected by any changes to the variable in the > test), so stays 0 through to the call to nsk_aod_agentFinished. So the test > fails. > > The split in the change causes the updated post-ObjectFree event success > value of 1 to be passed to agentFinished. So the test passes. > > That required some head scratching to find at the time. That's the point of > the comment about flushing pending events. Maybe the comment should be > improved. @kimbarrett Okay, thank you for explanation. I agree, it'd make sense to improve the comment a little bit. Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 12:39:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 12:39:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> Message-ID: <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> On Thu, 19 Nov 2020 10:10:06 GMT, Serguei Spitsyn wrote: >>> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: >>> >>> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: >>> >>> ``` >>> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); >>> + >>> + /* Flush any pending ObjectFree events, which may set success to 1 */ >>> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, >>> + JVMTI_EVENT_OBJECT_FREE, >>> + NULL) != JVMTI_ERROR_NONE) { >>> + success = 0; >>> + } >>> + >>> + nsk_aod_agentFinished(jni, agentName, success); >>> } >>> ``` >> >> This change really is needed. >> >> The success variable in the test is a global, initially 0, set to 1 by the >> ObjectFree handler. >> >> In the old code, if the ObjectFree event hasn't been posted yet, we pass the >> initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where >> it's a local variable (so unaffected by any changes to the variable in the >> test), so stays 0 through to the call to nsk_aod_agentFinished. So the test >> fails. >> >> The split in the change causes the updated post-ObjectFree event success >> value of 1 to be passed to agentFinished. So the test passes. >> >> That required some head scratching to find at the time. That's the point of >> the comment about flushing pending events. Maybe the comment should be >> improved. > > @kimbarrett > Okay, thank you for explanation. > I agree, it'd make sense to improve the comment a little bit. > Thanks, > Serguei So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change? ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 12:57:25 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 12:57:25 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> Message-ID: On Thu, 19 Nov 2020 12:36:46 GMT, Coleen Phillimore wrote: >> @kimbarrett >> Okay, thank you for explanation. >> I agree, it'd make sense to improve the comment a little bit. >> Thanks, >> Serguei > > So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change? > Ok, with this change, it's not posted yet and the success variable for nsk_aod_agentFinished is the local variable. We should fix this in an RFE filed: https://bugs.openjdk.java.net/browse/JDK-8256651 /* Flush any pending ObjectFree events, which will set global success variable to 1 for any pending ObjectFree events. */ How about this? The word 'global' helps me. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 12:57:24 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 12:57:24 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v13] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Update comment in jvmti test. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/9ef44f28..40168e63 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=12 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=11-12 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 12:57:25 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 12:57:25 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> Message-ID: On Thu, 19 Nov 2020 12:51:11 GMT, Coleen Phillimore wrote: >> So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change? >> Ok, with this change, it's not posted yet and the success variable for nsk_aod_agentFinished is the local variable. We should fix this in an RFE filed: https://bugs.openjdk.java.net/browse/JDK-8256651 > > /* Flush any pending ObjectFree events, which will set global success variable to 1 > for any pending ObjectFree events. */ > How about this? The word 'global' helps me. With remerging into shenandoah, all the jdi tests pass with shenandoah also. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 13:01:22 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 13:01:22 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v14] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Fix copyrights in test changes. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/40168e63..589e4c5b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=13 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=12-13 Stats: 5 lines in 5 files changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 14:33:16 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 14:33:16 GMT Subject: Integrated: 8212879: Make JVMTI TagMap table concurrent In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore wrote: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. This pull request has now been integrated. Changeset: ba721f5f Author: Coleen Phillimore URL: https://git.openjdk.java.net/jdk/commit/ba721f5f Stats: 1891 lines in 49 files changed: 769 ins; 992 del; 130 mod 8212879: Make JVMTI TagMap table concurrent Co-authored-by: Kim Barrett Co-authored-by: Coleen Phillimore Reviewed-by: stefank, ihse, zgu, eosterlund, sspitsyn, kbarrett ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 14:33:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 14:33:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> Message-ID: On Thu, 19 Nov 2020 12:54:23 GMT, Coleen Phillimore wrote: >> /* Flush any pending ObjectFree events, which will set global success variable to 1 >> for any pending ObjectFree events. */ >> How about this? The word 'global' helps me. > > With remerging into shenandoah, all the jdi tests pass with shenandoah also. Thank you to all the reviewers. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From github.com+13688759+lgxbslgx at openjdk.java.net Thu Nov 19 14:46:09 2020 From: github.com+13688759+lgxbslgx at openjdk.java.net (Guoxiong Li) Date: Thu, 19 Nov 2020 14:46:09 GMT Subject: RFR: 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails Message-ID: Hi all, `TestDriver.java` used `sun.hotspot.WhiteBox.getBooleanVMFlag("Use*GC")` which could return null. This patch uses `sun.hotspot.gc.GC` instead of `sun.hotspot.WhiteBox` to avoid the NullPointerException. Thank you for taking the time to review. Best Regards. ------------- Commit messages: - 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails Changes: https://git.openjdk.java.net/jdk/pull/1319/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1319&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8250888 Stats: 7 lines in 1 file changed: 2 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/1319.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1319/head:pull/1319 PR: https://git.openjdk.java.net/jdk/pull/1319 From shade at openjdk.java.net Thu Nov 19 15:27:04 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 19 Nov 2020 15:27:04 GMT Subject: RFR: 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 14:40:45 GMT, Guoxiong Li wrote: > Hi all, > > `TestDriver.java` used `sun.hotspot.WhiteBox.getBooleanVMFlag("Use*GC")` which could return null. > This patch uses `sun.hotspot.gc.GC` instead of `sun.hotspot.WhiteBox` to avoid the NullPointerException. > Thank you for taking the time to review. > > Best Regards. I believe it should be `isSelected()` rather than `isSupported()`. Try to run the test with different GCs like this to verify: make run-test TEST=nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java TEST_VM_OPTS=-XX:+UseShenandoahGC ------------- Changes requested by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1319 From github.com+13688759+lgxbslgx at openjdk.java.net Thu Nov 19 15:53:15 2020 From: github.com+13688759+lgxbslgx at openjdk.java.net (Guoxiong Li) Date: Thu, 19 Nov 2020 15:53:15 GMT Subject: RFR: 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails [v2] In-Reply-To: References: Message-ID: > Hi all, > > `TestDriver.java` used `sun.hotspot.WhiteBox.getBooleanVMFlag("Use*GC")` which could return null. > This patch uses `sun.hotspot.gc.GC` instead of `sun.hotspot.WhiteBox` to avoid the NullPointerException. > Thank you for taking the time to review. > > Best Regards. Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: Use `GC.isSelected` instead of `GC.isSupported` ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1319/files - new: https://git.openjdk.java.net/jdk/pull/1319/files/13d243a4..97ad6ca6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1319&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1319&range=00-01 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/1319.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1319/head:pull/1319 PR: https://git.openjdk.java.net/jdk/pull/1319 From cjplummer at openjdk.java.net Thu Nov 19 16:54:14 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 19 Nov 2020 16:54:14 GMT Subject: RFR: 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails [v2] In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 15:53:15 GMT, Guoxiong Li wrote: >> Hi all, >> >> `TestDriver.java` used `sun.hotspot.WhiteBox.getBooleanVMFlag("Use*GC")` which could return null. >> This patch uses `sun.hotspot.gc.GC` instead of `sun.hotspot.WhiteBox` to avoid the NullPointerException. >> Thank you for taking the time to review. >> >> Best Regards. > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Use `GC.isSelected` instead of `GC.isSupported` The changes look good, but I think the fact that when you used `isSupported()` you didn't see any failures indicates that proper testing was not done with all GCs. Please be sure to do this testing. In fact you might want to first try testing with `isSupported()` to confirm that you do see failures. ------------- Marked as reviewed by cjplummer (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1319 From sspitsyn at openjdk.java.net Thu Nov 19 18:07:09 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 19 Nov 2020 18:07:09 GMT Subject: RFR: 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails [v2] In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 15:53:15 GMT, Guoxiong Li wrote: >> Hi all, >> >> `TestDriver.java` used `sun.hotspot.WhiteBox.getBooleanVMFlag("Use*GC")` which could return null. >> This patch uses `sun.hotspot.gc.GC` instead of `sun.hotspot.WhiteBox` to avoid the NullPointerException. >> Thank you for taking the time to review. >> >> Best Regards. > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Use `GC.isSelected` instead of `GC.isSupported` Hi Guoxiong, It looks good to me. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1319 From amenkov at openjdk.java.net Thu Nov 19 21:54:10 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Thu, 19 Nov 2020 21:54:10 GMT Subject: RFR: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" Message-ID: - fixed java part of the test to handle spurious wakeups in the debuggee thread; - fixed native part of the test to detect and report the case when debuggee thread not found. ------------- Commit messages: - fixed cm01t002 - Merge branch 'master' of github.com:openjdk/jdk into master - Merge branch 'master' of github.com:openjdk/jdk into master - Merge branch 'master' of github.com:openjdk/jdk into master - 8253372: [TESTBUG] update tests which require jvmti - hotspot Changes: https://git.openjdk.java.net/jdk/pull/1330/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1330&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256364 Stats: 15 lines in 2 files changed: 14 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1330.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1330/head:pull/1330 PR: https://git.openjdk.java.net/jdk/pull/1330 From lmesnik at openjdk.java.net Thu Nov 19 22:16:02 2020 From: lmesnik at openjdk.java.net (Leonid Mesnik) Date: Thu, 19 Nov 2020 22:16:02 GMT Subject: RFR: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 21:49:09 GMT, Alex Menkov wrote: > - fixed java part of the test to handle spurious wakeups in the debuggee thread; > - fixed native part of the test to detect and report the case when debuggee thread not found. Looks good. ------------- PR: https://git.openjdk.java.net/jdk/pull/1330 From cjplummer at openjdk.java.net Thu Nov 19 22:17:05 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 19 Nov 2020 22:17:05 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v4] In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 06:16:17 GMT, Lin Zang wrote: >> This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. >> >> example: >> jmap -dump:live,gz=1,file=dump.gz > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > fix indentation issue of BasicJmapTest.java Changes requested by cjplummer (Reviewer). test/jdk/sun/tools/jmap/BasicJMapTest.java line 270: > 268: } > 269: > 270: private static void verifyDumpFile(File dump, boolean compressed) { I don't think the `compressed` argument is needed. The rest of the changes look fine. ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From cjplummer at openjdk.java.net Thu Nov 19 22:29:00 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 19 Nov 2020 22:29:00 GMT Subject: RFR: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 22:13:35 GMT, Leonid Mesnik wrote: >> - fixed java part of the test to handle spurious wakeups in the debuggee thread; >> - fixed native part of the test to detect and report the case when debuggee thread not found. > > Looks good. Why does this PR show 5 commits, including one that looks like it was for a previously fixed CR? It looks maybe you reused a branch. In the end it looks like skara did the right thing, and the diff looks right, but this seems to be something it is best to avoid with a PR. A rebase and a forced push might resolve this. ------------- PR: https://git.openjdk.java.net/jdk/pull/1330 From lzang at openjdk.java.net Fri Nov 20 01:17:18 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Fri, 20 Nov 2020 01:17:18 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v5] In-Reply-To: References: Message-ID: > This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. > > example: > jmap -dump:live,gz=1,file=dump.gz Lin Zang has updated the pull request incrementally with one additional commit since the last revision: remove unnecessory arguments in verifyDump() ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1251/files - new: https://git.openjdk.java.net/jdk/pull/1251/files/f0bd8d81..e014a52f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1251&range=03-04 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1251.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1251/head:pull/1251 PR: https://git.openjdk.java.net/jdk/pull/1251 From cjplummer at openjdk.java.net Fri Nov 20 03:01:05 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 20 Nov 2020 03:01:05 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v5] In-Reply-To: References: Message-ID: <1hbNKihSyBpa4DqU29wDLspPPf-U7WckTC-yl4iACfE=.5f61ac04-6318-45e3-8a8f-ec2dbad51d95@github.com> On Fri, 20 Nov 2020 01:17:18 GMT, Lin Zang wrote: >> This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. >> >> example: >> jmap -dump:live,gz=1,file=dump.gz > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > remove unnecessory arguments in verifyDump() Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From github.com+13688759+lgxbslgx at openjdk.java.net Fri Nov 20 06:33:07 2020 From: github.com+13688759+lgxbslgx at openjdk.java.net (Guoxiong Li) Date: Fri, 20 Nov 2020 06:33:07 GMT Subject: RFR: 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails [v2] In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 18:03:59 GMT, Serguei Spitsyn wrote: >> Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: >> >> Use `GC.isSelected` instead of `GC.isSupported` > > Hi Guoxiong, > It looks good to me. > Thanks, > Serguei I run the test locally using the following command before I submit this patch. make test TEST="jtreg:hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java" TEST_VM_OPTS=-XX:+UseShenandoahGC This test passes which misleads me that my patch is right. In any case, I apologize for this mistake and will try my best to do more testing before submitting patches in the future. Thank you very much for taking the time to review. Best Regards. ------------- PR: https://git.openjdk.java.net/jdk/pull/1319 From shade at openjdk.java.net Fri Nov 20 07:06:06 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 20 Nov 2020 07:06:06 GMT Subject: RFR: 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails [v2] In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 06:30:31 GMT, Guoxiong Li wrote: >> Hi Guoxiong, >> It looks good to me. >> Thanks, >> Serguei > > I run the test locally using the following command before I submit this patch. > make test TEST="jtreg:hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java" TEST_VM_OPTS=-XX:+UseShenandoahGC > This test passes which misleads me that my patch is right. > > In any case, I apologize for this mistake and will try my best to do more testing before submitting patches in the future. > Thank you very much for taking the time to review. > > Best Regards. I tested this patch with Serial, Parallel, G1, Shenandoah, Z and it passes. ------------- PR: https://git.openjdk.java.net/jdk/pull/1319 From github.com+13688759+lgxbslgx at openjdk.java.net Fri Nov 20 07:06:07 2020 From: github.com+13688759+lgxbslgx at openjdk.java.net (Guoxiong Li) Date: Fri, 20 Nov 2020 07:06:07 GMT Subject: Integrated: 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 14:40:45 GMT, Guoxiong Li wrote: > Hi all, > > `TestDriver.java` used `sun.hotspot.WhiteBox.getBooleanVMFlag("Use*GC")` which could return null. > This patch uses `sun.hotspot.gc.GC` instead of `sun.hotspot.WhiteBox` to avoid the NullPointerException. > Thank you for taking the time to review. > > Best Regards. This pull request has now been integrated. Changeset: 5fedb69e Author: Guoxiong Li Committer: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/5fedb69e Stats: 7 lines in 1 file changed: 2 ins; 0 del; 5 mod 8250888: nsk/jvmti/scenarios/general_functions/GF08/gf08t001/TestDriver.java fails Reviewed-by: cjplummer, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/1319 From pliden at openjdk.java.net Fri Nov 20 13:51:12 2020 From: pliden at openjdk.java.net (Per Liden) Date: Fri, 20 Nov 2020 13:51:12 GMT Subject: RFR: 8255987: JDI tests fail with com.sun.jdi.ObjectCollectedException Message-ID: A number of JDI tests create objects on the debugger side with calls to `newInstance()`. However, on the debugee side, these new instances will only be held on to by a `JNIGlobalWeakRef`, which means they could be collected at any time, even before `newInstace()` returns. A number of JDI tests don't take this into account, and can hence get spurious `ObjectCollectedException` thrown at them, which results in test failures. To make these objects stick around, a call to `disableCollection()` is needed (but also note that the object could have been collected by the time we call `disableCollection()`). In addition, `SDEDebuggee::executeTestMethods()` creates class loaders, which shortly after dies (and potentially gets collected). This creates problems on the debugger side, since code locations in this (now potentially unloaded class/method) gets invalidated. We must ensure that these class loaders stay alive to avoid these problems. Normally, these problems are fairly hard to provoke, since you have to be unlucky and get the timing of a GC just right. However, it's fairly easy to provoke by forcing GC cycles to happen all the time (e.g. using ZGC with -XX:ZCollectionInterval=0.01) and/or inserting `Thread.sleep()` calls right after calls to `newInstance()'. This patch fixes all instances of this problem that I managed to find. Testing: All `vmTestbase/nsk/jdi/` tests now pass, even when using the above described measures to try to provoke the problem. ------------- Commit messages: - 8255987: JDI tests fail with com.sun.jdi.ObjectCollectedException Changes: https://git.openjdk.java.net/jdk/pull/1348/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1348&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255987 Stats: 168 lines in 8 files changed: 137 ins; 0 del; 31 mod Patch: https://git.openjdk.java.net/jdk/pull/1348.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1348/head:pull/1348 PR: https://git.openjdk.java.net/jdk/pull/1348 From redestad at openjdk.java.net Fri Nov 20 21:03:14 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Fri, 20 Nov 2020 21:03:14 GMT Subject: RFR: 8256741: Reduce footprint of compiler interface data structures Message-ID: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> A few data structure in the ci allocate unconditionally created GrowableArrays out-of-line, have fields that are newer updated/read, or are unnecessarily cached. By cleaning this up we can slightly reduce memory used for JIT compilations while slightly speeding them up. ------------- Commit messages: - Copyrights, syntax polish - Adjust vmStructs/SA to changes in ciObjectFactory - Inline unconditionally created GAs in ciObjectFactory - Remove sad assert - Remove ciMethod fields cached in ciTypeFlow - Reduce footprint of compiler interface data structures Changes: https://git.openjdk.java.net/jdk/pull/1346/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1346&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256741 Stats: 226 lines in 9 files changed: 28 ins; 88 del; 110 mod Patch: https://git.openjdk.java.net/jdk/pull/1346.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1346/head:pull/1346 PR: https://git.openjdk.java.net/jdk/pull/1346 From cjplummer at openjdk.java.net Fri Nov 20 21:59:07 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 20 Nov 2020 21:59:07 GMT Subject: RFR: 8255934: JConsole 14 and greater fails to connect to older JVM In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 23:00:05 GMT, Alex Menkov wrote: > OperatingSystemMXBean was changed in jdk14 (see JDK-8226575): > New methods getTotalMemorySize and getFreeMemorySize were added, old getTotalPhysicalMemorySize and getFreePhysicalMemorySize were deprecated. > > The fix adds fallbacks for the values (i.e. if new methods fail, jconsole calls old methods) src/jdk.jconsole/share/classes/sun/tools/jconsole/SummaryTab.java line 268: > 266: sunOSMBean::getTotalPhysicalMemorySize), > 267: tryToGet(sunOSMBean::getFreeMemorySize, > 268: sunOSMBean::getFreePhysicalMemorySize), What happens if/when `getTotalPhysicalMemorySize()` and `getFreePhysicalMemorySize()` are removed. This code will fail to compile, right? I guess at that point the fix would be to try the new API, and if it throws an exception, then just use -1. In other words, there would be no way to get the value from older JVMs. ------------- PR: https://git.openjdk.java.net/jdk/pull/1243 From amenkov at openjdk.java.net Fri Nov 20 22:13:11 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 20 Nov 2020 22:13:11 GMT Subject: RFR: 8255934: JConsole 14 and greater fails to connect to older JVM In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 21:55:43 GMT, Chris Plummer wrote: >> OperatingSystemMXBean was changed in jdk14 (see JDK-8226575): >> New methods getTotalMemorySize and getFreeMemorySize were added, old getTotalPhysicalMemorySize and getFreePhysicalMemorySize were deprecated. >> >> The fix adds fallbacks for the values (i.e. if new methods fail, jconsole calls old methods) > > src/jdk.jconsole/share/classes/sun/tools/jconsole/SummaryTab.java line 268: > >> 266: sunOSMBean::getTotalPhysicalMemorySize), >> 267: tryToGet(sunOSMBean::getFreeMemorySize, >> 268: sunOSMBean::getFreePhysicalMemorySize), > > What happens if/when `getTotalPhysicalMemorySize()` and `getFreePhysicalMemorySize()` are removed. This code will fail to compile, right? I guess at that point the fix would be to try the new API, and if it throws an exception, then just use -1. In other words, there would be no way to get the value from older JVMs. Yes, that's right. If/when the methods are removed, this code will require an update too. But I think it would be better to show the values as long as the methods exist ------------- PR: https://git.openjdk.java.net/jdk/pull/1243 From cjplummer at openjdk.java.net Fri Nov 20 22:24:16 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 20 Nov 2020 22:24:16 GMT Subject: RFR: 8256741: Reduce footprint of compiler interface data structures In-Reply-To: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> References: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> Message-ID: On Fri, 20 Nov 2020 12:19:48 GMT, Claes Redestad wrote: > A few data structure in the ci allocate unconditionally created GrowableArrays out-of-line, have fields that are newer updated/read, or are unnecessarily cached. By cleaning this up we can slightly reduce memory used for JIT compilations while slightly speeding them up. src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/ci/ciObjectFactory.java line 82: > 80: public GrowableArray symbols() { > 81: Address addr = getAddress().addOffsetTo(symbolsField.getOffset()); > 82: return GrowableArray.create(addr, ciSymbolConstructor); It's unclear to me why these two changes were needed. Don't they produce the same result before and after? ------------- PR: https://git.openjdk.java.net/jdk/pull/1346 From cjplummer at openjdk.java.net Fri Nov 20 22:25:08 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 20 Nov 2020 22:25:08 GMT Subject: RFR: 8255934: JConsole 14 and greater fails to connect to older JVM In-Reply-To: References: Message-ID: <1J2e5-5yaBTkT8udi3xfGziFRM4ycTEUtFwPin04vME=.a7f675b3-cad8-4c89-9dc8-338b01f0b326@github.com> On Mon, 16 Nov 2020 23:00:05 GMT, Alex Menkov wrote: > OperatingSystemMXBean was changed in jdk14 (see JDK-8226575): > New methods getTotalMemorySize and getFreeMemorySize were added, old getTotalPhysicalMemorySize and getFreePhysicalMemorySize were deprecated. > > The fix adds fallbacks for the values (i.e. if new methods fail, jconsole calls old methods) Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1243 From redestad at openjdk.java.net Fri Nov 20 22:42:02 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Fri, 20 Nov 2020 22:42:02 GMT Subject: RFR: 8256741: Reduce footprint of compiler interface data structures In-Reply-To: References: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> Message-ID: On Fri, 20 Nov 2020 22:21:04 GMT, Chris Plummer wrote: >> A few data structure in the ci allocate unconditionally created GrowableArrays out-of-line, have fields that are newer updated/read, or are unnecessarily cached. By cleaning this up we can slightly reduce memory used for JIT compilations while slightly speeding them up. > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/ci/ciObjectFactory.java line 82: > >> 80: public GrowableArray symbols() { >> 81: Address addr = getAddress().addOffsetTo(symbolsField.getOffset()); >> 82: return GrowableArray.create(addr, ciSymbolConstructor); > > It's unclear to me why these two changes were needed. Don't they produce the same result before and after? When changing the fields from `GrowableArray<..>*` to `GrowableArray<..>` I looked around and found another use of an embedded `GrowableArray` in `InlineTree.java` and took my cues from that. If you're certain these approaches are semantically identical I can drop these changes, but as I'm not exactly sure how to verify and test this I went the safe(?) route of leaning on prior art here. ------------- PR: https://git.openjdk.java.net/jdk/pull/1346 From cjplummer at openjdk.java.net Fri Nov 20 22:53:59 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 20 Nov 2020 22:53:59 GMT Subject: RFR: 8256741: Reduce footprint of compiler interface data structures In-Reply-To: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> References: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> Message-ID: On Fri, 20 Nov 2020 12:19:48 GMT, Claes Redestad wrote: > A few data structure in the ci allocate unconditionally created GrowableArrays out-of-line, have fields that are newer updated/read, or are unnecessarily cached. By cleaning this up we can slightly reduce memory used for JIT compilations while slightly speeding them up. Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1346 From cjplummer at openjdk.java.net Fri Nov 20 22:54:01 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 20 Nov 2020 22:54:01 GMT Subject: RFR: 8256741: Reduce footprint of compiler interface data structures In-Reply-To: References: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> Message-ID: On Fri, 20 Nov 2020 22:39:19 GMT, Claes Redestad wrote: >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/ci/ciObjectFactory.java line 82: >> >>> 80: public GrowableArray symbols() { >>> 81: Address addr = getAddress().addOffsetTo(symbolsField.getOffset()); >>> 82: return GrowableArray.create(addr, ciSymbolConstructor); >> >> It's unclear to me why these two changes were needed. Don't they produce the same result before and after? > > When changing the fields from `GrowableArray<..>*` to `GrowableArray<..>` I looked around and found another use of an embedded `GrowableArray` in `InlineTree.java` and took my cues from that. If you're certain these approaches are semantically identical I can drop these changes, but as I'm not exactly sure how to verify and test this I went the safe(?) route of leaning on prior art here. Ah, I see now. I didn't pick up on the change from a pointer type to an embedded type. Yes, I think your changes are correct and necessary. Rather than plucking the pointer value out of the field, you are now computing the address of the field, which seems like the right thing to be doing when changing the field from a pointer type to an embedded type. Consider the SA changes reviewed. I'm not reviewing any of the hotspot changes. ------------- PR: https://git.openjdk.java.net/jdk/pull/1346 From kvn at openjdk.java.net Fri Nov 20 23:14:15 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 20 Nov 2020 23:14:15 GMT Subject: RFR: 8256741: Reduce footprint of compiler interface data structures In-Reply-To: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> References: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> Message-ID: <4Ifnx25KPdlz3Q8zgyahVXCgLoz3F4kYijl1tBzy__g=.21062af5-be49-485c-8c16-30e69feef668@github.com> On Fri, 20 Nov 2020 12:19:48 GMT, Claes Redestad wrote: > A few data structure in the ci allocate unconditionally created GrowableArrays out-of-line, have fields that are newer updated/read, or are unnecessarily cached. By cleaning this up we can slightly reduce memory used for JIT compilations while slightly speeding them up. Good. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1346 From sspitsyn at openjdk.java.net Fri Nov 20 23:30:10 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 20 Nov 2020 23:30:10 GMT Subject: RFR: 8255934: JConsole 14 and greater fails to connect to older JVM In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 23:00:05 GMT, Alex Menkov wrote: > OperatingSystemMXBean was changed in jdk14 (see JDK-8226575): > New methods getTotalMemorySize and getFreeMemorySize were added, old getTotalPhysicalMemorySize and getFreePhysicalMemorySize were deprecated. > > The fix adds fallbacks for the values (i.e. if new methods fail, jconsole calls old methods) Hi Alex, It looks good to me. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1243 From amenkov at openjdk.java.net Fri Nov 20 23:36:02 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 20 Nov 2020 23:36:02 GMT Subject: Integrated: 8255934: JConsole 14 and greater fails to connect to older JVM In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 23:00:05 GMT, Alex Menkov wrote: > OperatingSystemMXBean was changed in jdk14 (see JDK-8226575): > New methods getTotalMemorySize and getFreeMemorySize were added, old getTotalPhysicalMemorySize and getFreePhysicalMemorySize were deprecated. > > The fix adds fallbacks for the values (i.e. if new methods fail, jconsole calls old methods) This pull request has now been integrated. Changeset: 14de791d Author: Alex Menkov URL: https://git.openjdk.java.net/jdk/commit/14de791d Stats: 22 lines in 1 file changed: 20 ins; 0 del; 2 mod 8255934: JConsole 14 and greater fails to connect to older JVM Reviewed-by: cjplummer, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/1243 From sspitsyn at openjdk.java.net Fri Nov 20 23:51:13 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 20 Nov 2020 23:51:13 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v5] In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 01:17:18 GMT, Lin Zang wrote: >> This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. >> >> example: >> jmap -dump:live,gz=1,file=dump.gz > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > remove unnecessory arguments in verifyDump() Hi Lin, The fix looks okay to me. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1251 From amenkov at openjdk.java.net Fri Nov 20 23:55:01 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 20 Nov 2020 23:55:01 GMT Subject: RFR: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 22:26:38 GMT, Chris Plummer wrote: >> Looks good. > > Why does this PR show 5 commits, including one that looks like it was for a previously fixed CR? It looks maybe you reused a branch. In the end it looks like skara did the right thing, and the diff looks right, but this seems to be something it is best to avoid with a PR. A rebase and a forced push might resolve this. Something is wrong with my personal fork. To avoid issues I closing this PR and will recreate PF ------------- PR: https://git.openjdk.java.net/jdk/pull/1330 From amenkov at openjdk.java.net Fri Nov 20 23:55:02 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 20 Nov 2020 23:55:02 GMT Subject: Withdrawn: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 21:49:09 GMT, Alex Menkov wrote: > - fixed java part of the test to handle spurious wakeups in the debuggee thread; > - fixed native part of the test to detect and report the case when debuggee thread not found. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/1330 From lzang at openjdk.java.net Sat Nov 21 01:02:03 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Sat, 21 Nov 2020 01:02:03 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v5] In-Reply-To: <1hbNKihSyBpa4DqU29wDLspPPf-U7WckTC-yl4iACfE=.5f61ac04-6318-45e3-8a8f-ec2dbad51d95@github.com> References: <1hbNKihSyBpa4DqU29wDLspPPf-U7WckTC-yl4iACfE=.5f61ac04-6318-45e3-8a8f-ec2dbad51d95@github.com> Message-ID: <-xM2Hw4tXC8UUkkDuyrvNWTkIHhfulTmn_8WKsUpLZU=.796b2cb4-5695-47ef-8c42-229698f70df9@github.com> On Fri, 20 Nov 2020 02:57:55 GMT, Chris Plummer wrote: >> Lin Zang has updated the pull request incrementally with one additional commit since the last revision: >> >> remove unnecessory arguments in verifyDump() > > Marked as reviewed by cjplummer (Reviewer). Thank @plummercj and @sspitsyn for your help reviewing this PR. I will add /integrate label, and may I ask your help to sponse pushing it? BRs, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From amenkov at openjdk.java.net Sat Nov 21 02:00:22 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Sat, 21 Nov 2020 02:00:22 GMT Subject: RFR: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" Message-ID: Resending the RFR without unexpected extra commits - fixed java part of the test to handle spurious wakeups in the debuggee thread; - fixed native part of the test to detect and report the case when debuggee thread not found. ------------- Commit messages: - Fixed cm01t002 Changes: https://git.openjdk.java.net/jdk/pull/1361/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1361&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256364 Stats: 17 lines in 2 files changed: 14 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1361.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1361/head:pull/1361 PR: https://git.openjdk.java.net/jdk/pull/1361 From cjplummer at openjdk.java.net Sat Nov 21 03:25:01 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Sat, 21 Nov 2020 03:25:01 GMT Subject: RFR: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" In-Reply-To: References: Message-ID: On Sat, 21 Nov 2020 01:54:58 GMT, Alex Menkov wrote: > Resending the RFR without unexpected extra commits > > - fixed java part of the test to handle spurious wakeups in the debuggee thread; > - fixed native part of the test to detect and report the case when debuggee thread not found. Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1361 From sspitsyn at openjdk.java.net Sat Nov 21 05:09:17 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Sat, 21 Nov 2020 05:09:17 GMT Subject: RFR: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" In-Reply-To: References: Message-ID: On Sat, 21 Nov 2020 01:54:58 GMT, Alex Menkov wrote: > Resending the RFR without unexpected extra commits > > - fixed java part of the test to handle spurious wakeups in the debuggee thread; > - fixed native part of the test to detect and report the case when debuggee thread not found. It looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1361 From redestad at openjdk.java.net Mon Nov 23 10:20:59 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Mon, 23 Nov 2020 10:20:59 GMT Subject: RFR: 8256741: Reduce footprint of compiler interface data structures In-Reply-To: References: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> Message-ID: On Fri, 20 Nov 2020 22:51:23 GMT, Chris Plummer wrote: >> A few data structure in the ci allocate unconditionally created GrowableArrays out-of-line, have fields that are newer updated/read, or are unnecessarily cached. By cleaning this up we can slightly reduce memory used for JIT compilations while slightly speeding them up. > > Marked as reviewed by cjplummer (Reviewer). @plummercj @vnkozlov - thank you for reviewing ------------- PR: https://git.openjdk.java.net/jdk/pull/1346 From redestad at openjdk.java.net Mon Nov 23 10:21:00 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Mon, 23 Nov 2020 10:21:00 GMT Subject: Integrated: 8256741: Reduce footprint of compiler interface data structures In-Reply-To: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> References: <_ueFk7sJ0AZFFkyoMgEl-st9DaN-8PRJWPnBHfXpRn0=.cd50148b-3ed0-437a-8508-adbe2a104a11@github.com> Message-ID: On Fri, 20 Nov 2020 12:19:48 GMT, Claes Redestad wrote: > A few data structure in the ci allocate unconditionally created GrowableArrays out-of-line, have fields that are newer updated/read, or are unnecessarily cached. By cleaning this up we can slightly reduce memory used for JIT compilations while slightly speeding them up. This pull request has now been integrated. Changeset: c0689d25 Author: Claes Redestad URL: https://git.openjdk.java.net/jdk/commit/c0689d25 Stats: 226 lines in 9 files changed: 28 ins; 88 del; 110 mod 8256741: Reduce footprint of compiler interface data structures Reviewed-by: cjplummer, kvn ------------- PR: https://git.openjdk.java.net/jdk/pull/1346 From sgehwolf at openjdk.java.net Mon Nov 23 15:55:09 2020 From: sgehwolf at openjdk.java.net (Severin Gehwolf) Date: Mon, 23 Nov 2020 15:55:09 GMT Subject: RFR: 8254001: [Metrics] Enhance parsing of cgroup interface files for version detection Message-ID: <52vN8j9B_6s6amMHb8b_1QKyhwjCvThIz5T17aNix28=.31113ef4-61d0-4525-a3df-9aa20d408935@github.com> This is an enhancement which solves two issues: 1. Multiple reads of relevant cgroup interface files. Now interface files are only read once per file (just like Hotspot). 2. Proxies creation of the impl specific subsystem via `determineType()` as before, but now reads all relevant interface files: `/proc/cgroups`, `/proc/self/mountinfo` and `/proc/self/cgroup`. Once read it passes the parsed information to the impl specific subsystem classes for instantiation. This allows for more flexibility of testing as interface files can be mocked and, thus, more cases can be tested that way without having access to these specific systems. For example, proper regression tests for JDK-8217766 and JDK-8253435 have been added now with this in place. * [x] Tested on Linux x86_64 on cgroups v1 and cgroups v2. Container tests pass. ------------- Commit messages: - 8254001: [Metrics] Enhance parsing of cgroup interface files for version detection Changes: https://git.openjdk.java.net/jdk/pull/1393/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1393&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8254001 Stats: 537 lines in 5 files changed: 268 ins; 193 del; 76 mod Patch: https://git.openjdk.java.net/jdk/pull/1393.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1393/head:pull/1393 PR: https://git.openjdk.java.net/jdk/pull/1393 From sgehwolf at openjdk.java.net Mon Nov 23 15:55:10 2020 From: sgehwolf at openjdk.java.net (Severin Gehwolf) Date: Mon, 23 Nov 2020 15:55:10 GMT Subject: RFR: 8254001: [Metrics] Enhance parsing of cgroup interface files for version detection In-Reply-To: <52vN8j9B_6s6amMHb8b_1QKyhwjCvThIz5T17aNix28=.31113ef4-61d0-4525-a3df-9aa20d408935@github.com> References: <52vN8j9B_6s6amMHb8b_1QKyhwjCvThIz5T17aNix28=.31113ef4-61d0-4525-a3df-9aa20d408935@github.com> Message-ID: On Mon, 23 Nov 2020 15:46:56 GMT, Severin Gehwolf wrote: > This is an enhancement which solves two issues: > > 1. Multiple reads of relevant cgroup interface files. Now interface files are only read once per file (just like Hotspot). > 2. Proxies creation of the impl specific subsystem via `determineType()` as before, but now reads all relevant interface files: `/proc/cgroups`, `/proc/self/mountinfo` and `/proc/self/cgroup`. Once read it passes the parsed information to the impl specific subsystem classes for instantiation. This allows for more flexibility of testing as interface files can be mocked and, thus, more cases can be tested that way without having access to these specific systems. For example, proper regression tests for JDK-8217766 and JDK-8253435 have been added now with this in place. > > * [x] Tested on Linux x86_64 on cgroups v1 and cgroups v2. Container tests pass. @poonamparhar Perhaps you want to take a look at this in the context of not regressing JDK-8255908? Thanks! src/java.base/linux/classes/jdk/internal/platform/CgroupSubsystemFactory.java line 268: > 266: info.setMountPoint(mountPath); > 267: info.setMountRoot(mountRoot); > 268: } This is the actual fix of JDK-8253435 for the Java side. ------------- PR: https://git.openjdk.java.net/jdk/pull/1393 From sgehwolf at openjdk.java.net Mon Nov 23 16:01:58 2020 From: sgehwolf at openjdk.java.net (Severin Gehwolf) Date: Mon, 23 Nov 2020 16:01:58 GMT Subject: RFR: 8254001: [Metrics] Enhance parsing of cgroup interface files for version detection In-Reply-To: <52vN8j9B_6s6amMHb8b_1QKyhwjCvThIz5T17aNix28=.31113ef4-61d0-4525-a3df-9aa20d408935@github.com> References: <52vN8j9B_6s6amMHb8b_1QKyhwjCvThIz5T17aNix28=.31113ef4-61d0-4525-a3df-9aa20d408935@github.com> Message-ID: On Mon, 23 Nov 2020 15:46:56 GMT, Severin Gehwolf wrote: > This is an enhancement which solves two issues: > > 1. Multiple reads of relevant cgroup interface files. Now interface files are only read once per file (just like Hotspot). > 2. Proxies creation of the impl specific subsystem via `determineType()` as before, but now reads all relevant interface files: `/proc/cgroups`, `/proc/self/mountinfo` and `/proc/self/cgroup`. Once read it passes the parsed information to the impl specific subsystem classes for instantiation. This allows for more flexibility of testing as interface files can be mocked and, thus, more cases can be tested that way without having access to these specific systems. For example, proper regression tests for JDK-8217766 and JDK-8253435 have been added now with this in place. > > * [x] Tested on Linux x86_64 on cgroups v1 and cgroups v2. Container tests pass. test/jdk/jdk/internal/platform/cgroup/TestCgroupSubsystemFactory.java line 238: > 236: assertFalse("Join controller combination expected as cgroups v1", res.isCgroupV2()); > 237: CgroupInfo memoryInfo = res.getInfos().get("memory"); > 238: assertEquals("/user.slice/user-1000.slice/session-3.scope", memoryInfo.getCgroupPath()); The gist of the Java equivalent of JDK-8253939 (which fixed this for Hotspot). I.e. a regression test for JDK-8217766 and JDK-8254854. test/jdk/jdk/internal/platform/cgroup/TestCgroupSubsystemFactory.java line 267: > 265: assertFalse("Duplicate cpusets should not influence detection heuristic", res.isCgroupV2()); > 266: CgroupInfo cpuSetInfo = res.getInfos().get("cpuset"); > 267: assertEquals("/sys/fs/cgroup/cpuset", cpuSetInfo.getMountPoint()); We can now assert the proper mount point is being used for multiple cpuset mounts. ------------- PR: https://git.openjdk.java.net/jdk/pull/1393 From poonam at openjdk.java.net Mon Nov 23 19:41:57 2020 From: poonam at openjdk.java.net (Poonam Bajaj) Date: Mon, 23 Nov 2020 19:41:57 GMT Subject: RFR: 8254001: [Metrics] Enhance parsing of cgroup interface files for version detection In-Reply-To: References: <52vN8j9B_6s6amMHb8b_1QKyhwjCvThIz5T17aNix28=.31113ef4-61d0-4525-a3df-9aa20d408935@github.com> Message-ID: On Mon, 23 Nov 2020 15:50:18 GMT, Severin Gehwolf wrote: >> This is an enhancement which solves two issues: >> >> 1. Multiple reads of relevant cgroup interface files. Now interface files are only read once per file (just like Hotspot). >> 2. Proxies creation of the impl specific subsystem via `determineType()` as before, but now reads all relevant interface files: `/proc/cgroups`, `/proc/self/mountinfo` and `/proc/self/cgroup`. Once read it passes the parsed information to the impl specific subsystem classes for instantiation. This allows for more flexibility of testing as interface files can be mocked and, thus, more cases can be tested that way without having access to these specific systems. For example, proper regression tests for JDK-8217766 and JDK-8253435 have been added now with this in place. >> >> * [x] Tested on Linux x86_64 on cgroups v1 and cgroups v2. Container tests pass. > > @poonamparhar Perhaps you want to take a look at this in the context of not regressing JDK-8255908? Thanks! With respect to JDK-8255908, the changes look good to me. ------------- PR: https://git.openjdk.java.net/jdk/pull/1393 From sgehwolf at openjdk.java.net Mon Nov 23 20:00:59 2020 From: sgehwolf at openjdk.java.net (Severin Gehwolf) Date: Mon, 23 Nov 2020 20:00:59 GMT Subject: RFR: 8254001: [Metrics] Enhance parsing of cgroup interface files for version detection In-Reply-To: References: <52vN8j9B_6s6amMHb8b_1QKyhwjCvThIz5T17aNix28=.31113ef4-61d0-4525-a3df-9aa20d408935@github.com> Message-ID: On Mon, 23 Nov 2020 19:56:56 GMT, Severin Gehwolf wrote: >> With respect to JDK-8255908, the changes look good to me. > >> With respect to JDK-8255908, the changes look good to me. > > Thanks! @bobvandette Please review when you've got some cycles to spare. Much appreciated! ------------- PR: https://git.openjdk.java.net/jdk/pull/1393 From sgehwolf at openjdk.java.net Mon Nov 23 20:00:58 2020 From: sgehwolf at openjdk.java.net (Severin Gehwolf) Date: Mon, 23 Nov 2020 20:00:58 GMT Subject: RFR: 8254001: [Metrics] Enhance parsing of cgroup interface files for version detection In-Reply-To: References: <52vN8j9B_6s6amMHb8b_1QKyhwjCvThIz5T17aNix28=.31113ef4-61d0-4525-a3df-9aa20d408935@github.com> Message-ID: On Mon, 23 Nov 2020 19:39:18 GMT, Poonam Bajaj wrote: > With respect to JDK-8255908, the changes look good to me. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1393 From ysuenaga at openjdk.java.net Tue Nov 24 05:12:26 2020 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 24 Nov 2020 05:12:26 GMT Subject: RFR: 8252657: JVMTI agent is not unloaded when Agent_OnAttach is failed [v2] In-Reply-To: <1H1wUQdxCLU2qddqEIYSx2iOhIKL3b5etUmjsS6NBlU=.0bf1fe0c-8dcf-4ca0-bd57-b8794d5f2810@github.com> References: <1H1wUQdxCLU2qddqEIYSx2iOhIKL3b5etUmjsS6NBlU=.0bf1fe0c-8dcf-4ca0-bd57-b8794d5f2810@github.com> Message-ID: > If `Agent_OnAttach()` in JVMTI agent which is attempted to load via JVMTI.agent_load dcmd is failed, it would not be unloaded. > We've [discussed it on serviceability-dev](https://mail.openjdk.java.net/pipermail/serviceability-dev/2020-September/032839.html). This PR is a continuation of that. > > This PR also includes to call `Agent_OnUnload()` when `Agent_OnAttach()` failed. > > How to reproduce: > > 1. Build JVMTI agent for test > $ git clone https://github.com/YaSuenag/jvmti-examples.git > $ cd jvmti-examples/helloworld/out/build > $ cmake ../.. > > 2. Run JShell > > 3. Load JVMTI agent via `jcmd JVMTI.agent_load` with "error" ("error" means `Agent_OnAttach()` returns JNI_ERR) > $ jcmd > 89456 jdk.jshell.execution.RemoteExecutionControl 45651 > 89547 sun.tools.jcmd.JCmd > 89436 jdk.jshell/jdk.internal.jshell.tool.JShellToolProvider > $ jcmd 89436 JVMTI.agent_load `pwd`/libhelloworld.so error > 89436: > return code: -1 > > 4. Check loaded libraries via `jcmd VM.dynlibs` > $ jcmd 89436 VM.dynlibs | grep libhelloworld > 7f2f8b06b000-7f2f8b06c000 r--p 00000000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so > 7f2f8b06c000-7f2f8b06d000 r-xp 00001000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so > 7f2f8b06d000-7f2f8b06e000 r--p 00002000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so > 7f2f8b06e000-7f2f8b06f000 r--p 00002000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so > 7f2f8b06f000-7f2f8b070000 rw-p 00003000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so Yasumasa Suenaga has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Update patch - Merge remote-tracking branch 'upstream/master' into JDK-8252657 - revert - JVMTI agent is not unloaded when Agent_OnAttach is failed ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/19/files - new: https://git.openjdk.java.net/jdk/pull/19/files/3cc731a9..640014dc Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=19&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=19&range=00-01 Stats: 715494 lines in 6740 files changed: 549418 ins; 117424 del; 48652 mod Patch: https://git.openjdk.java.net/jdk/pull/19.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/19/head:pull/19 PR: https://git.openjdk.java.net/jdk/pull/19 From ysuenaga at openjdk.java.net Tue Nov 24 05:45:55 2020 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 24 Nov 2020 05:45:55 GMT Subject: RFR: 8252657: JVMTI agent is not unloaded when Agent_OnAttach is failed In-Reply-To: References: <1H1wUQdxCLU2qddqEIYSx2iOhIKL3b5etUmjsS6NBlU=.0bf1fe0c-8dcf-4ca0-bd57-b8794d5f2810@github.com> <80LJDTCsT_y-KlThryd5Bxu5RRyrjmKfs5p9vJUn61E=.68b594a0-fe58-4f4d-a49c-eec2e90f9373@github.com> <-Xrp6c94000-jE1p6NvzjsxFUW5ILrH_F1eT1i7esw8=.9d609f81-1b61-4ebf-9afd-73b834c1b18c@github.com> Message-ID: <96QB1WTG_puWZM9TqQusV_43lTbHoGxCIJKBpAG43o0=.0e67d662-7fd6-46d1-a571-81604669efb7@github.com> On Thu, 12 Nov 2020 01:34:33 GMT, Yasumasa Suenaga wrote: >> If we can change the spec that agent library would not be unloaded when `Agent_OnAttach()` failed, we can change like [webrev.00](https://cr.openjdk.java.net/~ysuenaga/JDK-8252657/webrev.00/). It is simple, and similar behavior with `Agent_OnLoad()`. It might be prefer for JVMTI agent developers. > > In case of `Agent_OnLoad()`, if it is failed (it returns other than zero), JVM is aborted and `Agent_OnUnload()` is not called. I think it is compliant with [JVMTI spec of Agent_OnUnload()](https://docs.oracle.com/en/java/javase/15/docs/specs/jvmti.html#onunload) which says uncontrolled shutdown (aborting JVM) is an exception to this rule. > > I will add CSR for this fix, but I want to discuss what we should do before. I like that `Agent_OnUnload()` wouldn't be called when `Agent_OnAttach()` is failed if we can change the spec - it is consistent and friendly with `Agent_OnLoad()`. I added [CSR for this PR](https://wiki.openjdk.java.net/display/csr/Main) and updated patch. Could you review it? I think it is reasonable to clarify the spec not to call `Agent_OnUnload()` when `Agent_OnLoad()` or `Agent_OnAttach()` are failed. It is nature for me, and it does not change current behavior in HotSpot. ------------- PR: https://git.openjdk.java.net/jdk/pull/19 From amenkov at openjdk.java.net Tue Nov 24 17:53:58 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Tue, 24 Nov 2020 17:53:58 GMT Subject: Integrated: 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" In-Reply-To: References: Message-ID: On Sat, 21 Nov 2020 01:54:58 GMT, Alex Menkov wrote: > Resending the RFR without unexpected extra commits > > - fixed java part of the test to handle spurious wakeups in the debuggee thread; > - fixed native part of the test to detect and report the case when debuggee thread not found. This pull request has now been integrated. Changeset: 2a1e9be6 Author: Alex Menkov URL: https://git.openjdk.java.net/jdk/commit/2a1e9be6 Stats: 17 lines in 2 files changed: 14 ins; 0 del; 3 mod 8256364: vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002 failed with "assert(handle != __null) failed: JNI handle should not be null" Reviewed-by: cjplummer, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/1361 From phh at openjdk.java.net Wed Nov 25 16:54:58 2020 From: phh at openjdk.java.net (Paul Hohensee) Date: Wed, 25 Nov 2020 16:54:58 GMT Subject: RFR: 8256450: Add gz option to jmap to write a gzipped heap dump [v5] In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 01:17:18 GMT, Lin Zang wrote: >> This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. >> >> example: >> jmap -dump:live,gz=1,file=dump.gz > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > remove unnecessory arguments in verifyDump() Marked as reviewed by phh (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From lzang at openjdk.java.net Wed Nov 25 16:54:59 2020 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 25 Nov 2020 16:54:59 GMT Subject: Integrated: 8256450: Add gz option to jmap to write a gzipped heap dump In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 08:39:43 GMT, Lin Zang wrote: > This PR add "gz" option to jmap -dump command to support generate gzipped heap dump. > > example: > jmap -dump:live,gz=1,file=dump.gz This pull request has now been integrated. Changeset: 461c5fc6 Author: Lin Zang Committer: Paul Hohensee URL: https://git.openjdk.java.net/jdk/commit/461c5fc6 Stats: 56 lines in 3 files changed: 47 ins; 0 del; 9 mod 8256450: Add gz option to jmap to write a gzipped heap dump Reviewed-by: cjplummer, sspitsyn, phh ------------- PR: https://git.openjdk.java.net/jdk/pull/1251 From coleenp at openjdk.java.net Wed Nov 25 18:47:08 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 25 Nov 2020 18:47:08 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" Message-ID: The ServiceThread cleaning used a stale ObjectFree state when calling remove_dead_entries, because another thread had concurrently set is_enabled to false. Add a lock around setting/resetting the lock event state and retest the state under a lock. Ran the test 100s of time without failure, where otherwise it fails very quickly. Tested with tier2,3 and running tiers 4,5,6 in progress. Thanks to Kim for his previous feedback. ------------- Commit messages: - 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" Changes: https://git.openjdk.java.net/jdk/pull/1439/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1439&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256830 Stats: 25 lines in 3 files changed: 23 ins; 1 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1439.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1439/head:pull/1439 PR: https://git.openjdk.java.net/jdk/pull/1439 From amenkov at openjdk.java.net Wed Nov 25 21:41:02 2020 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 25 Nov 2020 21:41:02 GMT Subject: RFR: 8256808: com/sun/jdi/CatchAllTest.java failed with "NullPointerException: Cannot invoke "lib.jdb.Jdb.log(String)" because "this.jdb" is null" Message-ID: JdbTest can get exception before jdb field is initialized. As Jdb logging does not depend on the instance, made Jdb.log method static ------------- Commit messages: - Fixed copyright years - Fixed test initialization logging Changes: https://git.openjdk.java.net/jdk/pull/1443/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1443&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256808 Stats: 6 lines in 2 files changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/1443.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1443/head:pull/1443 PR: https://git.openjdk.java.net/jdk/pull/1443 From kbarrett at openjdk.java.net Wed Nov 25 23:34:01 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 25 Nov 2020 23:34:01 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" In-Reply-To: References: Message-ID: <1NGa5DMWUHpmh3JWelXH4X2bQRt7pymebSQ19tywORM=.95e3ef25-e234-4daf-89f3-ea481b9c2266@github.com> On Wed, 25 Nov 2020 18:40:49 GMT, Coleen Phillimore wrote: > The ServiceThread cleaning used a stale ObjectFree state when calling remove_dead_entries, because another thread had concurrently set is_enabled to false. Add a lock around setting/resetting the lock event state and retest the state under a lock. Ran the test 100s of time without failure, where otherwise it fails very quickly. > Tested with tier2,3 and running tiers 4,5,6 in progress. > Thanks to Kim for his previous feedback. Marked as reviewed by kbarrett (Reviewer). src/hotspot/share/prims/jvmtiEventController.cpp line 463: > 461: ((now_enabled & OBJECT_FREE_BIT)) != 0) { > 462: // Set/reset the event enabled under the tagmap lock. > 463: set_enabled_events_with_lock(env, now_enabled); You could tighten up the test to only handle specially when the state of the ObjectFree event is changing, i.e. (((was_enabled ^ now_enabled) & OBJECT_FREE_BIT) != 0)``` Or you could not bother with the conditionalization at all, and just always call set_enabled_events_with_lock; I bet nobody would notice any performance difference. That would eliminate "benign" races between unlocked bit setting here and bit testing in remove_dead_entries_locked. Of course, the current implementation has such races on these bits all over the place; what's one race more or less among friends... Or you could just leave it as you have it. Your call. ------------- PR: https://git.openjdk.java.net/jdk/pull/1439 From pliden at openjdk.java.net Thu Nov 26 10:35:58 2020 From: pliden at openjdk.java.net (Per Liden) Date: Thu, 26 Nov 2020 10:35:58 GMT Subject: RFR: 8255987: JDI tests fail with com.sun.jdi.ObjectCollectedException In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 13:23:28 GMT, Per Liden wrote: > A number of JDI tests create objects on the debugger side with calls to `newInstance()`. However, on the debugee side, these new instances will only be held on to by a `JNIGlobalWeakRef`, which means they could be collected at any time, even before `newInstace()` returns. A number of JDI tests don't take this into account, and can hence get spurious `ObjectCollectedException` thrown at them, which results in test failures. To make these objects stick around, a call to `disableCollection()` is needed (but also note that the object could have been collected by the time we call `disableCollection()`). > > In addition, `SDEDebuggee::executeTestMethods()` creates class loaders, which shortly after dies (and potentially gets collected). This creates problems on the debugger side, since code locations in this (now potentially unloaded class/method) gets invalidated. We must ensure that these class loaders stay alive to avoid these problems. > > Normally, these problems are fairly hard to provoke, since you have to be unlucky and get the timing of a GC just right. However, it's fairly easy to provoke by forcing GC cycles to happen all the time (e.g. using ZGC with -XX:ZCollectionInterval=0.01) and/or inserting `Thread.sleep()` calls right after calls to `newInstance()`. > > This patch fixes all instances of this problem that I managed to find. > > Testing: All `vmTestbase/nsk/jdi/` tests now pass, even when using the above described measures to try to provoke the problem. Just a friendly ping. Still looking for reviewers for this fix. ------------- PR: https://git.openjdk.java.net/jdk/pull/1348 From shade at openjdk.java.net Thu Nov 26 18:06:59 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 26 Nov 2020 18:06:59 GMT Subject: RFR: 8256808: com/sun/jdi/CatchAllTest.java failed with "NullPointerException: Cannot invoke "lib.jdb.Jdb.log(String)" because "this.jdb" is null" In-Reply-To: References: Message-ID: On Wed, 25 Nov 2020 21:36:12 GMT, Alex Menkov wrote: > JdbTest can get exception before jdb field is initialized. > As Jdb logging does not depend on the instance, made Jdb.log method static test/jdk/com/sun/jdi/lib/jdb/JdbTest.java line 99: > 97: Jdb.log("======================================="); > 98: Jdb.log("Exception thrown during test execution: " + e.getMessage()); > 99: Jdb.log("======================================="); Why not just print it to `System.out` then, eliminating the method altogether? ------------- PR: https://git.openjdk.java.net/jdk/pull/1443 From dholmes at openjdk.java.net Fri Nov 27 02:29:57 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Fri, 27 Nov 2020 02:29:57 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" In-Reply-To: References: Message-ID: <4GRPSMKxwFONrO-YzHXh7z8WxPHJM2tP0Y5X5uWnV10=.03265f2f-a4f7-4ef0-99e3-37701aa6b7fa@github.com> On Wed, 25 Nov 2020 18:40:49 GMT, Coleen Phillimore wrote: > The ServiceThread cleaning used a stale ObjectFree state when calling remove_dead_entries, because another thread had concurrently set is_enabled to false. Add a lock around setting/resetting the lock event state and retest the state under a lock. Ran the test 100s of time without failure, where otherwise it fails very quickly. > Tested with tier2,3 and running tiers 4,5,6 in progress. > Thanks to Kim for his previous feedback. src/hotspot/share/prims/jvmtiTagMap.cpp line 1162: > 1160: if (_needs_cleaning) { > 1161: // Recheck whether to post object free events under the lock. > 1162: post_object_free = post_object_free && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE); Where is `is_enabled` called without the lock being held in a caller of `remove_dead_entries()`? ------------- PR: https://git.openjdk.java.net/jdk/pull/1439 From coleenp at openjdk.java.net Mon Nov 30 12:59:58 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 30 Nov 2020 12:59:58 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" In-Reply-To: <1NGa5DMWUHpmh3JWelXH4X2bQRt7pymebSQ19tywORM=.95e3ef25-e234-4daf-89f3-ea481b9c2266@github.com> References: <1NGa5DMWUHpmh3JWelXH4X2bQRt7pymebSQ19tywORM=.95e3ef25-e234-4daf-89f3-ea481b9c2266@github.com> Message-ID: On Wed, 25 Nov 2020 23:25:55 GMT, Kim Barrett wrote: >> The ServiceThread cleaning used a stale ObjectFree state when calling remove_dead_entries, because another thread had concurrently set is_enabled to false. Add a lock around setting/resetting the lock event state and retest the state under a lock. Ran the test 100s of time without failure, where otherwise it fails very quickly. >> Tested with tier2,3 and running tiers 4,5,6 in progress. >> Thanks to Kim for his previous feedback. > > src/hotspot/share/prims/jvmtiEventController.cpp line 463: > >> 461: ((now_enabled & OBJECT_FREE_BIT)) != 0) { >> 462: // Set/reset the event enabled under the tagmap lock. >> 463: set_enabled_events_with_lock(env, now_enabled); > > You could tighten up the test to only handle specially when the state of the ObjectFree event is changing, i.e. > > (((was_enabled ^ now_enabled) & OBJECT_FREE_BIT) != 0)``` > > Or you could not bother with the conditionalization at all, and just always call set_enabled_events_with_lock; I bet nobody would notice any performance difference. That would eliminate "benign" races between unlocked bit setting here and bit testing in remove_dead_entries_locked. Of course, the current implementation has such races on these bits all over the place; what's one race more or less among friends... > > Or you could just leave it as you have it. Your call. I like the idea of reseting the enabled bits under a lock unconditionally. I'll do that. ------------- PR: https://git.openjdk.java.net/jdk/pull/1439 From coleenp at openjdk.java.net Mon Nov 30 13:00:00 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 30 Nov 2020 13:00:00 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" In-Reply-To: <4GRPSMKxwFONrO-YzHXh7z8WxPHJM2tP0Y5X5uWnV10=.03265f2f-a4f7-4ef0-99e3-37701aa6b7fa@github.com> References: <4GRPSMKxwFONrO-YzHXh7z8WxPHJM2tP0Y5X5uWnV10=.03265f2f-a4f7-4ef0-99e3-37701aa6b7fa@github.com> Message-ID: On Fri, 27 Nov 2020 02:26:44 GMT, David Holmes wrote: >> The ServiceThread cleaning used a stale ObjectFree state when calling remove_dead_entries, because another thread had concurrently set is_enabled to false. Add a lock around setting/resetting the lock event state and retest the state under a lock. Ran the test 100s of time without failure, where otherwise it fails very quickly. >> Tested with tier2,3 and running tiers 4,5,6 in progress. >> Thanks to Kim for his previous feedback. > > src/hotspot/share/prims/jvmtiTagMap.cpp line 1162: > >> 1160: if (_needs_cleaning) { >> 1161: // Recheck whether to post object free events under the lock. >> 1162: post_object_free = post_object_free && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE); > > Where is `is_enabled` called without the lock being held in a caller of `remove_dead_entries()`? void JvmtiTagMap::flush_object_free_events() { assert_not_at_safepoint(); if (env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { Called by JVMTI to disable events and called by the service thread. And here for get_objects_with_tags: if (collector.some_dead_found() && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { post_dead_objects_on_vm_thread(); } ------------- PR: https://git.openjdk.java.net/jdk/pull/1439 From coleenp at openjdk.java.net Mon Nov 30 13:57:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 30 Nov 2020 13:57:14 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" [v2] In-Reply-To: References: Message-ID: <5Ed2eVORB8pJ0BmKISasvgLZCT4HIjJt-YA4sjZZW0k=.96a9d20c-f931-493a-83c2-2444e2646896@github.com> > The ServiceThread cleaning used a stale ObjectFree state when calling remove_dead_entries, because another thread had concurrently set is_enabled to false. Add a lock around setting/resetting the lock event state and retest the state under a lock. Ran the test 100s of time without failure, where otherwise it fails very quickly. > Tested with tier2,3 and running tiers 4,5,6 in progress. > Thanks to Kim for his previous feedback. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Make enable events lock unconditionally if tagmap present. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1439/files - new: https://git.openjdk.java.net/jdk/pull/1439/files/1d9e6bef..ca1715b9 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1439&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1439&range=00-01 Stats: 8 lines in 1 file changed: 0 ins; 6 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1439.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1439/head:pull/1439 PR: https://git.openjdk.java.net/jdk/pull/1439 From cjplummer at openjdk.java.net Mon Nov 30 20:06:58 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 30 Nov 2020 20:06:58 GMT Subject: RFR: 8255987: JDI tests fail with com.sun.jdi.ObjectCollectedException In-Reply-To: References: Message-ID: On Thu, 26 Nov 2020 10:33:21 GMT, Per Liden wrote: > Just a friendly ping. Still looking for reviewers for this fix. Until we resolve the discussion in [JDK-8255987](https://bugs.openjdk.java.net/browse/JDK-8255987), I don't think your suggested fix should be applied since it could be viewed as a workaround to a debug agent issue (not shutting down GC during `VM.suspendAll`) or as something that needs to be clarified in the JDI and JDWP specs (checking for `ObjectReference.disableCollection` failures, even when under `VM.suspendAll`, and retrying the allocation). I'd like to see the discussion resolved and follow-on bugs files. ------------- PR: https://git.openjdk.java.net/jdk/pull/1348 From sspitsyn at openjdk.java.net Mon Nov 30 20:08:57 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Mon, 30 Nov 2020 20:08:57 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" [v2] In-Reply-To: <5Ed2eVORB8pJ0BmKISasvgLZCT4HIjJt-YA4sjZZW0k=.96a9d20c-f931-493a-83c2-2444e2646896@github.com> References: <5Ed2eVORB8pJ0BmKISasvgLZCT4HIjJt-YA4sjZZW0k=.96a9d20c-f931-493a-83c2-2444e2646896@github.com> Message-ID: <8XNFwVFqjXQ95gjVxcRvmopfjzqkhxeLwWpVv488Grs=.2c0f8a1d-7a51-487a-a3da-4e0ffa34acd7@github.com> On Mon, 30 Nov 2020 13:57:14 GMT, Coleen Phillimore wrote: >> The ServiceThread cleaning used a stale ObjectFree state when calling remove_dead_entries, because another thread had concurrently set is_enabled to false. Add a lock around setting/resetting the lock event state and retest the state under a lock. Ran the test 100s of time without failure, where otherwise it fails very quickly. >> Tested with tier2,3 and running tiers 4,5,6 in progress. >> Thanks to Kim for his previous feedback. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Make enable events lock unconditionally if tagmap present. Hi Coleen, The fix looks okay to me. Of course, there is some overhead but I'm not sure the impact is significant. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1439 From coleenp at openjdk.java.net Mon Nov 30 20:42:58 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 30 Nov 2020 20:42:58 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" [v2] In-Reply-To: <8XNFwVFqjXQ95gjVxcRvmopfjzqkhxeLwWpVv488Grs=.2c0f8a1d-7a51-487a-a3da-4e0ffa34acd7@github.com> References: <5Ed2eVORB8pJ0BmKISasvgLZCT4HIjJt-YA4sjZZW0k=.96a9d20c-f931-493a-83c2-2444e2646896@github.com> <8XNFwVFqjXQ95gjVxcRvmopfjzqkhxeLwWpVv488Grs=.2c0f8a1d-7a51-487a-a3da-4e0ffa34acd7@github.com> Message-ID: <6gQu7Xd92VxquOSm5PHSFQ_OOYMiffQGaLuc_gT3H0o=.dc861e81-28a0-4510-9526-9339dc95e5d7@github.com> On Mon, 30 Nov 2020 20:05:43 GMT, Serguei Spitsyn wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Make enable events lock unconditionally if tagmap present. > > Hi Coleen, > The fix looks okay to me. > Of course, there is some overhead but I'm not sure the impact is significant. > Thanks, > Serguei Thank you Serguei! ------------- PR: https://git.openjdk.java.net/jdk/pull/1439 From coleenp at openjdk.java.net Mon Nov 30 20:55:00 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 30 Nov 2020 20:55:00 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" [v2] In-Reply-To: <6gQu7Xd92VxquOSm5PHSFQ_OOYMiffQGaLuc_gT3H0o=.dc861e81-28a0-4510-9526-9339dc95e5d7@github.com> References: <5Ed2eVORB8pJ0BmKISasvgLZCT4HIjJt-YA4sjZZW0k=.96a9d20c-f931-493a-83c2-2444e2646896@github.com> <8XNFwVFqjXQ95gjVxcRvmopfjzqkhxeLwWpVv488Grs=.2c0f8a1d-7a51-487a-a3da-4e0ffa34acd7@github.com> <6gQu7Xd92VxquOSm5PHSFQ_OOYMiffQGaLuc_gT3H0o=.dc861e81-28a0-4510-9526-9339dc95e5d7@github.com> Message-ID: On Mon, 30 Nov 2020 20:40:16 GMT, Coleen Phillimore wrote: >> Hi Coleen, >> The fix looks okay to me. >> Of course, there is some overhead but I'm not sure the impact is significant. >> Thanks, >> Serguei > > Thank you Serguei! I don't know how to measure impact, to be honest. ------------- PR: https://git.openjdk.java.net/jdk/pull/1439 From hseigel at openjdk.java.net Mon Nov 30 21:20:06 2020 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 30 Nov 2020 21:20:06 GMT Subject: RFR: 8256718: Obsolete the long term deprecated and aliased Trace flags Message-ID: Please review this change to obsolete the deprecated and aliased Trace flags. The now empty aliased_logging_flags support was left in arguments.cpp for use by trace flags that get deprecated and aliased in the future. With this change, users will get the following example messages when using these obsolete flags, depending on whether -XX:+... or -XX:-... was specified: VM warning: Ignoring option TraceClassPaths; support was removed in 16.0. Please use -Xlog:class+path=info instead. VM warning: Ignoring option TraceClassPaths; support was removed in 16.0. Please use -Xlog:class+path=off instead. The change was tested with tiers1and 2 on Linux, Windows, and MacOS, and tiers 3-5 on Linux x64 and with JCK lang and vm tests. Thanks, Harold ------------- Commit messages: - 8256718: Obsolete the long term deprecated and aliased Trace flags Changes: https://git.openjdk.java.net/jdk/pull/1525/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1525&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256718 Stats: 190 lines in 22 files changed: 51 ins; 112 del; 27 mod Patch: https://git.openjdk.java.net/jdk/pull/1525.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1525/head:pull/1525 PR: https://git.openjdk.java.net/jdk/pull/1525 From sspitsyn at openjdk.java.net Mon Nov 30 23:24:55 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Mon, 30 Nov 2020 23:24:55 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" [v2] In-Reply-To: References: <5Ed2eVORB8pJ0BmKISasvgLZCT4HIjJt-YA4sjZZW0k=.96a9d20c-f931-493a-83c2-2444e2646896@github.com> <8XNFwVFqjXQ95gjVxcRvmopfjzqkhxeLwWpVv488Grs=.2c0f8a1d-7a51-487a-a3da-4e0ffa34acd7@github.com> <6gQu7Xd92VxquOSm5PHSFQ_OOYMiffQGaLuc_gT3H0o=.dc861e81-28a0-4510-9526-9339dc95e5d7@github.com> Message-ID: On Mon, 30 Nov 2020 20:51:58 GMT, Coleen Phillimore wrote: >> Thank you Serguei! > > I don't know how to measure impact, to be honest. I'd suggest to separate this potential issue and work on it only if the impact is noticeable. ------------- PR: https://git.openjdk.java.net/jdk/pull/1439 From cjplummer at openjdk.java.net Mon Nov 30 23:32:55 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 30 Nov 2020 23:32:55 GMT Subject: RFR: 8256808: com/sun/jdi/CatchAllTest.java failed with "NullPointerException: Cannot invoke "lib.jdb.Jdb.log(String)" because "this.jdb" is null" In-Reply-To: References: Message-ID: On Wed, 25 Nov 2020 21:36:12 GMT, Alex Menkov wrote: > JdbTest can get exception before jdb field is initialized. > As Jdb logging does not depend on the instance, made Jdb.log method static Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1443 From coleenp at openjdk.java.net Mon Nov 30 23:32:56 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 30 Nov 2020 23:32:56 GMT Subject: RFR: 8256830: misc tests failed with "assert(env->is_enabled(JVMTI_EVENT_OBJECT_FREE)) failed: checking" [v2] In-Reply-To: References: <5Ed2eVORB8pJ0BmKISasvgLZCT4HIjJt-YA4sjZZW0k=.96a9d20c-f931-493a-83c2-2444e2646896@github.com> <8XNFwVFqjXQ95gjVxcRvmopfjzqkhxeLwWpVv488Grs=.2c0f8a1d-7a51-487a-a3da-4e0ffa34acd7@github.com> <6gQu7Xd92VxquOSm5PHSFQ_OOYMiffQGaLuc_gT3H0o=.dc861e81-28a0-4510-9526-9339dc95e5d7@github.com> Message-ID: <_oTCrSz5kPvuLdlthK7hLfIsMjFl8fZd6FNllFbjV0k=.b3dc9d22-4ed7-40cf-8e43-e944369f52b8@github.com> On Mon, 30 Nov 2020 23:22:14 GMT, Serguei Spitsyn wrote: >> I don't know how to measure impact, to be honest. > > I'd suggest to separate this potential issue and work on it only if the impact is noticeable. Ok, yes, Serguei. I don't think this is noticeable since one doesn't enable and disable events over and over in a loop. thanks for the code review and comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/1439 From cjplummer at openjdk.java.net Mon Nov 30 23:32:57 2020 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 30 Nov 2020 23:32:57 GMT Subject: RFR: 8256808: com/sun/jdi/CatchAllTest.java failed with "NullPointerException: Cannot invoke "lib.jdb.Jdb.log(String)" because "this.jdb" is null" In-Reply-To: References: Message-ID: On Thu, 26 Nov 2020 18:04:18 GMT, Aleksey Shipilev wrote: > Why not just print it to `System.out` then, eliminating the method altogether? There are a 2 of other references to log() within the Jdb class. I think the only real value log() has is in case you need to (temporarily) run a test with Jdb output directed somewhere else, or maybe you want to add a prefix to the output. I'm ok with the fix as is, but also wouldn't object to removing log() and changing all references to System.out.println. ------------- PR: https://git.openjdk.java.net/jdk/pull/1443 From sspitsyn at openjdk.java.net Mon Nov 30 23:44:55 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Mon, 30 Nov 2020 23:44:55 GMT Subject: RFR: 8256718: Obsolete the long term deprecated and aliased Trace flags In-Reply-To: References: Message-ID: On Mon, 30 Nov 2020 21:13:05 GMT, Harold Seigel wrote: > Please review this change to obsolete the deprecated and aliased Trace flags. The now empty aliased_logging_flags support was left in arguments.cpp for use by trace flags that get deprecated and aliased in the future. > > With this change, users will get the following example messages when using these obsolete flags, depending on whether -XX:+... or -XX:-... was specified: > > VM warning: Ignoring option TraceClassPaths; support was removed in 16.0. Please use -Xlog:class+path=info instead. > > VM warning: Ignoring option TraceClassPaths; support was removed in 16.0. Please use -Xlog:class+path=off instead. > > The change was tested with tiers1and 2 on Linux, Windows, and MacOS, and tiers 3-5 on Linux x64 and with JCK lang and vm tests. > > Thanks, Harold Hi Harold, The fix looks okay to me. I was more focusing on the serviceability flags. I'm not sure the flag `TraceJVMTIObjectTagging` should be mentioned in the `src/java.base/share/man/java.1` the same way as the `TraceRedefineClasses`. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1525 From sspitsyn at openjdk.java.net Mon Nov 30 23:53:55 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Mon, 30 Nov 2020 23:53:55 GMT Subject: RFR: 8256808: com/sun/jdi/CatchAllTest.java failed with "NullPointerException: Cannot invoke "lib.jdb.Jdb.log(String)" because "this.jdb" is null" In-Reply-To: References: Message-ID: On Wed, 25 Nov 2020 21:36:12 GMT, Alex Menkov wrote: > JdbTest can get exception before jdb field is initialized. > As Jdb logging does not depend on the instance, made Jdb.log method static Hi Alex, It looks good. I do not object against of getting rid of the Jdb.log method if there are no other dependencies. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1443