From coleenp at openjdk.java.net Thu Apr 1 00:29:19 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 00:29:19 GMT Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 19:02:02 GMT, Ioi Lam wrote: >> src/hotspot/share/runtime/flags/debug_globals.hpp line 38: >> >>> 36: // have any MANAGEABLE flags of the ccstr type, but we really need to >>> 37: // make sure the implementation is correct (in terms of memory allocation) >>> 38: // just in case someone may add such a flag in the future. >> >> Could you have just added a develop flag to the manageable flags instead? > > I had to use a `product` flag due to the following code, which should have been removed as part of [JDK-8243208](https://bugs.openjdk.java.net/browse/JDK-8243208), but I was afraid to do so because I didn't have a test case. I.e., all of our diagnostic/manageable/experimental flags were `product` flags. > > With this PR, now I have a test case -- I changed `DummyManageableStringFlag` to a `notproduct` flag, and removed the following code. I am re-running tiers1-4 now. > > void JVMFlag::check_all_flag_declarations() { > for (JVMFlag* current = &flagTable[0]; current->_name != NULL; current++) { > int flags = static_cast(current->_flags); > // Backwards compatibility. This will be relaxed/removed in JDK-7123237. > int mask = JVMFlag::KIND_DIAGNOSTIC | JVMFlag::KIND_MANAGEABLE | JVMFlag::KIND_EXPERIMENTAL; > if ((flags & mask) != 0) { > assert((flags & mask) == JVMFlag::KIND_DIAGNOSTIC || > (flags & mask) == JVMFlag::KIND_MANAGEABLE || > (flags & mask) == JVMFlag::KIND_EXPERIMENTAL, > "%s can be declared with at most one of " > "DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL", current->_name); > assert((flags & KIND_NOT_PRODUCT) == 0 && > (flags & KIND_DEVELOP) == 0, > "%s has an optional DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL " > "attribute; it must be declared as a product flag", current->_name); > } > } > } What's the difference between notproduct and develop again? Do we run tests with the optimized build and why would this flag be available in that build? ie. why not develop? ------------- PR: https://git.openjdk.java.net/jdk/pull/3254 From coleenp at openjdk.java.net Thu Apr 1 00:29:19 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 00:29:19 GMT Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 19:01:48 GMT, Ioi Lam wrote: >> There are two versions of JVMFlagAccess::ccstrAtPut() for modifying JVM flags of the ccstr type (i.e., strings). >> >> - One version requires the caller to free the old value, but some callers don't do that (writeableFlags.cpp). >> - The other version frees the old value on behalf of the caller. However, this version is accessible only via FLAG_SET_XXX macros and is currently unused. So it's unclear whether it actually works. >> >> We should combine these two versions into a single function, fix problems in the callers, and add test cases. The old value should be freed automatically, because typically the caller isn't interested in the old value. >> >> Note that the FLAG_SET_XXX macros do not return the old value. Requiring the caller of FLAG_SET_XXX to free the old value would be tedious and error prone. > > Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: > > relax flag attributions (ala JDK-7123237) Marked as reviewed by coleenp (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3254 From coleenp at openjdk.java.net Thu Apr 1 02:11:38 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 02:11:38 GMT Subject: RFR: 8264150: CDS dumping code calls TRAPS functions in VM thread Message-ID: This change initializes the vtables/itables without checking constraints. After initializing but before the class is visible in the SystemDictionary, constraints are checked in a separate loop. Tested with tier1-6 which includes jck tests. ------------- Commit messages: - 8264150: CDS dumping code calls TRAPS functions in VM thread Changes: https://git.openjdk.java.net/jdk/pull/3277/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3277&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264150 Stats: 290 lines in 10 files changed: 146 ins; 84 del; 60 mod Patch: https://git.openjdk.java.net/jdk/pull/3277.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3277/head:pull/3277 PR: https://git.openjdk.java.net/jdk/pull/3277 From ksakata at openjdk.java.net Thu Apr 1 02:34:23 2021 From: ksakata at openjdk.java.net (Koichi Sakata) Date: Thu, 1 Apr 2021 02:34:23 GMT Subject: RFR: 8176026: SA: Huge heap sizes cause a negative value to be displayed in the jhisto heap total [v2] In-Reply-To: References: <-OxCoBAqyBOqOS5DP3C4NPkq8PFLwwCL6wBXmiqeEzU=.f62e8e0f-418e-4129-a91a-c10e1571466e@github.com> Message-ID: On Fri, 26 Mar 2021 13:57:00 GMT, Koichi Sakata wrote: >> Marked as reviewed by mgkwill at github.com (no known OpenJDK username). > > I just updated the copyright. Could someone sponsor this pull request? It is ready. ------------- PR: https://git.openjdk.java.net/jdk/pull/3087 From pchilanomate at openjdk.java.net Thu Apr 1 03:12:33 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Thu, 1 Apr 2021 03:12:33 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 08:27:35 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: > > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) Hi Robbin, Great work! I only have minor comments. ------------- Marked as reviewed by pchilanomate (Committer). PR: https://git.openjdk.java.net/jdk/pull/3191 From pchilanomate at openjdk.java.net Thu Apr 1 03:24:37 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Thu, 1 Apr 2021 03:24:37 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 13:26:13 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.hpp line 142: > >> 140: private: >> 141: volatile bool _suspended; >> 142: volatile bool _suspend_requested; > > According to the PR description `_suspend_requested` is an optimization. > >> The "suspend requested" flag is an optimization, without it a dormant thread >> could be suspended and resumed many times and each would add a new >> asynchronous handshake. Suspend requested flag means there is already an >> asynchronous suspend handshake in queue which can be re-used, only the suspend >> flag needs to be set. > > I think there are a few places where _suspend_requested is queried by mistake instead of _suspended. Maybe it would help to prevent this if _suspend_requested was renamed to something that better describes its purpose, e.g. _has_async_suspend_handshake or similar. I also think it would be a good idea to rename it with something along those lines. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From pchilanomate at openjdk.java.net Thu Apr 1 03:29:38 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Thu, 1 Apr 2021 03:29:38 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 08:27:35 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: > > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) src/hotspot/share/runtime/handshake.hpp line 151: > 149: > 150: bool is_suspended() { return Atomic::load(&_suspended); } > 151: void set_suspend(bool to) { return Atomic::store(&_suspended, to); } Maybe set_suspended() instead()? src/hotspot/share/runtime/objectMonitor.cpp line 436: > 434: current->set_current_pending_monitor(NULL); > 435: > 436: // We cleared the pending monitor info since we've just gotten past Comment below needs updating since it mentions ThreadBlockInVM but we are not using it anymore. src/hotspot/share/runtime/thread.cpp line 277: > 275: #endif > 276: > 277: _SR_lock = new Monitor(Mutex::suspend_resume, "SR_lock", true, Mutex::suspend_resume rank is not used by any other monitor so we could remove it. src/hotspot/share/runtime/thread.hpp line 1142: > 1140: bool java_resume(); // higher-level resume logic called by the public APIs > 1141: bool is_suspended() { return _handshake.is_suspended(); } > 1142: bool suspend_request_pending() { return _handshake.suspend_request_pending(); } If you use is_suspended() to check for suspension in objectMonitor::enter() then we can remove this method. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From pchilanomate at openjdk.java.net Thu Apr 1 03:29:40 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Thu, 1 Apr 2021 03:29:40 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:32:40 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/thread.cpp line 667: > >> 665: >> 666: // We wait for the thread to transition to a more usable state. >> 667: for (int i = 1; i <= SuspendRetryCount; i++) { > > You will need to obsolete SuspendRetryCount and SuspendRetryDelay. This will need a CSR request indicating why there is no deprecation step. I think we will need to do the same for flags AssertOnSuspendWaitFailure and TraceSuspendWaitFailures. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From ysuenaga at openjdk.java.net Thu Apr 1 04:09:25 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Thu, 1 Apr 2021 04:09:25 GMT Subject: RFR: 8176026: SA: Huge heap sizes cause a negative value to be displayed in the jhisto heap total [v3] In-Reply-To: <2nwnR_h9DuOC_2rdT9z7j3ickPchjK9ozWv9obWCP_M=.018c90ea-1fc8-4ee5-b1a6-6b1a65a308e4@github.com> References: <2nwnR_h9DuOC_2rdT9z7j3ickPchjK9ozWv9obWCP_M=.018c90ea-1fc8-4ee5-b1a6-6b1a65a308e4@github.com> Message-ID: On Fri, 26 Mar 2021 13:51:40 GMT, Koichi Sakata wrote: >> When a heap is used more than about 2.1GB, clhsdb jhisto shows a negative number in the total field. >> >> $ java -Xmx20g Sample >> >> $ jhsdb clhsdb --pid 5773 >> Attaching to process 5773, please wait... >> hsdb> jhisto >> ... >> 299: 1 16 jdk.internal.misc.Unsafe >> 300: 3402 10737610256 byte[] >> Total : 15823 -2146661280 >> Heap traversal took 1.793 seconds. >> (Incidentally, the Sample is a program that only allocates many objects.) >> >> #### Details >> This is because in ObjectHistogram class the totalSize variable is int type. >> >> The total size is the total of ObjectHistogramElement#getSize() and getSize() returns long. So I changed int to long in the ObjectHistogram class. >> >> Additionally, I changed the type of the totalCount. This doesn't cause a bug, but ObjectHistogramElement#getCount() also returns long. So it doesn't need to treat it as int, I think. ObjectHistogramElement#compare() also do the same thing, so a class that is a huge size show the bottom of the list. >> >> #### Tests >> The jtreg test was successful. >> $ sudo make run-test TEST=serviceability/sa/ClhsdbJhisto.java >> >> $ cat build/linux-x86_64-server-fastdebug/test-results/jtreg_test_hotspot_jtreg_serviceability_sa_ClhsdbJhisto_java/text/summary.txt >> serviceability/sa/ClhsdbJhisto.java Passed. Execution successful >> >> I confirmed the output with the same program. >> >> $ ./jdk/build/linux-x86_64-server-fastdebug/jdk/bin/java -Xmx20g Sample >> $ ./jdk/build/linux-x86_64-server-fastdebug/jdk/bin/jhsdb clhsdb --pid 10196 >> Attaching to process 10196, please wait... >> hsdb> jhisto >> Object Histogram: >> >> num #instances #bytes Class description >> -------------------------------------------------------------------------- >> 1: 3405 13958838400 byte[] >> 2: 887 109032 java.lang.Class >> ... >> 300: 1 16 jdk.internal.misc.Unsafe >> Total : 15827 13959470288 >> Heap traversal took 1.72 seconds. > > Koichi Sakata has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright Looks good. I will sponsor you. ------------- Marked as reviewed by ysuenaga (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3087 From ksakata at openjdk.java.net Thu Apr 1 04:13:30 2021 From: ksakata at openjdk.java.net (Koichi Sakata) Date: Thu, 1 Apr 2021 04:13:30 GMT Subject: Integrated: 8176026: SA: Huge heap sizes cause a negative value to be displayed in the jhisto heap total In-Reply-To: References: Message-ID: On Fri, 19 Mar 2021 11:49:39 GMT, Koichi Sakata wrote: > When a heap is used more than about 2.1GB, clhsdb jhisto shows a negative number in the total field. > > $ java -Xmx20g Sample > > $ jhsdb clhsdb --pid 5773 > Attaching to process 5773, please wait... > hsdb> jhisto > ... > 299: 1 16 jdk.internal.misc.Unsafe > 300: 3402 10737610256 byte[] > Total : 15823 -2146661280 > Heap traversal took 1.793 seconds. > (Incidentally, the Sample is a program that only allocates many objects.) > > #### Details > This is because in ObjectHistogram class the totalSize variable is int type. > > The total size is the total of ObjectHistogramElement#getSize() and getSize() returns long. So I changed int to long in the ObjectHistogram class. > > Additionally, I changed the type of the totalCount. This doesn't cause a bug, but ObjectHistogramElement#getCount() also returns long. So it doesn't need to treat it as int, I think. ObjectHistogramElement#compare() also do the same thing, so a class that is a huge size show the bottom of the list. > > #### Tests > The jtreg test was successful. > $ sudo make run-test TEST=serviceability/sa/ClhsdbJhisto.java > > $ cat build/linux-x86_64-server-fastdebug/test-results/jtreg_test_hotspot_jtreg_serviceability_sa_ClhsdbJhisto_java/text/summary.txt > serviceability/sa/ClhsdbJhisto.java Passed. Execution successful > > I confirmed the output with the same program. > > $ ./jdk/build/linux-x86_64-server-fastdebug/jdk/bin/java -Xmx20g Sample > $ ./jdk/build/linux-x86_64-server-fastdebug/jdk/bin/jhsdb clhsdb --pid 10196 > Attaching to process 10196, please wait... > hsdb> jhisto > Object Histogram: > > num #instances #bytes Class description > -------------------------------------------------------------------------- > 1: 3405 13958838400 byte[] > 2: 887 109032 java.lang.Class > ... > 300: 1 16 jdk.internal.misc.Unsafe > Total : 15827 13959470288 > Heap traversal took 1.72 seconds. This pull request has now been integrated. Changeset: 39f0b27a Author: Koichi Sakata Committer: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/39f0b27a Stats: 5 lines in 2 files changed: 0 ins; 0 del; 5 mod 8176026: SA: Huge heap sizes cause a negative value to be displayed in the jhisto heap total Reviewed-by: cjplummer, kevinw, ysuenaga ------------- PR: https://git.openjdk.java.net/jdk/pull/3087 From iklam at openjdk.java.net Thu Apr 1 04:31:26 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 1 Apr 2021 04:31:26 GMT Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 00:25:53 GMT, Coleen Phillimore wrote: >> I had to use a `product` flag due to the following code, which should have been removed as part of [JDK-8243208](https://bugs.openjdk.java.net/browse/JDK-8243208), but I was afraid to do so because I didn't have a test case. I.e., all of our diagnostic/manageable/experimental flags were `product` flags. >> >> With this PR, now I have a test case -- I changed `DummyManageableStringFlag` to a `notproduct` flag, and removed the following code. I am re-running tiers1-4 now. >> >> void JVMFlag::check_all_flag_declarations() { >> for (JVMFlag* current = &flagTable[0]; current->_name != NULL; current++) { >> int flags = static_cast(current->_flags); >> // Backwards compatibility. This will be relaxed/removed in JDK-7123237. >> int mask = JVMFlag::KIND_DIAGNOSTIC | JVMFlag::KIND_MANAGEABLE | JVMFlag::KIND_EXPERIMENTAL; >> if ((flags & mask) != 0) { >> assert((flags & mask) == JVMFlag::KIND_DIAGNOSTIC || >> (flags & mask) == JVMFlag::KIND_MANAGEABLE || >> (flags & mask) == JVMFlag::KIND_EXPERIMENTAL, >> "%s can be declared with at most one of " >> "DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL", current->_name); >> assert((flags & KIND_NOT_PRODUCT) == 0 && >> (flags & KIND_DEVELOP) == 0, >> "%s has an optional DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL " >> "attribute; it must be declared as a product flag", current->_name); >> } >> } >> } > > What's the difference between notproduct and develop again? Do we run tests with the optimized build and why would this flag be available in that build? ie. why not develop? >From the top of globals.hpp: - `develop` flags are settable / visible only during development and are constant in the PRODUCT version - `notproduct` flags are settable / visible only during development and are not declared in the PRODUCT version Since this flag is only used in test cases, and specifically for modifying its value, it doesn't make sense to expose this flag to the PRODUCT version at all. So I chose `notproduct`, so we can save a few bytes for the PRODUCT as well. ------------- PR: https://git.openjdk.java.net/jdk/pull/3254 From david.holmes at oracle.com Thu Apr 1 04:35:45 2021 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Apr 2021 14:35:45 +1000 Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: <41950cd6-80d6-685d-58fb-90a74efd4470@oracle.com> On 1/04/2021 1:29 pm, Patricio Chilano Mateo wrote: > On Wed, 31 Mar 2021 06:32:40 GMT, David Holmes wrote: > >>> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >>> >>> - Merge branch 'master' into SuspendInHandshake >>> - Merge branch 'master' into SuspendInHandshake >>> - 8257831: Suspend with handshake (review baseline) >> >> src/hotspot/share/runtime/thread.cpp line 667: >> >>> 665: >>> 666: // We wait for the thread to transition to a more usable state. >>> 667: for (int i = 1; i <= SuspendRetryCount; i++) { >> >> You will need to obsolete SuspendRetryCount and SuspendRetryDelay. This will need a CSR request indicating why there is no deprecation step. > > I think we will need to do the same for flags AssertOnSuspendWaitFailure and TraceSuspendWaitFailures. Well spotted - both of those flags are unused already - their use was deleted with JDK-8223312, but we overlooked dropping the flags themselves. Cheers, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From dholmes at openjdk.java.net Thu Apr 1 04:56:28 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 1 Apr 2021 04:56:28 GMT Subject: RFR: 8264150: CDS dumping code calls TRAPS functions in VM thread In-Reply-To: References: Message-ID: <0pLA3mqjez1hR9bG5z5D9wCWvo3JGFvN6S6HyT3uRsI=.156eafb0-c895-40ee-a46c-f68e2f013ca0@github.com> On Tue, 30 Mar 2021 22:25:06 GMT, Coleen Phillimore wrote: > This change initializes the vtables/itables without checking constraints. After initializing but before the class is visible in the SystemDictionary, constraints are checked in a separate loop. > > Tested with tier1-6 which includes jck tests. Thanks for fixing this one Coleen! The refactoring was a bit more extensive than I had envisaged but the end result looks good. Thanks, David ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3277 From iklam at openjdk.java.net Thu Apr 1 05:12:22 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 1 Apr 2021 05:12:22 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v8] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 20:58:46 GMT, Yumin Qi wrote: >> Hi, Please review >> >> Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with >> `java -XX:DumpLoadedClassList= .... ` >> to collect shareable class names and saved in file `` , then >> `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` >> With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. >> The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. >> New added jcmd option: >> `jcmd VM.cds static_dump ` >> or >> `jcmd VM.cds dynamic_dump ` >> To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. >> The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. >> >> Tests: tier1,tier2,tier3,tier4 >> >> Thanks >> Yumin > > Yumin Qi has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits: > > - Fix revert unintentionally comment, merge master. > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Remove CDS.getVMArguments, changed to use VM.getRuntimeVMArguments. Removed unused function from ClassLoader. Improved InstanceKlass::is_shareable() and related test. Added more test scenarios. > - Remove redundant check for if a class is shareable > - Fix according to review comment and add more tests > - Fix filter more flags to exclude in static dump, add more test cases > - Merge branch 'master' into jdk-8259070 > - Fix white space in CDS.java > - Add function CDS.dumpSharedArchive in java to dump shared archive > - 8259070: Add jcmd option to dump CDS Changes requested by iklam (Reviewer). src/hotspot/share/services/diagnosticCommand.cpp line 1106: > 1104: output()->print_cr("Dynamic dump:"); > 1105: if (!UseSharedSpaces) { > 1106: output()->print_cr("CDS is not available for the JDK"); I think we should use a message similar to this: $ java -Xshare:off -XX:ArchiveClassesAtExit=xxx -version Error occurred during initialization of VM DynamicDumpSharedSpaces is unsupported when base CDS archive is not loaded How about "dynamic_dump is unsupported when base CDS archive is not loaded". src/hotspot/share/services/diagnosticCommand.cpp line 1125: > 1123: Symbol* cds_name = vmSymbols::jdk_internal_misc_CDS(); > 1124: Klass* cds_klass = SystemDictionary::resolve_or_fail(cds_name, true /*throw error*/, CHECK); > 1125: JavaValue result(T_OBJECT); Should be `result(T_VOID)` to match the signature of `CDS.dumpSharedArchive()` src/hotspot/share/services/diagnosticCommand.cpp line 1133: > 1131: vmSymbols::dumpSharedArchive(), > 1132: vmSymbols::dumpSharedArchive_signature(), > 1133: &args, THREAD); Should be `&args, CHECK);`. Recently, we have started to to avoid using `THREAD` if the callee function may throw an exception. src/java.base/share/classes/jdk/internal/misc/CDS.java line 218: > 216: try { > 217: PrintStream prt = new PrintStream(fileName); > 218: prt.println("Command:"); [Try-with-resources](https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html) should be used to make sure the streams are closed when the method exits (normally or on exception). try (InputStreamReader isr = new InputStreamReader(stream); BufferedReader rdr = new BufferedReader(isr); PrintStream prt = new PrintStream(fileName);) { prt.println("Command:"); .... src/java.base/share/classes/jdk/internal/misc/CDS.java line 307: > 305: outputStdStream(proc.getErrorStream(), stdErrFile, cmds); > 306: }).start(); > 307: I think the above code can be refactored to avoid duplication. Something like: String stdOutFile = drainOutput(proc.getInputStream(), "stdout"); String stdErrFile = drainOutput(proc.getErrorStream(), "stderr"); and move the common code into drainOutput(). ------------- PR: https://git.openjdk.java.net/jdk/pull/2737 From david.holmes at oracle.com Thu Apr 1 05:22:53 2021 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Apr 2021 15:22:53 +1000 Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: References: Message-ID: <928358f3-5976-8e60-dbf2-c647d8fe3e02@oracle.com> On 1/04/2021 5:05 am, Ioi Lam wrote: > On Wed, 31 Mar 2021 12:58:50 GMT, Coleen Phillimore wrote: > >>> Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: >>> >>> relax flag attributions (ala JDK-7123237) >> >> src/hotspot/share/runtime/flags/debug_globals.hpp line 38: >> >>> 36: // have any MANAGEABLE flags of the ccstr type, but we really need to >>> 37: // make sure the implementation is correct (in terms of memory allocation) >>> 38: // just in case someone may add such a flag in the future. >> >> Could you have just added a develop flag to the manageable flags instead? > > I had to use a `product` flag due to the following code, which should have been removed as part of [JDK-8243208](https://bugs.openjdk.java.net/browse/JDK-8243208), but I was afraid to do so because I didn't have a test case. I.e., all of our diagnostic/manageable/experimental flags were `product` flags. > > With this PR, now I have a test case -- I changed `DummyManageableStringFlag` to a `notproduct` flag, and removed the following code. I am re-running tiers1-4 now. Sorry but I don't understand how a test involving one flag replaces a chunk of code that validated every flag declaration ?? David ----- > void JVMFlag::check_all_flag_declarations() { > for (JVMFlag* current = &flagTable[0]; current->_name != NULL; current++) { > int flags = static_cast(current->_flags); > // Backwards compatibility. This will be relaxed/removed in JDK-7123237. > int mask = JVMFlag::KIND_DIAGNOSTIC | JVMFlag::KIND_MANAGEABLE | JVMFlag::KIND_EXPERIMENTAL; > if ((flags & mask) != 0) { > assert((flags & mask) == JVMFlag::KIND_DIAGNOSTIC || > (flags & mask) == JVMFlag::KIND_MANAGEABLE || > (flags & mask) == JVMFlag::KIND_EXPERIMENTAL, > "%s can be declared with at most one of " > "DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL", current->_name); > assert((flags & KIND_NOT_PRODUCT) == 0 && > (flags & KIND_DEVELOP) == 0, > "%s has an optional DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL " > "attribute; it must be declared as a product flag", current->_name); > } > } > } > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3254 > From ioi.lam at oracle.com Thu Apr 1 05:29:59 2021 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 31 Mar 2021 22:29:59 -0700 Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: <928358f3-5976-8e60-dbf2-c647d8fe3e02@oracle.com> References: <928358f3-5976-8e60-dbf2-c647d8fe3e02@oracle.com> Message-ID: <8134d2f8-af9e-12ff-7381-62a91eeeec72@oracle.com> On 3/31/21 10:22 PM, David Holmes wrote: > On 1/04/2021 5:05 am, Ioi Lam wrote: >> On Wed, 31 Mar 2021 12:58:50 GMT, Coleen Phillimore >> wrote: >> >>> >>>> 36: // have any MANAGEABLE flags of the ccstr type, but we really >>>> need to >>>> 37: // make sure the implementation is correct (in terms of memory >>>> allocation) >>>> 38: // just in case someone may add such a flag in the future. >>> >>> Could you have just added a develop flag to the manageable flags >>> instead? >> >> I had to use a `product` flag due to the following code, which should >> have been removed as part of >> [JDK-8243208](https://bugs.openjdk.java.net/browse/JDK-8243208), but >> I was afraid to do so because I didn't have a test case. I.e., all of >> our diagnostic/manageable/experimental flags were `product` flags. >> >> With this PR, now I have a test case -- I changed >> `DummyManageableStringFlag` to a `notproduct` flag, and removed the >> following code. I am re-running tiers1-4 now. > > Sorry but I don't understand how a test involving one flag replaces a > chunk of code that validated every flag declaration ?? > When I did JDK-8243208, I wasn't sure if the VM would actually support diagnostic/manageable/experimental flags that were not `product`. So I added check_all_flag_declarations() to prevent anyone from adding such a flag "casually" without thinking about. Now that I have added such a flag, and I believe I have tested pretty thoroughly (tiers 1-4), I think the VM indeed supports these flags. So now I feel it's safe to remove check_all_flag_declarations(). Thanks - Ioi > David > ----- > >> void JVMFlag::check_all_flag_declarations() { >> ?? for (JVMFlag* current = &flagTable[0]; current->_name != NULL; >> current++) { >> ???? int flags = static_cast(current->_flags); >> ???? // Backwards compatibility. This will be relaxed/removed in >> JDK-7123237. >> ???? int mask = JVMFlag::KIND_DIAGNOSTIC | JVMFlag::KIND_MANAGEABLE | >> JVMFlag::KIND_EXPERIMENTAL; >> ???? if ((flags & mask) != 0) { >> ?????? assert((flags & mask) == JVMFlag::KIND_DIAGNOSTIC || >> ????????????? (flags & mask) == JVMFlag::KIND_MANAGEABLE || >> ????????????? (flags & mask) == JVMFlag::KIND_EXPERIMENTAL, >> ????????????? "%s can be declared with at most one of " >> ????????????? "DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL", current->_name); >> ?????? assert((flags & KIND_NOT_PRODUCT) == 0 && >> ????????????? (flags & KIND_DEVELOP) == 0, >> ????????????? "%s has an optional DIAGNOSTIC, MANAGEABLE or >> EXPERIMENTAL " >> ????????????? "attribute; it must be declared as a product flag", >> current->_name); >> ???? } >> ?? } >> } >> >> ------------- >> >> PR: https://git.openjdk.java.net/jdk/pull/3254 >> From david.holmes at oracle.com Thu Apr 1 05:47:48 2021 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Apr 2021 15:47:48 +1000 Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: <8134d2f8-af9e-12ff-7381-62a91eeeec72@oracle.com> References: <928358f3-5976-8e60-dbf2-c647d8fe3e02@oracle.com> <8134d2f8-af9e-12ff-7381-62a91eeeec72@oracle.com> Message-ID: <56fca089-7f8b-1eff-e4cd-9eb1719c1c39@oracle.com> On 1/04/2021 3:29 pm, Ioi Lam wrote: > > > On 3/31/21 10:22 PM, David Holmes wrote: >> On 1/04/2021 5:05 am, Ioi Lam wrote: >>> On Wed, 31 Mar 2021 12:58:50 GMT, Coleen Phillimore >>> wrote: >>> >>>> >>>>> 36: // have any MANAGEABLE flags of the ccstr type, but we really >>>>> need to >>>>> 37: // make sure the implementation is correct (in terms of memory >>>>> allocation) >>>>> 38: // just in case someone may add such a flag in the future. >>>> >>>> Could you have just added a develop flag to the manageable flags >>>> instead? >>> >>> I had to use a `product` flag due to the following code, which should >>> have been removed as part of >>> [JDK-8243208](https://bugs.openjdk.java.net/browse/JDK-8243208), but >>> I was afraid to do so because I didn't have a test case. I.e., all of >>> our diagnostic/manageable/experimental flags were `product` flags. >>> >>> With this PR, now I have a test case -- I changed >>> `DummyManageableStringFlag` to a `notproduct` flag, and removed the >>> following code. I am re-running tiers1-4 now. >> >> Sorry but I don't understand how a test involving one flag replaces a >> chunk of code that validated every flag declaration ?? >> > > When I did JDK-8243208, I wasn't sure if the VM would actually support > diagnostic/manageable/experimental flags that were not `product`. So I > added check_all_flag_declarations() to prevent anyone from adding such a > flag "casually" without thinking about. > > Now that I have added such a flag, and I believe I have tested pretty > thoroughly (tiers 1-4), I think the VM indeed supports these flags. So > now I feel it's safe to remove check_all_flag_declarations(). But the check was checking two things: 1. That diagnostic/manageable/experimental are mutually exclusive 2. That they only apply to product flags I believe both of these rules still stand based on what we defined such attributes to mean. And even if you think #2 should not apply, #1 still does and so could still be checked. And if #2 is no longer our rule then the documentation in globals.hpp should be updated - though IMHO #2 should remain as-is. David ----- > Thanks > - Ioi > > > >> David >> ----- >> >>> void JVMFlag::check_all_flag_declarations() { >>> ?? for (JVMFlag* current = &flagTable[0]; current->_name != NULL; >>> current++) { >>> ???? int flags = static_cast(current->_flags); >>> ???? // Backwards compatibility. This will be relaxed/removed in >>> JDK-7123237. >>> ???? int mask = JVMFlag::KIND_DIAGNOSTIC | JVMFlag::KIND_MANAGEABLE | >>> JVMFlag::KIND_EXPERIMENTAL; >>> ???? if ((flags & mask) != 0) { >>> ?????? assert((flags & mask) == JVMFlag::KIND_DIAGNOSTIC || >>> ????????????? (flags & mask) == JVMFlag::KIND_MANAGEABLE || >>> ????????????? (flags & mask) == JVMFlag::KIND_EXPERIMENTAL, >>> ????????????? "%s can be declared with at most one of " >>> ????????????? "DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL", current->_name); >>> ?????? assert((flags & KIND_NOT_PRODUCT) == 0 && >>> ????????????? (flags & KIND_DEVELOP) == 0, >>> ????????????? "%s has an optional DIAGNOSTIC, MANAGEABLE or >>> EXPERIMENTAL " >>> ????????????? "attribute; it must be declared as a product flag", >>> current->_name); >>> ???? } >>> ?? } >>> } >>> >>> ------------- >>> >>> PR: https://git.openjdk.java.net/jdk/pull/3254 >>> > From iklam at openjdk.java.net Thu Apr 1 06:08:52 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 1 Apr 2021 06:08:52 GMT Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v3] In-Reply-To: References: Message-ID: > There are two versions of JVMFlagAccess::ccstrAtPut() for modifying JVM flags of the ccstr type (i.e., strings). > > - One version requires the caller to free the old value, but some callers don't do that (writeableFlags.cpp). > - The other version frees the old value on behalf of the caller. However, this version is accessible only via FLAG_SET_XXX macros and is currently unused. So it's unclear whether it actually works. > > We should combine these two versions into a single function, fix problems in the callers, and add test cases. The old value should be freed automatically, because typically the caller isn't interested in the old value. > > Note that the FLAG_SET_XXX macros do not return the old value. Requiring the caller of FLAG_SET_XXX to free the old value would be tedious and error prone. Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: Revert "relax flag attributions (ala JDK-7123237)" This reverts commit 673aaafc4860dd7f70a3f222422ae84e85fd4219. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3254/files - new: https://git.openjdk.java.net/jdk/pull/3254/files/673aaafc..2a5e2b19 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3254&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3254&range=01-02 Stats: 37 lines in 4 files changed: 36 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3254.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3254/head:pull/3254 PR: https://git.openjdk.java.net/jdk/pull/3254 From ioi.lam at oracle.com Thu Apr 1 06:19:12 2021 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 31 Mar 2021 23:19:12 -0700 Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: <56fca089-7f8b-1eff-e4cd-9eb1719c1c39@oracle.com> References: <928358f3-5976-8e60-dbf2-c647d8fe3e02@oracle.com> <8134d2f8-af9e-12ff-7381-62a91eeeec72@oracle.com> <56fca089-7f8b-1eff-e4cd-9eb1719c1c39@oracle.com> Message-ID: On 3/31/21 10:47 PM, David Holmes wrote: > On 1/04/2021 3:29 pm, Ioi Lam wrote: >> >> >> On 3/31/21 10:22 PM, David Holmes wrote: >>> On 1/04/2021 5:05 am, Ioi Lam wrote: >>>> On Wed, 31 Mar 2021 12:58:50 GMT, Coleen Phillimore >>>> wrote: >>>> >>>>> >>>>>> 36: // have any MANAGEABLE flags of the ccstr type, but we really >>>>>> need to >>>>>> 37: // make sure the implementation is correct (in terms of >>>>>> memory allocation) >>>>>> 38: // just in case someone may add such a flag in the future. >>>>> >>>>> Could you have just added a develop flag to the manageable flags >>>>> instead? >>>> >>>> I had to use a `product` flag due to the following code, which >>>> should have been removed as part of >>>> [JDK-8243208](https://bugs.openjdk.java.net/browse/JDK-8243208), >>>> but I was afraid to do so because I didn't have a test case. I.e., >>>> all of our diagnostic/manageable/experimental flags were `product` >>>> flags. >>>> >>>> With this PR, now I have a test case -- I changed >>>> `DummyManageableStringFlag` to a `notproduct` flag, and removed the >>>> following code. I am re-running tiers1-4 now. >>> >>> Sorry but I don't understand how a test involving one flag replaces >>> a chunk of code that validated every flag declaration ?? >>> >> >> When I did JDK-8243208, I wasn't sure if the VM would actually >> support diagnostic/manageable/experimental flags that were not >> `product`. So I added check_all_flag_declarations() to prevent anyone >> from adding such a flag "casually" without thinking about. >> >> Now that I have added such a flag, and I believe I have tested pretty >> thoroughly (tiers 1-4), I think the VM indeed supports these flags. >> So now I feel it's safe to remove check_all_flag_declarations(). > > But the check was checking two things: > > 1. That diagnostic/manageable/experimental are mutually exclusive > 2. That they only apply to product flags > > I believe both of these rules still stand based on what we defined > such attributes to mean. And even if you think #2 should not apply, #1 > still does and so could still be checked. And if #2 is no longer our > rule then the documentation in globals.hpp should be updated - though > IMHO #2 should remain as-is. > I think neither #1 and #2 make sense. These were limitation introduced by the old flags implementation, where you had to declare a flag using one of these macros ??? diagnostic(type, name, .... ??? manageable(type, name, .... ??? experimental(type, name, .... That's why you have #1 (mutual exclusion). #2 was also due to the implementation. It makes no sense that you can't have an develop flag for an experimental feature. However, in the old implementation, adding more variations would cause macro explosion. See https://github.com/openjdk/jdk/blame/7d8519fffe46b6b5139b3aa51b18fcf0249a9e14/src/hotspot/share/runtime/flags/jvmFlag.cpp#L776 Anyway, changing this should be done in a separate RFE. I have reverted [v2] from this PR, so we are back to [v1]. Thanks - Ioi > David > ----- > > >> Thanks >> - Ioi >> >> >> >>> David >>> ----- >>> >>>> void JVMFlag::check_all_flag_declarations() { >>>> ?? for (JVMFlag* current = &flagTable[0]; current->_name != NULL; >>>> current++) { >>>> ???? int flags = static_cast(current->_flags); >>>> ???? // Backwards compatibility. This will be relaxed/removed in >>>> JDK-7123237. >>>> ???? int mask = JVMFlag::KIND_DIAGNOSTIC | JVMFlag::KIND_MANAGEABLE >>>> | JVMFlag::KIND_EXPERIMENTAL; >>>> ???? if ((flags & mask) != 0) { >>>> ?????? assert((flags & mask) == JVMFlag::KIND_DIAGNOSTIC || >>>> ????????????? (flags & mask) == JVMFlag::KIND_MANAGEABLE || >>>> ????????????? (flags & mask) == JVMFlag::KIND_EXPERIMENTAL, >>>> ????????????? "%s can be declared with at most one of " >>>> ????????????? "DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL", >>>> current->_name); >>>> ?????? assert((flags & KIND_NOT_PRODUCT) == 0 && >>>> ????????????? (flags & KIND_DEVELOP) == 0, >>>> ????????????? "%s has an optional DIAGNOSTIC, MANAGEABLE or >>>> EXPERIMENTAL " >>>> ????????????? "attribute; it must be declared as a product flag", >>>> current->_name); >>>> ???? } >>>> ?? } >>>> } >>>> >>>> ------------- >>>> >>>> PR: https://git.openjdk.java.net/jdk/pull/3254 >>>> >> From iklam at openjdk.java.net Thu Apr 1 06:28:25 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 1 Apr 2021 06:28:25 GMT Subject: RFR: 8264150: CDS dumping code calls TRAPS functions in VM thread In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 22:25:06 GMT, Coleen Phillimore wrote: > This change initializes the vtables/itables without checking constraints. After initializing but before the class is visible in the SystemDictionary, constraints are checked in a separate loop. > > Tested with tier1-6 which includes jck tests. Marked as reviewed by iklam (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3277 From david.holmes at oracle.com Thu Apr 1 06:35:59 2021 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Apr 2021 16:35:59 +1000 Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: References: <928358f3-5976-8e60-dbf2-c647d8fe3e02@oracle.com> <8134d2f8-af9e-12ff-7381-62a91eeeec72@oracle.com> <56fca089-7f8b-1eff-e4cd-9eb1719c1c39@oracle.com> Message-ID: <131a0ea9-b491-f38e-d846-11af9db13897@oracle.com> On 1/04/2021 4:19 pm, Ioi Lam wrote: > > > On 3/31/21 10:47 PM, David Holmes wrote: >> On 1/04/2021 3:29 pm, Ioi Lam wrote: >>> >>> >>> On 3/31/21 10:22 PM, David Holmes wrote: >>>> On 1/04/2021 5:05 am, Ioi Lam wrote: >>>>> On Wed, 31 Mar 2021 12:58:50 GMT, Coleen Phillimore >>>>> wrote: >>>>> >>>>>> >>>>>>> 36: // have any MANAGEABLE flags of the ccstr type, but we really >>>>>>> need to >>>>>>> 37: // make sure the implementation is correct (in terms of >>>>>>> memory allocation) >>>>>>> 38: // just in case someone may add such a flag in the future. >>>>>> >>>>>> Could you have just added a develop flag to the manageable flags >>>>>> instead? >>>>> >>>>> I had to use a `product` flag due to the following code, which >>>>> should have been removed as part of >>>>> [JDK-8243208](https://bugs.openjdk.java.net/browse/JDK-8243208), >>>>> but I was afraid to do so because I didn't have a test case. I.e., >>>>> all of our diagnostic/manageable/experimental flags were `product` >>>>> flags. >>>>> >>>>> With this PR, now I have a test case -- I changed >>>>> `DummyManageableStringFlag` to a `notproduct` flag, and removed the >>>>> following code. I am re-running tiers1-4 now. >>>> >>>> Sorry but I don't understand how a test involving one flag replaces >>>> a chunk of code that validated every flag declaration ?? >>>> >>> >>> When I did JDK-8243208, I wasn't sure if the VM would actually >>> support diagnostic/manageable/experimental flags that were not >>> `product`. So I added check_all_flag_declarations() to prevent anyone >>> from adding such a flag "casually" without thinking about. >>> >>> Now that I have added such a flag, and I believe I have tested pretty >>> thoroughly (tiers 1-4), I think the VM indeed supports these flags. >>> So now I feel it's safe to remove check_all_flag_declarations(). >> >> But the check was checking two things: >> >> 1. That diagnostic/manageable/experimental are mutually exclusive >> 2. That they only apply to product flags >> >> I believe both of these rules still stand based on what we defined >> such attributes to mean. And even if you think #2 should not apply, #1 >> still does and so could still be checked. And if #2 is no longer our >> rule then the documentation in globals.hpp should be updated - though >> IMHO #2 should remain as-is. >> > > I think neither #1 and #2 make sense. These were limitation introduced > by the old flags implementation, where you had to declare a flag using > one of these macros > > ??? diagnostic(type, name, .... > ??? manageable(type, name, .... > ??? experimental(type, name, .... > > That's why you have #1 (mutual exclusion). > > #2 was also due to the implementation. It makes no sense that you can't > have an develop flag for an experimental feature. I don't agree with either of those claims. This is about semantics not implementation. diagnostic/manageable/experimental are distinct kinds of product flags with different levels of "support" based on their intended use in the product. We don't need such distinctions with non-product flags because the level of "support" is all the same and all flags are equal. David ----- > However, in the old implementation, adding more variations would cause > macro explosion. See > https://github.com/openjdk/jdk/blame/7d8519fffe46b6b5139b3aa51b18fcf0249a9e14/src/hotspot/share/runtime/flags/jvmFlag.cpp#L776 > > > Anyway, changing this should be done in a separate RFE. I have reverted > [v2] from this PR, so we are back to [v1]. > > Thanks > - Ioi > > >> David >> ----- >> >> >>> Thanks >>> - Ioi >>> >>> >>> >>>> David >>>> ----- >>>> >>>>> void JVMFlag::check_all_flag_declarations() { >>>>> ?? for (JVMFlag* current = &flagTable[0]; current->_name != NULL; >>>>> current++) { >>>>> ???? int flags = static_cast(current->_flags); >>>>> ???? // Backwards compatibility. This will be relaxed/removed in >>>>> JDK-7123237. >>>>> ???? int mask = JVMFlag::KIND_DIAGNOSTIC | JVMFlag::KIND_MANAGEABLE >>>>> | JVMFlag::KIND_EXPERIMENTAL; >>>>> ???? if ((flags & mask) != 0) { >>>>> ?????? assert((flags & mask) == JVMFlag::KIND_DIAGNOSTIC || >>>>> ????????????? (flags & mask) == JVMFlag::KIND_MANAGEABLE || >>>>> ????????????? (flags & mask) == JVMFlag::KIND_EXPERIMENTAL, >>>>> ????????????? "%s can be declared with at most one of " >>>>> ????????????? "DIAGNOSTIC, MANAGEABLE or EXPERIMENTAL", >>>>> current->_name); >>>>> ?????? assert((flags & KIND_NOT_PRODUCT) == 0 && >>>>> ????????????? (flags & KIND_DEVELOP) == 0, >>>>> ????????????? "%s has an optional DIAGNOSTIC, MANAGEABLE or >>>>> EXPERIMENTAL " >>>>> ????????????? "attribute; it must be declared as a product flag", >>>>> current->_name); >>>>> ???? } >>>>> ?? } >>>>> } >>>>> >>>>> ------------- >>>>> >>>>> PR: https://git.openjdk.java.net/jdk/pull/3254 >>>>> >>> > From shade at openjdk.java.net Thu Apr 1 07:22:25 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 1 Apr 2021 07:22:25 GMT Subject: RFR: 8264484: Replace uses of StringBuffer with StringBuilder in jdk.hotspot.agent [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 23:47:41 GMT, Yasumasa Suenaga wrote: >> Andrey Turbanov has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264484: Replace uses of StringBuffer with StringBuilder in jdk.hotspot.agent >> >> fix indentation > > Looks good! > BTW have you found sponsor? I can sponsor you if you need. @YaSuenag, you can just sponsor without asking, I think. ------------- PR: https://git.openjdk.java.net/jdk/pull/3275 From github.com+741251+turbanoff at openjdk.java.net Thu Apr 1 07:27:08 2021 From: github.com+741251+turbanoff at openjdk.java.net (Andrey Turbanov) Date: Thu, 1 Apr 2021 07:27:08 GMT Subject: Integrated: 8264484: Replace uses of StringBuffer with StringBuilder in jdk.hotspot.agent In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 18:58:24 GMT, Andrey Turbanov wrote: > Found by IntelliJ IDEA inspection `Java | Java language level migration aids | Java 5 | 'StringBuffer' may be 'StringBuilder'` > As suggested in https://github.com/openjdk/jdk/pull/1507#issuecomment-757369003 I've created separate PR for module `jdk.hotspot.agent` > Similar cleanups in the past: https://bugs.openjdk.java.net/browse/JDK-8041679 https://bugs.openjdk.java.net/browse/JDK-8264029 This pull request has now been integrated. Changeset: 6cf10950 Author: Andrey Turbanov Committer: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/6cf10950 Stats: 116 lines in 37 files changed: 0 ins; 7 del; 109 mod 8264484: Replace uses of StringBuffer with StringBuilder in jdk.hotspot.agent Reviewed-by: kevinw, amenkov, ysuenaga ------------- PR: https://git.openjdk.java.net/jdk/pull/3275 From rehn at openjdk.java.net Thu Apr 1 07:33:36 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 1 Apr 2021 07:33:36 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 03:09:41 GMT, Patricio Chilano Mateo wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > Hi Robbin, > > Great work! I only have minor comments. I have a long easter weekend and I'm testing some stuff, so there will be a small delay, thanks for all the comments! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From coleenp at openjdk.java.net Thu Apr 1 11:40:28 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 11:40:28 GMT Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v3] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 06:08:52 GMT, Ioi Lam wrote: >> There are two versions of JVMFlagAccess::ccstrAtPut() for modifying JVM flags of the ccstr type (i.e., strings). >> >> - One version requires the caller to free the old value, but some callers don't do that (writeableFlags.cpp). >> - The other version frees the old value on behalf of the caller. However, this version is accessible only via FLAG_SET_XXX macros and is currently unused. So it's unclear whether it actually works. >> >> We should combine these two versions into a single function, fix problems in the callers, and add test cases. The old value should be freed automatically, because typically the caller isn't interested in the old value. >> >> Note that the FLAG_SET_XXX macros do not return the old value. Requiring the caller of FLAG_SET_XXX to free the old value would be tedious and error prone. > > Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: > > Revert "relax flag attributions (ala JDK-7123237)" > > This reverts commit 673aaafc4860dd7f70a3f222422ae84e85fd4219. src/hotspot/share/runtime/flags/debug_globals.hpp line 63: > 61: \ > 62: product(ccstr, DummyManageableStringFlag, NULL, MANAGEABLE, \ > 63: "Dummy flag for testing string handling in WriteableFlags") \ So this is in essence a product/manageable flag in debug only mode, which doesn't make sense at all, necessitating another macro that looks honestly bizarre. So I think that means a non-product manageable flag or even a develop manageable flag is something that we want, we want to be able to write a develop flag by the management interface. I agree diagnostic and experimental flags only seem to make sense as product flags. The doc could simply be updated. Also the doc at the top of this file refers to CCC which is no longer -> CSR. // MANAGEABLE flags are writeable external product flags. // They are dynamically writeable through the JDK management interface // (com.sun.management.HotSpotDiagnosticMXBean API) and also through JConsole. // These flags are external exported interface (see CCC). The list of // manageable flags can be queried programmatically through the management // interface. I'm not going to spend time typing about this minor point. The improvement is good and should be checked in. ------------- PR: https://git.openjdk.java.net/jdk/pull/3254 From hseigel at openjdk.java.net Thu Apr 1 12:28:29 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Thu, 1 Apr 2021 12:28:29 GMT Subject: RFR: 8264538: Rename SystemDictionary::parse_stream [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 21:41:39 GMT, Coleen Phillimore wrote: >> This function is used to call the classfile parser for hidden or anonymous classes, and for use with jvmti RedefineClasses. The latter only calls KlassFactory::create_from_stream and skips the rest of the code in SystemDictionary::parse_stream. >> >> Renamed SystemDictionary::parse_stream -> resolve_hidden_class_from_stream >> resolve_from_stream -> resolve_class_from_stream >> and have SystemDictionary::resolve_from_stream() call the right version depending on ClassLoadInfo flags. Callers of resolve_from_stream now pass protection domain via. ClassLoadInfo. >> >> So the external API is resolve_from_stream. >> >> Tested with tier1 on 4 Oracle supported platforms. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Add and remove comments. The changes look good! Thanks, Harold ------------- Marked as reviewed by hseigel (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3289 From lfoltan at openjdk.java.net Thu Apr 1 13:48:22 2021 From: lfoltan at openjdk.java.net (Lois Foltan) Date: Thu, 1 Apr 2021 13:48:22 GMT Subject: RFR: 8264538: Rename SystemDictionary::parse_stream [v2] In-Reply-To: References: <5XkcECxtI-f304vVc6TXjJNXHA5kpzcHORV8q8mPvzk=.da25a174-4c29-4f5a-a639-695043a06067@github.com> Message-ID: On Wed, 31 Mar 2021 21:22:39 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvm.cpp line 950: >> >>> 948: InstanceKlass* ik = NULL; >>> 949: if (!is_hidden) { >>> 950: ClassLoadInfo cl_info(protection_domain); >> >> Minor comment, you could pull the creation of ClassLoadInfo out of this if statement since both the the if and the else sections create a ClassLoadInfo with pretty much the same information. > > That other ClassLoadInfo cl_info(protection_domain) you see is from another function, so I can't pull it out. > The other side of the 'if' statement creates a ClassLoadInfo with all the hidden class goodies. In jvm_lookup_define_class there are 2 ClassLoadInfo cl_info constructions on line #950 and line #961 that could be pull out of the if statement. Again comment is minor and I am ok with how you decide to go forward with it. ------------- PR: https://git.openjdk.java.net/jdk/pull/3289 From coleenp at openjdk.java.net Thu Apr 1 14:58:13 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 14:58:13 GMT Subject: RFR: 8264538: Rename SystemDictionary::parse_stream [v3] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 12:24:56 GMT, Harold Seigel wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Add and remove comments. > > The changes look good! > Thanks, Harold Thanks Harold and Lois! ------------- PR: https://git.openjdk.java.net/jdk/pull/3289 From coleenp at openjdk.java.net Thu Apr 1 14:58:13 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 14:58:13 GMT Subject: Integrated: 8264538: Rename SystemDictionary::parse_stream In-Reply-To: References: Message-ID: <2AZFt45184wSKISLEKuBb3QwRjqRIcguukA5aA87Ciw=.84d2da62-7101-41d2-934f-5c158d0fdb62@github.com> On Wed, 31 Mar 2021 19:46:24 GMT, Coleen Phillimore wrote: > This function is used to call the classfile parser for hidden or anonymous classes, and for use with jvmti RedefineClasses. The latter only calls KlassFactory::create_from_stream and skips the rest of the code in SystemDictionary::parse_stream. > > Renamed SystemDictionary::parse_stream -> resolve_hidden_class_from_stream > resolve_from_stream -> resolve_class_from_stream > and have SystemDictionary::resolve_from_stream() call the right version depending on ClassLoadInfo flags. Callers of resolve_from_stream now pass protection domain via. ClassLoadInfo. > > So the external API is resolve_from_stream. > > Tested with tier1 on 4 Oracle supported platforms. This pull request has now been integrated. Changeset: 1dc75e9e Author: Coleen Phillimore URL: https://git.openjdk.java.net/jdk/commit/1dc75e9e Stats: 138 lines in 7 files changed: 37 ins; 29 del; 72 mod 8264538: Rename SystemDictionary::parse_stream Reviewed-by: lfoltan, hseigel ------------- PR: https://git.openjdk.java.net/jdk/pull/3289 From coleenp at openjdk.java.net Thu Apr 1 15:13:27 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 15:13:27 GMT Subject: RFR: 8264150: CDS dumping code calls TRAPS functions in VM thread In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 06:25:08 GMT, Ioi Lam wrote: >> This change initializes the vtables/itables without checking constraints. After initializing but before the class is visible in the SystemDictionary, constraints are checked in a separate loop. >> >> Tested with tier1-6 which includes jck tests. > > Marked as reviewed by iklam (Reviewer). Thanks David and Ioi. Yes, the refactoring was a bit more involved than anticipated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3277 From coleenp at openjdk.java.net Thu Apr 1 15:13:28 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 15:13:28 GMT Subject: Integrated: 8264150: CDS dumping code calls TRAPS functions in VM thread In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 22:25:06 GMT, Coleen Phillimore wrote: > This change initializes the vtables/itables without checking constraints. After initializing but before the class is visible in the SystemDictionary, constraints are checked in a separate loop. > > Tested with tier1-6 which includes jck tests. This pull request has now been integrated. Changeset: 4b197714 Author: Coleen Phillimore URL: https://git.openjdk.java.net/jdk/commit/4b197714 Stats: 290 lines in 10 files changed: 146 ins; 84 del; 60 mod 8264150: CDS dumping code calls TRAPS functions in VM thread Reviewed-by: dholmes, iklam ------------- PR: https://git.openjdk.java.net/jdk/pull/3277 From iklam at openjdk.java.net Thu Apr 1 15:27:54 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 1 Apr 2021 15:27:54 GMT Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v4] In-Reply-To: References: Message-ID: > There are two versions of JVMFlagAccess::ccstrAtPut() for modifying JVM flags of the ccstr type (i.e., strings). > > - One version requires the caller to free the old value, but some callers don't do that (writeableFlags.cpp). > - The other version frees the old value on behalf of the caller. However, this version is accessible only via FLAG_SET_XXX macros and is currently unused. So it's unclear whether it actually works. > > We should combine these two versions into a single function, fix problems in the callers, and add test cases. The old value should be freed automatically, because typically the caller isn't interested in the old value. > > Note that the FLAG_SET_XXX macros do not return the old value. Requiring the caller of FLAG_SET_XXX to free the old value would be tedious and error prone. Ioi Lam has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: - Merge branch 'master' into 8264285-clean-up-setting-ccstr-jvm-flags - Revert "relax flag attributions (ala JDK-7123237)" This reverts commit 673aaafc4860dd7f70a3f222422ae84e85fd4219. - relax flag attributions (ala JDK-7123237) - restored SET_FLAG_XXX for ccstr type, and fixed bugs in existing ccstr modification code - 8264285: Do not support FLAG_SET_XXX for VM flags of string type ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3254/files - new: https://git.openjdk.java.net/jdk/pull/3254/files/2a5e2b19..5f912221 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3254&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3254&range=02-03 Stats: 8053 lines in 376 files changed: 5492 ins; 1012 del; 1549 mod Patch: https://git.openjdk.java.net/jdk/pull/3254.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3254/head:pull/3254 PR: https://git.openjdk.java.net/jdk/pull/3254 From coleenp at openjdk.java.net Thu Apr 1 17:06:38 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 1 Apr 2021 17:06:38 GMT Subject: RFR: 8262280: Incorrect exception handling for VMThread in class redefinition Message-ID: This is a trivial change to remove the last TRAPS from redefine_single_class which is called by the VM thread during a safepoint. Tested with serviceability/jvmti/RedefineClasses, vmTestbase/nsk/jvmti,jdi and jdk/java/lang/instrument tests. And tier1 sanity in progress. ------------- Commit messages: - 8262280: Incorrect exception handling for VMThread in class redefinition Changes: https://git.openjdk.java.net/jdk/pull/3310/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3310&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8262280 Stats: 19 lines in 2 files changed: 0 ins; 0 del; 19 mod Patch: https://git.openjdk.java.net/jdk/pull/3310.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3310/head:pull/3310 PR: https://git.openjdk.java.net/jdk/pull/3310 From iklam at openjdk.java.net Thu Apr 1 18:28:24 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 1 Apr 2021 18:28:24 GMT Subject: Integrated: 8264285: Clean the modification of ccstr JVM flags In-Reply-To: References: Message-ID: On Mon, 29 Mar 2021 21:35:52 GMT, Ioi Lam wrote: > There are two versions of JVMFlagAccess::ccstrAtPut() for modifying JVM flags of the ccstr type (i.e., strings). > > - One version requires the caller to free the old value, but some callers don't do that (writeableFlags.cpp). > - The other version frees the old value on behalf of the caller. However, this version is accessible only via FLAG_SET_XXX macros and is currently unused. So it's unclear whether it actually works. > > We should combine these two versions into a single function, fix problems in the callers, and add test cases. The old value should be freed automatically, because typically the caller isn't interested in the old value. > > Note that the FLAG_SET_XXX macros do not return the old value. Requiring the caller of FLAG_SET_XXX to free the old value would be tedious and error prone. This pull request has now been integrated. Changeset: 58583990 Author: Ioi Lam URL: https://git.openjdk.java.net/jdk/commit/58583990 Stats: 205 lines in 9 files changed: 160 ins; 24 del; 21 mod 8264285: Clean the modification of ccstr JVM flags Reviewed-by: dholmes, coleenp ------------- PR: https://git.openjdk.java.net/jdk/pull/3254 From iklam at openjdk.java.net Thu Apr 1 18:28:22 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 1 Apr 2021 18:28:22 GMT Subject: RFR: 8264285: Clean the modification of ccstr JVM flags [v2] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 00:25:59 GMT, Coleen Phillimore wrote: >> Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: >> >> relax flag attributions (ala JDK-7123237) > > Marked as reviewed by coleenp (Reviewer). Thanks @coleenp and @dholmes-ora for reviewing. ------------- PR: https://git.openjdk.java.net/jdk/pull/3254 From dcubed at openjdk.java.net Thu Apr 1 19:46:39 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 1 Apr 2021 19:46:39 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Tue, 23 Mar 2021 14:10:29 GMT, Daniel D. Daugherty wrote: >> Hi Dan, >>> I figured you would enjoy this 20 year old blast from the past! >> Yes, of course, but these tests look like they have been written today! :) >> >> Thank you for making changes! >> I've just noticed one minor issue: >> >> libSuspendWithObjectMonitorEnter.cpp >> libSuspendWithObjectMonitorWait.cpp: >> The static variables below are not used and can be removed: >> 32 static jrawMonitorID threadLock = NULL; >> 33 static char threadLockName[] = "threadLock"; >> >> Thanks, >> Serguei > > @sspitsyn - Thanks for the re-review. I'll take care of the unused variables > and I'll do an audit of all three tests and look for more. Changes for the next version of the tests: - @robehn CR changes: - changed the JVM/TI function wrappers to be much simpler and just return the JVM/TI return code to the Java code caller; all error checking is now on the Java side of the test. - dropped the 'id' parameter; deleted many native support functions. @robehn - I kept the catch of UnsatisfiedLinkError because what I'm doing there is printing a nice error message and then rethrowing the same exception; it makes it easier to debug the build process for the test. @robehn - I moved the argument parsing code to the main() method; while the default configuration of the test doesn't use command line arguments, I have stress wrappers for these tests that use the command line args. @lyndseyBeil - I renamed the remaining native methods to `camelCase()` style. @sspitsyn - I've removed the unused variables from the three tests. @robehn, @lyndseyBeil and @sspitsyn - thanks for your reviews! New commit coming shortly. ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From dcubed at openjdk.java.net Thu Apr 1 19:46:38 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 1 Apr 2021 19:46:38 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: > Add three tests from JDK-4413752 ported to JVM/TI: > > - RawMonitorEnter() with SuspendThread() > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithRawMonitorEnter/SuspendWithRawMonitorEnter.java > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithRawMonitorEnter/libSuspendWithRawMonitorEnter.cpp > > - ObjectMonitor enter() with SuspendThread() > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorEnter/SuspendWithObjectMonitorEnter.java > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorEnter/libSuspendWithObjectMonitorEnter.cpp > > - ObjectMonitor wait() with SuspendThread > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/SuspendWithObjectMonitorWait.java > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/libSuspendWithObjectMonitorWait.cpp > > The Java files have a transaction diagram to show what each of the > threads in the test is doing. Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: Address lyndseyBeil, robehn and sspitsyn CR comments. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2899/files - new: https://git.openjdk.java.net/jdk/pull/2899/files/d548c7b5..28ac7069 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2899&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2899&range=02-03 Stats: 640 lines in 6 files changed: 211 ins; 309 del; 120 mod Patch: https://git.openjdk.java.net/jdk/pull/2899.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2899/head:pull/2899 PR: https://git.openjdk.java.net/jdk/pull/2899 From dcubed at openjdk.java.net Thu Apr 1 19:46:39 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 1 Apr 2021 19:46:39 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 19:39:07 GMT, Daniel D. Daugherty wrote: >> @sspitsyn - Thanks for the re-review. I'll take care of the unused variables >> and I'll do an audit of all three tests and look for more. > > Changes for the next version of the tests: > > - @robehn CR changes: > - changed the JVM/TI function wrappers to be much simpler and just return the JVM/TI return code to the Java code caller; all error checking is now on the Java side of the test. > - dropped the 'id' parameter; deleted many native support functions. > > @robehn - I kept the catch of UnsatisfiedLinkError because what I'm doing there is printing a nice error message and then rethrowing the same exception; it makes it easier to debug the build process for the test. > @robehn - I moved the argument parsing code to the main() method; while the default configuration of the test doesn't use command line arguments, I have stress wrappers for these tests that use the command line args. > > @lyndseyBeil - I renamed the remaining native methods to `camelCase()` style. > > @sspitsyn - I've removed the unused variables from the three tests. > > @robehn, @lyndseyBeil and @sspitsyn - thanks for your reviews! New commit coming shortly. The v03 version was tested with Mach5 Tier[134567] testing. The three new tests passed in all configurations in all of those tiers. I also used the latest version of the tests to reproduce the failure mode that I'm hunting in "JDK-8264393 JDK-8258284 introduced dangling TLH race" so these tests still help reproduce that bug. ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From dcubed at openjdk.java.net Thu Apr 1 19:46:39 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 1 Apr 2021 19:46:39 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Mon, 22 Mar 2021 09:02:02 GMT, Robbin Ehn wrote: >> @sspitsyn - Thanks for the review! I figured you would enjoy this 20 year old >> blast from the past! I'm tempted to ping @karenkinnear just to see if she'll >> remember these tests! >> >> I'll fix the indents in the .cpp files. Right now I'm using these tests to shake >> out @robehn's work on "JDK-8257831 Suspend with handshakes". > > Hi Dan, > > If you change the native methods, e.g. : > native static void RawMonitorEnter(int id); > To something like: > native static int RawMonitorEnter(); > > You can then handle the jvmti error in the Java code, so you don't need to pass down the 'id' of the thread. > You then remove all debug code from the C-code agent, which removes some native methods also. > Which also leads to that you don't need the thread 'id', instead you can just use the thread name, which removes some Java code. > > Also you shouldn't catch the UnsatisfiedLinkError, as docs also says: "An Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch." > > You also have a lot of code for argument parsing which is not used by the test inside the test methods. > Can you either remove that or if you want it put it in a separate class/method, so e.g. "doWork()" takes parsed arguments instead. > > Thanks, Robbin @robehn, @lyndseyBeil and @sspitsyn - please re-review when you get the chance. ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From lzang at openjdk.java.net Fri Apr 2 09:25:09 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Fri, 2 Apr 2021 09:25:09 GMT Subject: RFR: 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out In-Reply-To: References: <_4pksW9FB4VgM4sosjHs7q9lbPZtyGGy_wc_G53zLVg=.918f3888-1f52-4d30-bf1c-38e344af9e20@github.com> <7p7PD78B0fKuSonRZ44rQWvBKHabsz4n9VT4WbVluFI=.e4248320-f134-4ed0-b073-a3466970932c@github.com> <69h5uaXSdgc4Fkhla6QdtoezGPyVcFMNasVI8iNxYWI=.3a7c3e6a-8dfa-4ff1-a7cf-db71d3ea7bbf@github.com> <2J1CEmVaXJuvU3fseIBYDcJCihqoeAf07bt0hgvJWWE=.259496ed-1883-4035-a4e5-e6c45f511707@github.com> Message-ID: On Tue, 23 Mar 2021 23:25:10 GMT, Chris Plummer wrote: >> Dear Chris? >> >>> I guess I don't understand why you would want write-through for small arrays but not large objects. >> >> I think it is because that the current implementation does not have code which could calculate the object size before scan it. but it has the logic for calculate array length (`calculateArrayMaxLength()`) ahead of scaning. ( BTW, I am planing to add the logic for object, and then rewrite the whole heap dump impl to use writeThrough also for compressed dump, I think that should be in a separate PR) >> >>> But all this seems to be doing is grouping the HPROF_HEAP_DUMP records into an array rather than having them interspersed with other types of records. How does this help, and why would this mode not always be enabled? >> >> I think the original pupose of SEGMENT heap dump is to handle the case for large heap. In the hprof format spec, the size slot of the heap dump is 32bit, which means it has limitation of dump 4GB used heap. so use segmental dump could help resolve the problem. And IMO the reason for not always enable it is because every segment has a header and tail, which may introduce extra memory, althogh it is not much. >> >>> It looks like both HPROF_HEAP_DUMP_SEGMENT and HPROF_HEAP_DUMP_END are followed by a 4-byte size and 4-byte timestamp, although the timestamp seems to always be 0, and the size for END is also always 0. >> >> My understand from code is that 4-byte size and 4-byte timestamp only exist after HPROF_HEAP_DUMP_SEGMENT, and the HPROF_HEAP_DUMP_END actually is the end of the segment. So I think the specification is not accurate. >> >> Thanks, >> Lin > >> > I guess I don't understand why you would want write-through for small arrays but not large objects. >> >> I think it is because that the current implementation does not have code which could calculate the object size before scan it. but it has the logic for calculate array length (`calculateArrayMaxLength()`) ahead of scaning. ( BTW, I am planing to add the logic for object, and then rewrite the whole heap dump impl to use writeThrough also for compressed dump, I think that should be in a separate PR) > > Can't you just call `Oop.getObjectSize()`? > >> > But all this seems to be doing is grouping the HPROF_HEAP_DUMP records into an array rather than having them interspersed with other types of records. How does this help, and why would this mode not always be enabled? >> >> I think the original pupose of SEGMENT heap dump is to handle the case for large heap. In the hprof format spec, the size slot of the heap dump is 32bit, which means it has limitation of dump 4GB used heap. so use segmental dump could help resolve the problem. And IMO the reason for not always enable it is because every segment has a header and tail, which may introduce extra memory, althogh it is not much. > > Ok. So `HPROF_HEAP_DUMP` is just a record, and records have a 32-bit size limit. I assume that previously only one such record was allowed. So `HPROF_HEAP_DUMP_SEGMENT` was created, and the only difference between it and `HPROF_HEAP_DUMP` is that you can have more than one `HPROF_HEAP_DUMP_SEGMENT`. Am I understanding it correctly? Dear @plummercj, I am planing to rewrite the whole implementation with Oop.getObjectSize(), by which most of the object dump code could be write to underlying outputstream directly. And then there is no need to use segmental dump for gziped heap dump. And hence this issue could be fixed at all. Do you think it is OK to include the re-write change in this PR? BRs, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/2803 From ysuenaga at openjdk.java.net Fri Apr 2 12:28:42 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Fri, 2 Apr 2021 12:28:42 GMT Subject: RFR: 8264639: SA: Worker thread for ptrace on Linux should be more robustness Message-ID: LinuxDebuggerLocal has worker thread (LinuxDebuggerLocalWorkerThread) to call ptrace(2) because it have to call from same thread. LinuxDebuggerLocalWorkerThread does not have queue, so it depends on synchronized statement as following: https://github.com/openjdk/jdk/blob/66d9961cbd83dbfca20b0af3c20693438d4aff3f/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxDebuggerLocal.java#L156-L192 If `execute()` is called in same time, `task` would be broken. Fortunately all of caller of `execute()` are guarded with `synchronized` now. But task worker should be more robustness. I used `Executors::newSingleThreadExecutor`, so we don't need to take care task queue. I tested this change with serviceability/sa jtreg tests on Linux x64. ------------- Commit messages: - 8264639: SA: Worker thread for ptrace on Linux should be more robustness Changes: https://git.openjdk.java.net/jdk/pull/3321/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3321&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264639 Stats: 172 lines in 1 file changed: 17 ins; 108 del; 47 mod Patch: https://git.openjdk.java.net/jdk/pull/3321.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3321/head:pull/3321 PR: https://git.openjdk.java.net/jdk/pull/3321 From hseigel at openjdk.java.net Fri Apr 2 13:01:11 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Fri, 2 Apr 2021 13:01:11 GMT Subject: RFR: 8262280: Incorrect exception handling for VMThread in class redefinition In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 16:58:00 GMT, Coleen Phillimore wrote: > This is a trivial change to remove the last TRAPS from redefine_single_class which is called by the VM thread during a safepoint. > Tested with serviceability/jvmti/RedefineClasses, vmTestbase/nsk/jvmti,jdi and jdk/java/lang/instrument tests. And tier1 sanity in progress. The changes look good and trivial. Thank! Harold ------------- Marked as reviewed by hseigel (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3310 From coleenp at openjdk.java.net Fri Apr 2 13:09:29 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 2 Apr 2021 13:09:29 GMT Subject: RFR: 8262280: Incorrect exception handling for VMThread in class redefinition In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 12:58:18 GMT, Harold Seigel wrote: >> This is a trivial change to remove the last TRAPS from redefine_single_class which is called by the VM thread during a safepoint. >> Tested with serviceability/jvmti/RedefineClasses, vmTestbase/nsk/jvmti,jdi and jdk/java/lang/instrument tests. And tier1 sanity in progress. > > The changes look good and trivial. > Thank! Harold Thanks Harold! ------------- PR: https://git.openjdk.java.net/jdk/pull/3310 From coleenp at openjdk.java.net Fri Apr 2 13:09:30 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 2 Apr 2021 13:09:30 GMT Subject: Integrated: 8262280: Incorrect exception handling for VMThread in class redefinition In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 16:58:00 GMT, Coleen Phillimore wrote: > This is a trivial change to remove the last TRAPS from redefine_single_class which is called by the VM thread during a safepoint. > Tested with serviceability/jvmti/RedefineClasses, vmTestbase/nsk/jvmti,jdi and jdk/java/lang/instrument tests. And tier1 sanity in progress. This pull request has now been integrated. Changeset: 885916ed Author: Coleen Phillimore URL: https://git.openjdk.java.net/jdk/commit/885916ed Stats: 19 lines in 2 files changed: 0 ins; 0 del; 19 mod 8262280: Incorrect exception handling for VMThread in class redefinition Reviewed-by: hseigel ------------- PR: https://git.openjdk.java.net/jdk/pull/3310 From sspitsyn at openjdk.java.net Fri Apr 2 17:17:24 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 2 Apr 2021 17:17:24 GMT Subject: RFR: 8264148: Update spec for exceptions retrofitted for exception chaining In-Reply-To: References: Message-ID: On Wed, 24 Mar 2021 23:17:46 GMT, Joe Darcy wrote: > 8264148: Update spec for exceptions retrofitted for exception chaining Joe, The Serviceability part looks good. Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/3182 From sspitsyn at openjdk.java.net Fri Apr 2 17:43:36 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 2 Apr 2021 17:43:36 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v2] In-Reply-To: <653aJgBTTTvZWbRDrROT_5QzhkI6HO074qhVBkM-3ik=.aacec9d5-fd2f-407e-9d44-605664d30689@github.com> References: <653aJgBTTTvZWbRDrROT_5QzhkI6HO074qhVBkM-3ik=.aacec9d5-fd2f-407e-9d44-605664d30689@github.com> Message-ID: On Tue, 30 Mar 2021 23:51:41 GMT, Hui Shi wrote: >> ?ue to large TLAB size >> >> serviceability/jvmti/HeapMonitor tests intermittently fail when using PS/Serial GC, original test has implicit assumptions on TLAB size and depends on allocate fix amount of objects to consume TLAB and trigger object sampling. These tests will fail if TLAB is above 20M (this can easily happen when using PS/Serial GC and heap is large), when allocation can not consume current TLAB and _byte_until_sample. >> >> Fix in tests is adding an explicit GC to consume current TLAB. >> Running on 256G memory machine, make run-test CONF=release TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/" 'JTREG=JOBS=12;VM_OPTIONS=-XX:ActiveProcessorCount=1' >> >> before fix: 6 or 7 tests in 20 tests intermittently fail >> after fix: no failure in 100 runs release/fastdebug >> >> This might also fix https://bugs.openjdk.java.net/browse/JDK-8225313 > > Hui Shi has updated the pull request incrementally with one additional commit since the last revision: > > update copyright year Hi Hui, It looks good to me. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3265 From sspitsyn at openjdk.java.net Fri Apr 2 17:49:21 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 2 Apr 2021 17:49:21 GMT Subject: RFR: 8264311: Heap object statistics In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 21:46:12 GMT, Mikhailo Seledtsov wrote: >> I'm not involved in the Lilliput project, but as long as the stakeholders are ok with that approach, it's probably the way to go, especially since you seem to think the feature might still evolve some. > > If you are planning to integrate this change to jdk/jdk, please make sure to provide tests for this enhancement. Hi Roman, Do you have any test case verifying this? Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/3271 From dfuchs at openjdk.java.net Fri Apr 2 18:00:57 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Fri, 2 Apr 2021 18:00:57 GMT Subject: RFR: 8264124: Update MXBean specification and implementation to extend mapping of CompositeType to records [v7] In-Reply-To: <7Oco_BzQDkrrNFy6FRoUFAUx5IuHu9YtuEfNuLverUI=.b57d8a09-8b39-4db1-a439-57285dedb7f7@github.com> References: <7Oco_BzQDkrrNFy6FRoUFAUx5IuHu9YtuEfNuLverUI=.b57d8a09-8b39-4db1-a439-57285dedb7f7@github.com> Message-ID: > This RFE proposes to extend the MXBean framework to define a mapping to records. > > The MXBean framework already defines a mapping of `CompositeType` to plain java objects. Records are a natural representation of CompositeTypes. A record can be easily reconstructed from a `CompositeData` through the record canonical constructor. A clear advantage of records over plain java objects is that the canonical constructor will not need to be annotated in order to map composite data property names to constructor parameter names. > > With this RFE, here is an example comparing coding a composite type `NamedNumber` that consists of an `int` and a `String`, using records and using a plain java class. In both case, the `CompositeType` looks like this: > > CompositeType( > "NamedNumber", // typeName > "NamedNumber", // description > new String[] {"number", "name"}, // itemNames > new String[] {"number", "name"}, // itemDescriptions > new OpenType[] {SimpleType.INTEGER, > SimpleType.STRING} // itemTypes > ); > > The plain Java class needs a public constructor annotated with `@ConstructorParameters` annotation: > > public class NamedNumber { > public int getNumber() {return number;} > public String getName() {return name;} > @ConstructorParameters({"number", "name"}) > public NamedNumber(int number, String name) { > this.number = number; > this.name = name; > } > private final int number; > private final String name; > } > > And the equivalent with a record class: > > public record NamedNumber(int number, String name) {} Daniel Fuchs has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 13 additional commits since the last revision: - Merge - minor style issue - minor style issue - Integrated review feedback - Improved test - Moved mapping for records just before Mapping for MXBean interface; Improved test. - Merge branch 'master' into mxbeans-8264124 - Minor tweaks. Improved test. - Merge branch 'master' into mxbeans-8264124 - Integrated review feedback. Updated the section for Mapping to records. - ... and 3 more: https://git.openjdk.java.net/jdk/compare/5fe7d4c4...257a6ed8 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3201/files - new: https://git.openjdk.java.net/jdk/pull/3201/files/a624a763..257a6ed8 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3201&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3201&range=05-06 Stats: 5593 lines in 174 files changed: 3784 ins; 623 del; 1186 mod Patch: https://git.openjdk.java.net/jdk/pull/3201.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3201/head:pull/3201 PR: https://git.openjdk.java.net/jdk/pull/3201 From rkennke at openjdk.java.net Fri Apr 2 18:59:21 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Fri, 2 Apr 2021 18:59:21 GMT Subject: RFR: 8264311: Heap object statistics In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 17:46:35 GMT, Serguei Spitsyn wrote: > Hi Roman, > Do you have any test case verifying this? > Thanks, > Serguei Hi Serguei, no, not yet, but I will add some. Thanks for the suggestion! ------------- PR: https://git.openjdk.java.net/jdk/pull/3271 From minqi at openjdk.java.net Fri Apr 2 19:27:59 2021 From: minqi at openjdk.java.net (Yumin Qi) Date: Fri, 2 Apr 2021 19:27:59 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v9] In-Reply-To: References: Message-ID: <_jXjltEgPfLS6x74d5Q3UfwNXMsRvqyM_9-sDzVDcAg=.38d5f001-5351-4208-98d9-42002aecdd19@github.com> > Hi, Please review > > Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with > `java -XX:DumpLoadedClassList= .... ` > to collect shareable class names and saved in file `` , then > `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` > With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. > The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. > New added jcmd option: > `jcmd VM.cds static_dump ` > or > `jcmd VM.cds dynamic_dump ` > To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. > The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. > > Tests: tier1,tier2,tier3,tier4 > > Thanks > Yumin Yumin Qi has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits: - Restructed code and rename LingeredTestApp.java to JCmdTestLingeredApp.java - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Fix revert unintentionally comment, merge master. - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Remove CDS.getVMArguments, changed to use VM.getRuntimeVMArguments. Removed unused function from ClassLoader. Improved InstanceKlass::is_shareable() and related test. Added more test scenarios. - Remove redundant check for if a class is shareable - Fix according to review comment and add more tests - Fix filter more flags to exclude in static dump, add more test cases - Merge branch 'master' into jdk-8259070 - ... and 3 more: https://git.openjdk.java.net/jdk/compare/328e9514...79822e79 ------------- Changes: https://git.openjdk.java.net/jdk/pull/2737/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2737&range=08 Stats: 846 lines in 23 files changed: 772 ins; 58 del; 16 mod Patch: https://git.openjdk.java.net/jdk/pull/2737.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2737/head:pull/2737 PR: https://git.openjdk.java.net/jdk/pull/2737 From cjplummer at openjdk.java.net Fri Apr 2 19:46:13 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 2 Apr 2021 19:46:13 GMT Subject: RFR: 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out In-Reply-To: References: <_4pksW9FB4VgM4sosjHs7q9lbPZtyGGy_wc_G53zLVg=.918f3888-1f52-4d30-bf1c-38e344af9e20@github.com> <7p7PD78B0fKuSonRZ44rQWvBKHabsz4n9VT4WbVluFI=.e4248320-f134-4ed0-b073-a3466970932c@github.com> <69h5uaXSdgc4Fkhla6QdtoezGPyVcFMNasVI8iNxYWI=.3a7c3e6a-8dfa-4ff1-a7cf-db71d3ea7bbf@github.com> <2J1CEmVaXJuvU3fseIBYDcJCihqoeAf07bt0hgvJWWE=.259496ed-1883-4035-a4e5-e6c45f511707@github.com> Message-ID: <9Q5r2cTgc6Nx1FGbC44HAU1pJNdC9O5ifxg0Dhqh4iQ=.060314d7-6b48-435b-94a2-d458085cf48a@github.com> On Fri, 2 Apr 2021 09:22:26 GMT, Lin Zang wrote: >>> > I guess I don't understand why you would want write-through for small arrays but not large objects. >>> >>> I think it is because that the current implementation does not have code which could calculate the object size before scan it. but it has the logic for calculate array length (`calculateArrayMaxLength()`) ahead of scaning. ( BTW, I am planing to add the logic for object, and then rewrite the whole heap dump impl to use writeThrough also for compressed dump, I think that should be in a separate PR) >> >> Can't you just call `Oop.getObjectSize()`? >> >>> > But all this seems to be doing is grouping the HPROF_HEAP_DUMP records into an array rather than having them interspersed with other types of records. How does this help, and why would this mode not always be enabled? >>> >>> I think the original pupose of SEGMENT heap dump is to handle the case for large heap. In the hprof format spec, the size slot of the heap dump is 32bit, which means it has limitation of dump 4GB used heap. so use segmental dump could help resolve the problem. And IMO the reason for not always enable it is because every segment has a header and tail, which may introduce extra memory, althogh it is not much. >> >> Ok. So `HPROF_HEAP_DUMP` is just a record, and records have a 32-bit size limit. I assume that previously only one such record was allowed. So `HPROF_HEAP_DUMP_SEGMENT` was created, and the only difference between it and `HPROF_HEAP_DUMP` is that you can have more than one `HPROF_HEAP_DUMP_SEGMENT`. Am I understanding it correctly? > > Dear @plummercj, > I am planing to rewrite the whole implementation with Oop.getObjectSize(), by which most of the object dump code could be write to underlying outputstream directly. And then there is no need to use segmental dump for gziped heap dump. And hence this issue could be fixed at all. > Do you think it is OK to include the re-write change in this PR? > > BRs, > Lin If your rewrite has the side affect of fixing this issue, what I would suggest is a new CR and PR that just address the rewrite, especially since this PR has a lot of history that I think you are saying will pretty much become obsolete, and therefore is just noise to anyone wanting to review your changes. In the new PR you can note that the changes will also address 8262386, and you can include an `/issue 8262386` comment so it will be closed out with new PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/2803 From cjplummer at openjdk.java.net Fri Apr 2 20:23:22 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 2 Apr 2021 20:23:22 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v2] In-Reply-To: <653aJgBTTTvZWbRDrROT_5QzhkI6HO074qhVBkM-3ik=.aacec9d5-fd2f-407e-9d44-605664d30689@github.com> References: <653aJgBTTTvZWbRDrROT_5QzhkI6HO074qhVBkM-3ik=.aacec9d5-fd2f-407e-9d44-605664d30689@github.com> Message-ID: On Tue, 30 Mar 2021 23:51:41 GMT, Hui Shi wrote: >> ?ue to large TLAB size >> >> serviceability/jvmti/HeapMonitor tests intermittently fail when using PS/Serial GC, original test has implicit assumptions on TLAB size and depends on allocate fix amount of objects to consume TLAB and trigger object sampling. These tests will fail if TLAB is above 20M (this can easily happen when using PS/Serial GC and heap is large), when allocation can not consume current TLAB and _byte_until_sample. >> >> Fix in tests is adding an explicit GC to consume current TLAB. >> Running on 256G memory machine, make run-test CONF=release TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/" 'JTREG=JOBS=12;VM_OPTIONS=-XX:ActiveProcessorCount=1' >> >> before fix: 6 or 7 tests in 20 tests intermittently fail >> after fix: no failure in 100 runs release/fastdebug >> >> This might also fix https://bugs.openjdk.java.net/browse/JDK-8225313 > > Hui Shi has updated the pull request incrementally with one additional commit since the last revision: > > update copyright year Changes requested by cjplummer (Reviewer). test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 176: > 174: // and set _bytes_until_sample to 0. > 175: // initial _bytes_until_sample is geometric variable with the specified mean > 176: // (512K by default), check ThreadHeapSampler::pick_next_geometric_sample() I think there is too much detail about the hotspot implementation in these comments. I think for (1) it is enough to say that the current TLAB needs to be consumed, and that this can be accomplished with a System.gc(). For (2) I don't really understand what you are saying. If you can explaining without referring to specific fields in hotspot, that would be better. I also don't understand how you are making sure that (2) is triggered. Does that happen during the allocation loop? test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 180: > 178: // trigger GC to consume current TLAB in step1 > 179: // consume initial _bytes_until_sample in following loops, each iteration consume > 180: // about 1600KB, 10 iterations can definitly consume intitial _bytes_until_sample I think this comment should be after the System.gc() call. test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 215: > 213: enableSamplingEvents(); > 214: } > 215: // similar reason with sampleEverything, consume TLAB and trigger sample How about: ` // Use System.gc() to consume TLAB and trigger sampling as described above in sampleEverything` test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatObjectCorrectnessTest.java line 49: > 47: > 48: HeapMonitor.enableSamplingEvents(); > 49: // retire TLAB with GC, ensure sample happens, not assume TLAB size // Instead of relying on the allocation loop to fill and retire the TLAB, which might not happen, // use System.gc() to retire the TLAB and ensure sampling happens ------------- PR: https://git.openjdk.java.net/jdk/pull/3265 From cjplummer at openjdk.java.net Fri Apr 2 20:30:29 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 2 Apr 2021 20:30:29 GMT Subject: RFR: 8264639: SA: Worker thread for ptrace on Linux should be more robustness In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 08:27:05 GMT, Yasumasa Suenaga wrote: > LinuxDebuggerLocal has worker thread (LinuxDebuggerLocalWorkerThread) to call ptrace(2) because it have to call from same thread. > > LinuxDebuggerLocalWorkerThread does not have queue, so it depends on synchronized statement as following: > > https://github.com/openjdk/jdk/blob/66d9961cbd83dbfca20b0af3c20693438d4aff3f/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxDebuggerLocal.java#L156-L192 > > If `execute()` is called in same time, `task` would be broken. > Fortunately all of caller of `execute()` are guarded with `synchronized` now. But task worker should be more robustness. > > I used `Executors::newSingleThreadExecutor`, so we don't need to take care task queue. > > I tested this change with serviceability/sa jtreg tests on Linux x64. My first reaction to these changes is that they are fairly extensive, will take a lot of reviewing time, and don't seem to actually fix something that in real life is an issue. So I'm unsure of your motivation for all the work put into this issue. ------------- PR: https://git.openjdk.java.net/jdk/pull/3321 From cjplummer at openjdk.java.net Fri Apr 2 20:49:26 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 2 Apr 2021 20:49:26 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd In-Reply-To: References: Message-ID: On Sun, 28 Mar 2021 07:53:31 GMT, Yasumasa Suenaga wrote: > `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. > > This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. Overall the changes look good. I just have some minor suggestions for comments, code formatting, and help output. src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/SALauncher.java line 89: > 87: if (canConnectToRemote) { > 88: System.out.println(" or jhsdb " + mode + " --connect debugserver"); > 89: System.out.println(" or jhsdb " + mode + " --connect id at debugserver:1234"); You change here makes it look like if you specify `id@` then you also need to specify the port. I'd suggest also including the original line that just has `id at debugserver`. src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/SALauncher.java line 103: > 101: " This option overrides the system property 'sun.jvm.hotspot.rmi.port'. If not specified," + > 102: " the system property is used. If the system property is not set, the default port 1099 is used."); > 103: System.out.println(" --disableregistry Do not start RMI registry (to use RMI registry that exists)"); "to use RMI registry that exists" doesn't read well. Do you mean "use already started RMI registry"? src/jdk.hotspot.agent/share/man/jhsdb.1 line 188: > 186: If not specified, RMI registry will be started on startup. > 187: Otherwise will it not start, the RMI registry (rmid, etc) is needed > 188: before starting debugd. Is this what you mean: `Otherwise it will not be started, and the already started RMI registry will be used instead.` test/hotspot/jtreg/serviceability/sa/sadebugd/DebugdRmidTest.java line 37: > 35: * @test > 36: * @bug 8263636 > 37: * @requires vm.hasSA Can you add an `@summary` comment? test/hotspot/jtreg/serviceability/sa/sadebugd/DebugdRmidTest.java line 58: > 56: .redirectError(ProcessBuilder.Redirect.INHERIT) > 57: .start(); > 58: Thread.sleep(3000); // Sleep 3 sec for waiting to start rmid. "Sleep 3 sec to wait for rmid to start". test/hotspot/jtreg/serviceability/sa/sadebugd/DebugdUtils.java line 41: > 39: > 40: private boolean disableRegistry; > 41: I'd suggest removing the existing empty lines rather than follow the pattern of adding them. ------------- Changes requested by cjplummer (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3233 From hshi at openjdk.java.net Sat Apr 3 04:09:53 2021 From: hshi at openjdk.java.net (Hui Shi) Date: Sat, 3 Apr 2021 04:09:53 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v3] In-Reply-To: References: Message-ID: > ?ue to large TLAB size > > serviceability/jvmti/HeapMonitor tests intermittently fail when using PS/Serial GC, original test has implicit assumptions on TLAB size and depends on allocate fix amount of objects to consume TLAB and trigger object sampling. These tests will fail if TLAB is above 20M (this can easily happen when using PS/Serial GC and heap is large), when allocation can not consume current TLAB and _byte_until_sample. > > Fix in tests is adding an explicit GC to consume current TLAB. > Running on 256G memory machine, make run-test CONF=release TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/" 'JTREG=JOBS=12;VM_OPTIONS=-XX:ActiveProcessorCount=1' > > before fix: 6 or 7 tests in 20 tests intermittently fail > after fix: no failure in 100 runs release/fastdebug > > This might also fix https://bugs.openjdk.java.net/browse/JDK-8225313 Hui Shi has updated the pull request incrementally with one additional commit since the last revision: clarify comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3265/files - new: https://git.openjdk.java.net/jdk/pull/3265/files/3dd108d5..158b54c1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3265&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3265&range=01-02 Stats: 19 lines in 2 files changed: 3 ins; 8 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3265.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3265/head:pull/3265 PR: https://git.openjdk.java.net/jdk/pull/3265 From cjplummer at openjdk.java.net Sat Apr 3 04:21:28 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Sat, 3 Apr 2021 04:21:28 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v3] In-Reply-To: References: Message-ID: On Sat, 3 Apr 2021 04:09:53 GMT, Hui Shi wrote: >> ?ue to large TLAB size >> >> serviceability/jvmti/HeapMonitor tests intermittently fail when using PS/Serial GC, original test has implicit assumptions on TLAB size and depends on allocate fix amount of objects to consume TLAB and trigger object sampling. These tests will fail if TLAB is above 20M (this can easily happen when using PS/Serial GC and heap is large), when allocation can not consume current TLAB and _byte_until_sample. >> >> Fix in tests is adding an explicit GC to consume current TLAB. >> Running on 256G memory machine, make run-test CONF=release TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/" 'JTREG=JOBS=12;VM_OPTIONS=-XX:ActiveProcessorCount=1' >> >> before fix: 6 or 7 tests in 20 tests intermittently fail >> after fix: no failure in 100 runs release/fastdebug >> >> This might also fix https://bugs.openjdk.java.net/browse/JDK-8225313 > > Hui Shi has updated the pull request incrementally with one additional commit since the last revision: > > clarify comments Changes requested by cjplummer (Reviewer). test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 169: > 167: // Trigger GC then loop around an allocation loop and wait until Object Sampling > 168: // is enabled for every later allocation. It takes two steps: > 169: // 1. Consume current TLAB, whose size can be varies with heap/GC configuration "whose size can vary" test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 175: > 173: System.gc(); > 174: // Step2 loop allocation consume "bytes until sample", each iteration allocate > 175: // about 1600KB, 10 iterations can definitly consume initial "bytes until sample" // Step2 loop allocation consumes "bytes until sample". Each iteration allocates // about 1600KB, so 10 iterations will definitely consume initial "bytes until sample" ------------- PR: https://git.openjdk.java.net/jdk/pull/3265 From hshi at openjdk.java.net Sat Apr 3 04:27:36 2021 From: hshi at openjdk.java.net (Hui Shi) Date: Sat, 3 Apr 2021 04:27:36 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v3] In-Reply-To: References: Message-ID: On Sat, 3 Apr 2021 04:18:07 GMT, Chris Plummer wrote: >> Hui Shi has updated the pull request incrementally with one additional commit since the last revision: >> >> clarify comments > > test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 169: > >> 167: // Trigger GC then loop around an allocation loop and wait until Object Sampling >> 168: // is enabled for every later allocation. It takes two steps: >> 169: // 1. Consume current TLAB, whose size can be varies with heap/GC configuration > > "whose size can vary" sorry for so many typo and spell issues. ------------- PR: https://git.openjdk.java.net/jdk/pull/3265 From hshi at openjdk.java.net Sat Apr 3 04:27:38 2021 From: hshi at openjdk.java.net (Hui Shi) Date: Sat, 3 Apr 2021 04:27:38 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v2] In-Reply-To: References: <653aJgBTTTvZWbRDrROT_5QzhkI6HO074qhVBkM-3ik=.aacec9d5-fd2f-407e-9d44-605664d30689@github.com> Message-ID: On Fri, 2 Apr 2021 20:08:50 GMT, Chris Plummer wrote: >> Hui Shi has updated the pull request incrementally with one additional commit since the last revision: >> >> update copyright year > > test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 176: > >> 174: // and set _bytes_until_sample to 0. >> 175: // initial _bytes_until_sample is geometric variable with the specified mean >> 176: // (512K by default), check ThreadHeapSampler::pick_next_geometric_sample() > > I think there is too much detail about the hotspot implementation in these comments. I think for (1) it is enough to say that the current TLAB needs to be consumed, and that this can be accomplished with a System.gc(). For (2) I don't really understand what you are saying. If you can explaining without referring to specific fields in hotspot, that would be better. I also don't understand how you are making sure that (2) is triggered. Does that happen during the allocation loop? Thanks for your patient review?New comment is simplified. Step1 is retiring current TLAB. When allocating new TLAB, runtime checks if JVMTI object sample is enabled and report event. After report runtime also calculates how many bytes to skip before perform next object sample, which is stored in ThreadHeapSampler's field _bytes_until_sample. Also need consume these bytes before runtime picks up user specified object sample interval bytes (0 in this case). Step2 is an allocation loop consumes current ThreadHeapSampler's _bytes_until_sample bytes (initial value is randome size about 512K). > test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 180: > >> 178: // trigger GC to consume current TLAB in step1 >> 179: // consume initial _bytes_until_sample in following loops, each iteration consume >> 180: // about 1600KB, 10 iterations can definitly consume intitial _bytes_until_sample > > I think this comment should be after the System.gc() call. Thanks and accepted. ------------- PR: https://git.openjdk.java.net/jdk/pull/3265 From hshi at openjdk.java.net Sat Apr 3 04:35:46 2021 From: hshi at openjdk.java.net (Hui Shi) Date: Sat, 3 Apr 2021 04:35:46 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v4] In-Reply-To: References: Message-ID: <6NQUyu5ntGWkUU0CkTp6Xerp8j5Ode7je9pm6qPB4VA=.b63d1e41-fccf-4e65-8445-a633be77e577@github.com> > ?ue to large TLAB size > > serviceability/jvmti/HeapMonitor tests intermittently fail when using PS/Serial GC, original test has implicit assumptions on TLAB size and depends on allocate fix amount of objects to consume TLAB and trigger object sampling. These tests will fail if TLAB is above 20M (this can easily happen when using PS/Serial GC and heap is large), when allocation can not consume current TLAB and _byte_until_sample. > > Fix in tests is adding an explicit GC to consume current TLAB. > Running on 256G memory machine, make run-test CONF=release TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/" 'JTREG=JOBS=12;VM_OPTIONS=-XX:ActiveProcessorCount=1' > > before fix: 6 or 7 tests in 20 tests intermittently fail > after fix: no failure in 100 runs release/fastdebug > > This might also fix https://bugs.openjdk.java.net/browse/JDK-8225313 Hui Shi has updated the pull request incrementally with one additional commit since the last revision: Fix comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3265/files - new: https://git.openjdk.java.net/jdk/pull/3265/files/158b54c1..e9d2a8f4 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3265&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3265&range=02-03 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3265.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3265/head:pull/3265 PR: https://git.openjdk.java.net/jdk/pull/3265 From hshi at openjdk.java.net Sat Apr 3 04:51:29 2021 From: hshi at openjdk.java.net (Hui Shi) Date: Sat, 3 Apr 2021 04:51:29 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v3] In-Reply-To: References: Message-ID: On Sat, 3 Apr 2021 04:24:47 GMT, Hui Shi wrote: >> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 169: >> >>> 167: // Trigger GC then loop around an allocation loop and wait until Object Sampling >>> 168: // is enabled for every later allocation. It takes two steps: >>> 169: // 1. Consume current TLAB, whose size can be varies with heap/GC configuration >> >> "whose size can vary" > > sorry for so many typo and spell issues. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3265 From hshi at openjdk.java.net Sat Apr 3 04:51:30 2021 From: hshi at openjdk.java.net (Hui Shi) Date: Sat, 3 Apr 2021 04:51:30 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v3] In-Reply-To: References: Message-ID: <1IDSYMGOnBISDK1psAzF3xPBmUUe9kDPmkKNlOwjnBo=.2336b885-5965-4aee-ae68-83056e9a5c42@github.com> On Sat, 3 Apr 2021 04:17:14 GMT, Chris Plummer wrote: >> Hui Shi has updated the pull request incrementally with one additional commit since the last revision: >> >> clarify comments > > test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 175: > >> 173: System.gc(); >> 174: // Step2 loop allocation consume "bytes until sample", each iteration allocate >> 175: // about 1600KB, 10 iterations can definitly consume initial "bytes until sample" > > // Step2 loop allocation consumes "bytes until sample". Each iteration allocates > // about 1600KB, so 10 iterations will definitely consume initial "bytes until sample" Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3265 From ysuenaga at openjdk.java.net Sat Apr 3 12:29:33 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Sat, 3 Apr 2021 12:29:33 GMT Subject: RFR: 8264639: SA: Worker thread for ptrace on Linux should be more robustness In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 20:27:04 GMT, Chris Plummer wrote: > I'm unsure of your motivation for all the work put into this issue. When I modified LinuxDebuggerLocal to add ptrace accessor, I couldn't understand why the method should be qualified `synchronized`. ptrace caller should only one thread, so I understand the need of worker thread. The comment of WorkerThreadTask says the need of worker thread, but it does not mention the caller should be synchronized. We can just add the comment of course, but we can simplify the worker with modern API. We might add new worker tasks in future, so I thought it's better to refactor in this opportunity. For example, following the current approach, we need to add new worker task in below: class CreateDwarfParserTask implements WorkerThreadTask { DwarfParser result; public void doit(LinuxDebuggerLocal debugger) { result = new DwarfParser(libptr, rip, rbp, rsp, debugger); } } CreateDwarfParserTask task = new CreateDwarfParserTask(); workerThread.execute(task); return task.result; But we can simplify it after this PR: try { return ptraceWorkerThreadPool.submit(() -> new DwarfParser(libptr, rip, rbp, rsp, LinuxDebuggerLocal.this)) .get(); } catch (InterruptedException | ExecutionException e) { throw new DebuggerException(e); } This PR will take a lot of reviewing time as you said. If it does not worth to work, I want to add the comment about `synchronized` at least. ------------- PR: https://git.openjdk.java.net/jdk/pull/3321 From cjplummer at openjdk.java.net Sun Apr 4 01:31:24 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Sun, 4 Apr 2021 01:31:24 GMT Subject: RFR: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size [v4] In-Reply-To: <6NQUyu5ntGWkUU0CkTp6Xerp8j5Ode7je9pm6qPB4VA=.b63d1e41-fccf-4e65-8445-a633be77e577@github.com> References: <6NQUyu5ntGWkUU0CkTp6Xerp8j5Ode7je9pm6qPB4VA=.b63d1e41-fccf-4e65-8445-a633be77e577@github.com> Message-ID: On Sat, 3 Apr 2021 04:35:46 GMT, Hui Shi wrote: >> ?ue to large TLAB size >> >> serviceability/jvmti/HeapMonitor tests intermittently fail when using PS/Serial GC, original test has implicit assumptions on TLAB size and depends on allocate fix amount of objects to consume TLAB and trigger object sampling. These tests will fail if TLAB is above 20M (this can easily happen when using PS/Serial GC and heap is large), when allocation can not consume current TLAB and _byte_until_sample. >> >> Fix in tests is adding an explicit GC to consume current TLAB. >> Running on 256G memory machine, make run-test CONF=release TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/" 'JTREG=JOBS=12;VM_OPTIONS=-XX:ActiveProcessorCount=1' >> >> before fix: 6 or 7 tests in 20 tests intermittently fail >> after fix: no failure in 100 runs release/fastdebug >> >> This might also fix https://bugs.openjdk.java.net/browse/JDK-8225313 > > Hui Shi has updated the pull request incrementally with one additional commit since the last revision: > > Fix comments Looks good. Thanks for the updates. ------------- Marked as reviewed by cjplummer (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3265 From cjplummer at openjdk.java.net Sun Apr 4 01:50:28 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Sun, 4 Apr 2021 01:50:28 GMT Subject: RFR: 8264639: SA: Worker thread for ptrace on Linux should be more robustness In-Reply-To: References: Message-ID: On Sat, 3 Apr 2021 12:26:33 GMT, Yasumasa Suenaga wrote: >> My first reaction to these changes is that they are fairly extensive, will take a lot of reviewing time, and don't seem to actually fix something that in real life is an issue. So I'm unsure of your motivation for all the work put into this issue. > >> I'm unsure of your motivation for all the work put into this issue. > > When I modified LinuxDebuggerLocal to add ptrace accessor, I couldn't understand why the method should be qualified `synchronized`. ptrace caller should only one thread, so I understand the need of worker thread. > The comment of WorkerThreadTask says the need of worker thread, but it does not mention the caller should be synchronized. > > We can just add the comment of course, but we can simplify the worker with modern API. We might add new worker tasks in future, so I thought it's better to refactor in this opportunity. > > For example, following the current approach, we need to add new worker task in below: > > class CreateDwarfParserTask implements WorkerThreadTask { > DwarfParser result; > public void doit(LinuxDebuggerLocal debugger) { > result = new DwarfParser(libptr, rip, rbp, rsp, debugger); > } > } > > CreateDwarfParserTask task = new CreateDwarfParserTask(); > workerThread.execute(task); > return task.result; > > But we can simplify it after this PR: > > try { > return ptraceWorkerThreadPool.submit(() -> new DwarfParser(libptr, rip, rbp, rsp, LinuxDebuggerLocal.this)) > .get(); > } catch (InterruptedException | ExecutionException e) { > throw new DebuggerException(e); > } > > This PR will take a lot of reviewing time as you said. If it does not worth to work, I want to add the comment about `synchronized` at least. Can't this issue also be fixed by simply making `execute()` synchronized? Then you don't need to worry about the caller being synchronized. My suggestion would to not make this change at this point, since it's not addressing an actual bug the user can run into. You can add documentation in the code, which can also reference the CR. The CR should be changed to an RFE, and for now closed as "Will Not Fix". You can archive a reference to your current change from the CR in case there ever becomes a real need to address this. Does that sound ok? ------------- PR: https://git.openjdk.java.net/jdk/pull/3321 From pavel.rappo at oracle.com Sun Apr 4 10:50:45 2021 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Sun, 4 Apr 2021 10:50:45 +0000 Subject: *Address.hashCode ignore the upper 32 bits of a long value In-Reply-To: <9611e414ab996d463c07e17867252980@oss.nttdata.com> References: <9611e414ab996d463c07e17867252980@oss.nttdata.com> Message-ID: <6D1AB0C4-C6A6-470D-AB4F-DE3F488A7B99@oracle.com> A better place for this email might be the serviceability-dev mailing list (CC'ed). > On 4 Apr 2021, at 09:31, kariyam wrote: > > Hi, > > I found that sun.jvm.hotspot.debugger.*.*Address.hashCode ignore the upper 32 bits of a long value. > > e.g. https://github.com/openjdk/jdk/blob/3789983e89c9de252ef546a1b98a732a7d066650/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxAddress.java#L56 > > I don't think the upper 32 bits of a long value should be ignored. > IMHO, the Long.hashCode static method is suitable for such cases. > > If it's worth making this change, could anyone submit this issue to JBS? > > I'm ready to submit a pull request, but I don't have an Author role. > > Please let me know if there is a better place to do so. > > Thanks From kariyam at oss.nttdata.com Sun Apr 4 16:38:50 2021 From: kariyam at oss.nttdata.com (Mitsuru Kariya) Date: Mon, 05 Apr 2021 01:38:50 +0900 Subject: *Address.hashCode ignore the upper 32 bits of a long value In-Reply-To: <6D1AB0C4-C6A6-470D-AB4F-DE3F488A7B99@oracle.com> References: <9611e414ab996d463c07e17867252980@oss.nttdata.com> <6D1AB0C4-C6A6-470D-AB4F-DE3F488A7B99@oracle.com> Message-ID: <0722133b150866bb147f4ec353323878@oss.nttdata.com> Thank you for your advice. I subscribed serviceability-dev mailing list. 2021-04-04 19:50 ? Pavel Rappo ????????: > A better place for this email might be the serviceability-dev mailing > list (CC'ed). > >> On 4 Apr 2021, at 09:31, kariyam wrote: >> >> Hi, >> >> I found that sun.jvm.hotspot.debugger.*.*Address.hashCode ignore the >> upper 32 bits of a long value. >> >> e.g. >> https://github.com/openjdk/jdk/blob/3789983e89c9de252ef546a1b98a732a7d066650/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxAddress.java#L56 >> >> I don't think the upper 32 bits of a long value should be ignored. >> IMHO, the Long.hashCode static method is suitable for such cases. >> >> If it's worth making this change, could anyone submit this issue to >> JBS? >> >> I'm ready to submit a pull request, but I don't have an Author role. >> >> Please let me know if there is a better place to do so. >> >> Thanks From ysuenaga at openjdk.java.net Mon Apr 5 00:00:46 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 5 Apr 2021 00:00:46 GMT Subject: RFR: 8264639: SA: Worker thread for ptrace on Linux should be more robustness In-Reply-To: References: Message-ID: <2L398AuADveor0zC_DFvGusMJogXtsv83316jUVb5Cc=.c6d0866f-7e9e-4e0c-8671-2fda1690cfca@github.com> On Sun, 4 Apr 2021 01:47:40 GMT, Chris Plummer wrote: >>> I'm unsure of your motivation for all the work put into this issue. >> >> When I modified LinuxDebuggerLocal to add ptrace accessor, I couldn't understand why the method should be qualified `synchronized`. ptrace caller should only one thread, so I understand the need of worker thread. >> The comment of WorkerThreadTask says the need of worker thread, but it does not mention the caller should be synchronized. >> >> We can just add the comment of course, but we can simplify the worker with modern API. We might add new worker tasks in future, so I thought it's better to refactor in this opportunity. >> >> For example, following the current approach, we need to add new worker task in below: >> >> class CreateDwarfParserTask implements WorkerThreadTask { >> DwarfParser result; >> public void doit(LinuxDebuggerLocal debugger) { >> result = new DwarfParser(libptr, rip, rbp, rsp, debugger); >> } >> } >> >> CreateDwarfParserTask task = new CreateDwarfParserTask(); >> workerThread.execute(task); >> return task.result; >> >> But we can simplify it after this PR: >> >> try { >> return ptraceWorkerThreadPool.submit(() -> new DwarfParser(libptr, rip, rbp, rsp, LinuxDebuggerLocal.this)) >> .get(); >> } catch (InterruptedException | ExecutionException e) { >> throw new DebuggerException(e); >> } >> >> This PR will take a lot of reviewing time as you said. If it does not worth to work, I want to add the comment about `synchronized` at least. > > Can't this issue also be fixed by simply making `execute()` synchronized? Then you don't need to worry about the caller being synchronized. > > My suggestion would to not make this change at this point, since it's not addressing an actual bug the user can run into. You can add documentation in the code, which can also reference the CR. The CR should be changed to an RFE, and for now closed as "Will Not Fix". You can archive a reference to your current change from the CR in case there ever becomes a real need to address this. Does that sound ok? Ok, I will change issue type to RFE, and will close both it and PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/3321 From ysuenaga at openjdk.java.net Mon Apr 5 00:00:47 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 5 Apr 2021 00:00:47 GMT Subject: Withdrawn: 8264639: SA: Worker thread for ptrace on Linux should be more robustness In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 08:27:05 GMT, Yasumasa Suenaga wrote: > LinuxDebuggerLocal has worker thread (LinuxDebuggerLocalWorkerThread) to call ptrace(2) because it have to call from same thread. > > LinuxDebuggerLocalWorkerThread does not have queue, so it depends on synchronized statement as following: > > https://github.com/openjdk/jdk/blob/66d9961cbd83dbfca20b0af3c20693438d4aff3f/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxDebuggerLocal.java#L156-L192 > > If `execute()` is called in same time, `task` would be broken. > Fortunately all of caller of `execute()` are guarded with `synchronized` now. But task worker should be more robustness. > > I used `Executors::newSingleThreadExecutor`, so we don't need to take care task queue. > > I tested this change with serviceability/sa jtreg tests on Linux x64. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3321 From ysuenaga at openjdk.java.net Mon Apr 5 00:25:38 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 5 Apr 2021 00:25:38 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: > `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. > > This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Fix comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3233/files - new: https://git.openjdk.java.net/jdk/pull/3233/files/c09b2177..02d2b259 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3233&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3233&range=00-01 Stats: 11 lines in 4 files changed: 2 ins; 5 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3233.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3233/head:pull/3233 PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Mon Apr 5 00:31:14 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 5 Apr 2021 00:31:14 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 20:34:37 GMT, Chris Plummer wrote: >> Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix comments > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/SALauncher.java line 89: > >> 87: if (canConnectToRemote) { >> 88: System.out.println(" or jhsdb " + mode + " --connect debugserver"); >> 89: System.out.println(" or jhsdb " + mode + " --connect id at debugserver:1234"); > > Your change here makes it look like if you specify `id@` then you also need to specify the port. I'd suggest also including the original line that just has `id at debugserver`. I reverted the original line in new commit. We can also specify port number without `id@` (e.g. `--connect debugserver:1234`). Is it ok not to describe on help message? IMHO it is not good to describe all patterns because it might be verbosely. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From suenaga at oss.nttdata.com Mon Apr 5 00:31:51 2021 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Mon, 5 Apr 2021 09:31:51 +0900 Subject: 8263565: NPE was thrown when sun.jvm.hotspot.rmi.serverNamePrefix was set Message-ID: Hi all, Could you review JDK-8263565? JBS: https://bugs.openjdk.java.net/browse/JDK-8263565 PR: https://github.com/openjdk/jdk/pull/3000 This PR fixes NPE when the user specify sun.jvm.hotspot.rmi.serverNamePrefix system property. It has already been reviewed. I need one more reviewer to push. Thanks, Yasumasa From ysuenaga at openjdk.java.net Mon Apr 5 05:24:05 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 5 Apr 2021 05:24:05 GMT Subject: RFR: 8264686: ClhsdbTestConnectArgument.java should use SATestUtils::validateSADebugDPrivileges Message-ID: [JDK-8263670](https://bugs.openjdk.java.net/browse/JDK-8263670) has introduced `SATestUtils::validateSADebugDPrivileges` to check privileges for on OS X. https://github.com/openjdk/jdk/blob/a209ed01bafb7721d6b733d1c4bd3f1776463b5e/test/lib/jdk/test/lib/SA/SATestUtils.java#L211-L225 Most of testcases in serviceability/sa/sadebugd use it, however ClhsdbTestConnectArgument.java isn't so yet. ------------- Commit messages: - 8264686: ClhsdbTestConnectArgument.java should use SATestUtils::validateSADebugDPrivileges Changes: https://git.openjdk.java.net/jdk/pull/3339/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3339&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264686 Stats: 12 lines in 1 file changed: 0 ins; 11 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3339.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3339/head:pull/3339 PR: https://git.openjdk.java.net/jdk/pull/3339 From hseigel at openjdk.java.net Mon Apr 5 18:05:42 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 5 Apr 2021 18:05:42 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups Message-ID: Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. Thanks, Harold ------------- Commit messages: - 8264711: More runtime TRAPS cleanups Changes: https://git.openjdk.java.net/jdk/pull/3345/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3345&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264711 Stats: 146 lines in 32 files changed: 5 ins; 19 del; 122 mod Patch: https://git.openjdk.java.net/jdk/pull/3345.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3345/head:pull/3345 PR: https://git.openjdk.java.net/jdk/pull/3345 From lfoltan at openjdk.java.net Mon Apr 5 19:07:19 2021 From: lfoltan at openjdk.java.net (Lois Foltan) Date: Mon, 5 Apr 2021 19:07:19 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 17:57:13 GMT, Harold Seigel wrote: > Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Looks good Harold. One minor comment for the file jfrJavaSupport.cpp. Thanks, Lois src/hotspot/share/jfr/jni/jfrJavaSupport.cpp line 144: > 142: ObjectSynchronizer::jni_enter(h_obj, THREAD->as_Java_thread()); > 143: ObjectSynchronizer::notifyall(h_obj, THREAD); > 144: ObjectSynchronizer::jni_exit(THREAD->as_Java_thread(), h_obj()); For consistency can you switch the parameter order for jni_enter as well in this change? It looks a little bit odd that jni_exit was changed and not jni_enter. ------------- Marked as reviewed by lfoltan (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3345 From pchilanomate at openjdk.java.net Mon Apr 5 19:14:26 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Mon, 5 Apr 2021 19:14:26 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 17:57:13 GMT, Harold Seigel wrote: > Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Hi Harold, I looked at the changes to synchronization code and looks good except for one issue below. Thanks, Patricio src/hotspot/share/prims/jni.cpp line 2738: > 2736: > 2737: Handle obj(THREAD, JNIHandles::resolve_non_null(jobj)); > 2738: ObjectSynchronizer::jni_exit(THREAD->as_Java_thread(), obj()); Here we would return JNI_ERR if we throw IMSE from jni_exit(). src/hotspot/share/runtime/synchronizer.cpp line 609: > 607: // intentionally do not use CHECK on check_owner because we must exit the > 608: // monitor even if an exception was already pending. > 609: if (monitor->check_owner(current)) { We can actually throw IMSE from check_owner() if this thread is not the real owner. ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From amenkov at openjdk.java.net Mon Apr 5 19:16:27 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Mon, 5 Apr 2021 19:16:27 GMT Subject: RFR: 8263565: NPE was thrown when sun.jvm.hotspot.rmi.serverNamePrefix was set In-Reply-To: References: Message-ID: On Mon, 15 Mar 2021 02:01:12 GMT, Yasumasa Suenaga wrote: > NPE was thrown when I set server name prefix for debugd as following: > > $ jhsdb -J-Dsun.jvm.hotspot.rmi.serverNamePrefix=test debugd --pid 781 > Attaching to process ID 781 and starting RMI services, please wait... > Error attaching to process or starting server: sun.jvm.hotspot.debugger.DebuggerException: java.lang.NullPointerException: Cannot invoke "String.length()" because "this.input" is null > at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:78) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:379) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:329) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.startServer(HotSpotAgent.java:215) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runDEBUGD(SALauncher.java:431) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:493) > Caused by: java.lang.NullPointerException: Cannot invoke "String.length()" because "this.input" is null > at java.base/java.net.URI$Parser.parse(URI.java:3166) > at java.base/java.net.URI.(URI.java:623) > at java.rmi/java.rmi.Naming.intParseURL(Naming.java:273) > at java.rmi/java.rmi.Naming.parseURL(Naming.java:237) > at java.rmi/java.rmi.Naming.rebind(Naming.java:171) > at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:64) > ... 5 more > > `serverNamePrefix` would not affect due to trivial bug. Marked as reviewed by amenkov (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3000 From hseigel at openjdk.java.net Mon Apr 5 20:27:54 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 5 Apr 2021 20:27:54 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v2] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 19:11:49 GMT, Patricio Chilano Mateo wrote: >> Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: >> >> Undo change to ObjectSynchronizer::jni_exit() > > Hi Harold, > > I looked at the changes to synchronization code and looks good except for one issue below. > > Thanks, > Patricio Thanks Lois and Patricio for reviewing the change! I removed my bogus change to ObjectSynchronizer::jni_exit() which also made calls consistent in jfrJavaSupport.cpp. Harold ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From hseigel at openjdk.java.net Mon Apr 5 20:27:53 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 5 Apr 2021 20:27:53 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v2] In-Reply-To: References: Message-ID: > Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: Undo change to ObjectSynchronizer::jni_exit() ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3345/files - new: https://git.openjdk.java.net/jdk/pull/3345/files/b83f1404..e488fb8a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3345&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3345&range=00-01 Stats: 6 lines in 4 files changed: 1 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/3345.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3345/head:pull/3345 PR: https://git.openjdk.java.net/jdk/pull/3345 From cjplummer at openjdk.java.net Mon Apr 5 20:41:21 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 5 Apr 2021 20:41:21 GMT Subject: RFR: 8264686: ClhsdbTestConnectArgument.java should use SATestUtils::validateSADebugDPrivileges In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 01:45:15 GMT, Yasumasa Suenaga wrote: > [JDK-8263670](https://bugs.openjdk.java.net/browse/JDK-8263670) has introduced `SATestUtils::validateSADebugDPrivileges` to check privileges for on OS X. > > https://github.com/openjdk/jdk/blob/a209ed01bafb7721d6b733d1c4bd3f1776463b5e/test/lib/jdk/test/lib/SA/SATestUtils.java#L211-L225 > > Most of testcases in serviceability/sa/sadebugd use it, however ClhsdbTestConnectArgument.java isn't so yet. Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3339 From cjplummer at openjdk.java.net Mon Apr 5 21:03:18 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 5 Apr 2021 21:03:18 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 00:27:35 GMT, Yasumasa Suenaga wrote: >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/SALauncher.java line 89: >> >>> 87: if (canConnectToRemote) { >>> 88: System.out.println(" or jhsdb " + mode + " --connect debugserver"); >>> 89: System.out.println(" or jhsdb " + mode + " --connect id at debugserver:1234"); >> >> Your change here makes it look like if you specify `id@` then you also need to specify the port. I'd suggest also including the original line that just has `id at debugserver`. > > I reverted the original line in new commit. > > We can also specify port number without `id@` (e.g. `--connect debugserver:1234`). Is it ok not to describe on help message? IMHO it is not good to describe all patterns because it might be verbosely. It's hard to say what's best here. You don't want to give examples that may be misleading, but in some cases including every possible example can become too verbose. It looks like there are 4 possible examples here; with and w/o `id` and with and w/o the port. Including 3 of the 4 makes me think you should just add the 1 missing one to make it complete. On the other hand, maybe you could go with just the `id at debugserver:1234` example and leave the other 3 off. The syntax does clearly show the `id` and port are optional. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From cjplummer at openjdk.java.net Mon Apr 5 21:03:19 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 5 Apr 2021 21:03:19 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 00:25:38 GMT, Yasumasa Suenaga wrote: >> `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. >> >> This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Fix comments test/hotspot/jtreg/serviceability/sa/sadebugd/DebugdRmidTest.java line 51: > 49: JDKToolLauncher rmidLauncher = JDKToolLauncher.createUsingTestJDK("rmid"); > 50: rmidLauncher.addToolArg("-J-Dsun.rmi.activation.execPolicy=none"); > 51: rmidLauncher.addToolArg("-J--add-modules=jdk.hotspot.agent"); Is this really needed? Although SA will be using rmid, I don't understand why rmid needs to know about SA. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From chris.plummer at oracle.com Mon Apr 5 21:17:45 2021 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 5 Apr 2021 14:17:45 -0700 Subject: *Address.hashCode ignore the upper 32 bits of a long value In-Reply-To: <9611e414ab996d463c07e17867252980@oss.nttdata.com> References: <9611e414ab996d463c07e17867252980@oss.nttdata.com> Message-ID: [moving to serviceability-dev] Hi, I'm not sure if Address hashcodes are even used by SA, and if they are, I doubt this slightly improved hash would make a noticeable difference. However, if you want to pursue this change just to get started with making OpenJDK contributions, I'm ok with that. I've filed https://bugs.openjdk.java.net/browse/JDK-8264734 thanks, Chris On 4/4/21 1:31 AM, kariyam wrote: > Hi, > > I found that sun.jvm.hotspot.debugger.*.*Address.hashCode ignore the > upper 32 bits of a long value. > > e.g. > https://github.com/openjdk/jdk/blob/3789983e89c9de252ef546a1b98a732a7d066650/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxAddress.java#L56 > > I don't think the upper 32 bits of a long value should be ignored. > IMHO, the Long.hashCode static method is suitable for such cases. > > If it's worth making this change, could anyone submit this issue to JBS? > > I'm ready to submit a pull request, but I don't have an Author role. > > Please let me know if there is a better place to do so. > > Thanks From pchilanomate at openjdk.java.net Mon Apr 5 21:58:18 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Mon, 5 Apr 2021 21:58:18 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v2] In-Reply-To: References: Message-ID: <2bT7H0E5An2OdtaDZvm3JWsrGlbSGxhSNdlfIp81N4s=.75e61e4b-1ab3-48cc-be1d-5e027a56e079@github.com> On Mon, 5 Apr 2021 20:27:53 GMT, Harold Seigel wrote: >> Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. >> >> Thanks, Harold > > Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: > > Undo change to ObjectSynchronizer::jni_exit() Marked as reviewed by pchilanomate (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From pchilanomate at openjdk.java.net Mon Apr 5 21:58:19 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Mon, 5 Apr 2021 21:58:19 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v2] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 20:24:31 GMT, Harold Seigel wrote: > Thanks Lois and Patricio for reviewing the change! > I removed my bogus change to ObjectSynchronizer::jni_exit() which also made calls consistent in jfrJavaSupport.cpp. > Harold Thanks Harold! Fix looks good. ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From dholmes at openjdk.java.net Mon Apr 5 23:31:27 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Mon, 5 Apr 2021 23:31:27 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v2] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 20:27:53 GMT, Harold Seigel wrote: >> Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. >> >> Thanks, Harold > > Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: > > Undo change to ObjectSynchronizer::jni_exit() Hi Harold, Lots of good cleanup here but also a couple of things that I think are incorrect. Platform string creation can throw exceptions; and I also believe the ClassFileLoadHook can too. Thanks, David src/hotspot/share/classfile/javaClasses.cpp line 396: > 394: > 395: // Converts a C string to a Java String based on current encoding > 396: Handle java_lang_String::create_from_platform_dependent_str(JavaThread* current, const char* str) { This change is incorrect. JNU_NewStringPlatform can raise an exception so TRAPS here is correct. src/hotspot/share/classfile/klassFactory.cpp line 117: > 115: Handle protection_domain, > 116: JvmtiCachedClassFileData** cached_class_file, > 117: TRAPS) { I don't think this removal of TRAPS is correct. The ClassFileLoadHook could result in an exception becoming pending. src/hotspot/share/prims/jvmtiEnv.cpp line 709: > 707: > 708: // need the path as java.lang.String > 709: Handle path = java_lang_String::create_from_platform_dependent_str(THREAD, segment); This change will need reverting as an exception is possible as previously noted. But note that if there was no possibility of exception here then the following check of HAS_PENDING_EXCEPTION should also have been removed. ------------- Changes requested by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3345 From david.holmes at oracle.com Mon Apr 5 23:42:43 2021 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Apr 2021 09:42:43 +1000 Subject: RFR: 8264711: More runtime TRAPS cleanups In-Reply-To: References: Message-ID: On 6/04/2021 5:14 am, Patricio Chilano Mateo wrote: > src/hotspot/share/prims/jni.cpp line 2738: > >> 2736: >> 2737: Handle obj(THREAD, JNIHandles::resolve_non_null(jobj)); >> 2738: ObjectSynchronizer::jni_exit(THREAD->as_Java_thread(), obj()); > > Here we would return JNI_ERR if we throw IMSE from jni_exit(). Strictly speaking we probably should return JNI_ERR in that case, but the spec (as usual) is non-specific about the relationship between error codes and throwing exceptions. I would not suggest making such a change now. Note that we would have to be careful to only return JNI_ERR in the single case of IMSE, and even then we would have to be certain that the IMSE came from the actual "exit" and not e.g. err = MonitorEnter(obj); ... throwIMSE() ... err = MonitorExit(obj) Cheers, David > src/hotspot/share/runtime/synchronizer.cpp line 609: > >> 607: // intentionally do not use CHECK on check_owner because we must exit the >> 608: // monitor even if an exception was already pending. >> 609: if (monitor->check_owner(current)) { > > We can actually throw IMSE from check_owner() if this thread is not the real owner. > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3345 > From ysuenaga at openjdk.java.net Mon Apr 5 23:51:31 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 5 Apr 2021 23:51:31 GMT Subject: Integrated: 8264686: ClhsdbTestConnectArgument.java should use SATestUtils::validateSADebugDPrivileges In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 01:45:15 GMT, Yasumasa Suenaga wrote: > [JDK-8263670](https://bugs.openjdk.java.net/browse/JDK-8263670) has introduced `SATestUtils::validateSADebugDPrivileges` to check privileges for on OS X. > > https://github.com/openjdk/jdk/blob/a209ed01bafb7721d6b733d1c4bd3f1776463b5e/test/lib/jdk/test/lib/SA/SATestUtils.java#L211-L225 > > Most of testcases in serviceability/sa/sadebugd use it, however ClhsdbTestConnectArgument.java isn't so yet. This pull request has now been integrated. Changeset: c41cd152 Author: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/c41cd152 Stats: 12 lines in 1 file changed: 0 ins; 11 del; 1 mod 8264686: ClhsdbTestConnectArgument.java should use SATestUtils::validateSADebugDPrivileges Reviewed-by: cjplummer ------------- PR: https://git.openjdk.java.net/jdk/pull/3339 From ysuenaga at openjdk.java.net Tue Apr 6 00:12:24 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 6 Apr 2021 00:12:24 GMT Subject: Integrated: 8263565: NPE was thrown when sun.jvm.hotspot.rmi.serverNamePrefix was set In-Reply-To: References: Message-ID: On Mon, 15 Mar 2021 02:01:12 GMT, Yasumasa Suenaga wrote: > NPE was thrown when I set server name prefix for debugd as following: > > $ jhsdb -J-Dsun.jvm.hotspot.rmi.serverNamePrefix=test debugd --pid 781 > Attaching to process ID 781 and starting RMI services, please wait... > Error attaching to process or starting server: sun.jvm.hotspot.debugger.DebuggerException: java.lang.NullPointerException: Cannot invoke "String.length()" because "this.input" is null > at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:78) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:379) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:329) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.startServer(HotSpotAgent.java:215) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runDEBUGD(SALauncher.java:431) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:493) > Caused by: java.lang.NullPointerException: Cannot invoke "String.length()" because "this.input" is null > at java.base/java.net.URI$Parser.parse(URI.java:3166) > at java.base/java.net.URI.(URI.java:623) > at java.rmi/java.rmi.Naming.intParseURL(Naming.java:273) > at java.rmi/java.rmi.Naming.parseURL(Naming.java:237) > at java.rmi/java.rmi.Naming.rebind(Naming.java:171) > at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:64) > ... 5 more > > `serverNamePrefix` would not affect due to trivial bug. This pull request has now been integrated. Changeset: b1a225e1 Author: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/b1a225e1 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod 8263565: NPE was thrown when sun.jvm.hotspot.rmi.serverNamePrefix was set Reviewed-by: cjplummer, amenkov ------------- PR: https://git.openjdk.java.net/jdk/pull/3000 From ysuenaga at openjdk.java.net Tue Apr 6 00:59:38 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 6 Apr 2021 00:59:38 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v3] In-Reply-To: References: Message-ID: > `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. > > This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Fix help message for --connect ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3233/files - new: https://git.openjdk.java.net/jdk/pull/3233/files/02d2b259..93c6f13e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3233&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3233&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3233.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3233/head:pull/3233 PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Tue Apr 6 00:59:38 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 6 Apr 2021 00:59:38 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v3] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 20:57:34 GMT, Chris Plummer wrote: > On the other hand, maybe you could go with just the id at debugserver:1234 example and leave the other 3 off. The syntax does clearly show the id and port are optional. Agree, I left `id at debugserver:1234` only in new commit. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Tue Apr 6 00:59:39 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 6 Apr 2021 00:59:39 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 20:50:30 GMT, Chris Plummer wrote: >> Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix comments > > test/hotspot/jtreg/serviceability/sa/sadebugd/DebugdRmidTest.java line 51: > >> 49: JDKToolLauncher rmidLauncher = JDKToolLauncher.createUsingTestJDK("rmid"); >> 50: rmidLauncher.addToolArg("-J-Dsun.rmi.activation.execPolicy=none"); >> 51: rmidLauncher.addToolArg("-J--add-modules=jdk.hotspot.agent"); > > Is this really needed? Although SA will be using rmid, I don't understand why rmid needs to know about SA. They are needed. If we don't give `execPolicy=none`, we can see warning on console. SA code would run on rmid, so we need to add module. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From hshi at openjdk.java.net Tue Apr 6 01:40:37 2021 From: hshi at openjdk.java.net (Hui Shi) Date: Tue, 6 Apr 2021 01:40:37 GMT Subject: Integrated: 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 12:02:40 GMT, Hui Shi wrote: > ?ue to large TLAB size > > serviceability/jvmti/HeapMonitor tests intermittently fail when using PS/Serial GC, original test has implicit assumptions on TLAB size and depends on allocate fix amount of objects to consume TLAB and trigger object sampling. These tests will fail if TLAB is above 20M (this can easily happen when using PS/Serial GC and heap is large), when allocation can not consume current TLAB and _byte_until_sample. > > Fix in tests is adding an explicit GC to consume current TLAB. > Running on 256G memory machine, make run-test CONF=release TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/" 'JTREG=JOBS=12;VM_OPTIONS=-XX:ActiveProcessorCount=1' > > before fix: 6 or 7 tests in 20 tests intermittently fail > after fix: no failure in 100 runs release/fastdebug > > This might also fix https://bugs.openjdk.java.net/browse/JDK-8225313 This pull request has now been integrated. Changeset: dc608fd0 Author: Hui Shi Committer: Jie Fu URL: https://git.openjdk.java.net/jdk/commit/dc608fd0 Stats: 19 lines in 2 files changed: 12 ins; 0 del; 7 mod 8264411: serviceability/jvmti/HeapMonitor tests intermittently fail due to large TLAB size Reviewed-by: sspitsyn, cjplummer ------------- PR: https://git.openjdk.java.net/jdk/pull/3265 From cjplummer at openjdk.java.net Tue Apr 6 01:45:16 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 6 Apr 2021 01:45:16 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 00:56:08 GMT, Yasumasa Suenaga wrote: >> test/hotspot/jtreg/serviceability/sa/sadebugd/DebugdRmidTest.java line 51: >> >>> 49: JDKToolLauncher rmidLauncher = JDKToolLauncher.createUsingTestJDK("rmid"); >>> 50: rmidLauncher.addToolArg("-J-Dsun.rmi.activation.execPolicy=none"); >>> 51: rmidLauncher.addToolArg("-J--add-modules=jdk.hotspot.agent"); >> >> Is this really needed? Although SA will be using rmid, I don't understand why rmid needs to know about SA. > > They are needed. > > If we don't give `execPolicy=none`, we can see warning on console. > SA code would run on rmid, so we need to add module. I was actually just referring to `--add-modules`, but github added a few lines before the one I selected. I guess I'm not fully understanding what `rmid` is for in this context, and how it relates to the `rmiregistry` command. I thought it was starting the registry, but that does not seem to be it's primary purpose (maybe it starts it as a side affect). Also, it is deprecated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Tue Apr 6 02:50:21 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 6 Apr 2021 02:50:21 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 01:42:45 GMT, Chris Plummer wrote: >> They are needed. >> >> If we don't give `execPolicy=none`, we can see warning on console. >> SA code would run on rmid, so we need to add module. > > I was actually just referring to `--add-modules`, but github added a few lines before the one I selected. > > I guess I'm not fully understanding what `rmid` is for in this context, and how it relates to the `rmiregistry` command. I thought it was starting the registry, but that does not seem to be it's primary purpose (maybe it starts it as a side affect). Also, it is deprecated. SA has RMI remote object, it will be handled in RMI registry. We can see following exception without `--add-modules`. $ jhsdb debugd --pid 40339 --disableregistry --hostname localhost --registryport 1098 Attaching to process ID 40339 and starting RMI services, please wait... Error attaching to process or starting server: sun.jvm.hotspot.debugger.DebuggerException: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:75) at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:380) at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:330) at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.startServer(HotSpotAgent.java:216) at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runDEBUGD(SALauncher.java:437) at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:499) Caused by: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:389) at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200) at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197) at java.base/java.security.AccessController.doPrivileged(AccessController.java:691) at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196) at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587) at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828) at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705) at java.base/java.security.AccessController.doPrivileged(AccessController.java:391) at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:831) at java.rmi/sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:303) at java.rmi/sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:279) at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:380) at java.rmi/sun.rmi.registry.RegistryImpl_Stub.rebind(RegistryImpl_Stub.java:158) at java.rmi/java.rmi.Naming.rebind(Naming.java:177) at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:64) ... 5 more Caused by: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger at java.rmi/sun.rmi.registry.RegistryImpl_Skel.dispatch(RegistryImpl_Skel.java:157) at java.rmi/sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:466) at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:296) at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200) at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197) at java.base/java.security.AccessController.doPrivileged(AccessController.java:691) at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196) at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587) at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828) at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705) at java.base/java.security.AccessController.doPrivileged(AccessController.java:391) at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:831) Caused by: java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:432) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:586) at java.rmi/sun.rmi.server.LoaderHandler$Loader.loadClass(LoaderHandler.java:1207) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:466) at java.rmi/sun.rmi.server.LoaderHandler.loadClassForName(LoaderHandler.java:1221) at java.rmi/sun.rmi.server.LoaderHandler.loadProxyInterfaces(LoaderHandler.java:731) at java.rmi/sun.rmi.server.LoaderHandler.loadProxyClass(LoaderHandler.java:674) at java.rmi/sun.rmi.server.LoaderHandler.loadProxyClass(LoaderHandler.java:611) at java.rmi/java.rmi.server.RMIClassLoader$2.loadProxyClass(RMIClassLoader.java:646) at java.rmi/java.rmi.server.RMIClassLoader.loadProxyClass(RMIClassLoader.java:311) at java.rmi/sun.rmi.server.MarshalInputStream.resolveProxyClass(MarshalInputStream.java:254) at java.base/java.io.ObjectInputStream.readProxyDesc(ObjectInputStream.java:1944) at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1886) at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196) at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1706) at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:496) at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:454) at java.rmi/sun.rmi.registry.RegistryImpl_Skel.dispatch(RegistryImpl_Skel.java:154) ... 14 more As an option, we can also use RMI registry which is started by debugd as following: * console 1 * `jhsdb debugd --disableregistry --pid 40859` * console 2 * `jhsdb -J-Dsun.jvm.hotspot.rmi.serverNamePrefix=second debugd --disableregistry --pid 40860` ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From cjplummer at openjdk.java.net Tue Apr 6 03:18:29 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 6 Apr 2021 03:18:29 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 02:46:40 GMT, Yasumasa Suenaga wrote: >> I was actually just referring to `--add-modules`, but github added a few lines before the one I selected. >> >> I guess I'm not fully understanding what `rmid` is for in this context, and how it relates to the `rmiregistry` command. I thought it was starting the registry, but that does not seem to be it's primary purpose (maybe it starts it as a side affect). Also, it is deprecated. > > SA has RMI remote object, it will be handled in RMI registry. > We can see following exception without `--add-modules`. > > $ jhsdb debugd --pid 40339 --disableregistry --hostname localhost --registryport 1098 > Attaching to process ID 40339 and starting RMI services, please wait... > Error attaching to process or starting server: sun.jvm.hotspot.debugger.DebuggerException: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: > java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: > java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger > at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:75) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:380) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:330) > at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.startServer(HotSpotAgent.java:216) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runDEBUGD(SALauncher.java:437) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:499) > Caused by: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: > java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: > java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger > at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:389) > at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200) > at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197) > at java.base/java.security.AccessController.doPrivileged(AccessController.java:691) > at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196) > at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587) > at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828) > at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705) > at java.base/java.security.AccessController.doPrivileged(AccessController.java:391) > at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704) > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) > at java.base/java.lang.Thread.run(Thread.java:831) > at java.rmi/sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:303) > at java.rmi/sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:279) > at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:380) > at java.rmi/sun.rmi.registry.RegistryImpl_Stub.rebind(RegistryImpl_Stub.java:158) > at java.rmi/java.rmi.Naming.rebind(Naming.java:177) > at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:64) > ... 5 more > Caused by: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: > java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger > at java.rmi/sun.rmi.registry.RegistryImpl_Skel.dispatch(RegistryImpl_Skel.java:157) > at java.rmi/sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:466) > at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:296) > at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200) > at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197) > at java.base/java.security.AccessController.doPrivileged(AccessController.java:691) > at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196) > at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587) > at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828) > at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705) > at java.base/java.security.AccessController.doPrivileged(AccessController.java:391) > at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704) > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) > at java.base/java.lang.Thread.run(Thread.java:831) > Caused by: java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger > at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:432) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:586) > at java.rmi/sun.rmi.server.LoaderHandler$Loader.loadClass(LoaderHandler.java:1207) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) > at java.base/java.lang.Class.forName0(Native Method) > at java.base/java.lang.Class.forName(Class.java:466) > at java.rmi/sun.rmi.server.LoaderHandler.loadClassForName(LoaderHandler.java:1221) > at java.rmi/sun.rmi.server.LoaderHandler.loadProxyInterfaces(LoaderHandler.java:731) > at java.rmi/sun.rmi.server.LoaderHandler.loadProxyClass(LoaderHandler.java:674) > at java.rmi/sun.rmi.server.LoaderHandler.loadProxyClass(LoaderHandler.java:611) > at java.rmi/java.rmi.server.RMIClassLoader$2.loadProxyClass(RMIClassLoader.java:646) > at java.rmi/java.rmi.server.RMIClassLoader.loadProxyClass(RMIClassLoader.java:311) > at java.rmi/sun.rmi.server.MarshalInputStream.resolveProxyClass(MarshalInputStream.java:254) > at java.base/java.io.ObjectInputStream.readProxyDesc(ObjectInputStream.java:1944) > at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1886) > at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196) > at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1706) > at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:496) > at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:454) > at java.rmi/sun.rmi.registry.RegistryImpl_Skel.dispatch(RegistryImpl_Skel.java:154) > ... 14 more > > As an option, we can also use RMI registry which is started by debugd as following: > > * console 1 > * `jhsdb debugd --disableregistry --pid 40859` > * console 2 > * `jhsdb -J-Dsun.jvm.hotspot.rmi.serverNamePrefix=second debugd --disableregistry --pid 40860` Both of the above are using `--disableregistry`. Is that what you meant to do? I would think that you would not want that on the first one. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Tue Apr 6 03:35:25 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 6 Apr 2021 03:35:25 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 03:16:29 GMT, Chris Plummer wrote: >> SA has RMI remote object, it will be handled in RMI registry. >> We can see following exception without `--add-modules`. >> >> $ jhsdb debugd --pid 40339 --disableregistry --hostname localhost --registryport 1098 >> Attaching to process ID 40339 and starting RMI services, please wait... >> Error attaching to process or starting server: sun.jvm.hotspot.debugger.DebuggerException: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: >> java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: >> java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger >> at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:75) >> at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:380) >> at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:330) >> at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.startServer(HotSpotAgent.java:216) >> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runDEBUGD(SALauncher.java:437) >> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:499) >> Caused by: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: >> java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: >> java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger >> at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:389) >> at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200) >> at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197) >> at java.base/java.security.AccessController.doPrivileged(AccessController.java:691) >> at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196) >> at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587) >> at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828) >> at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705) >> at java.base/java.security.AccessController.doPrivileged(AccessController.java:391) >> at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704) >> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) >> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) >> at java.base/java.lang.Thread.run(Thread.java:831) >> at java.rmi/sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:303) >> at java.rmi/sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:279) >> at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:380) >> at java.rmi/sun.rmi.registry.RegistryImpl_Stub.rebind(RegistryImpl_Stub.java:158) >> at java.rmi/java.rmi.Naming.rebind(Naming.java:177) >> at jdk.hotspot.agent/sun.jvm.hotspot.RMIHelper.rebind(RMIHelper.java:64) >> ... 5 more >> Caused by: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: >> java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger >> at java.rmi/sun.rmi.registry.RegistryImpl_Skel.dispatch(RegistryImpl_Skel.java:157) >> at java.rmi/sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:466) >> at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:296) >> at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200) >> at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197) >> at java.base/java.security.AccessController.doPrivileged(AccessController.java:691) >> at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196) >> at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587) >> at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828) >> at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705) >> at java.base/java.security.AccessController.doPrivileged(AccessController.java:391) >> at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704) >> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) >> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) >> at java.base/java.lang.Thread.run(Thread.java:831) >> Caused by: java.lang.ClassNotFoundException: sun.jvm.hotspot.debugger.remote.RemoteDebugger >> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:432) >> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:586) >> at java.rmi/sun.rmi.server.LoaderHandler$Loader.loadClass(LoaderHandler.java:1207) >> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) >> at java.base/java.lang.Class.forName0(Native Method) >> at java.base/java.lang.Class.forName(Class.java:466) >> at java.rmi/sun.rmi.server.LoaderHandler.loadClassForName(LoaderHandler.java:1221) >> at java.rmi/sun.rmi.server.LoaderHandler.loadProxyInterfaces(LoaderHandler.java:731) >> at java.rmi/sun.rmi.server.LoaderHandler.loadProxyClass(LoaderHandler.java:674) >> at java.rmi/sun.rmi.server.LoaderHandler.loadProxyClass(LoaderHandler.java:611) >> at java.rmi/java.rmi.server.RMIClassLoader$2.loadProxyClass(RMIClassLoader.java:646) >> at java.rmi/java.rmi.server.RMIClassLoader.loadProxyClass(RMIClassLoader.java:311) >> at java.rmi/sun.rmi.server.MarshalInputStream.resolveProxyClass(MarshalInputStream.java:254) >> at java.base/java.io.ObjectInputStream.readProxyDesc(ObjectInputStream.java:1944) >> at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1886) >> at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196) >> at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1706) >> at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:496) >> at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:454) >> at java.rmi/sun.rmi.registry.RegistryImpl_Skel.dispatch(RegistryImpl_Skel.java:154) >> ... 14 more >> >> As an option, we can also use RMI registry which is started by debugd as following: >> >> * console 1 >> * `jhsdb debugd --disableregistry --pid 40859` >> * console 2 >> * `jhsdb -J-Dsun.jvm.hotspot.rmi.serverNamePrefix=second debugd --disableregistry --pid 40860` > > Both of the above are using `--disableregistry`. Is that what you meant to do? I would think that you would not want that on the first one. Sorry, the correct commands are as follows: * console 1 (start RMI registry) * `jhsdb debugd --pid 40859` * console 2 (add `--disableregistry` to use it in console 1) * `jhsdb -J-Dsun.jvm.hotspot.rmi.serverNamePrefix=second debugd --disableregistry --pid 40860` ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From cjplummer at openjdk.java.net Tue Apr 6 04:12:32 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 6 Apr 2021 04:12:32 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 03:32:25 GMT, Yasumasa Suenaga wrote: >> Both of the above are using `--disableregistry`. Is that what you meant to do? I would think that you would not want that on the first one. > > Sorry, the correct commands are as follows: > > * console 1 (start RMI registry) > * `jhsdb debugd --pid 40859` > * console 2 (add `--disableregistry` to use it in console 1) > * `jhsdb -J-Dsun.jvm.hotspot.rmi.serverNamePrefix=second debugd --disableregistry --pid 40860` Is this the actual use case you are trying to address (registry already started by a debugd process). If yes, then I think that's the best approach for the test. If no, what is the use case? If it's some random application starting the registry, will there be issue related to it not having been started with `--add-modules=jdk.hotspot.agent`? ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Tue Apr 6 07:30:59 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 6 Apr 2021 07:30:59 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v4] In-Reply-To: References: Message-ID: > `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. > > This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Update testcase ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3233/files - new: https://git.openjdk.java.net/jdk/pull/3233/files/93c6f13e..adadf240 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3233&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3233&range=02-03 Stats: 237 lines in 3 files changed: 123 ins; 114 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3233.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3233/head:pull/3233 PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Tue Apr 6 07:31:01 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 6 Apr 2021 07:31:01 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 04:08:51 GMT, Chris Plummer wrote: >> Sorry, the correct commands are as follows: >> >> * console 1 (start RMI registry) >> * `jhsdb debugd --pid 40859` >> * console 2 (add `--disableregistry` to use it in console 1) >> * `jhsdb -J-Dsun.jvm.hotspot.rmi.serverNamePrefix=second debugd --disableregistry --pid 40860` > > Is this the actual use case you are trying to address (registry already started by a debugd process). If yes, then I think that's the best approach for the test. If no, what is the use case? If it's some random application starting the registry, will there be issue related to it not having been started with `--add-modules=jdk.hotspot.agent`? I updated testcase. Could you review again? ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From rehn at openjdk.java.net Tue Apr 6 09:08:31 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 09:08:31 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <3hnHTIU7h2VZ8ZcubYBQV7L-T8yyoJ1VUdQH5RK8eWE=.afc11483-37d2-4cc6-9177-13c6cb709e88@github.com> Message-ID: On Wed, 31 Mar 2021 14:32:23 GMT, Robbin Ehn wrote: >> SR_handler is used for OS-level suspend/resume (not to conflict with this change-set). >> This feature is only used by JFR (AFAIK), and JFR only samples threads on it's ThreadsList. >> This means the JavaThread can never be terminated, hence this code will always pass. >> >> If someone else is using OS-level suspend/resume without a ThreadsList, the bug is there is no ThreadsList AFAICT. >> >> Since ThreadLocalStorage::thread() is cleared last in ~Thread with Thread::clear_thread_current(); may be in the ~Thread destructor. >> So the argument is that would be safe to read stuff from Thread but not JavaThread? >> Since the compiler (and CPU) may reorder and optimize away code, so I argue reading from a half destroyed object is not a great idea. >> E.g. Monitor* _SR_lock; is not a volatile pointer; since reading from this memory is UB after destruction, compiler is free to remove or not publish the store to NULL. >> >> So I suggest either to remove this check, since the only user is using a ThreadsList and any other should also be using that too. >> Or call Thread::clear_thread_current() before the JavaThread destructor is called, that way we can be certain that there is no UB. > > I got some offline input from David, there seem to be an issue with signal being delivered after the ThreadsListHandle destructor. If that is the case a ThreadsList doesn't help here. > > So I suggested we keep this out of this change-set and just take another suitable field to mirror what we have today. > > `ParkEvent * _ParkEvent;` ? Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 09:08:32 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 09:08:32 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 13:38:59 GMT, Richard Reingruber wrote: >> src/hotspot/share/runtime/handshake.cpp line 617: >> >>> 615: { >>> 616: MutexLocker ml(&_lock, Mutex::_no_safepoint_check_flag); >>> 617: if (!is_suspend_requested()) { >> >> The current thread is _thread_in_vm. Can it be suspended then? For a suspend it has to be in a safe thread state which it cannot leave while suspended. So it must have reached _thread_in_vm not suspended and it cannot be suspended while in that state. > > Shouldn't `is_suspend_requested()` be replaced with `is_suspended()`? See also comment on declaration of `_suspend_requested`. Your are right. Doing a different version. Need some testing. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 09:31:34 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 09:31:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 13:28:14 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.cpp line 715: > >> 713: bool HandshakeState::resume() { >> 714: // Can't be suspended if there is no suspend request. >> 715: if (!is_suspend_requested()) { > > Shouldn't `is_suspend_requested()` be replaced with `is_suspended()`? See also comment on declaration of `_suspend_requested`. Fixed > src/hotspot/share/runtime/handshake.cpp line 720: > >> 718: MutexLocker ml(&_lock, Mutex::_no_safepoint_check_flag); >> 719: // Can't be suspended if there is no suspend request. >> 720: if (!is_suspend_requested()) { > > Shouldn't `is_suspend_requested()` be replaced with `is_suspended()`? See also comment on declaration of `_suspend_requested`. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 09:31:35 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 09:31:35 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 03:21:31 GMT, Patricio Chilano Mateo wrote: >> src/hotspot/share/runtime/handshake.hpp line 142: >> >>> 140: private: >>> 141: volatile bool _suspended; >>> 142: volatile bool _suspend_requested; >> >> According to the PR description `_suspend_requested` is an optimization. >> >>> The "suspend requested" flag is an optimization, without it a dormant thread >>> could be suspended and resumed many times and each would add a new >>> asynchronous handshake. Suspend requested flag means there is already an >>> asynchronous suspend handshake in queue which can be re-used, only the suspend >>> flag needs to be set. >> >> I think there are a few places where _suspend_requested is queried by mistake instead of _suspended. Maybe it would help to prevent this if _suspend_requested was renamed to something that better describes its purpose, e.g. _has_async_suspend_handshake or similar. > > I also think it would be a good idea to rename it with something along those lines. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 09:43:24 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 09:43:24 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <7YguEQOtYiRSePMc_LMqRge_hC82d6W5YFRC3e4hnK0=.b4a7b678-a20b-4089-8a0c-4ed9a2e368b1@github.com> On Tue, 30 Mar 2021 13:36:17 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/prims/jvmtiRawMonitor.cpp line 323: > >> 321: // Suspend request flag can only be set in handshakes. >> 322: // By blocking handshakes, suspend request flag cannot change its value. >> 323: if (!jt->handshake_state()->is_suspend_requested()) { > > Shouldn't `jt->handshake_state()->is_suspend_requested()` be replaced with `jt->is_suspended()`? See also comment on declaration of `_suspend_requested`. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 09:55:34 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 09:55:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> On Tue, 6 Apr 2021 09:48:50 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/objectMonitor.cpp line 415: >> >>> 413: current->set_thread_state_fence(_thread_blocked_trans); >>> 414: if (SafepointMechanism::should_process(current) && >>> 415: current->suspend_request_pending()) { >> >> Shouldn't `suspend_request_pending()` be replaced with `is_suspended()`? See also comment on declaration of `_suspend_requested`. >> And isn't checking `SafepointMechanism::should_process(current)` redundant? > > We don't want to always to take a lock (handshake state lock), if the poll is disarmed there cannot be any suspend request. > We only need to unlock the OM for suspends that happened before we grabbed the OM lock in EnterI(). > > When we leave EnterI we are still blocked, this means a thread might be processing the suspend handshake, which was emitted before we did EnterI(). > To synchronize the suspend check with such a thread we need to grab the HandshakeState lock, e.g. we cannot racy check the suspended flag. > > I choosed to check for an async suspension handshake that needs to be processed. (We can have such without being suspended). We could ignored that async handshake by just checking is_suspended, it would be processed in the else statement instead with "SafepointMechanism::process_if_requested(current);" instead. > > But we avoid releasing the OM lock in cases where we already have been resumed. > > So I can change to is_suspended inside the the method, but we still must do it under the HandshakeState lock. To clarify, is_suspended() can only be checked when the JavaThread is _unsafe_, *after* you have transitioned from the safe state. In this case we are checking if we are suspended before we have completed the transition because we need to know if we should drop the OM lock. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 09:55:33 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 09:55:33 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 13:43:09 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/objectMonitor.cpp line 415: > >> 413: current->set_thread_state_fence(_thread_blocked_trans); >> 414: if (SafepointMechanism::should_process(current) && >> 415: current->suspend_request_pending()) { > > Shouldn't `suspend_request_pending()` be replaced with `is_suspended()`? See also comment on declaration of `_suspend_requested`. > And isn't checking `SafepointMechanism::should_process(current)` redundant? We don't want to always to take a lock (handshake state lock), if the poll is disarmed there cannot be any suspend request. We only need to unlock the OM for suspends that happened before we grabbed the OM lock in EnterI(). When we leave EnterI we are still blocked, this means a thread might be processing the suspend handshake, which was emitted before we did EnterI(). To synchronize the suspend check with such a thread we need to grab the HandshakeState lock, e.g. we cannot racy check the suspended flag. I choosed to check for an async suspension handshake that needs to be processed. (We can have such without being suspended). We could ignored that async handshake by just checking is_suspended, it would be processed in the else statement instead with "SafepointMechanism::process_if_requested(current);" instead. But we avoid releasing the OM lock in cases where we already have been resumed. So I can change to is_suspended inside the the method, but we still must do it under the HandshakeState lock. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 10:11:25 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 10:11:25 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 13:54:58 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/prims/jvmtiRawMonitor.cpp line 364: > >> 362: for (;;) { >> 363: simple_enter(jt); >> 364: if (!SafepointMechanism::should_process(jt)) { > > It seems to be likely that the condition is false in the first loop iteration and the thread has to do another iteration even if not suspended. Wouldn't it be ok to break from the loop if `!jt->is_suspended()`? > I'm asking because in ObjectMonitor::enter() L414 there is similar code and the condition there is `SafepointMechanism::should_process(current) && current->suspend_request_pending()` I didn't add that optimization here because I don't think it is needed, but I can add it. > src/hotspot/share/runtime/handshake.cpp line 486: > >> 484: // Asynchronous may block so they may not execute ~PreserveExceptionMark before safepointing >> 485: // in outer loop. >> 486: op->do_handshake(_handshakee); > > Maybe add PauseNoSafepointVerifier to document that the current thread can transition to a safe state? Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 10:16:26 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 10:16:26 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 14:42:48 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/mutex.cpp line 244: > >> 242: wait_status = _lock.wait(timeout); >> 243: in_flight_mutex = this; // save for ~ThreadBlockInVM >> 244: > > Empty line can be removed. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 10:21:26 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 10:21:26 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 14:55:35 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/mutexLocker.hpp line 260: > >> 258: >> 259: bool wait(int64_t timeout = 0, >> 260: bool as_suspend_equivalent = !Mutex::_as_suspend_equivalent_flag) { > > The declaration of Mutex::_as_suspend_equivalent_flag should be removed. You might want to grep for 'equivalent' to find more that can be removed. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 10:54:27 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 10:54:27 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:22:34 GMT, David Holmes wrote: >> src/hotspot/share/runtime/os.cpp line 874: >> >>> 872: >>> 873: void os::start_thread(Thread* thread) { >>> 874: if (thread->is_Java_thread()) { >> >> Then and else blocks seem to do the very same things. > > Agreed - no need to distinguish between thread types here. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 11:12:34 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 11:12:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <8oSXNMTncb-dM6m7GCF2nmSD8uoRzYktoMXcW9PxOZE=.a6cc5682-c5f2-4584-ba2c-6a69f2f107fb@github.com> On Tue, 30 Mar 2021 19:49:45 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/safepointMechanism.cpp line 101: > >> 99: !thread->handshake_state()->process_by_self()) { >> 100: need_rechecking = true; >> 101: } > > What about this version > if (thread->handshake_state()->should_process()) { > need_rechecking = !thread->handshake_state()->process_by_self(); > } > or even > need_rechecking = > thread->handshake_state()->should_process() && !thread->handshake_state()->process_by_self(); > With the latter you could eliminate L82 > need_rechecking = false; > Also I'd find it more natural if `process_by_self()` could return true if rechecking is needed. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 11:12:32 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 11:12:32 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:59:27 GMT, David Holmes wrote: > Hi Robbin, > > This is a great piece of work with a lot of code simplifications. Unfortunately some new complexities that need to be hidden by appropriate abstractions. Lots of comments, queries and suggestions below. > > Thanks, > David Thanks David! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 11:12:35 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 11:12:35 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 20:27:46 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/sweeper.cpp line 276: > >> 274: >> 275: ThreadBlockInVM tbivm(thread); >> 276: thread->java_suspend_self(); > > In the baseline version of NMethodSweeper::handle_safepoint_request() there is a call `thread->java_suspend_self()`. This call will immediately return, won't it? Do you know what the purpose of this call was? The destructor of ThreadBlockInVM will block the current thread for the safepoint. So `java_suspend_self()` seems redundant. I don't know. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Tue Apr 6 11:31:38 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Tue, 6 Apr 2021 11:31:38 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> Message-ID: On Tue, 6 Apr 2021 09:52:06 GMT, Robbin Ehn wrote: >> We don't want to always to take a lock (handshake state lock), if the poll is disarmed there cannot be any suspend request. >> We only need to unlock the OM for suspends that happened before we grabbed the OM lock in EnterI(). >> >> When we leave EnterI we are still blocked, this means a thread might be processing the suspend handshake, which was emitted before we did EnterI(). >> To synchronize the suspend check with such a thread we need to grab the HandshakeState lock, e.g. we cannot racy check the suspended flag. >> >> I choosed to check for an async suspension handshake that needs to be processed. (We can have such without being suspended). We could ignored that async handshake by just checking is_suspended, it would be processed in the else statement instead with "SafepointMechanism::process_if_requested(current);" instead. >> >> But we avoid releasing the OM lock in cases where we already have been resumed. >> >> So I can change to is_suspended inside the the method, but we still must do it under the HandshakeState lock. > > To clarify, is_suspended() can only be checked when the JavaThread is _unsafe_, *after* you have transitioned from the safe state. > In this case we are checking if we are suspended before we have completed the transition because we need to know if we should drop the OM lock. > > > We don't want to always to take a lock (handshake state lock), if the poll is disarmed there cannot be any suspend request. > We only need to unlock the OM for suspends that happened before we grabbed the OM lock in EnterI(). > > When we leave EnterI we are still blocked, this means a thread might be processing the suspend handshake, which was emitted before we did EnterI(). > To synchronize the suspend check with such a thread we need to grab the HandshakeState lock, e.g. we cannot racy check the suspended flag. I don't think that checking the suspended flag without holding the handshake state lock would be too racy. Holding the OM is sufficient synchronization for checking the suspended flag here. If the suspender thread S did the suspend request while holding the OM then the current thread T will see that it was suspended when it has entered the OM. If S did the suspend without holding OM, only then the check would be racy but that would be ok. > I choosed to check for an async suspension handshake that needs to be processed. (We can have such without being suspended). We could ignored that async handshake by just checking is_suspended, it would be processed in the else statement instead with "SafepointMechanism::process_if_requested(current);" instead. > > But we avoid releasing the OM lock in cases where we already have been resumed. > > So I can change to is_suspended inside the the method, but we still must do it under the HandshakeState lock. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Tue Apr 6 11:59:29 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Tue, 6 Apr 2021 11:59:29 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> Message-ID: On Tue, 6 Apr 2021 11:28:26 GMT, Richard Reingruber wrote: >> To clarify, is_suspended() can only be checked when the JavaThread is _unsafe_, *after* you have transitioned from the safe state. >> In this case we are checking if we are suspended before we have completed the transition because we need to know if we should drop the OM lock. > >> >> >> We don't want to always to take a lock (handshake state lock), if the poll is disarmed there cannot be any suspend request. >> We only need to unlock the OM for suspends that happened before we grabbed the OM lock in EnterI(). >> >> When we leave EnterI we are still blocked, this means a thread might be processing the suspend handshake, which was emitted before we did EnterI(). >> To synchronize the suspend check with such a thread we need to grab the HandshakeState lock, e.g. we cannot racy check the suspended flag. > > I don't think that checking the suspended flag without holding the handshake > state lock would be too racy. Holding the OM is sufficient synchronization for > checking the suspended flag here. > > If the suspender thread S did the suspend request while holding the OM then the > current thread T will see that it was suspended when it has entered the OM. > If S did the suspend without holding OM, only then the check would be racy but that would be ok. > >> I choosed to check for an async suspension handshake that needs to be processed. (We can have such without being suspended). We could ignored that async handshake by just checking is_suspended, it would be processed in the else statement instead with "SafepointMechanism::process_if_requested(current);" instead. >> >> But we avoid releasing the OM lock in cases where we already have been resumed. >> >> So I can change to is_suspended inside the the method, but we still must do it under the HandshakeState lock. > > > To clarify, is_suspended() can only be checked when the JavaThread is _unsafe_, _after_ you have transitioned from the safe state. For a minute I misunderstood this and thought you meant this as a general rule for calling `is_suspended()` while there are examples where it is called without any synchronization (e.g. in `JvmtiEnv::GetThreadState()`) which can be ok. > In this case we are checking if we are suspended before we have completed the transition because we need to know if we should drop the OM lock. I think it is sufficient to hold the OM when calling `is_suspended` to decide if OM has to be dropped. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Tue Apr 6 11:59:29 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Tue, 6 Apr 2021 11:59:29 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <6kAUUsRWTc5_nRULmaJN3HLT47eRR0az61k5BDdoTrE=.9cacc103-bac5-44f4-b61c-749f3f587f6f@github.com> On Tue, 6 Apr 2021 10:07:56 GMT, Robbin Ehn wrote: > > > Sorry, the PauseNoSafepointVerifier needs the nsv in it's constructor and it's not in this scope. > I can't easily add that. Sorry, I overlooked that. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 11:59:29 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 11:59:29 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> Message-ID: <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> On Tue, 6 Apr 2021 11:49:42 GMT, Richard Reingruber wrote: >>> >>> >>> We don't want to always to take a lock (handshake state lock), if the poll is disarmed there cannot be any suspend request. >>> We only need to unlock the OM for suspends that happened before we grabbed the OM lock in EnterI(). >>> >>> When we leave EnterI we are still blocked, this means a thread might be processing the suspend handshake, which was emitted before we did EnterI(). >>> To synchronize the suspend check with such a thread we need to grab the HandshakeState lock, e.g. we cannot racy check the suspended flag. >> >> I don't think that checking the suspended flag without holding the handshake >> state lock would be too racy. Holding the OM is sufficient synchronization for >> checking the suspended flag here. >> >> If the suspender thread S did the suspend request while holding the OM then the >> current thread T will see that it was suspended when it has entered the OM. >> If S did the suspend without holding OM, only then the check would be racy but that would be ok. >> >>> I choosed to check for an async suspension handshake that needs to be processed. (We can have such without being suspended). We could ignored that async handshake by just checking is_suspended, it would be processed in the else statement instead with "SafepointMechanism::process_if_requested(current);" instead. >>> >>> But we avoid releasing the OM lock in cases where we already have been resumed. >>> >>> So I can change to is_suspended inside the the method, but we still must do it under the HandshakeState lock. > >> >> >> To clarify, is_suspended() can only be checked when the JavaThread is _unsafe_, _after_ you have transitioned from the safe state. > > For a minute I misunderstood this and thought you meant this as a general rule > for calling `is_suspended()` while there are examples where it is called without > any synchronization (e.g. in `JvmtiEnv::GetThreadState()`) which can be ok. > >> In this case we are checking if we are suspended before we have completed the transition because we need to know if we should drop the OM lock. > > I think it is sufficient to hold the OM when calling `is_suspended` to decide if OM has to be dropped. I do not follow, the OM is unrelated. The suspender do not hold the OM. What happens is: - Thread A wait on OM X in blocked. - Thread Z suspends Thread A, thread Z have no relation to OM X. - Thread B unlocks OM X, thread B have no relation to the suspend request. - Thread A locks OM X while blocked. - Thread A was not allowed to lock OM X due to it is actually suspended. - Thread A unlocks OM X and waits until resumed. So while A and B fight over OM X, A is trying to figure out if Z suspended him while grabbing the lock. The suspend flag and 'async handshake' flag are protected by the HandshakeState lock. (this is not lockless protocol) The stores in SuspendThreadHandshake are not ordered, so they may happen in any order. To read the stores there you must hold the HandshakeState lock. Therefore if we want the exact answer to am A suspended we must read the flag under the lock. Makes sense? ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Tue Apr 6 12:26:35 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Tue, 6 Apr 2021 12:26:35 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> Message-ID: On Tue, 6 Apr 2021 11:56:36 GMT, Robbin Ehn wrote: > > > I do not follow, the OM is unrelated. > The suspender do not hold the OM. > > What happens is: > > * Thread A wait on OM X in blocked. > > * Thread Z suspends Thread A, thread Z have no relation to OM X. > > * Thread B unlocks OM X, thread B have no relation to the suspend request. > > * Thread A locks OM X while blocked. > > * Thread A was not allowed to lock OM X due to it is actually suspended. > > * Thread A unlocks OM X and waits until resumed. > > > So while A and B fight over OM X, A is trying to figure out if Z suspended him while grabbing the lock. If understand the example correctly then the suspend operation and the operations on OM X are unordered with respect to each other. If so, then we can put them into an order where the suspend happens after "Thread A locks OM X while blocked.". We are free to do this; there's no synchronization that would prevent it. Also when the suspend request happened while A was blocked then after `current->set_thread_state_fence(_thread_blocked_trans);` the check of `is_suspended()` will return true even if the handshake state lock is not acquired for the check. And if Z tried to suspend A after the state change then Z will block because A is not safe anymore. Sorry for insisting. I really hope I'm not wrong ;) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 12:26:35 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 12:26:35 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 10:22:28 GMT, Richard Reingruber wrote: >> My comment was about JVMTI_ERROR_THREAD_NOT_ALIVE > > Sure. I agree with your comment. I think we should add JVMTI_ERROR_THREAD_SUSPENDED as @reinrich says, it is possible for someone to sneak in a second suspend request before us. @dcubed-ojdk it seem like we could be posting JvmtiExport::post_monitor_contended_enter() from the ensure_join() which locks the threadObj. So it might be best to treat this the same way as the others? ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 12:48:35 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 12:48:35 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> Message-ID: On Tue, 6 Apr 2021 12:23:06 GMT, Richard Reingruber wrote: >> I do not follow, the OM is unrelated. >> The suspender do not hold the OM. >> >> What happens is: >> - Thread A wait on OM X in blocked. >> - Thread Z suspends Thread A, thread Z have no relation to OM X. >> - Thread B unlocks OM X, thread B have no relation to the suspend request. >> - Thread A locks OM X while blocked. >> - Thread A was not allowed to lock OM X due to it is actually suspended. >> - Thread A unlocks OM X and waits until resumed. >> >> So while A and B fight over OM X, A is trying to figure out if Z suspended him while grabbing the lock. >> >> The suspend flag and 'async handshake' flag are protected by the HandshakeState lock. (this is not lockless protocol) >> The stores in SuspendThreadHandshake are not ordered, so they may happen in any order. >> To read the stores there you must hold the HandshakeState lock. >> Therefore if we want the exact answer to am A suspended we must read the flag under the lock. >> >> Makes sense? > >> >> >> I do not follow, the OM is unrelated. >> The suspender do not hold the OM. >> >> What happens is: >> >> * Thread A wait on OM X in blocked. >> >> * Thread Z suspends Thread A, thread Z have no relation to OM X. >> >> * Thread B unlocks OM X, thread B have no relation to the suspend request. >> >> * Thread A locks OM X while blocked. >> >> * Thread A was not allowed to lock OM X due to it is actually suspended. >> >> * Thread A unlocks OM X and waits until resumed. >> >> >> So while A and B fight over OM X, A is trying to figure out if Z suspended him while grabbing the lock. > > If understand the example correctly then the suspend operation and the > operations on OM X are unordered with respect to each other. If so, then we can > put them into an order where the suspend happens after "Thread A locks OM X while > blocked.". We are free to do this; there's no synchronization that would prevent it. > > Also when the suspend request happened while A was blocked then after > `current->set_thread_state_fence(_thread_blocked_trans);` the check of > `is_suspended()` will return true even if the handshake state lock is not > acquired for the check. And if Z tried to suspend A after the state change then > Z will block because A is not safe anymore. > > Sorry for insisting. I really hope I'm not wrong ;) The only reason _suspended is volatile is to be able to make the the fast check in resume(). So disregard that early check and that it is volatile, the users of the flag uses HandshakeState lock for synchronizing reads and stores to that flag. E.g set_suspend(true); set_async_suspend_handshake(true); log_trace(thread, suspend)("JavaThread:" INTPTR_FORMAT " suspended, arming ThreadSuspension", p2i(_handshakee)); ThreadSuspensionHandshake* ts = new ThreadSuspensionHandshake(); Handshake::execute(ts, _handshakee); Before we have enqueued the async handshake there is no trap and but the flag set_suspend(true). This means we will miss the async handshake and continue to executed with suspended flag. Both flags and async handshake are set atomically by serializing over the HnadshakeState lock. In this case we want to both know if we are suspended if so process the suspension. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 12:57:32 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 12:57:32 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> Message-ID: On Tue, 6 Apr 2021 12:44:01 GMT, Robbin Ehn wrote: >>> >>> >>> I do not follow, the OM is unrelated. >>> The suspender do not hold the OM. >>> >>> What happens is: >>> >>> * Thread A wait on OM X in blocked. >>> >>> * Thread Z suspends Thread A, thread Z have no relation to OM X. >>> >>> * Thread B unlocks OM X, thread B have no relation to the suspend request. >>> >>> * Thread A locks OM X while blocked. >>> >>> * Thread A was not allowed to lock OM X due to it is actually suspended. >>> >>> * Thread A unlocks OM X and waits until resumed. >>> >>> >>> So while A and B fight over OM X, A is trying to figure out if Z suspended him while grabbing the lock. >> >> If understand the example correctly then the suspend operation and the >> operations on OM X are unordered with respect to each other. If so, then we can >> put them into an order where the suspend happens after "Thread A locks OM X while >> blocked.". We are free to do this; there's no synchronization that would prevent it. >> >> Also when the suspend request happened while A was blocked then after >> `current->set_thread_state_fence(_thread_blocked_trans);` the check of >> `is_suspended()` will return true even if the handshake state lock is not >> acquired for the check. And if Z tried to suspend A after the state change then >> Z will block because A is not safe anymore. >> >> Sorry for insisting. I really hope I'm not wrong ;) > > The only reason _suspended is volatile is to be able to make the the fast check in resume(). > So disregard that early check and that it is volatile, the users of the flag uses HandshakeState lock for synchronizing reads and stores to that flag. > > E.g > set_suspend(true); > set_async_suspend_handshake(true); > log_trace(thread, suspend)("JavaThread:" INTPTR_FORMAT " suspended, arming ThreadSuspension", p2i(_handshakee)); > ThreadSuspensionHandshake* ts = new ThreadSuspensionHandshake(); > Handshake::execute(ts, _handshakee); > > Before we have enqueued the async handshake there is no trap and but the flag set_suspend(true). > This means we will miss the async handshake and continue to executed with suspended flag. > > Both flags and async handshake are set atomically by serializing over the HnadshakeState lock. > > In this case we want to both know if we are suspended if so process the suspension. (technically this will be ordered by the poll is since polls are only disarmed by threads them selfs) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 13:10:31 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 13:10:31 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <3R504XyTsO1yytT_Mkt9EU1JFSP91nUdjtl0njFZFzw=.a0794436-09de-448a-b53e-8eb36c5a481f@github.com> On Wed, 31 Mar 2021 14:01:19 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/prims/jvmtiEnv.cpp line 946: >> >>> 944: // java_thread - pre-checked >>> 945: jvmtiError >>> 946: JvmtiEnv::SuspendThread(JavaThread* java_thread) { >> >> The comment above this still states: >> >> // Threads_lock NOT held, java_thread not protected by lock >> >> but the java_thread is protected by a TLH so we should document that so we know it is always safe to refer to java_thread below. > > These `Threads_lock NOT held ...` comments in JVM/TI are a left over > issue from the Thread-SMR project and I think I forgot to file a follow-up > bug to clean those up. > > I recommend handling that in a general cleanup issue for all of these > comments in JVM/TI (and possibly elsewhere). Please let me know if I > should go ahead and file that bug. Yes, please file that issue. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 6 13:10:34 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 6 Apr 2021 13:10:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <1WF1UhbRqexQnJon-jE6hzSAQSesYP9CSh1LyRQwprs=.6a57db9f-8e38-4554-ad32-941a9d0b6b5e@github.com> On Wed, 31 Mar 2021 03:51:02 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/prims/jvmtiImpl.cpp line 767: > >> 765: // >> 766: >> 767: bool JvmtiSuspendControl::suspend(JavaThread *java_thread) { > > Future RFE: JvmtiSuspendControl is no longer needed Yes ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Tue Apr 6 15:49:33 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Tue, 6 Apr 2021 15:49:33 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> Message-ID: On Tue, 6 Apr 2021 12:54:40 GMT, Robbin Ehn wrote: >> The only reason _suspended is volatile is to be able to make the the fast check in resume(). >> So disregard that early check and that it is volatile, the users of the flag uses HandshakeState lock for synchronizing reads and stores to that flag. >> >> E.g >> set_suspend(true); >> set_async_suspend_handshake(true); >> log_trace(thread, suspend)("JavaThread:" INTPTR_FORMAT " suspended, arming ThreadSuspension", p2i(_handshakee)); >> ThreadSuspensionHandshake* ts = new ThreadSuspensionHandshake(); >> Handshake::execute(ts, _handshakee); >> >> Before we have enqueued the async handshake there is no trap and but the flag set_suspend(true). >> This means we will miss the async handshake and continue to executed with suspended flag. >> >> Both flags and async handshake are set atomically by serializing over the HnadshakeState lock. >> >> In this case we want to both know if we are suspended if so process the suspension. > > (technically this will be ordered by the poll is since polls are only disarmed by threads them selfs) > (meaning if you really promise to call should_process() after you have seen "set_suspend(true);" you will see the async handshake since then we do take the HandshakeState lock.) > Also when the suspend request happened while A was blocked then after > current->set_thread_state_fence(_thread_blocked_trans); the check of > is_suspended() will return true even if the handshake state lock is not > acquired for the check That statement of mine is wrong. (Hopefully) correct is that after the complete state change from _thread_blocked to _thread_in_vm which includes being blocked for a safepoint/handshake the current thread would be able to check is_suspended() without holding the handshake state lock. It does not make a lot of sense then because in an unsafe state JavaThread::current()->is_suspended() will always return false. > Thread A wait on OM X in blocked. > Thread Z suspends Thread A, thread Z have no relation to OM X. > Thread B unlocks OM X, thread B have no relation to the suspend request. > Thread A locks OM X while blocked. > Thread A was not allowed to lock OM X due to it is actually suspended. > Thread A unlocks OM X and waits until resumed. I understand that example now too: OM and suspend operations are unrelated. So I thought it would be ok for A to enter OM X, but it is not. A thread must not leave a safe state if it was suspended in that state (with a handshake). If it did, e.g., its stack could become not walkable. And it must not enter the monitor either. Sorry for the confusion. The check is good. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Tue Apr 6 16:28:52 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Tue, 6 Apr 2021 16:28:52 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 10:05:08 GMT, Robbin Ehn wrote: >> src/hotspot/share/prims/jvmtiRawMonitor.cpp line 364: >> >>> 362: for (;;) { >>> 363: simple_enter(jt); >>> 364: if (!SafepointMechanism::should_process(jt)) { >> >> It seems to be likely that the condition is false in the first loop iteration and the thread has to do another iteration even if not suspended. Wouldn't it be ok to break from the loop if `!jt->is_suspended()`? >> I'm asking because in ObjectMonitor::enter() L414 there is similar code and the condition there is `SafepointMechanism::should_process(current) && current->suspend_request_pending()` > > I didn't add that optimization here because I don't think it is needed, but I can add it. Your version is good. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From pchilanomate at openjdk.java.net Tue Apr 6 17:31:36 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Tue, 6 Apr 2021 17:31:36 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> Message-ID: On Tue, 6 Apr 2021 15:46:17 GMT, Richard Reingruber wrote: >> (technically this will be ordered by the poll is since polls are only disarmed by threads them selfs) >> (meaning if you really promise to call should_process() after you have seen "set_suspend(true);" you will see the async handshake since then we do take the HandshakeState lock.) > >> Also when the suspend request happened while A was blocked then after >> current->set_thread_state_fence(_thread_blocked_trans); the check of >> is_suspended() will return true even if the handshake state lock is not >> acquired for the check > > That statement of mine is wrong. (Hopefully) correct is that after the complete > state change from _thread_blocked to _thread_in_vm which includes being blocked > for a safepoint/handshake the current thread would be able to check > is_suspended() without holding the handshake state lock. It does not make a lot > of sense then because in an unsafe state JavaThread::current()->is_suspended() > will always return false. > >> Thread A wait on OM X in blocked. >> Thread Z suspends Thread A, thread Z have no relation to OM X. >> Thread B unlocks OM X, thread B have no relation to the suspend request. >> Thread A locks OM X while blocked. >> Thread A was not allowed to lock OM X due to it is actually suspended. >> Thread A unlocks OM X and waits until resumed. > > I understand that example now too: OM and suspend operations are unrelated. So I > thought it would be ok for A to enter OM X, but it is not. A thread must not > leave a safe state if it was suspended in that state (with a handshake). If it > did, e.g., its stack could become not walkable. And it must not enter the > monitor either. > > Sorry for the confusion. The check is good. But I still think we can just call is_suspended() here. If is_suspended() returns false, then Z hasn't finished suspending A yet (either async operation was still not added to the queue in case this is the first suspend for A, or do_handshake() hasn't yet finished for that SuspendThreadHandshake operation in case there was already an async operation in the queue). Since we have already acquired OM, then Z will not see that after being suspended A switched from not owning OM to owning it. It will always see that A owned OM. And A will still block for the suspension in the 'else' clause since poll is armed due to SuspendThreadHandshake operation(even if async operation was still not added to the queue at the time A called is_suspended()), but we wouldn't need to release OM. If is_suspended() returns true then at most we will have a false positive, since it could be that after the check somebody already resumed A, but that can also happen even with suspend_request_pending() right after releasing HandshakeState::_lock. Is that correct or am I missing something? ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From pchilanomate at openjdk.java.net Tue Apr 6 18:40:29 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Tue, 6 Apr 2021 18:40:29 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 08:27:35 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: > > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) src/hotspot/share/runtime/handshake.hpp line 122: > 120: // must take slow path, process_by_self(), if _lock is held. > 121: bool should_process() { > 122: return !_queue.is_empty() || _lock.is_locked(); While trying to think of the objectMonitor case I realize the order of this should probably be reversed with a load fence in between. Otherwise we could have the following scenario: Thread A blocks in ~ThreadInVMfromJava() -> wait_for_object_deoptimization() with _thread_blocked state. Thread Z executes SuspendThreadHandshake on A. Gets context switched before adding the async handshake. Thread A unblocks and transitions back to _thread_in_java. Calls SafepointMechanism::process_if_requested()-> process_if_requested_slow() since poll is armed. Checks that the handshake queue is empty. Gets context switched. Thread Z installs async handshake in queue, releases _lock and continues. Thread A checks _lock is unlocked so it doesn't call HandshakeState::process_by_self() to suspend and continues back to Java. (Poll will still be armed though because we will see it in update_poll_values()). ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Tue Apr 6 18:51:42 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 6 Apr 2021 18:51:42 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 12:22:42 GMT, Robbin Ehn wrote: >> Sure. I agree with your comment. > > I think we should add JVMTI_ERROR_THREAD_SUSPENDED as @reinrich says, it is possible for someone to sneak in a second suspend request before us. > > @dcubed-ojdk it seem like we could be posting JvmtiExport::post_monitor_contended_enter() from the ensure_join() which locks the threadObj. > > So it might be best to treat this the same way as the others? By "treat this the same way as the others", you mean check and return either JVMTI_ERROR_THREAD_NOT_ALIVE or JVMTI_ERROR_THREAD_SUSPENDED as appropriate when we get a false back from JvmtiSuspendControl::suspend(current)? I'm not sure what this question is about: > it seem like we could be posting JvmtiExport::post_monitor_contended_enter() from the ensure_join() which locks the threadObj. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From sspitsyn at openjdk.java.net Tue Apr 6 19:00:43 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Tue, 6 Apr 2021 19:00:43 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 11:09:52 GMT, Robbin Ehn wrote: >> Hi Robbin, >> >> This is a great piece of work with a lot of code simplifications. Unfortunately some new complexities that need to be hidden by appropriate abstractions. Lots of comments, queries and suggestions below. >> >> Thanks, >> David > >> Hi Robbin, >> >> This is a great piece of work with a lot of code simplifications. Unfortunately some new complexities that need to be hidden by appropriate abstractions. Lots of comments, queries and suggestions below. >> >> Thanks, >> David > > Thanks David! Hi Robbin, I'm looking at it. I have a concern that it can significantly impact Loom implementation. There is a non-trivial update in Loom to support virtual threads in JVMTI Suspend/Resume. So we need to make sure your approach is going to work for virtual threads as well. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Tue Apr 6 19:00:43 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 6 Apr 2021 19:00:43 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: <3R504XyTsO1yytT_Mkt9EU1JFSP91nUdjtl0njFZFzw=.a0794436-09de-448a-b53e-8eb36c5a481f@github.com> References: <3R504XyTsO1yytT_Mkt9EU1JFSP91nUdjtl0njFZFzw=.a0794436-09de-448a-b53e-8eb36c5a481f@github.com> Message-ID: <9FA4KeXipEhxHnc_K8tpPPejVz3oj_MCzvn9iIhR554=.dae67fa0-9218-49a7-9062-74d082d4e35b@github.com> On Tue, 6 Apr 2021 13:06:39 GMT, Robbin Ehn wrote: >> These `Threads_lock NOT held ...` comments in JVM/TI are a left over >> issue from the Thread-SMR project and I think I forgot to file a follow-up >> bug to clean those up. >> >> I recommend handling that in a general cleanup issue for all of these >> comments in JVM/TI (and possibly elsewhere). Please let me know if I >> should go ahead and file that bug. > > Yes, please file that issue. Done. I filed: JDK-8264800 cleanup Threads_lock comments in JVM/TI function headers ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From cjplummer at openjdk.java.net Tue Apr 6 20:00:31 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 6 Apr 2021 20:00:31 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v4] In-Reply-To: References: Message-ID: <6OcUukKCZ3nRDwprImsbFGxmFCp3hUMsMzQW3-h6U2g=.4b15e61c-2c3a-48ce-8c8a-d1efe7c7f427@github.com> On Tue, 6 Apr 2021 07:30:59 GMT, Yasumasa Suenaga wrote: >> `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. >> >> This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Update testcase Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From cjplummer at openjdk.java.net Tue Apr 6 20:00:31 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 6 Apr 2021 20:00:31 GMT Subject: RFR: 8263636: Add --disableregistry option to jhsdb debugd [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 07:27:14 GMT, Yasumasa Suenaga wrote: >> Is this the actual use case you are trying to address (registry already started by a debugd process). If yes, then I think that's the best approach for the test. If no, what is the use case? If it's some random application starting the registry, will there be issue related to it not having been started with `--add-modules=jdk.hotspot.agent`? > > I updated testcase. Could you review again? Looks good. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Wed Apr 7 00:12:11 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Wed, 7 Apr 2021 00:12:11 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: > `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. > > This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Rename --disableregistry to --disable-registry ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3233/files - new: https://git.openjdk.java.net/jdk/pull/3233/files/adadf240..87d3639a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3233&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3233&range=03-04 Stats: 5 lines in 3 files changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/3233.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3233/head:pull/3233 PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Wed Apr 7 00:12:11 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Wed, 7 Apr 2021 00:12:11 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v4] In-Reply-To: <6OcUukKCZ3nRDwprImsbFGxmFCp3hUMsMzQW3-h6U2g=.4b15e61c-2c3a-48ce-8c8a-d1efe7c7f427@github.com> References: <6OcUukKCZ3nRDwprImsbFGxmFCp3hUMsMzQW3-h6U2g=.4b15e61c-2c3a-48ce-8c8a-d1efe7c7f427@github.com> Message-ID: On Tue, 6 Apr 2021 19:56:53 GMT, Chris Plummer wrote: >> Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: >> >> Update testcase > > Marked as reviewed by cjplummer (Reviewer). Thanks @plummercj for the review! I updated PR to rename `--disableregistry` to `--disable-registry` which is suggested in CSR. I will finalize CSR when it is reviewed. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From cjplummer at openjdk.java.net Wed Apr 7 04:22:26 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 7 Apr 2021 04:22:26 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: <5v7WPiMp87LHJihYmMiwmwMOmlNJEYpfBaf1wQ8IMH4=.58a839d3-dc82-4a93-8f97-e19f31b509b6@github.com> On Wed, 7 Apr 2021 00:12:11 GMT, Yasumasa Suenaga wrote: >> `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. >> >> This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Rename --disableregistry to --disable-registry Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From dholmes at openjdk.java.net Wed Apr 7 04:25:42 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 7 Apr 2021 04:25:42 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 18:47:58 GMT, Daniel D. Daugherty wrote: >> I think we should add JVMTI_ERROR_THREAD_SUSPENDED as @reinrich says, it is possible for someone to sneak in a second suspend request before us. >> >> @dcubed-ojdk it seem like we could be posting JvmtiExport::post_monitor_contended_enter() from the ensure_join() which locks the threadObj. >> >> So it might be best to treat this the same way as the others? > > By "treat this the same way as the others", you mean check and return either > JVMTI_ERROR_THREAD_NOT_ALIVE or JVMTI_ERROR_THREAD_SUSPENDED as > appropriate when we get a false back from JvmtiSuspendControl::suspend(current)? > > I'm not sure what this question is about: > >> it seem like we could be posting JvmtiExport::post_monitor_contended_enter() from the ensure_join() which locks the threadObj. I'm also unclear what Robbin is referring to. I go back to my original comment that surely JVMTI_ERROR_THREAD_NOT_ALIVE is impossible here? ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 06:57:38 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 06:57:38 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> Message-ID: On Tue, 6 Apr 2021 17:27:15 GMT, Patricio Chilano Mateo wrote: >>> Also when the suspend request happened while A was blocked then after >>> current->set_thread_state_fence(_thread_blocked_trans); the check of >>> is_suspended() will return true even if the handshake state lock is not >>> acquired for the check >> >> That statement of mine is wrong. (Hopefully) correct is that after the complete >> state change from _thread_blocked to _thread_in_vm which includes being blocked >> for a safepoint/handshake the current thread would be able to check >> is_suspended() without holding the handshake state lock. It does not make a lot >> of sense then because in an unsafe state JavaThread::current()->is_suspended() >> will always return false. >> >>> Thread A wait on OM X in blocked. >>> Thread Z suspends Thread A, thread Z have no relation to OM X. >>> Thread B unlocks OM X, thread B have no relation to the suspend request. >>> Thread A locks OM X while blocked. >>> Thread A was not allowed to lock OM X due to it is actually suspended. >>> Thread A unlocks OM X and waits until resumed. >> >> I understand that example now too: OM and suspend operations are unrelated. So I >> thought it would be ok for A to enter OM X, but it is not. A thread must not >> leave a safe state if it was suspended in that state (with a handshake). If it >> did, e.g., its stack could become not walkable. And it must not enter the >> monitor either. >> >> Sorry for the confusion. The check is good. > > But I still think we can just call is_suspended() here. If is_suspended() returns false, then Z hasn't finished suspending A yet (either async operation was still not added to the queue in case this is the first suspend for A, or do_handshake() hasn't yet finished for that SuspendThreadHandshake operation in case there was already an async operation in the queue). Since we have already acquired OM, then Z will not see that after being suspended A switched from not owning OM to owning it. It will always see that A owned OM. And A will still block for the suspension in the 'else' clause since poll is armed due to SuspendThreadHandshake operation(even if async operation was still not added to the queue at the time A called is_suspended()), but we wouldn't need to release OM. > If is_suspended() returns true then at most we will have a false positive, since it could be that after the check somebody already resumed A, but that can also happen even with suspend_request_pending() right after releasing HandshakeState::_lock. > Is that correct or am I missing something? I'm not saying it doesn't work, I'm saying it works by accident (I never intended to be able to read the flag without HandshakeState lock). So yes, if the flag is false (lockless read), it is not possible for the suspend to have happened before we locked the OM. And once poll is armed for the synchrone handshake it will stay armed until after we executed the asynchrone handshake trap. What is possible is that we have armed for the suspend but not yet set the flag. If we consider the flag as the tipping point and not the start of execution of the suspend sync handshake, we can do the lockless read. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 07:08:39 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 07:08:39 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 04:22:21 GMT, David Holmes wrote: >> By "treat this the same way as the others", you mean check and return either >> JVMTI_ERROR_THREAD_NOT_ALIVE or JVMTI_ERROR_THREAD_SUSPENDED as >> appropriate when we get a false back from JvmtiSuspendControl::suspend(current)? >> >> I'm not sure what this question is about: >> >>> it seem like we could be posting JvmtiExport::post_monitor_contended_enter() from the ensure_join() which locks the threadObj. > > I'm also unclear what Robbin is referring to. I go back to my original comment that surely JVMTI_ERROR_THREAD_NOT_ALIVE is impossible here? We do a callback with a terminated thread, if the thread then in the agent tries to suspend itself we should return JVMTI_ERROR_THREAD_NOT_ALIVE. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 07:16:43 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 07:16:43 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 03:59:35 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/prims/jvmtiRawMonitor.cpp line 319: > >> 317: jt = self->as_Java_thread(); >> 318: while (true) { >> 319: // To pause suspend requests while in native we must block handshakes. > > The earlier comment says we must be _thread_blocked in this code, so we won't be "native". But that is not a _thread_blocked that is managed by a thread-state transition so that is why this code has to "manually" manage the handshake state. Fixed comment ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 07:23:32 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 07:23:32 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 04:56:32 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/prims/jvmtiRawMonitor.cpp line 428: > >> 426: jt->set_thread_state_fence(_thread_in_native_trans); >> 427: SafepointMechanism::process_if_requested(jt); >> 428: if (jt->is_interrupted(true)) { > > A thread must be _thread_in_vm to safely query is_interrupted() as it accesses the threadObj. Any unsafe (not native/blocked) is fine, therefore I never completed the transition. I set the state to _thread_in_vm before. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 07:28:48 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 07:28:48 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 05:21:03 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.cpp line 463: > >> 461: ThreadInVMForHandshake tivm(_handshakee); >> 462: { >> 463: ttyLocker::break_tty_lock_for_safepoint(os::current_thread_id()); > > Why is this needed when it is inside ThreadInVMForHandshake constructor ?? If we process the async suspension handshake we can go to safepoint. And before safepoint we must drop the tty lock. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 07:28:45 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 07:28:45 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 05:03:47 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.cpp line 415: > >> 413: // Adds are done lock free and so is arming. >> 414: // Calling this method with lock held is considered an error. >> 415: assert(!_lock.owned_by_self(), "Lock should not be held"); > > If adds are still lock-free then why was this assertion removed? Because we add the async handshake inside the sync handshake and thus we already hold the _lock. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From david.holmes at oracle.com Wed Apr 7 07:36:06 2021 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Apr 2021 17:36:06 +1000 Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <1bcc8caa-6353-7b16-4e46-0388e8d3c1de@oracle.com> On 7/04/2021 5:08 pm, Robbin Ehn wrote: > On Wed, 7 Apr 2021 04:22:21 GMT, David Holmes wrote: > >>> By "treat this the same way as the others", you mean check and return either >>> JVMTI_ERROR_THREAD_NOT_ALIVE or JVMTI_ERROR_THREAD_SUSPENDED as >>> appropriate when we get a false back from JvmtiSuspendControl::suspend(current)? >>> >>> I'm not sure what this question is about: >>> >>>> it seem like we could be posting JvmtiExport::post_monitor_contended_enter() from the ensure_join() which locks the threadObj. >> >> I'm also unclear what Robbin is referring to. I go back to my original comment that surely JVMTI_ERROR_THREAD_NOT_ALIVE is impossible here? > > We do a callback with a terminated thread, if the thread then in the agent tries to suspend itself we should return JVMTI_ERROR_THREAD_NOT_ALIVE. How can you do a callback on a terminated thread here? The thread should only respond to the suspend request before it reaches the point in its exit where it would report "not alive". David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From rehn at openjdk.java.net Wed Apr 7 07:37:34 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 07:37:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 05:38:14 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.cpp line 633: > >> 631: bool HandshakeState::suspend_request_pending() { >> 632: assert(JavaThread::current()->thread_state() != _thread_blocked, "should not be in a blocked state"); >> 633: assert(JavaThread::current()->thread_state() != _thread_in_native, "should not be in native"); > > Not clear why we care about the state of the current thread here ?? I missed one assert, this must be asked on yourself. If you ask while in blocked/native the answer may change at any moment, since processing of handshake is ok. To get a stable result you must be in another state. Note @pchilano think we should remove this method, so either I add assert and comment or remove it. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From david.holmes at oracle.com Wed Apr 7 07:40:01 2021 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Apr 2021 17:40:01 +1000 Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <3bf9c296-8c9c-89af-87f7-29bc16d2d55b@oracle.com> On 7/04/2021 5:37 pm, Robbin Ehn wrote: > On Wed, 31 Mar 2021 05:38:14 GMT, David Holmes wrote: > >>> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >>> >>> - Merge branch 'master' into SuspendInHandshake >>> - 8257831: Suspend with handshake (review baseline) >> >> src/hotspot/share/runtime/handshake.cpp line 633: >> >>> 631: bool HandshakeState::suspend_request_pending() { >>> 632: assert(JavaThread::current()->thread_state() != _thread_blocked, "should not be in a blocked state"); >>> 633: assert(JavaThread::current()->thread_state() != _thread_in_native, "should not be in native"); >> >> Not clear why we care about the state of the current thread here ?? > > I missed one assert, this must be asked on yourself. > If you ask while in blocked/native the answer may change at any moment, since processing of handshake is ok. > To get a stable result you must be in another state. My point was more, shouldn't we be querying the state of the handshakee? The current thread's state can't change "at any moment". David > Note @pchilano think we should remove this method, so either I add assert and comment or remove it. > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From rehn at openjdk.java.net Wed Apr 7 08:06:59 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 08:06:59 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 07:04:50 GMT, Robbin Ehn wrote: >> I'm also unclear what Robbin is referring to. I go back to my original comment that surely JVMTI_ERROR_THREAD_NOT_ALIVE is impossible here? > > We do a callback with a terminated thread, if the thread then in the agent tries to suspend itself we should return JVMTI_ERROR_THREAD_NOT_ALIVE. "How can you do a callback on a terminated thread here? The thread should only respond to the suspend request before it reaches the point in its exit where it would report "not alive"." We need to keep the threadObj while we are not terminated. When terminated we need to clear threadObj and notify any waiters. ObjectLocker executes OM::enter(), on contention OM::enter() do a callback to your agent. If you on that callback execute suspendThreadList(), the correct return value for: `"JvmtiSuspendControl::suspend(current)"` would be JVMTI_ERROR_THREAD_NOT_ALIVE, since JavaThread::is_exiting() would return true here. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 08:24:36 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 08:24:36 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 15:50:56 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/objectMonitor.cpp line 973: > >> 971: if (SafepointMechanism::should_process(current)) { >> 972: if (_succ == current) { >> 973: _succ = NULL; > > IIUC then this is now not only executed if a thread is suspended but also when there's a safepoint / handshake. I tried to understand the effect of this but failed with timeout ;) On high-level clearing causes some extra work and contention when exiting. Maybe we should think about just doing this when suspended. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 08:37:33 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 08:37:33 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:08:49 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/objectMonitor.cpp line 411: > >> 409: OrderAccess::storestore(); >> 410: for (;;) { >> 411: current->set_thread_state(_thread_blocked); > > So IIUC because ThreadblockInVM is now responsible for handling thread suspension checks, but those checks need to be done at a lower level, we can no longer use ThreadBlockInVM in some places. That both undermines the value of ThreadBlockInVM and make this manual management code much more complex. > I agree with Richard that a TBIVM that took an unlock function would make this kind of code much clearer. The problem is the we lock the OM from blocked state and then regret it. Since we should not execute 'bytecode' during safe state but this implementation does that we end up in this mess. If this implementation did the correct thing we could use ThreadblockInVM straight forward. As I suggested if we are only blocked around parker it fixes it. The reason for grabbing the lock is that we cannot call exit on the OM which wakes next thread. Only the owner may call exit, this makes succession more complicate because you might wake a suspended thread. Now the suspended thread needs to wake another thread without owning the lock. And yes adding a method ThreadBlockInVM can help here. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 08:44:41 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 08:44:41 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 05:57:30 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/nonJavaThread.cpp line 326: > >> 324: >> 325: while (watcher_thread() != NULL) { >> 326: // This wait should make safepoint checks, wait without a timeout. > > Nit: s/checks, wait/checks and wait/ Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From david.holmes at oracle.com Wed Apr 7 09:45:41 2021 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Apr 2021 19:45:41 +1000 Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <93083fec-a0b8-5f64-f398-e3b1593825f9@oracle.com> On 7/04/2021 6:06 pm, Robbin Ehn wrote: > On Wed, 7 Apr 2021 07:04:50 GMT, Robbin Ehn wrote: > >>> I'm also unclear what Robbin is referring to. I go back to my original comment that surely JVMTI_ERROR_THREAD_NOT_ALIVE is impossible here? >> >> We do a callback with a terminated thread, if the thread then in the agent tries to suspend itself we should return JVMTI_ERROR_THREAD_NOT_ALIVE. > > "How can you do a callback on a terminated thread here? The thread should > only respond to the suspend request before it reaches the point in its > exit where it would report "not alive"." > > We need to keep the threadObj while we are not terminated. > When terminated we need to clear threadObj and notify any waiters. > ObjectLocker executes OM::enter(), on contention OM::enter() do a callback to your agent. > If you on that callback execute suspendThreadList(), the correct return value for: > `"JvmtiSuspendControl::suspend(current)"` would be JVMTI_ERROR_THREAD_NOT_ALIVE, since JavaThread::is_exiting() would return true here. Sorry yes I see what you mean. Thanks, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From david.holmes at oracle.com Wed Apr 7 09:54:27 2021 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Apr 2021 19:54:27 +1000 Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <789bc7b1-273c-bcab-4c85-6692445b934e@oracle.com> On 7/04/2021 6:37 pm, Robbin Ehn wrote: > On Wed, 31 Mar 2021 06:08:49 GMT, David Holmes wrote: > >>> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >>> >>> - Merge branch 'master' into SuspendInHandshake >>> - 8257831: Suspend with handshake (review baseline) >> >> src/hotspot/share/runtime/objectMonitor.cpp line 411: >> >>> 409: OrderAccess::storestore(); >>> 410: for (;;) { >>> 411: current->set_thread_state(_thread_blocked); >> >> So IIUC because ThreadblockInVM is now responsible for handling thread suspension checks, but those checks need to be done at a lower level, we can no longer use ThreadBlockInVM in some places. That both undermines the value of ThreadBlockInVM and make this manual management code much more complex. >> I agree with Richard that a TBIVM that took an unlock function would make this kind of code much clearer. > > The problem is the we lock the OM from blocked state and then regret it. > Since we should not execute 'bytecode' during safe state but this implementation does that we end up in this mess. > If this implementation did the correct thing we could use ThreadblockInVM straight forward. > As I suggested if we are only blocked around parker it fixes it. Pushing the thread-state transition closer to the actual part where we park/block might seem to avoid this issue, but if we were unparked and find ourselves suspended then we still have to wake another waiter, which we aren't allowed to do without owning the lock so ... > The reason for grabbing the lock is that we cannot call exit on the OM which wakes next thread. > Only the owner may call exit, this makes succession more complicate because you might wake a suspended thread. > Now the suspended thread needs to wake another thread without owning the lock. ... moving the scope of the TBIVM doesn't fix it. You've made the TBIVM responsible in part for suspension checks but the scope of the TBIVM as used and the scope where we need suspension checks don't match. > And yes adding a method ThreadBlockInVM can help here. So bottom line is that we can define a new TBIVM that can work in these cases and hide all the explicit state machinations? Thanks, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From rrich at openjdk.java.net Wed Apr 7 09:59:39 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Wed, 7 Apr 2021 09:59:39 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 08:27:35 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: > > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) src/hotspot/share/runtime/objectMonitor.cpp line 428: > 426: } else { > 427: // Only exit path from for loop > 428: SafepointMechanism::process_if_requested(current); I think the current thread can suspend here in L428. This seems problematic as entering OM has been started but not completed. - The current thread is set as owner in ObjectMonitor::_owner - The thread state will still be JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER because of the JavaThreadBlockedOnMonitorEnterState in L389. - Thread::_current_pending_monitor has not been reset to NULL. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Wed Apr 7 10:53:33 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Wed, 7 Apr 2021 10:53:33 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <4uRXV4EtReASGHnfiURatSQEaVR1AJEXyYAkDqbuqp4=.0740be86-2715-4f68-8e35-e3f4811b839f@github.com> <1ezluoPSA1_4PpSyTQKh2dNkBMGMTtlEg3BEzFxCLEo=.36b8a600-8148-4bfb-b8d7-1393b8cf29d7@github.com> Message-ID: <_RUR1QW_lPYryjMWvX15yX2-ivm9kUH28S3JWovYOZ8=.6d8e5bc2-2e5a-44a5-b6de-567e9582fe8d@github.com> On Wed, 7 Apr 2021 06:51:22 GMT, Robbin Ehn wrote: >> But I still think we can just call is_suspended() here. If is_suspended() returns false, then Z hasn't finished suspending A yet (either async operation was still not added to the queue in case this is the first suspend for A, or do_handshake() hasn't yet finished for that SuspendThreadHandshake operation in case there was already an async operation in the queue). Since we have already acquired OM, then Z will not see that after being suspended A switched from not owning OM to owning it. It will always see that A owned OM. And A will still block for the suspension in the 'else' clause since poll is armed due to SuspendThreadHandshake operation(even if async operation was still not added to the queue at the time A called is_suspended()), but we wouldn't need to release OM. >> If is_suspended() returns true then at most we will have a false positive, since it could be that after the check somebody already resumed A, but that can also happen even with suspend_request_pending() right after releasing HandshakeState::_lock. >> Is that correct or am I missing something? > > I'm not saying it doesn't work, I'm saying it works by accident (I never intended to be able to read the flag without HandshakeState lock). > So yes, if the flag is false (lockless read), it is not possible for the suspend to have happened before we locked the OM. > And once poll is armed for the synchrone handshake it will stay armed until after we executed the asynchrone handshake trap. > > What is possible is that we have armed for the suspend but not yet set the flag. > If we consider the flag as the tipping point and not the start of execution of the suspend sync handshake, we can do the lockless read. > But I still think we can just call is_suspended() here @pchilano, potentially I had similar thoughts when I asked about it before the long weekend: the current thread will suspend either way in the then or else block if the execution of the SuspendThreadHandshake was started while the current thread was _thread_blocked. If the suspend is ordered with operations on OM then is_suspended() will return true. Otherwise it is ok to become the owner. This is a more intuitive reasoning. I prefer yours. While looking at this I noticed that suspending in L428 could be problematic because the OM is just entered half way. Maybe we could use plain ThreadBlockInVM and deal with unlocking the OM very late just before waiting in HandshakeState::suspend_in_handshake(). ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 11:46:53 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 11:46:53 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 19:46:38 GMT, Daniel D. Daugherty wrote: >> Add three tests from JDK-4413752 ported to JVM/TI: >> >> - RawMonitorEnter() with SuspendThread() >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithRawMonitorEnter/SuspendWithRawMonitorEnter.java >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithRawMonitorEnter/libSuspendWithRawMonitorEnter.cpp >> >> - ObjectMonitor enter() with SuspendThread() >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorEnter/SuspendWithObjectMonitorEnter.java >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorEnter/libSuspendWithObjectMonitorEnter.cpp >> >> - ObjectMonitor wait() with SuspendThread >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/SuspendWithObjectMonitorWait.java >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/libSuspendWithObjectMonitorWait.cpp >> >> The Java files have a transaction diagram to show what each of the >> threads in the test is doing. > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > Address lyndseyBeil, robehn and sspitsyn CR comments. I really like simple agent code ?? Thanks! ------------- Marked as reviewed by rehn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/2899 From rehn at openjdk.java.net Wed Apr 7 11:52:19 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 11:52:19 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 09:56:19 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/objectMonitor.cpp line 428: > >> 426: } else { >> 427: // Only exit path from for loop >> 428: SafepointMechanism::process_if_requested(current); > > I think the current thread can suspend here in L428. This seems problematic as entering OM has been started but not completed. > > - The current thread is set as owner in ObjectMonitor::_owner > - The thread state will still be JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER because of the JavaThreadBlockedOnMonitorEnterState in L389. > - Thread::_current_pending_monitor has not been reset to NULL. Yes, I think you are correct. I'll look over it. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 12:07:44 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 12:07:44 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 11:50:09 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/objectMonitor.cpp line 428: >> >>> 426: } else { >>> 427: // Only exit path from for loop >>> 428: SafepointMechanism::process_if_requested(current); >> >> I think the current thread can suspend here in L428. This seems problematic as entering OM has been started but not completed. >> >> - The current thread is set as owner in ObjectMonitor::_owner >> - The thread state will still be JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER because of the JavaThreadBlockedOnMonitorEnterState in L389. >> - Thread::_current_pending_monitor has not been reset to NULL. > > Yes, I think you are correct. I'll look over it. Today the ThreadBlockInVM tbivm(current); is inside scope of JavaThreadBlockedOnMonitorEnterState jtbmes(current, this);. So this can happen today also. If you are context switch just before current->set_current_pending_monitor(NULL);. Someone suspends you and look at those states. If you agree that the issue is preexisting I prefer handling that outside scope of this. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From hseigel at openjdk.java.net Wed Apr 7 12:50:13 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Wed, 7 Apr 2021 12:50:13 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v3] In-Reply-To: References: Message-ID: > Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: restore improperly removed TRAPS ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3345/files - new: https://git.openjdk.java.net/jdk/pull/3345/files/e488fb8a..1ee327f0 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3345&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3345&range=01-02 Stats: 42 lines in 7 files changed: 5 ins; 3 del; 34 mod Patch: https://git.openjdk.java.net/jdk/pull/3345.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3345/head:pull/3345 PR: https://git.openjdk.java.net/jdk/pull/3345 From rehn at openjdk.java.net Wed Apr 7 13:01:20 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:01:20 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 05:44:35 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.cpp line 677: > >> 675: } else { >> 676: // Target is going to wake up and leave suspension. >> 677: // Let's just stop the thread from doing that. > > IIUC this would be the case where the target was hit with a suspend request but has not yet processed the actual suspension handshake op, then a resume comes in so suspended is no longer true, and then another suspend request is made (this one) which simply turns the suspended flag back on - is that right? > Overall I'm finding it very hard to see what the two suspend state flags actually signify. I'd like to see that written up somewhere. Sure ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:05:19 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:05:19 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 05:53:43 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.hpp line 148: > >> 146: // Called from the async handshake (the trap) >> 147: // to stop a thread from continuing executing when suspended. >> 148: void suspend_in_handshake(); > > I'm finding the terminology very confusing here: handshake_suspend versus suspend_in_handshake ? It isn't at all clear which method is used in which part of the protocol. The same goes for the different closures. Fixing ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:12:18 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:12:18 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:27:12 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/sweeper.cpp line 276: > >> 274: >> 275: ThreadBlockInVM tbivm(thread); >> 276: thread->java_suspend_self(); > > AFAICS the only change needed in this function was to delete the java_suspend_self as it is not handled in the TBIVM. Fixed > src/hotspot/share/runtime/thread.cpp line 460: > >> 458: delete metadata_handles(); >> 459: >> 460: // SR_handler uses this as a termination indicator - > > As noted previously we need a replacement for this as a proxy for a Thread (not JavaThread) terminating. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Wed Apr 7 13:12:14 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Wed, 7 Apr 2021 13:12:14 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 12:04:42 GMT, Robbin Ehn wrote: > Today the ThreadBlockInVM tbivm(current); is inside scope of JavaThreadBlockedOnMonitorEnterState jtbmes(current, this);. > So this can happen today also. > > If you are context switch just before current->set_current_pending_monitor(NULL);. > Someone suspends you and look at those states. > You mean the JVMTI agent suspends the current thread and then observes that the thread state first has the attribute JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER and in a later call it does not have it anymore (~ThreadBlockInVM doesn't check for suspend)? Yes that's problematic too. With the new code we could remain suspended with JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER. I think the OM would not be reported as owned monitor but another thread could not lock it. > If you agree that the issue is preexisting I prefer handling that outside scope of this. I'm ok with that. A simple solution could then be making use of ThreadBlockInVM. When returning from EnterI in L413 we would set a rollback function in the HandshakeState which can be called in HandshakeState::suspend_in_handshake() to exit the OM. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:12:19 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:12:19 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:27:38 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/sweeper.cpp line 276: > >> 274: } >> 275: MutexUnlocker mu(CodeCache_lock, Mutex::_no_safepoint_check_flag); >> 276: ThreadBlockInVM tbiv(JavaThread::current()); > > You already have the current thread. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:16:51 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:16:51 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 13:08:47 GMT, Richard Reingruber wrote: >> Today the ThreadBlockInVM tbivm(current); is inside scope of JavaThreadBlockedOnMonitorEnterState jtbmes(current, this);. >> So this can happen today also. >> >> If you are context switch just before current->set_current_pending_monitor(NULL);. >> Someone suspends you and look at those states. >> >> If you agree that the issue is preexisting I prefer handling that outside scope of this. > >> Today the ThreadBlockInVM tbivm(current); is inside scope of JavaThreadBlockedOnMonitorEnterState jtbmes(current, this);. >> So this can happen today also. >> >> If you are context switch just before current->set_current_pending_monitor(NULL);. >> Someone suspends you and look at those states. >> > > You mean the JVMTI agent suspends the current thread and then observes that the > thread state first has the attribute JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER and > in a later call it does not have it anymore (~ThreadBlockInVM doesn't check for > suspend)? Yes that's problematic too. > > With the new code we could remain suspended with > JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER. I think the OM would not be > reported as owned monitor but another thread could not lock it. > >> If you agree that the issue is preexisting I prefer handling that outside scope of this. > > I'm ok with that. > > A simple solution could then be making use of ThreadBlockInVM. When returning > from EnterI in L413 we would set a rollback function in the HandshakeState which > can be called in HandshakeState::suspend_in_handshake() to exit the OM. I'm mean the state you describe will be seen on that line: - The current thread is set as owner in ObjectMonitor::_owner - The thread state will still be JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER because of the JavaThreadBlockedOnMonitorEnterState in L389. - Thread::_current_pending_monitor has not been reset to NULL. Will be seen while we are context while in blocked state before clearing the _current_pending_monitor. Ok good. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:24:07 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:24:07 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: <8hNRJg0k8K1pqPObfAJGLkfSJVypvNeQfqSpBSRLvUE=.dadff50d-1895-4dca-a8cd-65eeae14126f@github.com> On Tue, 6 Apr 2021 18:37:38 GMT, Patricio Chilano Mateo wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.hpp line 122: > >> 120: // must take slow path, process_by_self(), if _lock is held. >> 121: bool should_process() { >> 122: return !_queue.is_empty() || _lock.is_locked(); > > While trying to think of the objectMonitor case I realize the order of this should probably be reversed with a load fence in between. Otherwise we could have the following scenario: > > Thread A blocks in ~ThreadInVMfromJava() -> wait_for_object_deoptimization() with _thread_blocked state. > Thread Z executes SuspendThreadHandshake on A. Gets context switched before adding the async handshake. > Thread A unblocks and transitions back to _thread_in_java. Calls SafepointMechanism::process_if_requested()-> process_if_requested_slow() since poll is armed. Checks that the handshake queue is empty. Gets context switched. > Thread Z installs async handshake in queue, releases _lock and continues. > Thread A checks _lock is unlocked so it doesn't call HandshakeState::process_by_self() to suspend and continues back to Java. (Poll will still be armed though because we will see it in update_poll_values()). You are correct. > src/hotspot/share/runtime/thread.hpp line 1142: > >> 1140: bool java_resume(); // higher-level resume logic called by the public APIs >> 1141: bool is_suspended() { return _handshake.is_suspended(); } >> 1142: bool suspend_request_pending() { return _handshake.suspend_request_pending(); } > > If you use is_suspended() to check for suspension in objectMonitor::enter() then we can remove this method. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:24:09 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:24:09 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: <8hNRJg0k8K1pqPObfAJGLkfSJVypvNeQfqSpBSRLvUE=.dadff50d-1895-4dca-a8cd-65eeae14126f@github.com> References: <8hNRJg0k8K1pqPObfAJGLkfSJVypvNeQfqSpBSRLvUE=.dadff50d-1895-4dca-a8cd-65eeae14126f@github.com> Message-ID: On Wed, 7 Apr 2021 13:17:40 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/handshake.hpp line 122: >> >>> 120: // must take slow path, process_by_self(), if _lock is held. >>> 121: bool should_process() { >>> 122: return !_queue.is_empty() || _lock.is_locked(); >> >> While trying to think of the objectMonitor case I realize the order of this should probably be reversed with a load fence in between. Otherwise we could have the following scenario: >> >> Thread A blocks in ~ThreadInVMfromJava() -> wait_for_object_deoptimization() with _thread_blocked state. >> Thread Z executes SuspendThreadHandshake on A. Gets context switched before adding the async handshake. >> Thread A unblocks and transitions back to _thread_in_java. Calls SafepointMechanism::process_if_requested()-> process_if_requested_slow() since poll is armed. Checks that the handshake queue is empty. Gets context switched. >> Thread Z installs async handshake in queue, releases _lock and continues. >> Thread A checks _lock is unlocked so it doesn't call HandshakeState::process_by_self() to suspend and continues back to Java. (Poll will still be armed though because we will see it in update_poll_values()). > > You are correct. Fixed, good catch! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:33:20 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:33:20 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:37:00 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/thread.cpp line 1442: > >> 1440: >> 1441: // The careful dance between thread suspension and exit is handled here: >> 1442: _handshake.thread_exit(); > > I don't like the fact this hides the set_terminating call. > > In fact I think I'd rather keep this in-line rather than tucking it away inside the handshake code. This looks too much like we're informing the handshake that the thread is exiting rather then coordinating thread termination with a late suspension request. The problem is that we need access to the private _lock to coordinate this. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:42:52 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:42:52 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:41:20 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/thread.cpp line 2102: > >> 2100: do { >> 2101: set_thread_state(_thread_blocked); >> 2102: // Check if _external_suspend was set in the previous loop iteration. > > Note that the comment for this method at line 1806 needs updating as it refers to a now deleted method. I removed that line, not sure if the comment needs additional fixing. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:42:55 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:42:55 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 09:37:37 GMT, Richard Reingruber wrote: >> src/hotspot/share/runtime/thread.cpp line 2105: >> >>> 2103: if (is_external_suspend()) { >>> 2104: java_suspend_self(); >>> 2105: } >> >> It is not at all clear to me how this method should interact with thread suspension ?? > > In JavaThread::wait_for_object_deoptimization() the current thread can transition to the safe state _thread_blocked. In that state it can be suspended. In the baseline version wait_for_object_deoptimization() checks for suspension explicitly and self suspends if positive. With this enhancement it happens implicitly by calling SafepointMechanism::process_if_requested(). Yes, thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 13:43:00 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 13:43:00 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:46:00 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/thread.hpp line 1146: > >> 1144: // if external|deopt suspend is present. >> 1145: bool is_suspend_after_native() const { >> 1146: return (_suspend_flags & (_obj_deopt JFR_ONLY(| _trace_flag))) != 0; > > Comment at line 1144 needs updating. Fixed > src/hotspot/share/runtime/thread.inline.hpp line 194: > >> 192: >> 193: inline void JavaThread::set_exiting() { >> 194: // use release-store so the setting of _terminated is seen more quickly > > Pre-existing: A release doesn't push changes to main memory more quickly it only establishes ordering with previous loads and stores. Why do we need release semantics here - what state are with ordering with? What load-acquire are we pairing with? I beilive this is legacy from pre-ThreadsList era. Now we _should_ not have any such situations where such race matters. A thread on a ThreadsList can become terminated but it is still perfectly reachable. If we filter out terminated threads from a ThreadsList, any of those can become terminated just after any ways. So Atomic::store/load is fine here IMHO. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 14:03:12 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 14:03:12 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:50:17 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/thread.inline.hpp line 207: > >> 205: } >> 206: >> 207: inline void JavaThread::set_terminated(TerminatedTypes t) { > > I prefer set_terminated(arg) over the new set of methods. We had two methods: void set_terminated(TerminatedTypes t); void set_terminated_value(); Terminated is part of the name of the method, the name of the flag, the name of the type and part of the names of two of the states, which is very confusing. Also the setters now match the queries: E.g. `bool is_exiting()` The queries do not indicate in any sense that they are queries on the terminated flag. The state flag is an implementation detail from query POV. So to be consistent e.g. "set_exiting()" also hides the fact that we keep track of this with a flag. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 14:07:13 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 14:07:13 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <8H3OiD6YuG9DcS5H-teFeDXvNeMNneFTWzshRVckA_c=.e7fd792c-c20f-4a91-b4dd-e888a83e8d3e@github.com> On Wed, 31 Mar 2021 06:51:19 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/vmStructs.cpp line 764: > >> 762: \ >> 763: /*********************************/ \ >> 764: /* JavaThread (NOTE: incomplete) */ \ > > ??? Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From hseigel at openjdk.java.net Wed Apr 7 14:19:50 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Wed, 7 Apr 2021 14:19:50 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v4] In-Reply-To: References: Message-ID: > Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: remove unneeded statement ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3345/files - new: https://git.openjdk.java.net/jdk/pull/3345/files/1ee327f0..0f1b4f6b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3345&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3345&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3345.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3345/head:pull/3345 PR: https://git.openjdk.java.net/jdk/pull/3345 From hseigel at openjdk.java.net Wed Apr 7 14:19:53 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Wed, 7 Apr 2021 14:19:53 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v4] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 19:08:38 GMT, Patricio Chilano Mateo wrote: >> Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: >> >> remove unneeded statement > > src/hotspot/share/runtime/synchronizer.cpp line 609: > >> 607: // intentionally do not use CHECK on check_owner because we must exit the >> 608: // monitor even if an exception was already pending. >> 609: if (monitor->check_owner(current)) { > > We can actually throw IMSE from check_owner() if this thread is not the real owner. I reverted this change. Thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From hseigel at openjdk.java.net Wed Apr 7 14:24:23 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Wed, 7 Apr 2021 14:24:23 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v2] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 23:28:38 GMT, David Holmes wrote: >> Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: >> >> Undo change to ObjectSynchronizer::jni_exit() > > Hi Harold, > > Lots of good cleanup here but also a couple of things that I think are incorrect. Platform string creation can throw exceptions; and I also believe the ClassFileLoadHook can too. > > Thanks, > David Please review this latest version of this change containing reviewer suggested changes. Thanks, Harold > src/hotspot/share/classfile/javaClasses.cpp line 396: > >> 394: >> 395: // Converts a C string to a Java String based on current encoding >> 396: Handle java_lang_String::create_from_platform_dependent_str(JavaThread* current, const char* str) { > > This change is incorrect. JNU_NewStringPlatform can raise an exception so TRAPS here is correct. Thanks for finding this. I reverted this change and the change to as_platform_dependent_str() because it calls GetStringPlatformChars() which calls JNU_GetStringPlatformChars() which can throw an exception. > src/hotspot/share/prims/jvmtiEnv.cpp line 709: > >> 707: >> 708: // need the path as java.lang.String >> 709: Handle path = java_lang_String::create_from_platform_dependent_str(THREAD, segment); > > This change will need reverting as an exception is possible as previously noted. > > But note that if there was no possibility of exception here then the following check of HAS_PENDING_EXCEPTION should also have been removed. Reverted. ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From hseigel at openjdk.java.net Wed Apr 7 14:24:24 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Wed, 7 Apr 2021 14:24:24 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v4] In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 23:15:48 GMT, David Holmes wrote: >> Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: >> >> remove unneeded statement > > src/hotspot/share/classfile/klassFactory.cpp line 117: > >> 115: ClassLoaderData* loader_data, >> 116: Handle protection_domain, >> 117: JvmtiCachedClassFileData** cached_class_file) { > > I don't think this removal of TRAPS is correct. The ClassFileLoadHook could result in an exception becoming pending. Thanks! This change was reverted. ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From rehn at openjdk.java.net Wed Apr 7 14:28:17 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 14:28:17 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:53:50 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/Thread.java line 79: > >> 77: } >> 78: >> 79: public boolean isExternalSuspend() { > > I assume a new SA API for thread suspension is needed here. If it is important to ask if a thread is suspended. The native stack-traces would easily show this. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Wed Apr 7 14:36:41 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Wed, 7 Apr 2021 14:36:41 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 13:33:45 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/thread.cpp line 2102: >> >>> 2100: do { >>> 2101: set_thread_state(_thread_blocked); >>> 2102: // Check if _external_suspend was set in the previous loop iteration. >> >> Note that the comment for this method at line 1806 needs updating as it refers to a now deleted method. > > I removed that line, not sure if the comment needs additional fixing. I think David requested updating of the comment for `JavaThread::java_suspend_self_with_safepoint_check()`. I hope to delete the whole method soon and build on the new suspend mechanism, if Robbin is ok with that. For now I'd like to propose the following: // Wait for another thread to perform object reallocation and relocking on behalf of // this thread. // Raw thread state transition to _thread_blocked and back again to the original // state before returning are performed. The current thread is required to // change to _thread_blocked in order to be seen to be safepoint/handshake safe // whilst suspended and only after becoming handshake safe, the other thread can // complete the handshake used to synchronize with this thread and then perform // the reallocation and relocking. We cannot use the thread state transition // helpers because we arrive here in various states and also because the helpers // indirectly call this method. After leaving _thread_blocked we have to check // for safepoint/handshake, except if _thread_in_native. The thread is safe // without blocking then. Allowed states are enumerated in // SafepointSynchronize::block(). See also EscapeBarrier::sync_and_suspend_*() ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 14:36:43 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 14:36:43 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 06:57:16 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java line 132: > >> 130: ThreadToSuspend.setAllThreadsReady(); >> 131: >> 132: while (!checkSuspendedStatus()) { > > The changes to this test are still quite unclear to me. Too hard to figure out what the test was originally trying to do! The test worked because we didn't check for suspend when leaving a safepoint request back to native. @pchilano have more info on why the test even works today. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 14:59:16 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 14:59:16 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 03:21:55 GMT, Patricio Chilano Mateo wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.hpp line 151: > >> 149: >> 150: bool is_suspended() { return Atomic::load(&_suspended); } >> 151: void set_suspend(bool to) { return Atomic::store(&_suspended, to); } > > Maybe set_suspended() instead()? Fixed > src/hotspot/share/runtime/objectMonitor.cpp line 436: > >> 434: current->set_current_pending_monitor(NULL); >> 435: >> 436: // We cleared the pending monitor info since we've just gotten past > > Comment below needs updating since it mentions ThreadBlockInVM but we are not using it anymore. Fixed > src/hotspot/share/runtime/thread.cpp line 277: > >> 275: #endif >> 276: >> 277: _SR_lock = new Monitor(Mutex::suspend_resume, "SR_lock", true, > > Mutex::suspend_resume rank is not used by any other monitor so we could remove it. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 15:04:41 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 15:04:41 GMT Subject: RFR: 8257831: Suspend with handshakes [v3] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 14:31:05 GMT, Richard Reingruber wrote: >> I removed that line, not sure if the comment needs additional fixing. > > I think David requested updating of the comment for `JavaThread::java_suspend_self_with_safepoint_check()`. > > I hope to delete the whole method soon and build on the new suspend mechanism, if Robbin is ok with that. For now I'd like to propose the following: > > // Wait for another thread to perform object reallocation and relocking on behalf of > // this thread. > // Raw thread state transition to _thread_blocked and back again to the original > // state before returning are performed. The current thread is required to > // change to _thread_blocked in order to be seen to be safepoint/handshake safe > // whilst suspended and only after becoming handshake safe, the other thread can > // complete the handshake used to synchronize with this thread and then perform > // the reallocation and relocking. We cannot use the thread state transition > // helpers because we arrive here in various states and also because the helpers > // indirectly call this method. After leaving _thread_blocked we have to check > // for safepoint/handshake, except if _thread_in_native. The thread is safe > // without blocking then. Allowed states are enumerated in > // SafepointSynchronize::block(). See also EscapeBarrier::sync_and_suspend_*() Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From minqi at openjdk.java.net Wed Apr 7 15:37:49 2021 From: minqi at openjdk.java.net (Yumin Qi) Date: Wed, 7 Apr 2021 15:37:49 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v10] In-Reply-To: References: Message-ID: <8Em0aTz_KxKneptL7dOuu1ldmzYI0kwDzEfHkz1kQrc=.2ad4881c-23db-4934-8a79-f84db54a1417@github.com> > Hi, Please review > > Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with > `java -XX:DumpLoadedClassList= .... ` > to collect shareable class names and saved in file `` , then > `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` > With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. > The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. > New added jcmd option: > `jcmd VM.cds static_dump ` > or > `jcmd VM.cds dynamic_dump ` > To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. > The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. > > Tests: tier1,tier2,tier3,tier4 > > Thanks > Yumin Yumin Qi has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 14 commits: - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Restructed code and rename LingeredTestApp.java to JCmdTestLingeredApp.java - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Fix revert unintentionally comment, merge master. - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Remove CDS.getVMArguments, changed to use VM.getRuntimeVMArguments. Removed unused function from ClassLoader. Improved InstanceKlass::is_shareable() and related test. Added more test scenarios. - Remove redundant check for if a class is shareable - Fix according to review comment and add more tests - Fix filter more flags to exclude in static dump, add more test cases - ... and 4 more: https://git.openjdk.java.net/jdk/compare/a756d8d7...b3d20348 ------------- Changes: https://git.openjdk.java.net/jdk/pull/2737/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2737&range=09 Stats: 835 lines in 23 files changed: 762 ins; 58 del; 15 mod Patch: https://git.openjdk.java.net/jdk/pull/2737.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2737/head:pull/2737 PR: https://git.openjdk.java.net/jdk/pull/2737 From dcubed at openjdk.java.net Wed Apr 7 15:44:33 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 7 Apr 2021 15:44:33 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v4] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 14:19:50 GMT, Harold Seigel wrote: >> Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. >> >> Thanks, Harold > > Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: > > remove unneeded statement Marked as reviewed by dcubed (Reviewer). src/hotspot/share/interpreter/bytecode.cpp line 160: > 158: } > 159: > 160: Handle Bytecode_invoke::appendix(TRAPS) { You might want to mention the removal of this function in the bug report so that it will show up in a JBS search. ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From dcubed at openjdk.java.net Wed Apr 7 15:44:34 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 7 Apr 2021 15:44:34 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v4] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 15:40:58 GMT, Daniel D. Daugherty wrote: >> Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: >> >> remove unneeded statement > > Marked as reviewed by dcubed (Reviewer). Thumbs up. ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From dcubed at openjdk.java.net Wed Apr 7 15:48:34 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 7 Apr 2021 15:48:34 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 11:43:15 GMT, Robbin Ehn wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> Address lyndseyBeil, robehn and sspitsyn CR comments. > > I really like simple agent code ?? > > Thanks! @robehn - Thanks for the re-review! And thank you for suggesting the much simpler agent code!! ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From pchilanomate at openjdk.java.net Wed Apr 7 15:52:45 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Wed, 7 Apr 2021 15:52:45 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 14:33:00 GMT, Robbin Ehn wrote: >> test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java line 132: >> >>> 130: ThreadToSuspend.setAllThreadsReady(); >>> 131: >>> 132: while (!checkSuspendedStatus()) { >> >> The changes to this test are still quite unclear to me. Too hard to figure out what the test was originally trying to do! > > The test worked because we didn't check for suspend when leaving a safepoint request back to native. > @pchilano have more info on why the test even works today. Right, the reason the test was working so far but not anymore without this changes is because before we didn't check for suspension on ~ThreadBlockInVM() after leaving a safepoint safe state. The interaction is kind of subtle: In the test the main thread creates 10 working threads and sets one of those to have the role of 'Suspender'. The 'Suspender' acquires agent_monitor(a jvmtirawmonitor), suspends all working threads including itself by calling jvmti->SuspendThreadList() and then it releases agent_monitor. In the mean time the main thread just blocks on agent_monitor until 'Suspender' releases it to check that everybody was indeed suspended. Now, at first glance this looks like(at least for me) the test should deadlock because 'Suspender' should never return from SuspendThreadList() to release agent_monitor since he was suspended too. The reason 'Suspender' returns from SuspendThreadList() is because he never actually checks for suspension after the safepoint operation (VM_ThreadsSuspendJVMTI) finished. 'Suspender' waits on the VMOperation_lock with as_suspend_equivalent=false (default) so it transitions back to _thread_in_vm without suspending. And then when going back to native in ~ThreadInVMfromNati ve() we also don't check for suspends (we now check it here too actually). The JVMTI specs says this about SuspendThreadList: "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." So this patch actually fixes that. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 16:25:10 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 16:25:10 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 18:53:54 GMT, Serguei Spitsyn wrote: > Hi Robbin, > I'm looking at it. I have a concern that it can significantly impact Loom implementation. There is a non-trivial update in Loom to support virtual threads in JVMTI Suspend/Resume. So we need to make sure your approach is going to work for virtual threads as well. Hi Serguei, I looked at loom implementation. I will help you do the merge. The main thing seem to split some logic into the handshake itself. Thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 16:25:14 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 16:25:14 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <04Dot4HzYGzkxX3_VISq-flvgW76ptGorPrp2fOrDrA=.f76f8484-ac79-4e8e-b219-6f3a0bf6f1e0@github.com> On Wed, 31 Mar 2021 05:52:14 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.hpp line 139: > >> 137: Thread* active_handshaker() const { return _active_handshaker; } >> 138: >> 139: // Suspend/resume support > > This would be a good place to describe the operation of these two state fields Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 7 16:29:12 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 7 Apr 2021 16:29:12 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <8XHYnrIhSOcB6qEpq4-1FlTw8LvfPTmIG3LyIs3BC8E=.301d2dff-0341-44e3-b066-e80b3945a9a6@github.com> On Wed, 7 Apr 2021 16:23:08 GMT, Robbin Ehn wrote: >> Hi Robbin, >> I'm looking at it. I have a concern that it can significantly impact Loom implementation. There is a non-trivial update in Loom to support virtual threads in JVMTI Suspend/Resume. So we need to make sure your approach is going to work for virtual threads as well. > >> Hi Robbin, >> I'm looking at it. I have a concern that it can significantly impact Loom implementation. There is a non-trivial update in Loom to support virtual threads in JVMTI Suspend/Resume. So we need to make sure your approach is going to work for virtual threads as well. > > Hi Serguei, I looked at loom implementation. > I will help you do the merge. The main thing seem to split some logic into the handshake itself. > > Thanks Working on pushing a new version, it will not have addressed all comments but many of them. I will build and do testing before push, and start deep testing in the background. I also merge with mainline. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From sspitsyn at openjdk.java.net Wed Apr 7 17:56:09 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 7 Apr 2021 17:56:09 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 19:46:38 GMT, Daniel D. Daugherty wrote: >> Add three tests from JDK-4413752 ported to JVM/TI: >> >> - RawMonitorEnter() with SuspendThread() >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithRawMonitorEnter/SuspendWithRawMonitorEnter.java >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithRawMonitorEnter/libSuspendWithRawMonitorEnter.cpp >> >> - ObjectMonitor enter() with SuspendThread() >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorEnter/SuspendWithObjectMonitorEnter.java >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorEnter/libSuspendWithObjectMonitorEnter.cpp >> >> - ObjectMonitor wait() with SuspendThread >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/SuspendWithObjectMonitorWait.java >> - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/libSuspendWithObjectMonitorWait.cpp >> >> The Java files have a transaction diagram to show what each of the >> threads in the test is doing. > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > Address lyndseyBeil, robehn and sspitsyn CR comments. Hi Dan, It looks good in general. But I think moving JVMTI error code checks to Java is a step in a wrong direction. First, it makes it inconsistent with all existing JVMTI tests. In this particular case, all the complexity is moved to Java side making it unbalanced. Also, we are working with Leonid toward creating relevant native testing lib for serviceability/jvmti tests (created it for Loom first) which has a support to print symbolic names (for error codes as well) if necessary. Not sure, I understood what wrong with the native check_jvmti_status(). It seems the id argument is not needed much and can be removed anyway. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/2899 From dcubed at openjdk.java.net Wed Apr 7 19:20:40 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 7 Apr 2021 19:20:40 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 17:52:16 GMT, Serguei Spitsyn wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> Address lyndseyBeil, robehn and sspitsyn CR comments. > > Hi Dan, > It looks good in general. > But I think moving JVMTI error code checks to Java is a step in a wrong direction. > First, it makes it inconsistent with all existing JVMTI tests. In this particular case, all the complexity is moved to Java side making it unbalanced. Also, we are working with Leonid toward creating relevant native testing lib for serviceability/jvmti tests (created it for Loom first) which has a support to print symbolic names (for error codes as well) if necessary. > Not sure, I understood what wrong with the native check_jvmti_status(). It seems the id argument is not needed much and can be removed anyway. > Thanks, > Serguei Hi @sspitsyn, I rather like the much simpler agent code and it puts the majority of logic in the .java file so the reader/reviewer doesn't have to jump back and forth between the two files nearly as much. The `id` argument had to be passed to the old version of the function calls in order for the logging messages generated by the agent code to (easily) have the same thread names as the Java side logging. Once all the error checking was moved to the Java side, that made the `id` argument no longer necessary and also allowed the native `check_jvmti_status()` function (along with other functions) to be removed. So there was nothing wrong with the native `check_jvmti_status()` function; it was just made obsolete by the code migration to the Java side. I consider @robehn's idea here to be good evolution for these kinds of tests to move code from the native side, where the code is more platform dependent, to the Java side, where the code is more platform independent. I hope I have convinced you that this is a good changeset for moving forward. ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From sspitsyn at openjdk.java.net Wed Apr 7 22:48:14 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 7 Apr 2021 22:48:14 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 19:17:33 GMT, Daniel D. Daugherty wrote: >> Hi Dan, >> It looks good in general. >> But I think moving JVMTI error code checks to Java is a step in a wrong direction. >> First, it makes it inconsistent with all existing JVMTI tests. In this particular case, all the complexity is moved to Java side making it unbalanced. Also, we are working with Leonid toward creating relevant native testing lib for serviceability/jvmti tests (created it for Loom first) which has a support to print symbolic names (for error codes as well) if necessary. >> Not sure, I understood what wrong with the native check_jvmti_status(). It seems the id argument is not needed much and can be removed anyway. >> Thanks, >> Serguei > > Hi @sspitsyn, > > I rather like the much simpler agent code and it puts the majority of logic > in the .java file so the reader/reviewer doesn't have to jump back and forth > between the two files nearly as much. > > The `id` argument had to be passed to the old version of the function calls > in order for the logging messages generated by the agent code to (easily) > have the same thread names as the Java side logging. Once all the error > checking was moved to the Java side, that made the `id` argument no longer > necessary and also allowed the native `check_jvmti_status()` function (along > with other functions) to be removed. So there was nothing wrong with the > native `check_jvmti_status()` function; it was just made obsolete by the code > migration to the Java side. > > I consider @robehn's idea here to be good evolution for these kinds of tests to > move code from the native side, where the code is more platform dependent, > to the Java side, where the code is more platform independent. > > I hope I have convinced you that this is a good changeset for moving forward. Dan, I'm sorry, I do not see much value in this approach going forward. One line with a call to check_jvmti_status() does not add much to the platform dependency no to complexity in native code. Also, more sophisticated analysis of the returned error code frequently is needed. Also, I consider it not feasible to move a thousand of JVMTI tests in this direction. So that we will end up with inconsistency over all tests. I've already marked this as reviewed, so you can push it. But I'm against following this pattern going forward. Or at least, we need to consult with the SQE engineers first. Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From dcubed at openjdk.java.net Thu Apr 8 01:30:17 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 8 Apr 2021 01:30:17 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 22:44:41 GMT, Serguei Spitsyn wrote: >> Hi @sspitsyn, >> >> I rather like the much simpler agent code and it puts the majority of logic >> in the .java file so the reader/reviewer doesn't have to jump back and forth >> between the two files nearly as much. >> >> The `id` argument had to be passed to the old version of the function calls >> in order for the logging messages generated by the agent code to (easily) >> have the same thread names as the Java side logging. Once all the error >> checking was moved to the Java side, that made the `id` argument no longer >> necessary and also allowed the native `check_jvmti_status()` function (along >> with other functions) to be removed. So there was nothing wrong with the >> native `check_jvmti_status()` function; it was just made obsolete by the code >> migration to the Java side. >> >> I consider @robehn's idea here to be good evolution for these kinds of tests to >> move code from the native side, where the code is more platform dependent, >> to the Java side, where the code is more platform independent. >> >> I hope I have convinced you that this is a good changeset for moving forward. > > Dan, > I'm sorry, I do not see much value in this approach going forward. One line with a call to check_jvmti_status() does not add much to the platform dependency no to complexity in native code. Also, more sophisticated analysis of the returned error code frequently is needed. Also, I consider it not feasible to move a thousand of JVMTI tests in this direction. So that we will end up with inconsistency over all tests. > I've already marked this as reviewed, so you can push it. But I'm against following this pattern going forward. Or at least, we need to consult with the SQE engineers first. > Thanks, > Serguei @sspitsyn - First, thanks for approving the changes! Second, I'm _not_ proposing changing any other JVM/TI tests to use this style. The only reason that this style works (for me and @robehn) is because none of these JVM/TI calls is expected to fail. The tests are carefully constructed to exercise the code for a race condition that will result in a hang or a crash (if something goes wrong). In the case of these tests, we don't need sophisticated analysis of the returned error codes. Again, thanks for approving these tests. ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From sspitsyn at openjdk.java.net Thu Apr 8 05:57:21 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 8 Apr 2021 05:57:21 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 15:48:55 GMT, Patricio Chilano Mateo wrote: >> The test worked because we didn't check for suspend when leaving a safepoint request back to native. >> @pchilano have more info on why the test even works today. > > Right, the reason the test was working so far but not anymore without this changes is because before we didn't check for suspension on ~ThreadBlockInVM() after leaving a safepoint safe state. The interaction is kind of subtle: > In the test the main thread creates 10 working threads and sets one of those to have the role of 'Suspender'. The 'Suspender' acquires agent_monitor(a jvmtirawmonitor), suspends all working threads including itself by calling jvmti->SuspendThreadList() and then it releases agent_monitor. In the mean time the main thread just blocks on agent_monitor until 'Suspender' releases it to check that everybody was indeed suspended. Now, at first glance this looks like(at least for me) the test should deadlock because 'Suspender' should never return from SuspendThreadList() to release agent_monitor since he was suspended too. The reason 'Suspender' returns from SuspendThreadList() is because he never actually checks for suspension after the safepoint operation (VM_ThreadsSuspendJVMTI) finished. 'Suspender' waits on the VMOperation_lock with as_suspend_equivalent=false (default) so it transitions back to _thread_in_vm without suspending. And then when going back to native in ~ThreadInVMfromNa tive() we also don't check for suspends (we now check it here too actually). > The JVMTI specs says this about SuspendThreadList: > > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > So this patch actually fixes that. We also had to fix deadlock in this test in Loom. The fix is almost the same, and I agree with it. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From sspitsyn at openjdk.java.net Thu Apr 8 06:08:45 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 8 Apr 2021 06:08:45 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: <8XHYnrIhSOcB6qEpq4-1FlTw8LvfPTmIG3LyIs3BC8E=.301d2dff-0341-44e3-b066-e80b3945a9a6@github.com> References: <8XHYnrIhSOcB6qEpq4-1FlTw8LvfPTmIG3LyIs3BC8E=.301d2dff-0341-44e3-b066-e80b3945a9a6@github.com> Message-ID: On Wed, 7 Apr 2021 16:25:28 GMT, Robbin Ehn wrote: >>> Hi Robbin, >>> I'm looking at it. I have a concern that it can significantly impact Loom implementation. There is a non-trivial update in Loom to support virtual threads in JVMTI Suspend/Resume. So we need to make sure your approach is going to work for virtual threads as well. >> >> Hi Serguei, I looked at loom implementation. >> I will help you do the merge. The main thing seem to split some logic into the handshake itself. >> >> Thanks > > Working on pushing a new version, it will not have addressed all comments but many of them. > I will build and do testing before push, and start deep testing in the background. > > I also merge with mainline. Hi Robbin, Thank you for the reply and suggestion to help with merge. I made the first pass through the webrev. The fix looks pretty good in general - nice simplifications! My estimation is that it should not be a big problem to merge it with Loom while it can require some effort to do so. I'm going for another pass to look on details. Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Thu Apr 8 07:17:48 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 8 Apr 2021 07:17:48 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: - White space fixes - Merge branch 'master' into SuspendInHandshake - Review fixes - Merge branch 'master' into SuspendInHandshake - Merge branch 'master' into SuspendInHandshake - 8257831: Suspend with handshake (review baseline) ------------- Changes: https://git.openjdk.java.net/jdk/pull/3191/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=03 Stats: 1341 lines in 39 files changed: 281 ins; 865 del; 195 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From sspitsyn at openjdk.java.net Thu Apr 8 08:15:33 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 8 Apr 2021 08:15:33 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: <8XHYnrIhSOcB6qEpq4-1FlTw8LvfPTmIG3LyIs3BC8E=.301d2dff-0341-44e3-b066-e80b3945a9a6@github.com> Message-ID: On Thu, 8 Apr 2021 06:02:35 GMT, Serguei Spitsyn wrote: >> Working on pushing a new version, it will not have addressed all comments but many of them. >> I will build and do testing before push, and start deep testing in the background. >> >> I also merge with mainline. > > Hi Robbin, > Thank you for the reply and suggestion to help with merge. I made the first pass through the webrev. The fix looks pretty good in general - nice simplifications! My estimation is that it should not be a big problem to merge it with Loom while it can require some effort to do so. I'm going for another pass to look on details. > Thanks, > Serguei The functions JvmtiSuspendControl::suspend and JvmtiSuspendControl::resume are not really needed anymore. You probably decided to keep them around to simplify merges a little bit. Is that right? ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From sspitsyn at openjdk.java.net Thu Apr 8 08:27:22 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 8 Apr 2021 08:27:22 GMT Subject: RFR: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI [v4] In-Reply-To: References: Message-ID: <-MO0WplIfGU2pEZifqv0Ow9Og7Gj-u2A8AqzYe_Bx8k=.e577f9ea-904f-4098-ac62-1a2a4f2218a4@github.com> On Thu, 8 Apr 2021 01:25:25 GMT, Daniel D. Daugherty wrote: >> Dan, >> I'm sorry, I do not see much value in this approach going forward. One line with a call to check_jvmti_status() does not add much to the platform dependency no to complexity in native code. Also, more sophisticated analysis of the returned error code frequently is needed. Also, I consider it not feasible to move a thousand of JVMTI tests in this direction. So that we will end up with inconsistency over all tests. >> I've already marked this as reviewed, so you can push it. But I'm against following this pattern going forward. Or at least, we need to consult with the SQE engineers first. >> Thanks, >> Serguei > > @sspitsyn - First, thanks for approving the changes! > > Second, I'm _not_ proposing changing any other JVM/TI tests to use this style. The only > reason that this style works (for me and @robehn) is because none of these JVM/TI calls > is expected to fail. The tests are carefully constructed to exercise the code for a race > condition that will result in a hang or a crash (if something goes wrong). In the case of > these tests, we don't need sophisticated analysis of the returned error codes. > > Again, thanks for approving these tests. Dan, thank you for extra explanation and for taking care to port these old tests! ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From dfuchs at openjdk.java.net Thu Apr 8 09:29:30 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 8 Apr 2021 09:29:30 GMT Subject: RFR: 8264124: Update MXBean specification and implementation to extend mapping of CompositeType to records [v6] In-Reply-To: References: <7Oco_BzQDkrrNFy6FRoUFAUx5IuHu9YtuEfNuLverUI=.b57d8a09-8b39-4db1-a439-57285dedb7f7@github.com> <-HFZga_QfdbwjZcwnYCmvWjaXEACUzdW3fd6D6BhX6I=.38ba198b-70e4-4682-a7f4-13965bb47162@github.com> Message-ID: On Wed, 31 Mar 2021 21:43:58 GMT, Mandy Chung wrote: >> Daniel Fuchs has updated the pull request incrementally with two additional commits since the last revision: >> >> - minor style issue >> - minor style issue > > Thanks for making the change. The spec change looks good to me. @mlchung @AlanBateman @ChrisHegarty would you formally approve the fix? The CSR has been approved. ------------- PR: https://git.openjdk.java.net/jdk/pull/3201 From dholmes at openjdk.java.net Thu Apr 8 10:13:21 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 8 Apr 2021 10:13:21 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v4] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 14:19:50 GMT, Harold Seigel wrote: >> Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. >> >> Thanks, Harold > > Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: > > remove unneeded statement Hi Harold, Updates seem fine. Thanks, David ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3345 From hseigel at openjdk.java.net Thu Apr 8 12:16:20 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Thu, 8 Apr 2021 12:16:20 GMT Subject: RFR: 8264711: More runtime TRAPS cleanups [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 10:10:00 GMT, David Holmes wrote: >> Harold Seigel has updated the pull request incrementally with one additional commit since the last revision: >> >> remove unneeded statement > > Hi Harold, > > Updates seem fine. > > Thanks, > David Thanks Lois, Patricio, David, and Dan for reviewing this! ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From hseigel at openjdk.java.net Thu Apr 8 12:16:21 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Thu, 8 Apr 2021 12:16:21 GMT Subject: Integrated: 8264711: More runtime TRAPS cleanups In-Reply-To: References: Message-ID: <3YKOgP4x6Fu-oS7-x_SuhvhDLWG3vH1cVzjNzKTHc6A=.5ffc2a72-2d23-46d1-9738-e59f318c636a@github.com> On Mon, 5 Apr 2021 17:57:13 GMT, Harold Seigel wrote: > Please review this additional cleanup of use of TRAPS in hotspot runtime code. The changes were tested with Mach5 tiers 1-2 on Linux, Mac OS, and Windows and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold This pull request has now been integrated. Changeset: af13c64f Author: Harold Seigel URL: https://git.openjdk.java.net/jdk/commit/af13c64f Stats: 97 lines in 26 files changed: 1 ins; 13 del; 83 mod 8264711: More runtime TRAPS cleanups Reviewed-by: lfoltan, pchilanomate, dholmes, dcubed ------------- PR: https://git.openjdk.java.net/jdk/pull/3345 From kvn at openjdk.java.net Thu Apr 8 15:32:40 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 8 Apr 2021 15:32:40 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler Message-ID: As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: - `jdk.aot` module ? the `jaotc` tool - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution - related HotSpot code guarded by `#if INCLUDE_AOT` Additionally, remove tests as well as code in makefiles related to AoT compilation. Tested hs-tier1-4 ------------- Commit messages: - 8264805: Remove the experimental Ahead-of-Time Compiler Changes: https://git.openjdk.java.net/jdk/pull/3398/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3398&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264805 Stats: 26903 lines in 375 files changed: 4 ins; 26703 del; 196 mod Patch: https://git.openjdk.java.net/jdk/pull/3398.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3398/head:pull/3398 PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Thu Apr 8 15:58:39 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 8 Apr 2021 15:58:39 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v2] In-Reply-To: References: Message-ID: > As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: > > - `jdk.aot` module ? the `jaotc` tool > - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution > - related HotSpot code guarded by `#if INCLUDE_AOT` > > Additionally, remove tests as well as code in makefiles related to AoT compilation. > > Tested hs-tier1-4 Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - Merge branch 'master' into JDK-8264805 - 8264805: Remove the experimental Ahead-of-Time Compiler ------------- Changes: https://git.openjdk.java.net/jdk/pull/3398/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3398&range=01 Stats: 26906 lines in 375 files changed: 4 ins; 26709 del; 193 mod Patch: https://git.openjdk.java.net/jdk/pull/3398.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3398/head:pull/3398 PR: https://git.openjdk.java.net/jdk/pull/3398 From rrich at openjdk.java.net Thu Apr 8 16:13:34 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Thu, 8 Apr 2021 16:13:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 07:17:48 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - White space fixes > - Merge branch 'master' into SuspendInHandshake > - Review fixes > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) Hi Robbin, thanks for the update. This is almost ready to be pushed from my pov. Richard. src/hotspot/share/runtime/handshake.cpp line 630: > 628: // exiting. > 629: } > 630: } I need a little help learning the steps of this dance :) We reach here in _thread_in_vm. We cannot be suspended in this state. There might be another thread waiting to handshake us to suspend us but why can't we just ignore that and do the `_handshakee->set_exiting();` even without locking HandshakeState::_lock? Adding a handshake operation is lock free, so even if the check `SafepointMechanism::should_process(_handshakee)` in L619 returns false a new operation to process could have been added concurrently such that `SafepointMechanism::should_process(_handshakee)` would return true when we execute `_handshakee->set_exiting();` in L620. What am I missing? ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From kvn at openjdk.java.net Thu Apr 8 16:17:33 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 8 Apr 2021 16:17:33 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v3] In-Reply-To: References: Message-ID: > As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: > > - `jdk.aot` module ? the `jaotc` tool > - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution > - related HotSpot code guarded by `#if INCLUDE_AOT` > > Additionally, remove tests as well as code in makefiles related to AoT compilation. > > Tested hs-tier1-4 Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: - Merge branch 'master' into JDK-8264805 - Merge branch 'master' into JDK-8264805 - 8264805: Remove the experimental Ahead-of-Time Compiler ------------- Changes: https://git.openjdk.java.net/jdk/pull/3398/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3398&range=02 Stats: 26906 lines in 375 files changed: 4 ins; 26709 del; 193 mod Patch: https://git.openjdk.java.net/jdk/pull/3398.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3398/head:pull/3398 PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Thu Apr 8 17:24:38 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 8 Apr 2021 17:24:38 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: > As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: > > - `jdk.aot` module ? the `jaotc` tool > - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution > - related HotSpot code guarded by `#if INCLUDE_AOT` > > Additionally, remove tests as well as code in makefiles related to AoT compilation. > > Tested hs-tier1-4 Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: Remove exports from Graal module to jdk.aot ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3398/files - new: https://git.openjdk.java.net/jdk/pull/3398/files/3dbc6210..6cce0f6c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3398&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3398&range=02-03 Stats: 39 lines in 3 files changed: 0 ins; 36 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3398.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3398/head:pull/3398 PR: https://git.openjdk.java.net/jdk/pull/3398 From chegar at openjdk.java.net Thu Apr 8 18:12:11 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Thu, 8 Apr 2021 18:12:11 GMT Subject: RFR: 8264124: Update MXBean specification and implementation to extend mapping of CompositeType to records [v7] In-Reply-To: References: <7Oco_BzQDkrrNFy6FRoUFAUx5IuHu9YtuEfNuLverUI=.b57d8a09-8b39-4db1-a439-57285dedb7f7@github.com> Message-ID: <9ImJYOGw4hDnvWrf0i88q9Onu3G5N7XSuKOamTmQrb0=.6d77eae9-f75e-4fa7-9a59-5201b6720121@github.com> On Fri, 2 Apr 2021 18:00:57 GMT, Daniel Fuchs wrote: >> This RFE proposes to extend the MXBean framework to define a mapping to records. >> >> The MXBean framework already defines a mapping of `CompositeType` to plain java objects. Records are a natural representation of CompositeTypes. A record can be easily reconstructed from a `CompositeData` through the record canonical constructor. A clear advantage of records over plain java objects is that the canonical constructor will not need to be annotated in order to map composite data property names to constructor parameter names. >> >> With this RFE, here is an example comparing coding a composite type `NamedNumber` that consists of an `int` and a `String`, using records and using a plain java class. In both case, the `CompositeType` looks like this: >> >> CompositeType( >> "NamedNumber", // typeName >> "NamedNumber", // description >> new String[] {"number", "name"}, // itemNames >> new String[] {"number", "name"}, // itemDescriptions >> new OpenType[] {SimpleType.INTEGER, >> SimpleType.STRING} // itemTypes >> ); >> >> The plain Java class needs a public constructor annotated with `@ConstructorParameters` annotation: >> >> public class NamedNumber { >> public int getNumber() {return number;} >> public String getName() {return name;} >> @ConstructorParameters({"number", "name"}) >> public NamedNumber(int number, String name) { >> this.number = number; >> this.name = name; >> } >> private final int number; >> private final String name; >> } >> >> And the equivalent with a record class: >> >> public record NamedNumber(int number, String name) {} > > Daniel Fuchs has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 13 additional commits since the last revision: > > - Merge > - minor style issue > - minor style issue > - Integrated review feedback > - Improved test > - Moved mapping for records just before Mapping for MXBean interface; Improved test. > - Merge branch 'master' into mxbeans-8264124 > - Minor tweaks. Improved test. > - Merge branch 'master' into mxbeans-8264124 > - Integrated review feedback. Updated the section for Mapping to records. > - ... and 3 more: https://git.openjdk.java.net/jdk/compare/8181cfe8...257a6ed8 Marked as reviewed by chegar (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3201 From coleenp at openjdk.java.net Thu Apr 8 19:35:06 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 8 Apr 2021 19:35:06 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 15:23:52 GMT, Vladimir Kozlov wrote: > As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: > > - `jdk.aot` module ? the `jaotc` tool > - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution > - related HotSpot code guarded by `#if INCLUDE_AOT` > > Additionally, remove tests as well as code in makefiles related to AoT compilation. > > Tested hs-tier1-4 void CodeCache::mark_for_evol_deoptimization(InstanceKlass* dependee) { This function, its caller and the function in jvmtiRedefineClasses.cpp that calls it can be deleted also, but you can file a separate RFE for that if you want. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From coleenp at openjdk.java.net Thu Apr 8 19:42:06 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 8 Apr 2021 19:42:06 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot I looked at the changes in oops, runtime, classfile and code directories. ------------- Marked as reviewed by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From erikj at openjdk.java.net Thu Apr 8 19:54:07 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Thu, 8 Apr 2021 19:54:07 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot Build changes look good. ------------- Marked as reviewed by erikj (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From stefank at openjdk.java.net Thu Apr 8 20:04:07 2021 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Thu, 8 Apr 2021 20:04:07 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot GC changes look good. ------------- Marked as reviewed by stefank (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Thu Apr 8 20:04:08 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 8 Apr 2021 20:04:08 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 19:58:11 GMT, Stefan Karlsson wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > GC changes look good. > void CodeCache::mark_for_evol_deoptimization(InstanceKlass* dependee) { > This function, its caller and the function in jvmtiRedefineClasses.cpp that calls it can be deleted also, but you can file a separate RFE for that if you want. Thank you, Coleen, for review. I will wait other comments and will remove this code. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Thu Apr 8 20:04:08 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 8 Apr 2021 20:04:08 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 20:00:30 GMT, Vladimir Kozlov wrote: >> GC changes look good. > >> void CodeCache::mark_for_evol_deoptimization(InstanceKlass* dependee) { >> This function, its caller and the function in jvmtiRedefineClasses.cpp that calls it can be deleted also, but you can file a separate RFE for that if you want. > > Thank you, Coleen, for review. I will wait other comments and will remove this code. Thank you, Erik and Stefan. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From iignatyev at openjdk.java.net Thu Apr 8 20:31:11 2021 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Thu, 8 Apr 2021 20:31:11 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot Test changes look good to me. ------------- Marked as reviewed by iignatyev (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From coleenp at openjdk.java.net Thu Apr 8 21:03:07 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 8 Apr 2021 21:03:07 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 20:28:27 GMT, Igor Ignatyev wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > Test changes look good to me. There's a comment above void VM_RedefineClasses::mark_dependent_code(InstanceKlass* ik) { that should be rewritten, so I think we should just file an RFE to remove it afterwards. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Thu Apr 8 21:57:07 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 8 Apr 2021 21:57:07 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 20:59:59 GMT, Coleen Phillimore wrote: > There's a comment above > void VM_RedefineClasses::mark_dependent_code(InstanceKlass* ik) { > that should be rewritten, so I think we should just file an RFE to remove it afterwards. https://bugs.openjdk.java.net/browse/JDK-8264941 ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From mchung at openjdk.java.net Thu Apr 8 22:06:28 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Thu, 8 Apr 2021 22:06:28 GMT Subject: RFR: 8264124: Update MXBean specification and implementation to extend mapping of CompositeType to records [v7] In-Reply-To: References: <7Oco_BzQDkrrNFy6FRoUFAUx5IuHu9YtuEfNuLverUI=.b57d8a09-8b39-4db1-a439-57285dedb7f7@github.com> Message-ID: On Fri, 2 Apr 2021 18:00:57 GMT, Daniel Fuchs wrote: >> This RFE proposes to extend the MXBean framework to define a mapping to records. >> >> The MXBean framework already defines a mapping of `CompositeType` to plain java objects. Records are a natural representation of CompositeTypes. A record can be easily reconstructed from a `CompositeData` through the record canonical constructor. A clear advantage of records over plain java objects is that the canonical constructor will not need to be annotated in order to map composite data property names to constructor parameter names. >> >> With this RFE, here is an example comparing coding a composite type `NamedNumber` that consists of an `int` and a `String`, using records and using a plain java class. In both case, the `CompositeType` looks like this: >> >> CompositeType( >> "NamedNumber", // typeName >> "NamedNumber", // description >> new String[] {"number", "name"}, // itemNames >> new String[] {"number", "name"}, // itemDescriptions >> new OpenType[] {SimpleType.INTEGER, >> SimpleType.STRING} // itemTypes >> ); >> >> The plain Java class needs a public constructor annotated with `@ConstructorParameters` annotation: >> >> public class NamedNumber { >> public int getNumber() {return number;} >> public String getName() {return name;} >> @ConstructorParameters({"number", "name"}) >> public NamedNumber(int number, String name) { >> this.number = number; >> this.name = name; >> } >> private final int number; >> private final String name; >> } >> >> And the equivalent with a record class: >> >> public record NamedNumber(int number, String name) {} > > Daniel Fuchs has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 13 additional commits since the last revision: > > - Merge > - minor style issue > - minor style issue > - Integrated review feedback > - Improved test > - Moved mapping for records just before Mapping for MXBean interface; Improved test. > - Merge branch 'master' into mxbeans-8264124 > - Minor tweaks. Improved test. > - Merge branch 'master' into mxbeans-8264124 > - Integrated review feedback. Updated the section for Mapping to records. > - ... and 3 more: https://git.openjdk.java.net/jdk/compare/522a6472...257a6ed8 This looks okay to me but I'm not that familiar with the existing implementation. The test case covers the important cases. ------------- Marked as reviewed by mchung (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3201 From dholmes at openjdk.java.net Fri Apr 9 04:35:15 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Fri, 9 Apr 2021 04:35:15 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot Hi Vladimir, This looks good to me - I went through all the files. It was nice to see how nicely compartmentalized the AOT feature was - that made checking the changes quite simple. The one exception was the fingerprinting code, which for some reason was AOT-only but not marked that way, so I'm not sure what the story is there. ?? I was also surprised to see some of the flags were not marked experimental, so there will need to a CSR request to remove those without going through the normal deprecation process. Thanks, David src/hotspot/cpu/x86/compiledIC_x86.cpp line 134: > 132: #ifdef ASSERT > 133: CodeBlob *cb = CodeCache::find_blob_unsafe((address) _call); > 134: assert(cb, "sanity"); Nit: implied boolean - use "cb != NULL" src/hotspot/share/memory/heap.hpp line 174: > 172: bool contains(const void* p) const { return low() <= p && p < high(); } > 173: bool contains_blob(const CodeBlob* blob) const { > 174: const void* start = (void*)blob; start seems redundant now ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From sspitsyn at openjdk.java.net Fri Apr 9 06:24:29 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 9 Apr 2021 06:24:29 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 16:09:52 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > Hi Robbin, > > thanks for the update. This is almost ready to be pushed from my pov. > > Richard. Hi Robbin, Just a couple of comments below. src/hotspot/share/runtime/handshake.hpp 154 // This flag is true while there is async handshake (trap) 155 // on queue. Since we do only need one, we can reuse it if 156 // thread thread gets suspended again (after a resume) 157 // and have not yet processed it. 158 bool _async_suspend_handshake; A couple of nits above: L156: "thread thread" => "thread" L157: "and have not yet processed it." => "and we have not yet processed it." src/hotspot/share/runtime/handshake.cpp 658 bool HandshakeState::suspend_with_handshake() { . . . 684 ThreadSuspensionHandshake* ts = new ThreadSuspensionHandshake(); 685 Handshake::execute(ts, _handshakee); I wonder why ThreadSuspensionHandshake is not stack allocated as SuspendThreadHandshake below: 701 bool HandshakeState::suspend() { 702 SuspendThreadHandshake st; 703 Handshake::execute(&st, _handshakee); Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From aph at openjdk.java.net Fri Apr 9 08:18:18 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Fri, 9 Apr 2021 08:18:18 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: <7ioJ2_GFIK53zxpqFcYLt9LYDd8WS7ronekRuo6O5Ac=.572d87d5-2f1f-41cf-8c9b-6d018926f0cf@github.com> On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot AArch64 looks fine. ------------- Marked as reviewed by aph (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From shade at openjdk.java.net Fri Apr 9 08:54:18 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 9 Apr 2021 08:54:18 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot Shenandoah parts look good. I have a few minor comments after reading the rest. src/hotspot/cpu/x86/globalDefinitions_x86.hpp line 73: > 71: > 72: #if INCLUDE_JVMCI > 73: #define COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS (EnableJVMCI) Minor nit: can probably drop parentheses here. src/hotspot/share/code/compiledIC.cpp line 715: > 713: tty->print("interpreted"); > 714: } else { > 715: tty->print("unknown"); We can probably split this cleanup into the minor issue, your call. The benefit of separate issue: backportability. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From dcubed at openjdk.java.net Fri Apr 9 15:05:20 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 9 Apr 2021 15:05:20 GMT Subject: Integrated: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI In-Reply-To: References: Message-ID: <9qxyrScT3Cqe9bdD5J67jEa6onF7X8rur8_uzcjLXwA=.5d2efa94-a4d3-434a-bad6-06ea5d5cea9b@github.com> On Tue, 9 Mar 2021 21:08:54 GMT, Daniel D. Daugherty wrote: > Add three tests from JDK-4413752 ported to JVM/TI: > > - RawMonitorEnter() with SuspendThread() > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithRawMonitorEnter/SuspendWithRawMonitorEnter.java > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithRawMonitorEnter/libSuspendWithRawMonitorEnter.cpp > > - ObjectMonitor enter() with SuspendThread() > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorEnter/SuspendWithObjectMonitorEnter.java > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorEnter/libSuspendWithObjectMonitorEnter.cpp > > - ObjectMonitor wait() with SuspendThread > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/SuspendWithObjectMonitorWait.java > - test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/libSuspendWithObjectMonitorWait.cpp > > The Java files have a transaction diagram to show what each of the > threads in the test is doing. This pull request has now been integrated. Changeset: 1ca4abe9 Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/1ca4abe9 Stats: 1501 lines in 6 files changed: 1501 ins; 0 del; 0 mod 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI Reviewed-by: sspitsyn, rehn ------------- PR: https://git.openjdk.java.net/jdk/pull/2899 From dcubed at openjdk.java.net Fri Apr 9 16:19:38 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 9 Apr 2021 16:19:38 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> On Thu, 8 Apr 2021 07:17:48 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - White space fixes > - Merge branch 'master' into SuspendInHandshake > - Review fixes > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) I'm still really happy with this fix! Thumbs up! Reminder: don't forget to decide what to do about the obsolete options. @dholmes-ora can provide guidance on how to handle this type of removal. src/hotspot/share/runtime/handshake.cpp line 632: > 630: } > 631: > 632: void HandshakeState::self_suspened() { Typo: s/self_suspened/self_suspended/ src/hotspot/share/runtime/handshake.cpp line 654: > 652: void do_thread(Thread* thr) { > 653: JavaThread* target = thr->as_Java_thread(); > 654: target->handshake_state()->self_suspened(); Typo: s/self_suspened/self_suspended/ src/hotspot/share/runtime/handshake.hpp line 131: > 129: return true; > 130: } > 131: OrderAccess::loadload(); Needs a comment explaining what the memory sync is for. src/hotspot/share/runtime/handshake.hpp line 156: > 154: // This flag is true while there is async handshake (trap) > 155: // on queue. Since we do only need one, we can reuse it if > 156: // thread thread gets suspended again (after a resume) typo: s/thread thread/thread/ src/hotspot/share/runtime/objectMonitor.cpp line 410: > 408: > 409: current->frame_anchor()->make_walkable(current); > 410: OrderAccess::storestore(); Needs a comment explaining what the memory sync is for. src/hotspot/share/runtime/objectMonitor.cpp line 970: > 968: > 969: current->frame_anchor()->make_walkable(current); > 970: OrderAccess::storestore(); Needs a comment explaining what the memory sync is for. src/hotspot/share/runtime/objectMonitor.cpp line 977: > 975: if (_succ == current) { > 976: _succ = NULL; > 977: OrderAccess::fence(); Please add a comment to this line: `OrderAccess::fence(); // always do a full fence when successor is cleared` src/hotspot/share/runtime/objectMonitor.cpp line 1540: > 1538: { > 1539: current->frame_anchor()->make_walkable(current); > 1540: OrderAccess::storestore(); Needs a comment explaining what the memory sync is for. src/hotspot/share/runtime/objectMonitor.cpp line 1555: > 1553: if (_succ == current) { > 1554: _succ = NULL; > 1555: OrderAccess::fence(); Please add a comment to this line: OrderAccess::fence(); // always do a full fence when successor is cleared src/hotspot/share/runtime/thread.cpp line 455: > 453: ParkEvent::Release(_ParkEvent); > 454: // Can be racingly loaded in signal handler via has_terminated() > 455: Atomic::store(&_ParkEvent, (ParkEvent*)NULL); Nice solution for that odd signal handler issue. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Fri Apr 9 16:19:39 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 9 Apr 2021 16:19:39 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 07:23:26 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/handshake.cpp line 415: >> >>> 413: // Adds are done lock free and so is arming. >>> 414: // Calling this method with lock held is considered an error. >>> 415: assert(!_lock.owned_by_self(), "Lock should not be held"); >> >> If adds are still lock-free then why was this assertion removed? > > Because we add the async handshake inside the sync handshake and thus we already hold the _lock. Doesn't that mean the comment on L415 needs updating? Should it be something like: // Synchronous adds are done lock free and so is arming, but some // asynchronous adds are done when we already hold the lock. I'm not sure about the "some asynchronous adds" part... ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Fri Apr 9 16:19:41 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 9 Apr 2021 16:19:41 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 16:08:06 GMT, Richard Reingruber wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.cpp line 630: > >> 628: // exiting. >> 629: } >> 630: } > > I need a little help learning the steps of this dance :) > > We reach here in _thread_in_vm. We cannot be suspended in this state. There > might be another thread waiting to handshake us to suspend us but why can't we > just ignore that and do the `_handshakee->set_exiting();` even without locking > HandshakeState::_lock? > > Adding a handshake operation is lock free, so even if the check > `SafepointMechanism::should_process(_handshakee)` in L619 returns false a new > operation to process could have been added concurrently such that > `SafepointMechanism::should_process(_handshakee)` would return true when we > execute `_handshakee->set_exiting();` in L620. What am I missing? `HandshakeState::suspend()` is a synchronous handshake and adding the handshake to the queue is lock free, but the execution of the synchronous handshake itself requires a `HandshakeState::claim_handshake()` call which does acquire the lock in question. We (the suspend requester) hold the lock while the handshake is being processed so we either detect that _handshakee->set_exiting() won the race (in the target thread) or we (the suspend requester) win the race of setting the suspend flag so the target thread can't exit yet. Hopefully that helps explain this dance. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Fri Apr 9 16:19:40 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 9 Apr 2021 16:19:40 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 07:25:29 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/handshake.cpp line 463: >> >>> 461: ThreadInVMForHandshake tivm(_handshakee); >>> 462: { >>> 463: ttyLocker::break_tty_lock_for_safepoint(os::current_thread_id()); >> >> Why is this needed when it is inside ThreadInVMForHandshake constructor ?? > > If we process the async suspension handshake we can go to safepoint. > And before safepoint we must drop the tty lock. Is this worth a comment above the `break_tty_lock_for_safepoint()` call? >> src/hotspot/share/runtime/thread.cpp line 1442: >> >>> 1440: >>> 1441: // The careful dance between thread suspension and exit is handled here: >>> 1442: _handshake.thread_exit(); >> >> I don't like the fact this hides the set_terminating call. >> >> In fact I think I'd rather keep this in-line rather than tucking it away inside the handshake code. This looks too much like we're informing the handshake that the thread is exiting rather then coordinating thread termination with a late suspension request. > > The problem is that we need access to the private _lock to coordinate this. Perhaps rename the function to something like handle_thread_exit_race()? ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Fri Apr 9 16:19:42 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 9 Apr 2021 16:19:42 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Fri, 9 Apr 2021 15:29:30 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.cpp line 632: > >> 630: } >> 631: >> 632: void HandshakeState::self_suspened() { > > Typo: s/self_suspened/self_suspended/ I'm not fond of `self_suspended` as the function name. Perhaps `do_self_suspend` or something like that... (What color would you like that bike shed?) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From iveresov at openjdk.java.net Fri Apr 9 16:41:20 2021 From: iveresov at openjdk.java.net (Igor Veresov) Date: Fri, 9 Apr 2021 16:41:20 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: <32eWo34nJ7czxicNNgoww6GpOpg0jKq8NZY_pPeQPpI=.e698cb8a-78e7-40a2-b54e-9b29ce1bedb1@github.com> On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot src/hotspot/share/jvmci/jvmciCodeInstaller.cpp line 1184: > 1182: } > 1183: } else if (jvmci_env()->isa_HotSpotMetaspaceConstantImpl(constant)) { > 1184: if (!_immutable_pic_compilation) { All _immutable_pic_compilation mentions can be removed as well. It is true only for AOT compiles produced by Graal. It's never going to be used without AOT. src/hotspot/share/oops/instanceKlass.hpp line 257: > 255: _misc_declares_nonstatic_concrete_methods = 1 << 6, // directly declares non-static, concrete methods > 256: _misc_has_been_redefined = 1 << 7, // class has been redefined > 257: _unused = 1 << 8, // Any particular reason we don't want to remove this gap? ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From iklam at openjdk.java.net Fri Apr 9 16:58:23 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Fri, 9 Apr 2021 16:58:23 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: <3zISek5YyT0zkmPX3UZtXKy_r63eOZoW6emV3SuRjPg=.4b805fb5-80c9-4b76-92f8-43f2335c525b@github.com> On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot LGTM. Just a small nit. src/hotspot/share/oops/methodCounters.cpp line 77: > 75: } > 76: > 77: void MethodCounters::metaspace_pointers_do(MetaspaceClosure* it) { MethodCounters::metaspace_pointers_do can be removed altogether (also need to remove the declaration in methodCounter.hpp). ------------- Marked as reviewed by iklam (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 17:13:17 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 17:13:17 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 04:32:14 GMT, David Holmes wrote: > Hi Vladimir, > > This looks good to me - I went through all the files. > > It was nice to see how nicely compartmentalized the AOT feature was - that made checking the changes quite simple. The one exception was the fingerprinting code, which for some reason was AOT-only but not marked that way, so I'm not sure what the story is there. ?? > > I was also surprised to see some of the flags were not marked experimental, so there will need to a CSR request to remove those without going through the normal deprecation process. > > Thanks, > David Thank you, David. We thought that we could use fingerprint code for other cases that is why we did not put it under `#if INCLUDE_AOT`. I filed CSR for AOT product flags removal: https://bugs.openjdk.java.net/browse/JDK-8265000 ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From sspitsyn at openjdk.java.net Fri Apr 9 17:26:22 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 9 Apr 2021 17:26:22 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Fri, 9 Apr 2021 16:15:28 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/handshake.cpp line 632: >> >>> 630: } >>> 631: >>> 632: void HandshakeState::self_suspened() { >> >> Typo: s/self_suspened/self_suspended/ > > I'm not fond of `self_suspended` as the function name. > Perhaps `do_self_suspend` or something like that... > (What color would you like that bike shed?) I like the suggestion to rename self_suspened to do_self_suspend. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From kvn at openjdk.java.net Fri Apr 9 17:42:27 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 17:42:27 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 17:09:58 GMT, Vladimir Kozlov wrote: >> Hi Vladimir, >> >> This looks good to me - I went through all the files. >> >> It was nice to see how nicely compartmentalized the AOT feature was - that made checking the changes quite simple. The one exception was the fingerprinting code, which for some reason was AOT-only but not marked that way, so I'm not sure what the story is there. ?? >> >> I was also surprised to see some of the flags were not marked experimental, so there will need to a CSR request to remove those without going through the normal deprecation process. >> >> Thanks, >> David > >> Hi Vladimir, >> >> This looks good to me - I went through all the files. >> >> It was nice to see how nicely compartmentalized the AOT feature was - that made checking the changes quite simple. The one exception was the fingerprinting code, which for some reason was AOT-only but not marked that way, so I'm not sure what the story is there. ?? >> >> I was also surprised to see some of the flags were not marked experimental, so there will need to a CSR request to remove those without going through the normal deprecation process. >> >> Thanks, >> David > > Thank you, David. > We thought that we could use fingerprint code for other cases that is why we did not put it under `#if INCLUDE_AOT`. > I filed CSR for AOT product flags removal: https://bugs.openjdk.java.net/browse/JDK-8265000 Thank you, Igor Ignatyev, Igor Veresov, Ioi, Aleksey and Andrew for reviews. I will update changes based on your comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 17:45:20 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 17:45:20 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 02:44:23 GMT, David Holmes wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > src/hotspot/cpu/x86/compiledIC_x86.cpp line 134: > >> 132: #ifdef ASSERT >> 133: CodeBlob *cb = CodeCache::find_blob_unsafe((address) _call); >> 134: assert(cb, "sanity"); > > Nit: implied boolean - use "cb != NULL" done ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 18:23:26 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 18:23:26 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 03:03:33 GMT, David Holmes wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > src/hotspot/share/memory/heap.hpp line 174: > >> 172: bool contains(const void* p) const { return low() <= p && p < high(); } >> 173: bool contains_blob(const CodeBlob* blob) const { >> 174: const void* start = (void*)blob; > > start seems redundant now Done: bool contains_blob(const CodeBlob* blob) const { - const void* start = (void*)blob; - return contains(start); + return contains((void*)blob); } ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 18:23:25 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 18:23:25 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 08:29:21 GMT, Aleksey Shipilev wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > src/hotspot/cpu/x86/globalDefinitions_x86.hpp line 73: > >> 71: >> 72: #if INCLUDE_JVMCI >> 73: #define COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS (EnableJVMCI) > > Minor nit: can probably drop parentheses here. done ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 18:32:26 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 18:32:26 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 08:32:59 GMT, Aleksey Shipilev wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > src/hotspot/share/code/compiledIC.cpp line 715: > >> 713: tty->print("interpreted"); >> 714: } else { >> 715: tty->print("unknown"); > > We can probably split this cleanup into the minor issue, your call. The benefit of separate issue: backportability. Reverted and filed https://bugs.openjdk.java.net/browse/JDK-8265013 ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 18:36:24 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 18:36:24 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: <32eWo34nJ7czxicNNgoww6GpOpg0jKq8NZY_pPeQPpI=.e698cb8a-78e7-40a2-b54e-9b29ce1bedb1@github.com> References: <32eWo34nJ7czxicNNgoww6GpOpg0jKq8NZY_pPeQPpI=.e698cb8a-78e7-40a2-b54e-9b29ce1bedb1@github.com> Message-ID: <8rFTTlGuCqN9XzBEbaAAf9R-YhTTqe45jv9Gh7F506w=.67e92697-68c4-4657-abb4-803231a9d1a9@github.com> On Fri, 9 Apr 2021 16:30:41 GMT, Igor Veresov wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > src/hotspot/share/oops/instanceKlass.hpp line 257: > >> 255: _misc_declares_nonstatic_concrete_methods = 1 << 6, // directly declares non-static, concrete methods >> 256: _misc_has_been_redefined = 1 << 7, // class has been redefined >> 257: _unused = 1 << 8, // > > Any particular reason we don't want to remove this gap? Less changes. We don't get any benefits from removing it. It is opposite - if we need a new value we will use it without changing following values again. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 19:09:25 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 19:09:25 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: <32eWo34nJ7czxicNNgoww6GpOpg0jKq8NZY_pPeQPpI=.e698cb8a-78e7-40a2-b54e-9b29ce1bedb1@github.com> References: <32eWo34nJ7czxicNNgoww6GpOpg0jKq8NZY_pPeQPpI=.e698cb8a-78e7-40a2-b54e-9b29ce1bedb1@github.com> Message-ID: On Fri, 9 Apr 2021 16:34:58 GMT, Igor Veresov wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > src/hotspot/share/jvmci/jvmciCodeInstaller.cpp line 1184: > >> 1182: } >> 1183: } else if (jvmci_env()->isa_HotSpotMetaspaceConstantImpl(constant)) { >> 1184: if (!_immutable_pic_compilation) { > > All _immutable_pic_compilation mentions can be removed as well. It is true only for AOT compiles produced by Graal. It's never going to be used without AOT. Done. I removed _immutable_pic_compilation in JVMCI in Hotspot. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 19:26:22 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 19:26:22 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: <3zISek5YyT0zkmPX3UZtXKy_r63eOZoW6emV3SuRjPg=.4b805fb5-80c9-4b76-92f8-43f2335c525b@github.com> References: <3zISek5YyT0zkmPX3UZtXKy_r63eOZoW6emV3SuRjPg=.4b805fb5-80c9-4b76-92f8-43f2335c525b@github.com> Message-ID: <9RiwxlBLMiREzNvRHU14RQBW33nieUT8-Pmkod_GvtA=.ad51b2f9-f244-4346-8844-9fee39c9aa5b@github.com> On Fri, 9 Apr 2021 16:54:35 GMT, Ioi Lam wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > src/hotspot/share/oops/methodCounters.cpp line 77: > >> 75: } >> 76: >> 77: void MethodCounters::metaspace_pointers_do(MetaspaceClosure* it) { > > MethodCounters::metaspace_pointers_do can be removed altogether (also need to remove the declaration in methodCounter.hpp). Done. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From mchung at openjdk.java.net Fri Apr 9 20:53:33 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Fri, 9 Apr 2021 20:53:33 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 17:24:38 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Remove exports from Graal module to jdk.aot I reviewed the module-loader-map and non-hotspot non-AOT tests. ------------- Marked as reviewed by mchung (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Fri Apr 9 21:59:04 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 9 Apr 2021 21:59:04 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v5] In-Reply-To: References: Message-ID: > As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: > > - `jdk.aot` module ? the `jaotc` tool > - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution > - related HotSpot code guarded by `#if INCLUDE_AOT` > > Additionally, remove tests as well as code in makefiles related to AoT compilation. > > Tested hs-tier1-4 Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: Address review comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3398/files - new: https://git.openjdk.java.net/jdk/pull/3398/files/6cce0f6c..71a166c1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3398&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3398&range=03-04 Stats: 36 lines in 9 files changed: 0 ins; 27 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/3398.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3398/head:pull/3398 PR: https://git.openjdk.java.net/jdk/pull/3398 From iveresov at openjdk.java.net Fri Apr 9 22:02:33 2021 From: iveresov at openjdk.java.net (Igor Veresov) Date: Fri, 9 Apr 2021 22:02:33 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v5] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 21:59:04 GMT, Vladimir Kozlov wrote: >> As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: >> >> - `jdk.aot` module ? the `jaotc` tool >> - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution >> - related HotSpot code guarded by `#if INCLUDE_AOT` >> >> Additionally, remove tests as well as code in makefiles related to AoT compilation. >> >> Tested hs-tier1-4 > > Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Address review comments Marked as reviewed by iveresov (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From amenkov at openjdk.java.net Sat Apr 10 01:22:45 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Sat, 10 Apr 2021 01:22:45 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" Message-ID: The test actually failed starting from jdk13, but the error is masked by JDK-8264667 (and shell part of the test didn't verify result of the java execution) The fix: - updates JvmtiClassFileReconstituter to add attributes in the same order as javac does - makes the test java instead of shell - added workaround for JDK-8264667 - test code is ready to ignore order of attributes ------------- Commit messages: - fixed spaces - JDK-8262002 Changes: https://git.openjdk.java.net/jdk/pull/3426/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3426&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8262002 Stats: 220 lines in 4 files changed: 113 ins; 96 del; 11 mod Patch: https://git.openjdk.java.net/jdk/pull/3426.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3426/head:pull/3426 PR: https://git.openjdk.java.net/jdk/pull/3426 From amenkov at openjdk.java.net Sat Apr 10 01:22:46 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Sat, 10 Apr 2021 01:22:46 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" In-Reply-To: References: Message-ID: <3C-yT6xsCeRk7U96EHpt2NuALsf2h3VqauXRGMENE80=.c8fa44d0-0ee0-4db4-8572-516b38586575@github.com> On Sat, 10 Apr 2021 01:10:37 GMT, Alex Menkov wrote: > The test actually failed starting from jdk13, but the error is masked by JDK-8264667 (and shell part of the test didn't verify result of the java execution) > The fix: > - updates JvmtiClassFileReconstituter to add attributes in the same order as javac does > - makes the test java instead of shell > - added workaround for JDK-8264667 > - test code is ready to ignore order of attributes label remove core-libs ------------- PR: https://git.openjdk.java.net/jdk/pull/3426 From dlong at openjdk.java.net Sat Apr 10 04:05:27 2021 From: dlong at openjdk.java.net (Dean Long) Date: Sat, 10 Apr 2021 04:05:27 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: <3zISek5YyT0zkmPX3UZtXKy_r63eOZoW6emV3SuRjPg=.4b805fb5-80c9-4b76-92f8-43f2335c525b@github.com> References: <3zISek5YyT0zkmPX3UZtXKy_r63eOZoW6emV3SuRjPg=.4b805fb5-80c9-4b76-92f8-43f2335c525b@github.com> Message-ID: On Fri, 9 Apr 2021 16:54:51 GMT, Ioi Lam wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > LGTM. Just a small nit. @iklam I thought the fingerprint code was also used by CDS. ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From iklam at openjdk.java.net Sat Apr 10 06:08:33 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Sat, 10 Apr 2021 06:08:33 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v4] In-Reply-To: <3zISek5YyT0zkmPX3UZtXKy_r63eOZoW6emV3SuRjPg=.4b805fb5-80c9-4b76-92f8-43f2335c525b@github.com> References: <3zISek5YyT0zkmPX3UZtXKy_r63eOZoW6emV3SuRjPg=.4b805fb5-80c9-4b76-92f8-43f2335c525b@github.com> Message-ID: On Fri, 9 Apr 2021 16:54:51 GMT, Ioi Lam wrote: >> Vladimir Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove exports from Graal module to jdk.aot > > LGTM. Just a small nit. > @iklam > I thought the fingerprint code was also used by CDS. CDS actually uses a different mechanism (CRC of the classfile bytes) to validate that a class has not changed between archive dumping time and runtime. See https://github.com/openjdk/jdk/blob/5784f6b7f74d0b8081ac107fea172539d57d6020/src/hotspot/share/classfile/systemDictionaryShared.cpp#L1126-L1130 ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From rrich at openjdk.java.net Sat Apr 10 07:41:25 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Sat, 10 Apr 2021 07:41:25 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: <-fsVAozVnQsZHcO1hEhjJi9oALuXdOCOg7q6wqgyvaA=.b0e4face-f58b-45ae-83ba-c6b249752272@github.com> On Fri, 9 Apr 2021 16:12:16 GMT, Daniel D. Daugherty wrote: >> src/hotspot/share/runtime/handshake.cpp line 630: >> >>> 628: // exiting. >>> 629: } >>> 630: } >> >> I need a little help learning the steps of this dance :) >> >> We reach here in _thread_in_vm. We cannot be suspended in this state. There >> might be another thread waiting to handshake us to suspend us but why can't we >> just ignore that and do the `_handshakee->set_exiting();` even without locking >> HandshakeState::_lock? >> >> Adding a handshake operation is lock free, so even if the check >> `SafepointMechanism::should_process(_handshakee)` in L619 returns false a new >> operation to process could have been added concurrently such that >> `SafepointMechanism::should_process(_handshakee)` would return true when we >> execute `_handshakee->set_exiting();` in L620. What am I missing? > > `HandshakeState::suspend()` is a synchronous handshake and adding the > handshake to the queue is lock free, but the execution of the synchronous > handshake itself requires a `HandshakeState::claim_handshake()` call which > does acquire the lock in question. We (the suspend requester) hold the lock > while the handshake is being processed so we either detect that > _handshakee->set_exiting() won the race (in the target thread) or we (the > suspend requester) win the race of setting the suspend flag so the target > thread can't exit yet. > > Hopefully that helps explain this dance. Hi Dan, thanks for picking up my question! > `HandshakeState::suspend()` is a synchronous handshake and adding the > handshake to the queue is lock free, but the execution of the synchronous > handshake itself requires a `HandshakeState::claim_handshake()` call which > does acquire the lock in question. My point would be that the attempt to execute the synchronous handshake for suspending a thread that is just about to call HandshakeState::thread_exit() cannot make progress (blocks) while the target thread is not safe (_thread_in_vm). A synchronous handshake requires the target thread to be in a safe state for the requester to execute the handshake operation. When executing HandshakeState::thread_exit() the suspendee is _thread_in_vm. And the requester will find it to be `_not_safe` when calling `possibly_can_process_handshake()` before calling `HandshakeState::claim_handshake()` or when calling `can_process_handshake()` afterwards. In both cases try_process() returns with failure _not_safe and the lock is not held. ++ 546 if (!possibly_can_process_handshake()) { 547 // JT is observed in an unsafe state, it must notice the handshake itself 548 return HandshakeState::_not_safe; 549 } 550 551 // Claim the mutex if there still an operation to be executed. 552 if (!claim_handshake()) { 553 return HandshakeState::_claim_failed; 554 } 555 556 // If we own the mutex at this point and while owning the mutex we 557 // can observe a safe state the thread cannot possibly continue without 558 // getting caught by the mutex. 559 if (!can_process_handshake()) { 560 _lock.unlock(); 561 return HandshakeState::_not_safe; 562 } So isn't being unsafe sufficient to sync with suspend requests? ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Sat Apr 10 13:59:38 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Sat, 10 Apr 2021 13:59:38 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: <-fsVAozVnQsZHcO1hEhjJi9oALuXdOCOg7q6wqgyvaA=.b0e4face-f58b-45ae-83ba-c6b249752272@github.com> References: <-fsVAozVnQsZHcO1hEhjJi9oALuXdOCOg7q6wqgyvaA=.b0e4face-f58b-45ae-83ba-c6b249752272@github.com> Message-ID: On Sat, 10 Apr 2021 07:38:37 GMT, Richard Reingruber wrote: >> `HandshakeState::suspend()` is a synchronous handshake and adding the >> handshake to the queue is lock free, but the execution of the synchronous >> handshake itself requires a `HandshakeState::claim_handshake()` call which >> does acquire the lock in question. We (the suspend requester) hold the lock >> while the handshake is being processed so we either detect that >> _handshakee->set_exiting() won the race (in the target thread) or we (the >> suspend requester) win the race of setting the suspend flag so the target >> thread can't exit yet. >> >> Hopefully that helps explain this dance. > > Hi Dan, > > thanks for picking up my question! > >> `HandshakeState::suspend()` is a synchronous handshake and adding the >> handshake to the queue is lock free, but the execution of the synchronous >> handshake itself requires a `HandshakeState::claim_handshake()` call which >> does acquire the lock in question. > > My point would be that the attempt to execute the synchronous handshake for > suspending a thread that is just about to call HandshakeState::thread_exit() > cannot make progress (blocks) while the target thread is not safe > (_thread_in_vm). > > A synchronous handshake requires the target thread to be in a safe state for the > requester to execute the handshake operation. When executing > HandshakeState::thread_exit() the suspendee is _thread_in_vm. And the requester > will find it to be `_not_safe` when calling `possibly_can_process_handshake()` > before calling `HandshakeState::claim_handshake()` or when calling > `can_process_handshake()` afterwards. In both cases try_process() returns with > failure _not_safe and the lock is not held. > > ++ > 546 if (!possibly_can_process_handshake()) { > 547 // JT is observed in an unsafe state, it must notice the handshake itself > 548 return HandshakeState::_not_safe; > 549 } > 550 > 551 // Claim the mutex if there still an operation to be executed. > 552 if (!claim_handshake()) { > 553 return HandshakeState::_claim_failed; > 554 } > 555 > 556 // If we own the mutex at this point and while owning the mutex we > 557 // can observe a safe state the thread cannot possibly continue without > 558 // getting caught by the mutex. > 559 if (!can_process_handshake()) { > 560 _lock.unlock(); > 561 return HandshakeState::_not_safe; > 562 } > > So isn't being unsafe sufficient to sync with suspend requests? Interesting point that I didn't pick up from your previous comment. Thanks for making it more clear for me. I need to mull on it for a bit. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From alanb at openjdk.java.net Sun Apr 11 13:37:03 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Sun, 11 Apr 2021 13:37:03 GMT Subject: RFR: 8264124: Update MXBean specification and implementation to extend mapping of CompositeType to records [v7] In-Reply-To: References: <7Oco_BzQDkrrNFy6FRoUFAUx5IuHu9YtuEfNuLverUI=.b57d8a09-8b39-4db1-a439-57285dedb7f7@github.com> Message-ID: On Fri, 2 Apr 2021 18:00:57 GMT, Daniel Fuchs wrote: >> This RFE proposes to extend the MXBean framework to define a mapping to records. >> >> The MXBean framework already defines a mapping of `CompositeType` to plain java objects. Records are a natural representation of CompositeTypes. A record can be easily reconstructed from a `CompositeData` through the record canonical constructor. A clear advantage of records over plain java objects is that the canonical constructor will not need to be annotated in order to map composite data property names to constructor parameter names. >> >> With this RFE, here is an example comparing coding a composite type `NamedNumber` that consists of an `int` and a `String`, using records and using a plain java class. In both case, the `CompositeType` looks like this: >> >> CompositeType( >> "NamedNumber", // typeName >> "NamedNumber", // description >> new String[] {"number", "name"}, // itemNames >> new String[] {"number", "name"}, // itemDescriptions >> new OpenType[] {SimpleType.INTEGER, >> SimpleType.STRING} // itemTypes >> ); >> >> The plain Java class needs a public constructor annotated with `@ConstructorParameters` annotation: >> >> public class NamedNumber { >> public int getNumber() {return number;} >> public String getName() {return name;} >> @ConstructorParameters({"number", "name"}) >> public NamedNumber(int number, String name) { >> this.number = number; >> this.name = name; >> } >> private final int number; >> private final String name; >> } >> >> And the equivalent with a record class: >> >> public record NamedNumber(int number, String name) {} > > Daniel Fuchs has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 13 additional commits since the last revision: > > - Merge > - minor style issue > - minor style issue > - Integrated review feedback > - Improved test > - Moved mapping for records just before Mapping for MXBean interface; Improved test. > - Merge branch 'master' into mxbeans-8264124 > - Minor tweaks. Improved test. > - Merge branch 'master' into mxbeans-8264124 > - Integrated review feedback. Updated the section for Mapping to records. > - ... and 3 more: https://git.openjdk.java.net/jdk/compare/49d8cb13...257a6ed8 Spec and implementation changes look good. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3201 From rehn at openjdk.java.net Mon Apr 12 06:50:21 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 06:50:21 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <-fsVAozVnQsZHcO1hEhjJi9oALuXdOCOg7q6wqgyvaA=.b0e4face-f58b-45ae-83ba-c6b249752272@github.com> Message-ID: On Sat, 10 Apr 2021 13:56:41 GMT, Daniel D. Daugherty wrote: >> Hi Dan, >> >> thanks for picking up my question! >> >>> `HandshakeState::suspend()` is a synchronous handshake and adding the >>> handshake to the queue is lock free, but the execution of the synchronous >>> handshake itself requires a `HandshakeState::claim_handshake()` call which >>> does acquire the lock in question. >> >> My point would be that the attempt to execute the synchronous handshake for >> suspending a thread that is just about to call HandshakeState::thread_exit() >> cannot make progress (blocks) while the target thread is not safe >> (_thread_in_vm). >> >> A synchronous handshake requires the target thread to be in a safe state for the >> requester to execute the handshake operation. When executing >> HandshakeState::thread_exit() the suspendee is _thread_in_vm. And the requester >> will find it to be `_not_safe` when calling `possibly_can_process_handshake()` >> before calling `HandshakeState::claim_handshake()` or when calling >> `can_process_handshake()` afterwards. In both cases try_process() returns with >> failure _not_safe and the lock is not held. >> >> ++ >> 546 if (!possibly_can_process_handshake()) { >> 547 // JT is observed in an unsafe state, it must notice the handshake itself >> 548 return HandshakeState::_not_safe; >> 549 } >> 550 >> 551 // Claim the mutex if there still an operation to be executed. >> 552 if (!claim_handshake()) { >> 553 return HandshakeState::_claim_failed; >> 554 } >> 555 >> 556 // If we own the mutex at this point and while owning the mutex we >> 557 // can observe a safe state the thread cannot possibly continue without >> 558 // getting caught by the mutex. >> 559 if (!can_process_handshake()) { >> 560 _lock.unlock(); >> 561 return HandshakeState::_not_safe; >> 562 } >> >> So isn't being unsafe sufficient to sync with suspend requests? > > Interesting point that I didn't pick up from your previous comment. > Thanks for making it more clear for me. I need to mull on it for a bit. Technically you are correct. I have tested to remove this is previously version and all tests passes fine. The reason I kept it is because the suspender have identified a thread for suspension and deemed it suspendable, so we play nice. To know why suspend failed the suspender must check if thread is exiting after a failed suspend. (originally I had a bug here which caused me to wrongfully introduce this from the beginning) I'll remove it, since it simplifies the code and David's comments about this code is now out-of-line can be fixed. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 07:12:43 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 07:12:43 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Fri, 9 Apr 2021 16:16:45 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > I'm still really happy with this fix! Thumbs up! > Reminder: don't forget to decide what to do about the obsolete options. > @dholmes-ora can provide guidance on how to handle this type of removal. Thanks Serguei, Fixed nits. > I wonder why ThreadSuspensionHandshake is not stack allocated as SuspendThreadHandshake below: Simplified: Suspender thread target another thread for suspension. It first allocates the synchronous handshake on stack and tries to execute it. When the target thread becomes blocked/native we install the asynchronous handshake (the trap). The trap will stop the target thread from leaving blocked/native until we resume it. Since the suspender now have successfully suspended the target (the trap is installed) it will return from this method call. Thus it's stack is not usable for the asynchronous handshake. The trap can even be executed after the suspender thread have terminated, therefore we need to but the trap on the heap. Hope this answers your question! Thanks, Robbin > Thanks, > Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 07:34:19 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 07:34:19 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Fri, 9 Apr 2021 17:23:36 GMT, Serguei Spitsyn wrote: >> I'm not fond of `self_suspended` as the function name. >> Perhaps `do_self_suspend` or something like that... >> (What color would you like that bike shed?) > > I like the suggestion to rename self_suspened to do_self_suspend. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 07:47:07 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 07:47:07 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: <82CqXQRu4OD8i9KFwuUswnYTHFaqktISMYKAsIt7A9k=.4208365c-9aa1-4cd2-aff9-89d14d239ecc@github.com> On Fri, 9 Apr 2021 15:34:50 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.hpp line 131: > >> 129: return true; >> 130: } >> 131: OrderAccess::loadload(); > > Needs a comment explaining what the memory sync is for. Fixed > src/hotspot/share/runtime/handshake.hpp line 156: > >> 154: // This flag is true while there is async handshake (trap) >> 155: // on queue. Since we do only need one, we can reuse it if >> 156: // thread thread gets suspended again (after a resume) > > typo: s/thread thread/thread/ Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 08:03:32 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 08:03:32 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Fri, 9 Apr 2021 16:16:45 GMT, Daniel D. Daugherty wrote: > I'm still really happy with this fix! Thumbs up! > Reminder: don't forget to decide what to do about the obsolete options. > @dholmes-ora can provide guidance on how to handle this type of removal. Thanks, yes fixing CSR! > src/hotspot/share/runtime/handshake.cpp line 654: > >> 652: void do_thread(Thread* thr) { >> 653: JavaThread* target = thr->as_Java_thread(); >> 654: target->handshake_state()->self_suspened(); > > Typo: s/self_suspened/self_suspended/ Fixed > src/hotspot/share/runtime/objectMonitor.cpp line 970: > >> 968: >> 969: current->frame_anchor()->make_walkable(current); >> 970: OrderAccess::storestore(); > > Needs a comment explaining what the memory sync is for. Fixed > src/hotspot/share/runtime/objectMonitor.cpp line 977: > >> 975: if (_succ == current) { >> 976: _succ = NULL; >> 977: OrderAccess::fence(); > > Please add a comment to this line: > `OrderAccess::fence(); // always do a full fence when successor is cleared` Fixed > src/hotspot/share/runtime/objectMonitor.cpp line 1540: > >> 1538: { >> 1539: current->frame_anchor()->make_walkable(current); >> 1540: OrderAccess::storestore(); > > Needs a comment explaining what the memory sync is for. Fixed > src/hotspot/share/runtime/objectMonitor.cpp line 1555: > >> 1553: if (_succ == current) { >> 1554: _succ = NULL; >> 1555: OrderAccess::fence(); > > Please add a comment to this line: > OrderAccess::fence(); // always do a full fence when successor is cleared Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Mon Apr 12 08:03:30 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Mon, 12 Apr 2021 08:03:30 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 07:17:48 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - White space fixes > - Merge branch 'master' into SuspendInHandshake > - Review fixes > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) Nice ?? Richard. ------------- Marked as reviewed by rrich (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 08:03:32 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 08:03:32 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <-fsVAozVnQsZHcO1hEhjJi9oALuXdOCOg7q6wqgyvaA=.b0e4face-f58b-45ae-83ba-c6b249752272@github.com> Message-ID: On Mon, 12 Apr 2021 07:44:38 GMT, Richard Reingruber wrote: > Also it does not eliminate the race as far as I can tell. You are correct. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Mon Apr 12 08:03:32 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Mon, 12 Apr 2021 08:03:32 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <-fsVAozVnQsZHcO1hEhjJi9oALuXdOCOg7q6wqgyvaA=.b0e4face-f58b-45ae-83ba-c6b249752272@github.com> Message-ID: On Mon, 12 Apr 2021 06:42:47 GMT, Robbin Ehn wrote: > > > Technically you are correct. > I have tested to remove this is previously version and all tests passes fine. > The reason I kept it is because the suspender have identified a thread for suspension and deemed it suspendable, so we play nice. > To know why suspend failed the suspender must check if thread is exiting after a failed suspend. (originally I had a bug here which caused me to wrongfully introduce this from the beginning) Thanks for the explanation. As I understand you want to be fault tolerant towards somewhat sloppy agents. That's of course ok. It should be explained in a comment though. > I'll remove it, since it simplifies the code and David's comments about this code is now out-of-line can be fixed. Also it does not eliminate the race as far as I can tell. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 08:09:28 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 08:09:28 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 07:56:36 GMT, Richard Reingruber wrote: > Nice > Richard. Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 08:09:29 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 08:09:29 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 03:24:45 GMT, Patricio Chilano Mateo wrote: >> src/hotspot/share/runtime/thread.cpp line 667: >> >>> 665: >>> 666: // We wait for the thread to transition to a more usable state. >>> 667: for (int i = 1; i <= SuspendRetryCount; i++) { >> >> You will need to obsolete SuspendRetryCount and SuspendRetryDelay. This will need a CSR request indicating why there is no deprecation step. > > I think we will need to do the same for flags AssertOnSuspendWaitFailure and TraceSuspendWaitFailures. I'll create CSR for the related to this change-set flags, SuspendRetryCount and SuspendRetryDelay. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 10:38:24 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 10:38:24 GMT Subject: RFR: 8257831: Suspend with handshakes [v5] In-Reply-To: References: Message-ID: <5lV-lT_joH_rjVXRphghGWLew_Cl7FfYbDh6xi4qXP4=.33c4a7ff-5038-4027-aded-19e185f738fe@github.com> > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: Review fixes 2 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3191/files - new: https://git.openjdk.java.net/jdk/pull/3191/files/ed8accf0..7e704326 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=03-04 Stats: 51 lines in 4 files changed: 18 ins; 24 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 10:38:25 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 10:38:25 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 12:57:01 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/handshake.cpp line 677: >> >>> 675: } else { >>> 676: // Target is going to wake up and leave suspension. >>> 677: // Let's just stop the thread from doing that. >> >> IIUC this would be the case where the target was hit with a suspend request but has not yet processed the actual suspension handshake op, then a resume comes in so suspended is no longer true, and then another suspend request is made (this one) which simply turns the suspended flag back on - is that right? >> Overall I'm finding it very hard to see what the two suspend state flags actually signify. I'd like to see that written up somewhere. > > Sure I pushed some changes with some more comments and renamed the flag. I hope that help, if not, let me know. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 10:53:46 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 10:53:46 GMT Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: - Merge branch 'master' into SuspendInHandshake - Review fixes 2 - White space fixes - Merge branch 'master' into SuspendInHandshake - Review fixes - Merge branch 'master' into SuspendInHandshake - Merge branch 'master' into SuspendInHandshake - 8257831: Suspend with handshake (review baseline) ------------- Changes: https://git.openjdk.java.net/jdk/pull/3191/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=05 Stats: 1332 lines in 39 files changed: 273 ins; 863 del; 196 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 10:53:47 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 10:53:47 GMT Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 13:57:50 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/thread.inline.hpp line 207: >> >>> 205: } >>> 206: >>> 207: inline void JavaThread::set_terminated(TerminatedTypes t) { >> >> I prefer set_terminated(arg) over the new set of methods. > > We had two methods: > > void set_terminated(TerminatedTypes t); > void set_terminated_value(); > > Terminated is part of the name of the method, the name of the flag, the name of the type and part of the names of two of the states, which is very confusing. > > Also the setters now match the queries: > E.g. > `bool is_exiting()` > > The queries do not indicate in any sense that they are queries on the terminated flag. > The state flag is an implementation detail from query POV. > So to be consistent e.g. "set_exiting()" also hides the fact that we keep track of this with a flag. Please advise :) , I can roll back if you insist! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 12 10:53:46 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 12 Apr 2021 10:53:46 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Mon, 12 Apr 2021 07:57:14 GMT, Robbin Ehn wrote: > I'm still really happy with this fix! Thumbs up! > Reminder: don't forget to decide what to do about the obsolete options. > @dholmes-ora can provide guidance on how to handle this type of removal. Thanks, yes I'll start a CSR! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From david.holmes at oracle.com Mon Apr 12 13:09:31 2021 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Apr 2021 23:09:31 +1000 Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On 12/04/2021 8:53 pm, Robbin Ehn wrote: > On Wed, 7 Apr 2021 13:57:50 GMT, Robbin Ehn wrote: > >>> src/hotspot/share/runtime/thread.inline.hpp line 207: >>> >>>> 205: } >>>> 206: >>>> 207: inline void JavaThread::set_terminated(TerminatedTypes t) { >>> >>> I prefer set_terminated(arg) over the new set of methods. >> >> We had two methods: >> >> void set_terminated(TerminatedTypes t); >> void set_terminated_value(); >> >> Terminated is part of the name of the method, the name of the flag, the name of the type and part of the names of two of the states, which is very confusing. I'll admit the API is not that clean but I would expect the method, the flag, the type, to all share a common name as they are all related to the same underlying thing: the termination state. set_terminated_value() seems completely unnecessary as a special-case. >> >> Also the setters now match the queries: >> E.g. >> `bool is_exiting()` >> >> The queries do not indicate in any sense that they are queries on the terminated flag. >> The state flag is an implementation detail from query POV. >> So to be consistent e.g. "set_exiting()" also hides the fact that we keep track of this with a flag. The point of the flag is that it is tracking a singular termination state with a progression from _not_terminated, through _thread_exiting, to _thread terminated, or the special state _vm_terminated (though I'm not sure why we need that ...) > Please advise :) , I can roll back if you insist! There is always room for improvement with these things, but the changes you are making to this part of the thread API seem completely unrelated to thread suspension, so I'd ask that you please roll them back, for now at least. Thanks, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From dfuchs at openjdk.java.net Mon Apr 12 16:34:38 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Mon, 12 Apr 2021 16:34:38 GMT Subject: Integrated: 8264124: Update MXBean specification and implementation to extend mapping of CompositeType to records In-Reply-To: <7Oco_BzQDkrrNFy6FRoUFAUx5IuHu9YtuEfNuLverUI=.b57d8a09-8b39-4db1-a439-57285dedb7f7@github.com> References: <7Oco_BzQDkrrNFy6FRoUFAUx5IuHu9YtuEfNuLverUI=.b57d8a09-8b39-4db1-a439-57285dedb7f7@github.com> Message-ID: On Thu, 25 Mar 2021 17:30:52 GMT, Daniel Fuchs wrote: > This RFE proposes to extend the MXBean framework to define a mapping to records. > > The MXBean framework already defines a mapping of `CompositeType` to plain java objects. Records are a natural representation of CompositeTypes. A record can be easily reconstructed from a `CompositeData` through the record canonical constructor. A clear advantage of records over plain java objects is that the canonical constructor will not need to be annotated in order to map composite data property names to constructor parameter names. > > With this RFE, here is an example comparing coding a composite type `NamedNumber` that consists of an `int` and a `String`, using records and using a plain java class. In both case, the `CompositeType` looks like this: > > > CompositeType( > "NamedNumber", // typeName > "NamedNumber", // description > new String[] {"number", "name"}, // itemNames > new String[] {"number", "name"}, // itemDescriptions > new OpenType[] {SimpleType.INTEGER, > SimpleType.STRING} // itemTypes > ); > > > The plain Java class needs a public constructor annotated with `@ConstructorParameters` annotation: > > > public class NamedNumber { > public int getNumber() {return number;} > public String getName() {return name;} > @ConstructorParameters({"number", "name"}) > public NamedNumber(int number, String name) { > this.number = number; > this.name = name; > } > private final int number; > private final String name; > } > > > And the equivalent with a record class: > > > public record NamedNumber(int number, String name) {} This pull request has now been integrated. Changeset: d84a7e55 Author: Daniel Fuchs URL: https://git.openjdk.java.net/jdk/commit/d84a7e55 Stats: 872 lines in 3 files changed: 838 ins; 12 del; 22 mod 8264124: Update MXBean specification and implementation to extend mapping of CompositeType to records Reviewed-by: mchung, chegar, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/3201 From dcubed at openjdk.java.net Mon Apr 12 21:02:49 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 12 Apr 2021 21:02:49 GMT Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 10:53:46 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge branch 'master' into SuspendInHandshake > - Review fixes 2 > - White space fixes > - Merge branch 'master' into SuspendInHandshake > - Review fixes > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) I only found nits so still looks good to me. src/hotspot/share/runtime/handshake.cpp line 466: > 464: // The exception to this rule is the asynchronous suspension handshake. > 465: // It by-passes the NSV by manully doing the transition. > 466: // Since we can have safepoints while suspeneded we need to release the tty locker if it is held. nit typo: s/manully/manually/ nit typo: s/suspeneded/suspended/ src/hotspot/share/runtime/handshake.hpp line 133: > 131: // check the _lock, if held go to slow path. > 132: // Since the handshakee is unsafe if _lock gets lock after this check > 133: // we know another threads cannot process any handshakes. nit typo: s/_lock gets lock/_lock gets locked/ nit typo: s/threads/thread/ src/hotspot/share/runtime/thread.cpp line 1448: > 1446: // The careful dance between thread suspension and exit is handled here. > 1447: // Since we are in vm and suspension is done with handshakes, > 1448: // we can just put in the exiting state and it will be corrcetly handled. Perhaps: s/Since we are in vm and/Since we are in thread_in_vm state and/ nit typo: s/corrcetly/correctly/ ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From cjplummer at openjdk.java.net Mon Apr 12 21:06:08 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 12 Apr 2021 21:06:08 GMT Subject: RFR: 8265028: JDWP debug agent thread lookup can be made faster Message-ID: Improved thread lookup speeds in the debug agent, especially when there is a large number of threads. See CR for details. ------------- Commit messages: - Faster debug agent thread lookups Changes: https://git.openjdk.java.net/jdk/pull/3444/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3444&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265028 Stats: 30 lines in 1 file changed: 8 ins; 10 del; 12 mod Patch: https://git.openjdk.java.net/jdk/pull/3444.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3444/head:pull/3444 PR: https://git.openjdk.java.net/jdk/pull/3444 From dholmes at openjdk.java.net Tue Apr 13 02:28:18 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 13 Apr 2021 02:28:18 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 07:20:15 GMT, Robbin Ehn wrote: >> src/hotspot/share/prims/jvmtiRawMonitor.cpp line 428: >> >>> 426: jt->set_thread_state_fence(_thread_in_native_trans); >>> 427: SafepointMechanism::process_if_requested(jt); >>> 428: if (jt->is_interrupted(true)) { >> >> A thread must be _thread_in_vm to safely query is_interrupted() as it accesses the threadObj. > > Any unsafe (not native/blocked) is fine, therefore I never completed the transition. > I set the state to _thread_in_vm before. Sorry I was misremembering the restriction, please remove the transition to _thread_in_vm again (it messes up the comments at lines 436 and 437.) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dholmes at openjdk.java.net Tue Apr 13 02:45:16 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 13 Apr 2021 02:45:16 GMT Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 10:53:46 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge branch 'master' into SuspendInHandshake > - Review fixes 2 > - White space fixes > - Merge branch 'master' into SuspendInHandshake > - Review fixes > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) src/hotspot/share/runtime/objectMonitor.cpp line 447: > 445: // Completed the tranisition. > 446: SafepointMechanism::process_if_requested(current); > 447: current->set_thread_state(_thread_in_vm); I feel very uncomfortable that we remain _thread_blocked_trans across such a lengthy chunk of code - particularly the call to exit()! This is an abuse of the trans states which are only supposed to exist and be used to ensure the correctness of the Dekker-duality when setting and reading the thread state. And I still would prefer to see these state changes and related safepoint-mechanism checks encapsulated somehow. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dholmes at openjdk.java.net Tue Apr 13 02:50:18 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 13 Apr 2021 02:50:18 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Fri, 9 Apr 2021 15:45:50 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/thread.cpp line 455: > >> 453: ParkEvent::Release(_ParkEvent); >> 454: // Can be racingly loaded in signal handler via has_terminated() >> 455: Atomic::store(&_ParkEvent, (ParkEvent*)NULL); > > Nice solution for that odd signal handler issue. Yes nice simple fix, but note that the comments at lines 451 and 452 are wrong - we can't ever encounter a null _ParkEvent, and we are not nulling for good hygiene any more. I suggest the comment at line 454 be changed to read: // Set to NULL as a termination indicator for has_terminated(). ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dholmes at openjdk.java.net Tue Apr 13 02:54:18 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 13 Apr 2021 02:54:18 GMT Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 10:53:46 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge branch 'master' into SuspendInHandshake > - Review fixes 2 > - White space fixes > - Merge branch 'master' into SuspendInHandshake > - Review fixes > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - 8257831: Suspend with handshake (review baseline) src/hotspot/share/runtime/thread.hpp line 746: > 744: > 745: // _ParkEvent is cleared > 746: bool has_terminated() { return Atomic::load(&_ParkEvent) == NULL; }; Please change the comment to: // Termination indicator used by the signal handler. _ParkEvent is just a convenient field // we can NULL out after setting the JavaThread termination state (which can't itself be read // from the signal handler if a signal hits during the Thread destructor. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From ysuenaga at openjdk.java.net Tue Apr 13 05:01:33 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 13 Apr 2021 05:01:33 GMT Subject: RFR: 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 Message-ID: I got -1.0 from both CpuLoad and SystemCpuLoad in OperatingSystem MXBean when I run the application on Fedora 33 x64 which is installed cgroups V2. ![jconsole-cpuload](https://user-images.githubusercontent.com/7421132/114485721-6372a180-9c47-11eb-81d9-568324cba336.png) I do not run the application in the container, nor do not run with resource limitation on cgroups, so JMX should report CPU load on host value in this case. ------------- Commit messages: - 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 Changes: https://git.openjdk.java.net/jdk/pull/3447/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3447&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265104 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3447.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3447/head:pull/3447 PR: https://git.openjdk.java.net/jdk/pull/3447 From dholmes at openjdk.java.net Tue Apr 13 06:17:58 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 13 Apr 2021 06:17:58 GMT Subject: RFR: 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 In-Reply-To: References: Message-ID: <-chfpbpw3z4u7MGrFp-XW0tyC1nStcvFQv33QkXLE64=.8f358a60-a6db-4ed7-888e-4176005f1ee8@github.com> On Tue, 13 Apr 2021 02:00:24 GMT, Yasumasa Suenaga wrote: > I got -1.0 from both CpuLoad and SystemCpuLoad in OperatingSystem MXBean when I run the application on Fedora 33 x64 which is installed cgroups V2. > > ![jconsole-cpuload](https://user-images.githubusercontent.com/7421132/114485721-6372a180-9c47-11eb-81d9-568324cba336.png) > > I do not run the application in the container, nor do not run with resource limitation on cgroups, so JMX should report CPU load on host value in this case. Hi Yasumasa, This strikes me as the wrong fix to the problem. isCpuSetSameAsHostCpuSet is only intended to be used as a simple optimization when the configured cpuset happens to match the hosts. What you are looking for is a fix to the problem when there is no cpuset set at all. It strikes me that in getCpuLoad() if there are no quotas and no effective-cpu-set and no cpusets.cpus value, then it should fallback to using the host values rather than returning -1. That said, the problem may also be that we have a containerMetrics object when no container is actually active! Perhaps that is the true bug here? Thanks, David David ------------- PR: https://git.openjdk.java.net/jdk/pull/3447 From ysuenaga at openjdk.java.net Tue Apr 13 06:55:20 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 13 Apr 2021 06:55:20 GMT Subject: RFR: 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 [v2] In-Reply-To: References: Message-ID: > I got -1.0 from both CpuLoad and SystemCpuLoad in OperatingSystem MXBean when I run the application on Fedora 33 x64 which is installed cgroups V2. > > ![jconsole-cpuload](https://user-images.githubusercontent.com/7421132/114485721-6372a180-9c47-11eb-81d9-568324cba336.png) > > I do not run the application in the container, nor do not run with resource limitation on cgroups, so JMX should report CPU load on host value in this case. Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Return host CPU load directly from getCpuLoad() ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3447/files - new: https://git.openjdk.java.net/jdk/pull/3447/files/f01564f3..8123fb36 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3447&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3447&range=00-01 Stats: 7 lines in 1 file changed: 4 ins; 1 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3447.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3447/head:pull/3447 PR: https://git.openjdk.java.net/jdk/pull/3447 From rehn at openjdk.java.net Tue Apr 13 06:58:12 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 06:58:12 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <8XHYnrIhSOcB6qEpq4-1FlTw8LvfPTmIG3LyIs3BC8E=.301d2dff-0341-44e3-b066-e80b3945a9a6@github.com> Message-ID: On Thu, 8 Apr 2021 08:12:11 GMT, Serguei Spitsyn wrote: > The functions JvmtiSuspendControl::suspend and JvmtiSuspendControl::resume are not really needed anymore. You probably decided to keep them around to simplify merges a little bit. Is that right? There is also the debug printing method, which I did not know were to place. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From ysuenaga at openjdk.java.net Tue Apr 13 07:03:04 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 13 Apr 2021 07:03:04 GMT Subject: RFR: 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 [v2] In-Reply-To: <-chfpbpw3z4u7MGrFp-XW0tyC1nStcvFQv33QkXLE64=.8f358a60-a6db-4ed7-888e-4176005f1ee8@github.com> References: <-chfpbpw3z4u7MGrFp-XW0tyC1nStcvFQv33QkXLE64=.8f358a60-a6db-4ed7-888e-4176005f1ee8@github.com> Message-ID: On Tue, 13 Apr 2021 06:14:42 GMT, David Holmes wrote: > This strikes me as the wrong fix to the problem. isCpuSetSameAsHostCpuSet is only intended to be used as a simple optimization when the configured cpuset happens to match the hosts. What you are looking for is a fix to the problem when there is no cpuset set at all. I pushed new commit to move the fix into `getCpuLoad()`. > It strikes me that in getCpuLoad() if there are no quotas and no effective-cpu-set and no cpusets.cpus value, then it should fallback to using the host values rather than returning -1. That said, the problem may also be that we have a containerMetrics object when no container is actually active! Perhaps that is the true bug here? I think this problem is similar to [JDK-8264482](https://bugs.openjdk.java.net/browse/JDK-8264482) (PR #3280 ). Currently both HotSpot and MBean check whether cgroups exists, but the the host (not into the container) might have it. We've discussed about it in #3280 , then I think it is difficult to modify to check whether the VM is in the container. Thus I think it is reasonable to just return host CPU load value at here. ------------- PR: https://git.openjdk.java.net/jdk/pull/3447 From rehn at openjdk.java.net Tue Apr 13 07:15:19 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 07:15:19 GMT Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 20:51:17 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes 2 >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/handshake.cpp line 466: > >> 464: // The exception to this rule is the asynchronous suspension handshake. >> 465: // It by-passes the NSV by manully doing the transition. >> 466: // Since we can have safepoints while suspeneded we need to release the tty locker if it is held. > > nit typo: s/manully/manually/ > nit typo: s/suspeneded/suspended/ Fixed > src/hotspot/share/runtime/handshake.hpp line 133: > >> 131: // check the _lock, if held go to slow path. >> 132: // Since the handshakee is unsafe if _lock gets lock after this check >> 133: // we know another threads cannot process any handshakes. > > nit typo: s/_lock gets lock/_lock gets locked/ > nit typo: s/threads/thread/ Fixed > src/hotspot/share/runtime/thread.cpp line 1448: > >> 1446: // The careful dance between thread suspension and exit is handled here. >> 1447: // Since we are in vm and suspension is done with handshakes, >> 1448: // we can just put in the exiting state and it will be corrcetly handled. > > Perhaps: s/Since we are in vm and/Since we are in thread_in_vm state and/ > nit typo: s/corrcetly/correctly/ Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 07:26:21 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 07:26:21 GMT Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 02:41:41 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes 2 >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/objectMonitor.cpp line 447: > >> 445: // Completed the tranisition. >> 446: SafepointMechanism::process_if_requested(current); >> 447: current->set_thread_state(_thread_in_vm); > > I feel very uncomfortable that we remain _thread_blocked_trans across such a lengthy chunk of code - particularly the call to exit()! This is an abuse of the trans states which are only supposed to exist and be used to ensure the correctness of the Dekker-duality when setting and reading the thread state. > > And I still would prefer to see these state changes and related safepoint-mechanism checks encapsulated somehow. Yes we should figure out something here. Note that we use to call exit() while blocked. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 07:36:20 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 07:36:20 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <1Tceht1hkeeh3E5H-JakAVgWPVhEehh2aORkLP6H0Z8=.940eeba1-3691-42e3-b4d9-1527ffb28dfd@github.com> On Tue, 13 Apr 2021 02:24:45 GMT, David Holmes wrote: >> Any unsafe (not native/blocked) is fine, therefore I never completed the transition. >> I set the state to _thread_in_vm before. > > Sorry I was misremembering the restriction, please remove the transition to _thread_in_vm again (it messes up the comments at lines 436 and 437.) Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 07:36:21 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 07:36:21 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Tue, 13 Apr 2021 02:46:46 GMT, David Holmes wrote: >> src/hotspot/share/runtime/thread.cpp line 455: >> >>> 453: ParkEvent::Release(_ParkEvent); >>> 454: // Can be racingly loaded in signal handler via has_terminated() >>> 455: Atomic::store(&_ParkEvent, (ParkEvent*)NULL); >> >> Nice solution for that odd signal handler issue. > > Yes nice simple fix, but note that the comments at lines 451 and 452 are wrong - we can't ever encounter a null _ParkEvent, and we are not nulling for good hygiene any more. I suggest the comment at line 454 be changed to read: > // Set to NULL as a termination indicator for has_terminated(). Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 07:36:23 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 07:36:23 GMT Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 02:51:32 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes 2 >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/thread.hpp line 746: > >> 744: >> 745: // _ParkEvent is cleared >> 746: bool has_terminated() { return Atomic::load(&_ParkEvent) == NULL; }; > > Please change the comment to: > > // Termination indicator used by the signal handler. _ParkEvent is just a convenient field > // we can NULL out after setting the JavaThread termination state (which can't itself be read > // from the signal handler if a signal hits during the Thread destructor. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From david.holmes at oracle.com Tue Apr 13 07:42:51 2021 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Apr 2021 17:42:51 +1000 Subject: RFR: 8257831: Suspend with handshakes [v6] In-Reply-To: References: Message-ID: On 13/04/2021 5:26 pm, Robbin Ehn wrote: > On Tue, 13 Apr 2021 02:41:41 GMT, David Holmes wrote: > >>> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: >>> >>> - Merge branch 'master' into SuspendInHandshake >>> - Review fixes 2 >>> - White space fixes >>> - Merge branch 'master' into SuspendInHandshake >>> - Review fixes >>> - Merge branch 'master' into SuspendInHandshake >>> - Merge branch 'master' into SuspendInHandshake >>> - 8257831: Suspend with handshake (review baseline) >> >> src/hotspot/share/runtime/objectMonitor.cpp line 447: >> >>> 445: // Completed the tranisition. >>> 446: SafepointMechanism::process_if_requested(current); >>> 447: current->set_thread_state(_thread_in_vm); >> >> I feel very uncomfortable that we remain _thread_blocked_trans across such a lengthy chunk of code - particularly the call to exit()! This is an abuse of the trans states which are only supposed to exist and be used to ensure the correctness of the Dekker-duality when setting and reading the thread state. >> >> And I still would prefer to see these state changes and related safepoint-mechanism checks encapsulated somehow. > > Yes we should figure out something here. Could be a neat use of a lambda "on_suspension_do" ... > Note that we use to call exit() while blocked. Yes, it is the use of trans that concerns me here as very little code expects to encounter a trans-state. Thanks, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From rehn at openjdk.java.net Tue Apr 13 07:56:21 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 07:56:21 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 11:57:36 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 11:57:36 GMT Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: References: Message-ID: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request incrementally with two additional commits since the last revision: - Obsolete unused flags - Review fixes 3 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3191/files - new: https://git.openjdk.java.net/jdk/pull/3191/files/6bd645c5..5034e8a5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=05-06 Stats: 50 lines in 9 files changed: 6 ins; 28 del; 16 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 11:57:36 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 11:57:36 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 11:58:19 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 11:58:19 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 12:07:15 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 12:07:15 GMT Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> References: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> Message-ID: On Tue, 13 Apr 2021 11:57:36 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with two additional commits since the last revision: > > - Obsolete unused flags > - Review fixes 3 Update from reviews and pushed flag obsoleting. CSR: https://bugs.openjdk.java.net/browse/JDK-8265124 Running test :) @dholmes-ora the changes to set_terminated is not exactly rollback. I notice the enum was private but set_terminated was public, so I changed the enum to public and only added set_terminated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 12:07:14 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 12:07:14 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From sgehwolf at openjdk.java.net Tue Apr 13 14:27:00 2021 From: sgehwolf at openjdk.java.net (Severin Gehwolf) Date: Tue, 13 Apr 2021 14:27:00 GMT Subject: RFR: 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 [v2] In-Reply-To: References: Message-ID: <_jJIKCWXPW0oO7y40HkRdDtbs-0OABS9rHAcf_M3qbo=.541bb951-9c7a-4f8d-80b4-718643a4fa4b@github.com> On Tue, 13 Apr 2021 06:55:20 GMT, Yasumasa Suenaga wrote: >> I got -1.0 from both CpuLoad and SystemCpuLoad in OperatingSystem MXBean when I run the application on Fedora 33 x64 which is installed cgroups V2. >> >> ![jconsole-cpuload](https://user-images.githubusercontent.com/7421132/114485721-6372a180-9c47-11eb-81d9-568324cba336.png) >> >> I do not run the application in the container, nor do not run with resource limitation on cgroups, so JMX should report CPU load on host value in this case. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Return host CPU load directly from getCpuLoad() This looks reasonable to me. ------------- Marked as reviewed by sgehwolf (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3447 From rehn at openjdk.java.net Tue Apr 13 16:51:23 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 16:51:23 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Tue Apr 13 16:59:16 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 13 Apr 2021 16:59:16 GMT Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> References: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> Message-ID: On Tue, 13 Apr 2021 11:57:36 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with two additional commits since the last revision: > > - Obsolete unused flags > - Review fixes 3 Still thumbs up. I only looked at the v06 incremental webrev. src/hotspot/share/runtime/arguments.cpp line 525: > 523: { "FlightRecorder", JDK_Version::jdk(13), JDK_Version::undefined(), JDK_Version::undefined() }, > 524: { "SuspendRetryCount", JDK_Version::jdk(16), JDK_Version::jdk(17), JDK_Version::jdk(18) }, > 525: { "SuspendRetryDelay", JDK_Version::jdk(16), JDK_Version::jdk(17), JDK_Version::jdk(18) }, I think both 'jdk(16)' values here need to be 'jdk(17)' since we didn't deprecate these option in JDK16. Or they might need to be 'JDK_Version::undefined()' since we didn't deprecate these options before obsoleting them. @dholmes-ora will know for sure. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 16:59:16 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 16:59:16 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 13 17:00:19 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 17:00:19 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From david.holmes at oracle.com Tue Apr 13 21:52:28 2021 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Apr 2021 07:52:28 +1000 Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: References: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> Message-ID: On 14/04/2021 2:59 am, Daniel D.Daugherty wrote: > On Tue, 13 Apr 2021 11:57:36 GMT, Robbin Ehn wrote: > >> Robbin Ehn has updated the pull request incrementally with two additional commits since the last revision: >> >> - Obsolete unused flags >> - Review fixes 3 > > Still thumbs up. > I only looked at the v06 incremental webrev. > > src/hotspot/share/runtime/arguments.cpp line 525: > >> 523: { "FlightRecorder", JDK_Version::jdk(13), JDK_Version::undefined(), JDK_Version::undefined() }, >> 524: { "SuspendRetryCount", JDK_Version::jdk(16), JDK_Version::jdk(17), JDK_Version::jdk(18) }, >> 525: { "SuspendRetryDelay", JDK_Version::jdk(16), JDK_Version::jdk(17), JDK_Version::jdk(18) }, > > I think both 'jdk(16)' values here need to be 'jdk(17)' since > we didn't deprecate these option in JDK16. Or they might > need to be 'JDK_Version::undefined()' since we didn't > deprecate these options before obsoleting them. > @dholmes-ora will know for sure. It should be set to undefined() as they were never deprecated. Thanks, David > ------------- > > Marked as reviewed by dcubed (Reviewer). > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From rehn at openjdk.java.net Tue Apr 13 21:55:25 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 13 Apr 2021 21:55:25 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From minqi at openjdk.java.net Tue Apr 13 23:04:24 2021 From: minqi at openjdk.java.net (Yumin Qi) Date: Tue, 13 Apr 2021 23:04:24 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v11] In-Reply-To: References: Message-ID: > Hi, Please review > > Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with > `java -XX:DumpLoadedClassList= .... ` > to collect shareable class names and saved in file `` , then > `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` > With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. > The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. > New added jcmd option: > `jcmd VM.cds static_dump ` > or > `jcmd VM.cds dynamic_dump ` > To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. > The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. > > Tests: tier1,tier2,tier3,tier4 > > Thanks > Yumin Yumin Qi has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: - Fix too much verbose output, make call to stopApp after test - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Restructed code and rename LingeredTestApp.java to JCmdTestLingeredApp.java - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Fix revert unintentionally comment, merge master. - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 - Remove CDS.getVMArguments, changed to use VM.getRuntimeVMArguments. Removed unused function from ClassLoader. Improved InstanceKlass::is_shareable() and related test. Added more test scenarios. - Remove redundant check for if a class is shareable - ... and 6 more: https://git.openjdk.java.net/jdk/compare/008fc75a...88c0f7d1 ------------- Changes: https://git.openjdk.java.net/jdk/pull/2737/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2737&range=10 Stats: 860 lines in 23 files changed: 786 ins; 58 del; 16 mod Patch: https://git.openjdk.java.net/jdk/pull/2737.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2737/head:pull/2737 PR: https://git.openjdk.java.net/jdk/pull/2737 From ysuenaga at openjdk.java.net Tue Apr 13 23:36:58 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 13 Apr 2021 23:36:58 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 00:12:11 GMT, Yasumasa Suenaga wrote: >> `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. >> >> This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Rename --disableregistry to --disable-registry Ping: Could you review this change? I need one more reviewer to push. CSR has been approved. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From dcubed at openjdk.java.net Wed Apr 14 00:25:25 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 14 Apr 2021 00:25:25 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() Message-ID: The synopsis pretty much says it all. There's a more detailed history in the RFE itself. Currently running the new test thru Mach5 Tier[1-7]. I've run the test thru several 12 hour runs on my MBP13 and on my Linux-X64 server. ------------- Commit messages: - 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() Changes: https://git.openjdk.java.net/jdk/pull/3478/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3478&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265153 Stats: 503 lines in 2 files changed: 503 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3478.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3478/head:pull/3478 PR: https://git.openjdk.java.net/jdk/pull/3478 From never at openjdk.java.net Wed Apr 14 00:42:11 2021 From: never at openjdk.java.net (Tom Rodriguez) Date: Wed, 14 Apr 2021 00:42:11 GMT Subject: RFR: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods Message-ID: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods ------------- Commit messages: - 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods Changes: https://git.openjdk.java.net/jdk/pull/3481/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3481&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265180 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3481.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3481/head:pull/3481 PR: https://git.openjdk.java.net/jdk/pull/3481 From iklam at openjdk.java.net Wed Apr 14 01:45:03 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Wed, 14 Apr 2021 01:45:03 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v11] In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 23:04:24 GMT, Yumin Qi wrote: >> Hi, Please review >> >> Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with >> `java -XX:DumpLoadedClassList= .... ` >> to collect shareable class names and saved in file `` , then >> `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` >> With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. >> The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. >> New added jcmd option: >> `jcmd VM.cds static_dump ` >> or >> `jcmd VM.cds dynamic_dump ` >> To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. >> The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. >> >> Tests: tier1,tier2,tier3,tier4 >> >> Thanks >> Yumin > > Yumin Qi has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: > > - Fix too much verbose output, make call to stopApp after test > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Restructed code and rename LingeredTestApp.java to JCmdTestLingeredApp.java > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Fix revert unintentionally comment, merge master. > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Remove CDS.getVMArguments, changed to use VM.getRuntimeVMArguments. Removed unused function from ClassLoader. Improved InstanceKlass::is_shareable() and related test. Added more test scenarios. > - Remove redundant check for if a class is shareable > - ... and 6 more: https://git.openjdk.java.net/jdk/compare/008fc75a...88c0f7d1 The latest versions looks good to me. ------------- Marked as reviewed by iklam (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/2737 From dholmes at openjdk.java.net Wed Apr 14 02:13:23 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 14 Apr 2021 02:13:23 GMT Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> References: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> Message-ID: On Tue, 13 Apr 2021 11:57:36 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with two additional commits since the last revision: > > - Obsolete unused flags > - Review fixes 3 src/hotspot/share/runtime/thread.cpp line 1446: > 1444: // The careful dance between thread suspension and exit is handled here. > 1445: // Since we are in thread_in_vm state and suspension is done with handshakes, > 1446: // we can just put in the exiting state and it will be corrcetly handled. typo: corrcetly ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 02:13:21 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 02:13:21 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dholmes at openjdk.java.net Wed Apr 14 02:18:13 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 14 Apr 2021 02:18:13 GMT Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> References: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> Message-ID: On Tue, 13 Apr 2021 11:57:36 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with two additional commits since the last revision: > > - Obsolete unused flags > - Review fixes 3 Latest updates look good - thanks. David ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 02:18:13 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 02:18:13 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 02:19:16 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 02:19:16 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dholmes at openjdk.java.net Wed Apr 14 02:53:59 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 14 Apr 2021 02:53:59 GMT Subject: RFR: 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 [v2] In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 06:55:20 GMT, Yasumasa Suenaga wrote: >> I got -1.0 from both CpuLoad and SystemCpuLoad in OperatingSystem MXBean when I run the application on Fedora 33 x64 which is installed cgroups V2. >> >> ![jconsole-cpuload](https://user-images.githubusercontent.com/7421132/114485721-6372a180-9c47-11eb-81d9-568324cba336.png) >> >> I do not run the application in the container, nor do not run with resource limitation on cgroups, so JMX should report CPU load on host value in this case. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Return host CPU load directly from getCpuLoad() Thanks Yasumasa, this seems a much better approach. David ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3447 From kvn at openjdk.java.net Wed Apr 14 03:29:55 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Wed, 14 Apr 2021 03:29:55 GMT Subject: RFR: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 00:35:55 GMT, Tom Rodriguez wrote: > 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods Good. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3481 From rehn at openjdk.java.net Wed Apr 14 06:34:19 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:34:19 GMT Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: References: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> Message-ID: <1196EWVVO0GIQHFzIFEdFCDB7JhUB14K27Ny0x04LEI=.d54726da-75ef-4c67-b0bf-7ccacd879d2a@github.com> On Wed, 14 Apr 2021 02:10:18 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request incrementally with two additional commits since the last revision: >> >> - Obsolete unused flags >> - Review fixes 3 > > src/hotspot/share/runtime/thread.cpp line 1446: > >> 1444: // The careful dance between thread suspension and exit is handled here. >> 1445: // Since we are in thread_in_vm state and suspension is done with handshakes, >> 1446: // we can just put in the exiting state and it will be corrcetly handled. > > typo: corrcetly Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:34:18 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:34:18 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:42:17 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:42:17 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:42:18 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:42:18 GMT Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: References: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> Message-ID: On Tue, 13 Apr 2021 16:49:53 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request incrementally with two additional commits since the last revision: >> >> - Obsolete unused flags >> - Review fixes 3 > > src/hotspot/share/runtime/arguments.cpp line 525: > >> 523: { "FlightRecorder", JDK_Version::jdk(13), JDK_Version::undefined(), JDK_Version::undefined() }, >> 524: { "SuspendRetryCount", JDK_Version::jdk(16), JDK_Version::jdk(17), JDK_Version::jdk(18) }, >> 525: { "SuspendRetryDelay", JDK_Version::jdk(16), JDK_Version::jdk(17), JDK_Version::jdk(18) }, > > I think both 'jdk(16)' values here need to be 'jdk(17)' since > we didn't deprecate these option in JDK16. Or they might > need to be 'JDK_Version::undefined()' since we didn't > deprecate these options before obsoleting them. > @dholmes-ora will know for sure. Fixed with undefined() as David said. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:48:40 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:48:40 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: Fixed flag undef dep + spelling error ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3191/files - new: https://git.openjdk.java.net/jdk/pull/3191/files/5034e8a5..451cdd00 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=06-07 Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:48:41 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:48:41 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:49:27 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:49:27 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:50:18 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:50:18 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:54:15 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:54:15 GMT Subject: RFR: 8257831: Suspend with handshakes [v7] In-Reply-To: References: <6lQLQ0naoM1J3saaG41ONZ_Fno8aZj8VuW8761B75qM=.3c69a5b3-d40b-4e85-9f66-7dfebc848608@github.com> Message-ID: On Wed, 14 Apr 2021 02:15:07 GMT, David Holmes wrote: > Latest updates look good - thanks. > > David Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 06:54:15 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 06:54:15 GMT Subject: RFR: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:44:58 GMT, David Holmes wrote: > Could be a neat use of a lambda "on_suspension_do" ... If no one else picks that up, I'll give it go after this :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From pliden at openjdk.java.net Wed Apr 14 07:21:19 2021 From: pliden at openjdk.java.net (Per Liden) Date: Wed, 14 Apr 2021 07:21:19 GMT Subject: RFR: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles Message-ID: JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. Testing: * Tier1-7 with ZGC * Manual testing with relevant jtreg tests ------------- Commit messages: - 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles Changes: https://git.openjdk.java.net/jdk/pull/3483/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3483&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265136 Stats: 179 lines in 9 files changed: 107 ins; 25 del; 47 mod Patch: https://git.openjdk.java.net/jdk/pull/3483.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3483/head:pull/3483 PR: https://git.openjdk.java.net/jdk/pull/3483 From ysuenaga at openjdk.java.net Wed Apr 14 07:41:58 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Wed, 14 Apr 2021 07:41:58 GMT Subject: Integrated: 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 In-Reply-To: References: Message-ID: <5KgtEP_CZzJjp2Qbq1K9fhS1uQtWM0kQPHEon90QoJw=.97b4bf87-c366-4030-bb89-6ad4892ab541@github.com> On Tue, 13 Apr 2021 02:00:24 GMT, Yasumasa Suenaga wrote: > I got -1.0 from both CpuLoad and SystemCpuLoad in OperatingSystem MXBean when I run the application on Fedora 33 x64 which is installed cgroups V2. > > ![jconsole-cpuload](https://user-images.githubusercontent.com/7421132/114485721-6372a180-9c47-11eb-81d9-568324cba336.png) > > I do not run the application in the container, nor do not run with resource limitation on cgroups, so JMX should report CPU load on host value in this case. This pull request has now been integrated. Changeset: e2106d5a Author: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/e2106d5a Stats: 6 lines in 1 file changed: 4 ins; 0 del; 2 mod 8265104: CpuLoad and SystemCpuLoad in OperatingSystem MXBean returns -1.0 Reviewed-by: dholmes, sgehwolf ------------- PR: https://git.openjdk.java.net/jdk/pull/3447 From stefank at openjdk.java.net Wed Apr 14 07:54:56 2021 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Wed, 14 Apr 2021 07:54:56 GMT Subject: RFR: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 00:35:55 GMT, Tom Rodriguez wrote: > 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods src/hotspot/share/prims/jvmtiExport.cpp line 1111: > 1109: : JvmtiMethodEventMark(thread,methodHandle(thread, nm->method())) { > 1110: _code_data = nm->code_begin(); > 1111: _code_size = (jint)pointer_delta(nm->code_end(), nm->code_begin(), sizeof(char)); Could CodeBlob::code_size be used instead?: int code_size() const { return code_end() - code_begin(); } ------------- PR: https://git.openjdk.java.net/jdk/pull/3481 From lzang at openjdk.java.net Wed Apr 14 12:29:26 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 14 Apr 2021 12:29:26 GMT Subject: RFR: 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out [v7] In-Reply-To: <9Q5r2cTgc6Nx1FGbC44HAU1pJNdC9O5ifxg0Dhqh4iQ=.060314d7-6b48-435b-94a2-d458085cf48a@github.com> References: <_4pksW9FB4VgM4sosjHs7q9lbPZtyGGy_wc_G53zLVg=.918f3888-1f52-4d30-bf1c-38e344af9e20@github.com> <7p7PD78B0fKuSonRZ44rQWvBKHabsz4n9VT4WbVluFI=.e4248320-f134-4ed0-b073-a3466970932c@github.com> <69h5uaXSdgc4Fkhla6QdtoezGPyVcFMNasVI8iNxYWI=.3a7c3e6a-8dfa-4ff1-a7cf-db71d3ea7bbf@github.com> <2J1CEmVaXJuvU3fseIBYDcJCihqoeAf07bt0hgvJWWE=.259496ed-1883-4035-a4e5-e6c45f511707@github.com> <9Q5r2cTgc6Nx1FGbC44HAU1pJNdC9O5ifxg0Dhqh4iQ=.060314d7-6b48-435b-94a2-d458085cf48a@github.com> Message-ID: On Fri, 2 Apr 2021 19:43:31 GMT, Chris Plummer wrote: >> Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: >> >> - Merge branch 'master' into sf >> - rename writeThrough to unbufferedMode and code refine >> - fix typo in comments >> - Merge branch 'master' into sf >> - Revert "reduce memory consumption" >> >> This reverts commit 70e43ddd453724ce36bf729fa6489c0027957b8e. >> - reduce memory consumption >> - code refine >> - 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out > > If your rewrite has the side affect of fixing this issue, what I would suggest is a new CR and PR that just address the rewrite, especially since this PR has a lot of history that I think you are saying will pretty much become obsolete, and therefore is just noise to anyone wanting to review your changes. In the new PR you can note that the changes will also address 8262386, and you can include an `/issue 8262386` comment so it will be closed out with new PR. Hi Chris(@plummercj ), Do you think this PR could be the proper fix of 8262386? After investigation, I think the rewrite may be more complicated than I thought: - The compressed heap dump, when the heap size is not larger than `2GB(Integer.MAX_VALUE)`, could be dumped directly with the dump size calculated by DataOutputStream.size(), same as the uncompressed heap dump. In this way, segmented heap dump could be used only for large heap dump. - When heap size is larger than `2GB`, segmented heap dump must be enabled to avoid the overflow of the `size` slot in segment, and also DataOutputStream.size() could not be used because of overflow. In this case, the `unbuffered mode` (or `write through mode`) should be used to avoid caching memory for compressed dump, especially for large objects. And the benefit of this rewriting is to reduce memory consumption. E.g. compressed heap dump does not require the segmental heap dump when heap is not large, and segmented heap dump could save memory by using `unbuffered mode`. But it doesn't have stronge relationship of the fix of 8262386. IMHO, this PR could be the fix of current implementation, and the fix code will be reused in the rewriting-solution. So maybe we could fix this issue first and then tracking the rewriting with a seperate JBS issue and PR? What do you think? BRs, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/2803 From lzang at openjdk.java.net Wed Apr 14 12:35:33 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 14 Apr 2021 12:35:33 GMT Subject: RFR: 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out [v8] In-Reply-To: <_4pksW9FB4VgM4sosjHs7q9lbPZtyGGy_wc_G53zLVg=.918f3888-1f52-4d30-bf1c-38e344af9e20@github.com> References: <_4pksW9FB4VgM4sosjHs7q9lbPZtyGGy_wc_G53zLVg=.918f3888-1f52-4d30-bf1c-38e344af9e20@github.com> Message-ID: > 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision: - Merge branch 'master' - Merge branch 'master' into sf - rename writeThrough to unbufferedMode and code refine - fix typo in comments - Merge branch 'master' into sf - Revert "reduce memory consumption" This reverts commit 70e43ddd453724ce36bf729fa6489c0027957b8e. - reduce memory consumption - code refine - 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2803/files - new: https://git.openjdk.java.net/jdk/pull/2803/files/a1732514..70dbbfea Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2803&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2803&range=06-07 Stats: 78603 lines in 2884 files changed: 49814 ins; 18293 del; 10496 mod Patch: https://git.openjdk.java.net/jdk/pull/2803.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2803/head:pull/2803 PR: https://git.openjdk.java.net/jdk/pull/2803 From lzang at openjdk.java.net Wed Apr 14 12:35:27 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 14 Apr 2021 12:35:27 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: References: Message-ID: > 8252842: Extend jmap to support parallel heap dump Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 25 commits: - Merge branch 'master' into par-dump - Merge branch 'master' into par-dump - Typo fix - remove parallel option and dumpheapext command - Revert "hide the dumpheapext error message" This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. - resize large object threshold - Merge branch 'master' into par-dump - Merge branch 'master' into par-dump - code clean - fix trailing white space issue - ... and 15 more: https://git.openjdk.java.net/jdk/compare/e2106d5a...997784b1 ------------- Changes: https://git.openjdk.java.net/jdk/pull/2261/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2261&range=19 Stats: 864 lines in 6 files changed: 662 ins; 60 del; 142 mod Patch: https://git.openjdk.java.net/jdk/pull/2261.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2261/head:pull/2261 PR: https://git.openjdk.java.net/jdk/pull/2261 From lzang at openjdk.java.net Wed Apr 14 12:37:56 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 14 Apr 2021 12:37:56 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v19] In-Reply-To: References: Message-ID: <5rHhgI7eFyvod0YgTaEPNvp-in8GEhqFVcwvsNxVggM=.bc280d77-f713-4b90-ab03-b1dcb828c27f@github.com> On Mon, 29 Mar 2021 11:27:49 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 24 commits: > > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - code clean > - fix trailing white space issue > - reduce memory consumption and fix memory leak issue > - ... and 14 more: https://git.openjdk.java.net/jdk/compare/aefc1560...301cf8cd Hi Chris, I am going to close the CSR https://bugs.openjdk.java.net/browse/JDK-8260424. But it seems this PR also revised the `parallel` option's help message for `jmap -histo`, Do you think it is ok to keep the change in the PR? FYI, here is the commits https://github.com/openjdk/jdk/pull/2261/commits/6789ba291a97a01fdc70258fb25dbf0d6464dba8 Thanks, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From dcubed at openjdk.java.net Wed Apr 14 15:37:00 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 14 Apr 2021 15:37:00 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 06:48:40 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed flag undef dep + spelling error Reviewed v07 incremental. Still looks good. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 14 15:37:00 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 14 Apr 2021 15:37:00 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: <6EyFHmoROs_E0zjIIZ-ilTWB3gixzWcYZhd9f0wnz6c=.461ee272-6b16-4519-b541-b66921d9b550@github.com> On Wed, 14 Apr 2021 15:31:21 GMT, Daniel D. Daugherty wrote: > Reviewed v07 incremental. Still looks good. Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From ccheung at openjdk.java.net Wed Apr 14 16:10:53 2021 From: ccheung at openjdk.java.net (Calvin Cheung) Date: Wed, 14 Apr 2021 16:10:53 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v11] In-Reply-To: References: Message-ID: <2MZBR7N1LLVc5ZpiuP9b6IBzhekgk4gCwz2EkY6xv7k=.60f0234f-61fb-4f29-b68e-08f0a92ca3de@github.com> On Tue, 13 Apr 2021 23:04:24 GMT, Yumin Qi wrote: >> Hi, Please review >> >> Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with >> `java -XX:DumpLoadedClassList= .... ` >> to collect shareable class names and saved in file `` , then >> `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` >> With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. >> The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. >> New added jcmd option: >> `jcmd VM.cds static_dump ` >> or >> `jcmd VM.cds dynamic_dump ` >> To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. >> The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. >> >> Tests: tier1,tier2,tier3,tier4 >> >> Thanks >> Yumin > > Yumin Qi has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: > > - Fix too much verbose output, make call to stopApp after test > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Restructed code and rename LingeredTestApp.java to JCmdTestLingeredApp.java > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Fix revert unintentionally comment, merge master. > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Remove CDS.getVMArguments, changed to use VM.getRuntimeVMArguments. Removed unused function from ClassLoader. Improved InstanceKlass::is_shareable() and related test. Added more test scenarios. > - Remove redundant check for if a class is shareable > - ... and 6 more: https://git.openjdk.java.net/jdk/compare/008fc75a...88c0f7d1 src/hotspot/share/memory/dynamicArchive.cpp line 348: > 346: set_has_been_dumped_once(); > 347: } > 348: assert(ArchiveClassesAtExit == nullptr, "already checked in arguments.cpp?"); Same assert has been done at line 338. test/hotspot/jtreg/runtime/cds/appcds/DumpClassList.java line 89: > 87: output.shouldContain("skip writing class java/lang/NewClass"); > 88: // but classes on -Xbootclasspath/a should not be skipped > 89: output.shouldNotContain("skip writing class boot/append/Foo"); Why was the above checks removed? ------------- PR: https://git.openjdk.java.net/jdk/pull/2737 From ccheung at openjdk.java.net Wed Apr 14 16:16:45 2021 From: ccheung at openjdk.java.net (Calvin Cheung) Date: Wed, 14 Apr 2021 16:16:45 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v11] In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 23:04:24 GMT, Yumin Qi wrote: >> Hi, Please review >> >> Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with >> `java -XX:DumpLoadedClassList= .... ` >> to collect shareable class names and saved in file `` , then >> `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` >> With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. >> The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. >> New added jcmd option: >> `jcmd VM.cds static_dump ` >> or >> `jcmd VM.cds dynamic_dump ` >> To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. >> The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. >> >> Tests: tier1,tier2,tier3,tier4 >> >> Thanks >> Yumin > > Yumin Qi has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: > > - Fix too much verbose output, make call to stopApp after test > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Restructed code and rename LingeredTestApp.java to JCmdTestLingeredApp.java > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Fix revert unintentionally comment, merge master. > - Merge branch 'master' of ssh://github.com/yminqi/jdk into jdk-8259070 > - Remove CDS.getVMArguments, changed to use VM.getRuntimeVMArguments. Removed unused function from ClassLoader. Improved InstanceKlass::is_shareable() and related test. Added more test scenarios. > - Remove redundant check for if a class is shareable > - ... and 6 more: https://git.openjdk.java.net/jdk/compare/008fc75a...88c0f7d1 Looks good. Just have a few minor comments. test/hotspot/jtreg/runtime/cds/appcds/jcmd/JCmdTest.java line 86: > 84: bootJar = JarBuilder.build("boot", BOOT_CLASSES); > 85: Path testJarPath = FileSystems.getDefault().getPath(testJar); > 86: Path bootJarPath = FileSystems.getDefault().getPath(bootJar); I think the above could be replaced with: String testJarPath = TestCommon.getTestJar(?test.jar?); String bootJarPath = TestCommon.getTestJar(?boot.jar?); Most of the CDS tests use the `getTestJar` method. You can leave it as is if you like. test/hotspot/jtreg/runtime/cds/appcds/jcmd/JCmdTest.java line 149: > 147: args.add("-Xlog:class+load"); > 148: > 149: LingeredApp app = createLingeredApp(args.toArray(new String[0])); Instead of `new String[0]`, I think it is more intuitive to use `new String[args.size()]`. ------------- Marked as reviewed by ccheung (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/2737 From dcubed at openjdk.java.net Wed Apr 14 16:22:58 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 14 Apr 2021 16:22:58 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 06:48:40 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed flag undef dep + spelling error Reviewed v07 incremental. Still looks good. Update: Did a crawl thru review of v07 full from the bottom up. I'm hoping the change in review order results in less "change blindness" since I've reviewed so many times. There are a few left overs that still need to be address. There are a couple buried in "resolved comments" where it looks like the issue that I raised is not actually resolved. src/hotspot/share/runtime/handshake.hpp line 99: > 97: // but we need to check for a safepoint before. > 98: // (this is due to a suspension handshake which put the JavaThread in blocked > 99: // state so a safepoint may be in-progress) nit: s/(this/(This/ nit: s/in-progress)/in-progress.)/ src/hotspot/share/runtime/handshake.hpp line 166: > 164: // thread gets suspended again (after a resume) > 165: // and we have not yet processed it. > 166: bool _async_suspend_handshake; Does this need to be `volatile`? src/hotspot/share/runtime/handshake.hpp line 177: > 175: void set_suspended(bool to) { return Atomic::store(&_suspended, to); } > 176: bool has_async_suspend_handshake() { return _async_suspend_handshake; } > 177: void set_async_suspend_handshake(bool to) { _async_suspend_handshake = to; } Does this need to be an Atomic::store? src/hotspot/share/runtime/objectMonitor.cpp line 424: > 422: // thread that suspended us. > 423: _recursions = 0; > 424: _succ = NULL; You might want to add a comment here: // Don't need a full fence after clearing successor here because of the call to exit(). ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Wed Apr 14 16:22:58 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 14 Apr 2021 16:22:58 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 07:23:26 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/handshake.cpp line 415: >> >>> 413: // Adds are done lock free and so is arming. >>> 414: // Calling this method with lock held is considered an error. >>> 415: assert(!_lock.owned_by_self(), "Lock should not be held"); >> >> If adds are still lock-free then why was this assertion removed? > > Because we add the async handshake inside the sync handshake and thus we already hold the _lock. @robehn - You still haven't addressed this one... ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Wed Apr 14 16:22:59 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 14 Apr 2021 16:22:59 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Mon, 12 Apr 2021 07:48:10 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/objectMonitor.cpp line 970: >> >>> 968: >>> 969: current->frame_anchor()->make_walkable(current); >>> 970: OrderAccess::storestore(); >> >> Needs a comment explaining what the memory sync is for. > > Fixed I'm not seeing the comment here. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From never at openjdk.java.net Wed Apr 14 16:31:43 2021 From: never at openjdk.java.net (Tom Rodriguez) Date: Wed, 14 Apr 2021 16:31:43 GMT Subject: RFR: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 07:51:13 GMT, Stefan Karlsson wrote: >> 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods > > src/hotspot/share/prims/jvmtiExport.cpp line 1111: > >> 1109: : JvmtiMethodEventMark(thread,methodHandle(thread, nm->method())) { >> 1110: _code_data = nm->code_begin(); >> 1111: _code_size = (jint)pointer_delta(nm->code_end(), nm->code_begin(), sizeof(char)); > > Could CodeBlob::code_size be used instead?: > > int code_size() const { return code_end() - code_begin(); } It certainly could. I was just following the pattern used by JvmtiExport::post_dynamic_code_generated but maybe it's better to use the real nmethod API. I'll change it. ------------- PR: https://git.openjdk.java.net/jdk/pull/3481 From dcubed at openjdk.java.net Wed Apr 14 16:36:56 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 14 Apr 2021 16:36:56 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 06:48:40 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed flag undef dep + spelling error I dug out the comments that I made in "resolved" sections where the original comment wasn't actually resolved. I posted new comments so let's see if those show up in the emails... src/hotspot/share/runtime/handshake.cpp line 415: > 413: // Adds are done lock free and so is arming. > 414: // Calling this method with lock held is considered an error. > 415: assert(!_lock.owned_by_self(), "Lock should not be held"); I originally added this comment inside the "resolved" conversation and that kept this comment from showing up. Let's try it as a new comment. Doesn't that mean the comment on L415 needs updating? Should it be something like: // Synchronous adds are done lock free and so is arming, but some // asynchronous adds are done when we already hold the lock. I'm not sure about the "some asynchronous adds" part... @dcubed-ojdk src/hotspot/share/runtime/objectMonitor.cpp line 972: > 970: > 971: current->frame_anchor()->make_walkable(current); > 972: OrderAccess::storestore(); Repeating this comment since my original is marked resolved, but I'm not seeing a new comment here. I originally pointed this out in the resolved conversation, but that doesn't make the comment show up in the review emails. Needs a comment explaining what the memory sync is for. src/hotspot/share/runtime/objectMonitor.cpp line 1559: > 1557: if (_succ == current) { > 1558: _succ = NULL; > 1559: OrderAccess::fence(); My original comment is marked "fixed", but I don't see the new comment: Please add a comment to this line: OrderAccess::fence(); // always do a full fence when successor is cleared ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From never at openjdk.java.net Wed Apr 14 16:41:01 2021 From: never at openjdk.java.net (Tom Rodriguez) Date: Wed, 14 Apr 2021 16:41:01 GMT Subject: RFR: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods [v2] In-Reply-To: References: Message-ID: > 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods Tom Rodriguez has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3481/files - new: https://git.openjdk.java.net/jdk/pull/3481/files/ef1e6929..7dbe88f9 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3481&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3481&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3481.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3481/head:pull/3481 PR: https://git.openjdk.java.net/jdk/pull/3481 From kariyam at oss.nttdata.com Wed Apr 14 16:59:29 2021 From: kariyam at oss.nttdata.com (Mitsuru Kariya) Date: Thu, 15 Apr 2021 01:59:29 +0900 Subject: *Address.hashCode ignore the upper 32 bits of a long value In-Reply-To: References: <9611e414ab996d463c07e17867252980@oss.nttdata.com> Message-ID: <6d4490ea51c59354acb6d518a232e8be@oss.nttdata.com> Thank you for your consideration. I would like to send a pull request soon. Thanks On 2021-04-06 06:17, Chris Plummer wrote: > [moving to serviceability-dev] > > Hi, > > I'm not sure if Address hashcodes are even used by SA, and if they > are, I doubt this slightly improved hash would make a noticeable > difference. However, if you want to pursue this change just to get > started with making OpenJDK contributions, I'm ok with that. I've > filed https://bugs.openjdk.java.net/browse/JDK-8264734 > > thanks, > > Chris > > On 4/4/21 1:31 AM, kariyam wrote: >> Hi, >> >> I found that sun.jvm.hotspot.debugger.*.*Address.hashCode ignore the >> upper 32 bits of a long value. >> >> e.g. >> https://github.com/openjdk/jdk/blob/3789983e89c9de252ef546a1b98a732a7d066650/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxAddress.java#L56 >> >> I don't think the upper 32 bits of a long value should be ignored. >> IMHO, the Long.hashCode static method is suitable for such cases. >> >> If it's worth making this change, could anyone submit this issue to >> JBS? >> >> I'm ready to submit a pull request, but I don't have an Author role. >> >> Please let me know if there is a better place to do so. >> >> Thanks From minqi at openjdk.java.net Wed Apr 14 19:08:25 2021 From: minqi at openjdk.java.net (Yumin Qi) Date: Wed, 14 Apr 2021 19:08:25 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v12] In-Reply-To: References: Message-ID: <3a_niaHCAlxycKq7lSyi6QWCntKhwGQXqjVN1d1D2SQ=.a85bfd81-7b5c-4416-b73a-d817e830aefd@github.com> > Hi, Please review > > Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with > `java -XX:DumpLoadedClassList= .... ` > to collect shareable class names and saved in file `` , then > `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` > With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. > The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. > New added jcmd option: > `jcmd VM.cds static_dump ` > or > `jcmd VM.cds dynamic_dump ` > To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. > The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. > > Tests: tier1,tier2,tier3,tier4 > > Thanks > Yumin Yumin Qi has updated the pull request incrementally with one additional commit since the last revision: Removed extra assert and fixed extra Path usage in test ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2737/files - new: https://git.openjdk.java.net/jdk/pull/2737/files/88c0f7d1..042ec8f3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2737&range=11 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2737&range=10-11 Stats: 5 lines in 2 files changed: 0 ins; 3 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/2737.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2737/head:pull/2737 PR: https://git.openjdk.java.net/jdk/pull/2737 From sspitsyn at openjdk.java.net Wed Apr 14 23:09:53 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 14 Apr 2021 23:09:53 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 06:48:40 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed flag undef dep + spelling error src/hotspot/share/prims/jvmtiEnv.cpp line 952: > 950: if (!JvmtiSuspendControl::suspend(java_thread)) { > 951: // Either the thread is already suspended or > 952: // the thread was in the process of exiting: Nit: replace this line with: "// it was in process of exiting." src/hotspot/share/prims/jvmtiEnv.cpp line 993: > 991: if (!JvmtiSuspendControl::suspend(java_thread)) { > 992: // Either the thread is already suspended or > 993: // the thread was in the process of exiting: Nit: replace this line with: "// it was in process of exiting." src/hotspot/share/prims/jvmtiEnv.cpp line 998: > 996: continue; > 997: } > 998: results[i] = JVMTI_ERROR_THREAD_SUSPENDED; Nit: Remove one extra space after '='. src/hotspot/share/prims/jvmtiEnv.cpp line 1006: > 1004: if (!JvmtiSuspendControl::suspend(current)) { > 1005: // Either the thread is already suspended or > 1006: // the thread was in the process of exiting: Nit: replace this line with: "// it was in process of exiting." src/hotspot/share/prims/jvmtiEnv.cpp line 1010: > 1008: results[self_index] = JVMTI_ERROR_THREAD_NOT_ALIVE; > 1009: } else { > 1010: results[self_index] = JVMTI_ERROR_THREAD_SUSPENDED; Nit: Remove one extra space after '='. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From sspitsyn at openjdk.java.net Wed Apr 14 23:14:52 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 14 Apr 2021 23:14:52 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: <7lr-BFiqmT_Zh6rPM8iawfNEbzkbp16Yq8kW_lx2vvs=.5f41e815-762e-4c76-a758-eca5fbe40f52@github.com> On Wed, 14 Apr 2021 06:48:40 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed flag undef dep + spelling error It looks good. I do not see any serviceability related related issues but posted some nits. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From sspitsyn at openjdk.java.net Wed Apr 14 23:35:32 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 14 Apr 2021 23:35:32 GMT Subject: RFR: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods [v2] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 16:41:01 GMT, Tom Rodriguez wrote: >> 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods > > Tom Rodriguez has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods Hi Thomas, The fix looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3481 From sspitsyn at openjdk.java.net Wed Apr 14 23:48:33 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 14 Apr 2021 23:48:33 GMT Subject: RFR: 8265028: JDWP debug agent thread lookup can be made faster In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 20:29:56 GMT, Chris Plummer wrote: > Improved thread lookup speeds in the debug agent, especially when there is a large number of threads. See CR for details. Looks good. ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3444 From david.holmes at oracle.com Wed Apr 14 23:53:58 2021 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Apr 2021 09:53:58 +1000 Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: On 15/04/2021 2:22 am, Daniel D.Daugherty wrote: > On Wed, 7 Apr 2021 07:23:26 GMT, Robbin Ehn wrote: > >>> src/hotspot/share/runtime/handshake.cpp line 415: >>> >>>> 413: // Adds are done lock free and so is arming. >>>> 414: // Calling this method with lock held is considered an error. >>>> 415: assert(!_lock.owned_by_self(), "Lock should not be held"); >>> >>> If adds are still lock-free then why was this assertion removed? >> >> Because we add the async handshake inside the sync handshake and thus we already hold the _lock. > > @robehn - You still haven't addressed this one... Assuming your comment is attributed to the correct comments, this is addressed. The assert has to be removed because we can own the lock. Cheers, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3191 > From mli at openjdk.java.net Thu Apr 15 03:14:41 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 15 Apr 2021 03:14:41 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v12] In-Reply-To: <3a_niaHCAlxycKq7lSyi6QWCntKhwGQXqjVN1d1D2SQ=.a85bfd81-7b5c-4416-b73a-d817e830aefd@github.com> References: <3a_niaHCAlxycKq7lSyi6QWCntKhwGQXqjVN1d1D2SQ=.a85bfd81-7b5c-4416-b73a-d817e830aefd@github.com> Message-ID: On Wed, 14 Apr 2021 19:08:25 GMT, Yumin Qi wrote: >> Hi, Please review >> >> Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with >> `java -XX:DumpLoadedClassList= .... ` >> to collect shareable class names and saved in file `` , then >> `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` >> With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. >> The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. >> New added jcmd option: >> `jcmd VM.cds static_dump ` >> or >> `jcmd VM.cds dynamic_dump ` >> To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. >> The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. >> >> Tests: tier1,tier2,tier3,tier4 >> >> Thanks >> Yumin > > Yumin Qi has updated the pull request incrementally with one additional commit since the last revision: > > Removed extra assert and fixed extra Path usage in test Marked as reviewed by mli (Reviewer). I looked back at the history, and find out why it calls from JVM into java then into JVM again, nice solution! ------------- PR: https://git.openjdk.java.net/jdk/pull/2737 From sspitsyn at openjdk.java.net Thu Apr 15 03:17:36 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 15 Apr 2021 03:17:36 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" In-Reply-To: References: Message-ID: On Sat, 10 Apr 2021 01:10:37 GMT, Alex Menkov wrote: > The test actually failed starting from jdk13, but the error is masked by JDK-8264667 (and shell part of the test didn't verify result of the java execution) > The fix: > - updates JvmtiClassFileReconstituter to add attributes in the same order as javac does > - makes the test java instead of shell > - added workaround for JDK-8264667 > - test code is ready to ignore order of attributes test/jdk/java/lang/instrument/ATransformerManagementTestCase.java line 125: > 123: manager.addTransformer(transformer, canRetransform); > 124: // verbosePrint("Added transformer " + transformer > 125: // + " with canRetransform=" + canRetransform); I'd suggest to remove the commented out lines. ------------- PR: https://git.openjdk.java.net/jdk/pull/3426 From sspitsyn at openjdk.java.net Thu Apr 15 03:30:34 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 15 Apr 2021 03:30:34 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" In-Reply-To: References: Message-ID: On Sat, 10 Apr 2021 01:10:37 GMT, Alex Menkov wrote: > The test actually failed starting from jdk13, but the error is masked by JDK-8264667 (and shell part of the test didn't verify result of the java execution) > The fix: > - updates JvmtiClassFileReconstituter to add attributes in the same order as javac does > - makes the test java instead of shell > - added workaround for JDK-8264667 > - test code is ready to ignore order of attributes test/jdk/java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.java line 152: > 150: // directory so there is no conflict with the file > 151: // in the test classes directory. > 152: String resourceName = fTargetClassName + ".class"; This name can be defined out of methods `originalTargetClassFile` and `transformClassFile`. The method name `transformClassFile` is confusing. I'd suggest to rename it to `transformedClassFile` or `modifiedClassFile`. Also, the name `originalTargetClassFile` can be shortened to `originalClassFile`. ------------- PR: https://git.openjdk.java.net/jdk/pull/3426 From sspitsyn at openjdk.java.net Thu Apr 15 04:05:35 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 15 Apr 2021 04:05:35 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" In-Reply-To: References: Message-ID: On Sat, 10 Apr 2021 01:10:37 GMT, Alex Menkov wrote: > The test actually failed starting from jdk13, but the error is masked by JDK-8264667 (and shell part of the test didn't verify result of the java execution) > The fix: > - updates JvmtiClassFileReconstituter to add attributes in the same order as javac does > - makes the test java instead of shell > - added workaround for JDK-8264667 > - test code is ready to ignore order of attributes test/jdk/java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.java line 186: > 184: int lineNum = 0; > 185: for (String line: out1) { > 186: if (expectedDiffenent(line)) { Is it possible to use the `expectedDiffenent()` as a filter to get only needed lines from `disassembleClassFile()`? ------------- PR: https://git.openjdk.java.net/jdk/pull/3426 From sspitsyn at openjdk.java.net Thu Apr 15 04:23:41 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 15 Apr 2021 04:23:41 GMT Subject: RFR: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 07:13:50 GMT, Per Liden wrote: > JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. > > Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. > > Testing: > * Tier1-7 with ZGC > * Manual testing with relevant jtreg tests Hi Per, It looks good to me. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3483 From minqi at openjdk.java.net Thu Apr 15 05:24:41 2021 From: minqi at openjdk.java.net (Yumin Qi) Date: Thu, 15 Apr 2021 05:24:41 GMT Subject: Integrated: 8259070: Add jcmd option to dump CDS In-Reply-To: References: Message-ID: On Fri, 26 Feb 2021 00:03:40 GMT, Yumin Qi wrote: > Hi, Please review > > Added jcmd option for dumping CDS archive during application runtime. Before this change, user has to dump shared archive in two steps: first run application with > `java -XX:DumpLoadedClassList= .... ` > to collect shareable class names and saved in file `` , then > `java -Xshare:dump -XX:SharedClassListFile= -XX:SharedArchiveFile= ...` > With this change, user can use jcmd to dump CDS without going through above steps. Also user can choose a moment during the app runtime to dump an archive. > The bug is associated with the CSR: https://bugs.openjdk.java.net/browse/JDK-8259798 which has been approved. > New added jcmd option: > `jcmd VM.cds static_dump ` > or > `jcmd VM.cds dynamic_dump ` > To dump dynamic archive, requires start app with newly added flag `-XX:+RecordDynamicDumpInfo`, with this flag, some information related to dynamic dump like loader constraints will be recorded. Note the dumping process changed some object memory locations so for dumping dynamic archive, can only done once for a running app. For static dump, user can dump multiple times against same process. > The file name is optional, if the file name is not supplied, the file name will take format of `java_pid_static.jsa` or `java_pid_dynamic.jsa` for static and dynamic respectively. The `` is the application process ID. > > Tests: tier1,tier2,tier3,tier4 > > Thanks > Yumin This pull request has now been integrated. Changeset: e7cbeba8 Author: Yumin Qi URL: https://git.openjdk.java.net/jdk/commit/e7cbeba8 Stats: 857 lines in 23 files changed: 783 ins; 58 del; 16 mod 8259070: Add jcmd option to dump CDS Reviewed-by: ccheung, iklam, mli ------------- PR: https://git.openjdk.java.net/jdk/pull/2737 From minqi at openjdk.java.net Thu Apr 15 05:24:38 2021 From: minqi at openjdk.java.net (Yumin Qi) Date: Thu, 15 Apr 2021 05:24:38 GMT Subject: RFR: 8259070: Add jcmd option to dump CDS [v12] In-Reply-To: References: Message-ID: <6rB1bQQTIMyzBIPTaK-GTib06tleC6MQlmM6Xy_m8VM=.cf46161f-872a-442f-9229-a02045bc4de9@github.com> On Sat, 27 Feb 2021 18:20:52 GMT, Ioi Lam wrote: >> Hi Yumin, >> >> This is a very useful addition. >> >> My biggest concern with this patch though is the use of `os::fork_and_exec()` in regular, non-fatal situations. I had a look at that function and I think that is very unsafe. >> >> - it does not close file descriptors, so the child vm will inherit all file descriptors not opened with FD_CLOEXEC. In Runtime.exec, we do this whole dance around safely closing off all unused file descriptors, which is missing here. Note that even though we mostly open all fds with CLOEXEC, this does not matter, since user code may not do that. >> - It uses vfork "if available", which is probably always, but that may be okay since the child exec's right away. Still, vfork makes me nervous. >> - Weirdly enough, it always spawns the child program via one indirection using a shell; so there is always the shell between you and your spawned VM, and you probably wont get the return code of your child vm process back. >> >> Until now, os::fork_and_exec() was only used in fatal situations where the VM was about to die anyway. And where it did not really matter if it worked or not. >> >> If we now want to use it in regular situations, we need to give that thing an overhaul and make it a first class fork api, and also test it better that it is today. The file descriptor issue has to be addressed at the very least. >> >> I really would consider rewriting the whole thing using posix_spawn. Since JDK15 I think posix_spawn is the default for Runtime.exec, so we know it works well. I would also do this in a separate RFE. >> >> Alternatively, you could call into java and use Runtime.exec(). >> >> Further remarks inline. >> >> Thanks, Thomas > >> I really would consider rewriting the whole thing using posix_spawn. Since JDK15 I think posix_spawn is the default for Runtime.exec, so we know it works well. I would also do this in a separate RFE. >> >> Alternatively, you could call into java and use Runtime.exec(). > > I think we should call into Java and use `Runtime.exec()`. Running a subprocess is very complicated, so we shouldn't try to duplicate the code in the VM. @iklam @calvinccheung @Hamlin-Li Thanks for review! ------------- PR: https://git.openjdk.java.net/jdk/pull/2737 From rehn at openjdk.java.net Thu Apr 15 06:47:59 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 15 Apr 2021 06:47:59 GMT Subject: RFR: 8257831: Suspend with handshakes [v4] In-Reply-To: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> References: <_3OlOvuhRPByRWmHYgNn1IxifEPGZhB2AzT3_x_85bk=.96400de6-8269-4814-a452-6bf409ab9b09@github.com> Message-ID: On Fri, 9 Apr 2021 15:39:08 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - Merge branch 'master' into SuspendInHandshake >> - Merge branch 'master' into SuspendInHandshake >> - 8257831: Suspend with handshake (review baseline) > > src/hotspot/share/runtime/objectMonitor.cpp line 410: > >> 408: >> 409: current->frame_anchor()->make_walkable(current); >> 410: OrderAccess::storestore(); > > Needs a comment explaining what the memory sync is for. Missed one, fixed. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Thu Apr 15 06:47:56 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 15 Apr 2021 06:47:56 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 16:05:39 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed flag undef dep + spelling error > > src/hotspot/share/runtime/handshake.hpp line 99: > >> 97: // but we need to check for a safepoint before. >> 98: // (this is due to a suspension handshake which put the JavaThread in blocked >> 99: // state so a safepoint may be in-progress) > > nit: s/(this/(This/ > nit: s/in-progress)/in-progress.)/ Fixed > src/hotspot/share/runtime/handshake.hpp line 166: > >> 164: // thread gets suspended again (after a resume) >> 165: // and we have not yet processed it. >> 166: bool _async_suspend_handshake; > > Does this need to be `volatile`? No, _async_suspend_handshake is now only loaded/stored with mutex held. > src/hotspot/share/runtime/handshake.hpp line 177: > >> 175: void set_suspended(bool to) { return Atomic::store(&_suspended, to); } >> 176: bool has_async_suspend_handshake() { return _async_suspend_handshake; } >> 177: void set_async_suspend_handshake(bool to) { _async_suspend_handshake = to; } > > Does this need to be an Atomic::store? No, _async_suspend_handshake is now only loaded/stored with mutex held. > src/hotspot/share/runtime/objectMonitor.cpp line 424: > >> 422: // thread that suspended us. >> 423: _recursions = 0; >> 424: _succ = NULL; > > You might want to add a comment here: > // Don't need a full fence after clearing successor here because of the call to exit(). Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Thu Apr 15 07:14:09 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 15 Apr 2021 07:14:09 GMT Subject: RFR: 8257831: Suspend with handshakes [v9] In-Reply-To: References: Message-ID: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits: - Merge branch 'master' into SuspendInHandshake - Review fixes 4 - Fixed flag undef dep + spelling error - Obsolete unused flags - Review fixes 3 - Merge branch 'master' into SuspendInHandshake - Review fixes 2 - White space fixes - Merge branch 'master' into SuspendInHandshake - Review fixes - ... and 3 more: https://git.openjdk.java.net/jdk/compare/b224b566...27bf041c ------------- Changes: https://git.openjdk.java.net/jdk/pull/3191/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=08 Stats: 1351 lines in 40 files changed: 273 ins; 882 del; 196 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Thu Apr 15 07:14:11 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 15 Apr 2021 07:14:11 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: <7lr-BFiqmT_Zh6rPM8iawfNEbzkbp16Yq8kW_lx2vvs=.5f41e815-762e-4c76-a758-eca5fbe40f52@github.com> References: <7lr-BFiqmT_Zh6rPM8iawfNEbzkbp16Yq8kW_lx2vvs=.5f41e815-762e-4c76-a758-eca5fbe40f52@github.com> Message-ID: On Wed, 14 Apr 2021 23:11:56 GMT, Serguei Spitsyn wrote: > It looks good. I do not see any serviceability related related issues but posted some nits. > Thanks, > Serguei Thank you Serguei! > src/hotspot/share/prims/jvmtiEnv.cpp line 952: > >> 950: if (!JvmtiSuspendControl::suspend(java_thread)) { >> 951: // Either the thread is already suspended or >> 952: // the thread was in the process of exiting: > > Nit: replace this line with: "// it was in process of exiting." Fixed > src/hotspot/share/prims/jvmtiEnv.cpp line 993: > >> 991: if (!JvmtiSuspendControl::suspend(java_thread)) { >> 992: // Either the thread is already suspended or >> 993: // the thread was in the process of exiting: > > Nit: replace this line with: "// it was in process of exiting." Fixed > src/hotspot/share/prims/jvmtiEnv.cpp line 998: > >> 996: continue; >> 997: } >> 998: results[i] = JVMTI_ERROR_THREAD_SUSPENDED; > > Nit: Remove one extra space after '='. Fixed > src/hotspot/share/prims/jvmtiEnv.cpp line 1006: > >> 1004: if (!JvmtiSuspendControl::suspend(current)) { >> 1005: // Either the thread is already suspended or >> 1006: // the thread was in the process of exiting: > > Nit: replace this line with: "// it was in process of exiting." Fixed > src/hotspot/share/prims/jvmtiEnv.cpp line 1010: > >> 1008: results[self_index] = JVMTI_ERROR_THREAD_NOT_ALIVE; >> 1009: } else { >> 1010: results[self_index] = JVMTI_ERROR_THREAD_SUSPENDED; > > Nit: Remove one extra space after '='. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Thu Apr 15 07:14:14 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 15 Apr 2021 07:14:14 GMT Subject: RFR: 8257831: Suspend with handshakes [v9] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 16:27:10 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits: >> >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes 4 >> - Fixed flag undef dep + spelling error >> - Obsolete unused flags >> - Review fixes 3 >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes 2 >> - White space fixes >> - Merge branch 'master' into SuspendInHandshake >> - Review fixes >> - ... and 3 more: https://git.openjdk.java.net/jdk/compare/b224b566...27bf041c > > src/hotspot/share/runtime/handshake.cpp line 415: > >> 413: // Adds are done lock free and so is arming. >> 414: // Calling this method with lock held is considered an error. >> 415: assert(!_lock.owned_by_self(), "Lock should not be held"); > > I originally added this comment inside the "resolved" conversation and that kept > this comment from showing up. Let's try it as a new comment. > > Doesn't that mean the comment on L415 needs updating? > Should it be something like: > > // Synchronous adds are done lock free and so is arming, but some > // asynchronous adds are done when we already hold the lock. > > I'm not sure about the "some asynchronous adds" part... > @dcubed-ojdk The mutex is unrelated to adds/inserts. Adds/inserts to the queue can always be done regardless of which locks a thread may or may not have. The reason for not allowing inserts while holding the HandshakeState lock was to eliminate that from the table, since it have other implications. As @pchilano found we needed to reverse the order in should_process() to make it work (which I missed). So now when we have looked at the case, it is okay to do it. Meaning that you can add a handshake to yourself from another handshake regardless of it is sync or async. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Thu Apr 15 07:14:21 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 15 Apr 2021 07:14:21 GMT Subject: RFR: 8257831: Suspend with handshakes [v8] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 16:30:56 GMT, Daniel D. Daugherty wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed flag undef dep + spelling error > > src/hotspot/share/runtime/objectMonitor.cpp line 972: > >> 970: >> 971: current->frame_anchor()->make_walkable(current); >> 972: OrderAccess::storestore(); > > Repeating this comment since my original is marked resolved, but I'm not seeing > a new comment here. I originally pointed this out in the resolved conversation, > but that doesn't make the comment show up in the review emails. > > Needs a comment explaining what the memory sync is for. I seem to have fixed 2 of out 3. > src/hotspot/share/runtime/objectMonitor.cpp line 1559: > >> 1557: if (_succ == current) { >> 1558: _succ = NULL; >> 1559: OrderAccess::fence(); > > My original comment is marked "fixed", but I don't see the new comment: > > Please add a comment to this line: > OrderAccess::fence(); // always do a full fence when successor is cleared I fixed the one at line 982, it was not clear there were two from your comment :) Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From lutz.schmidt at sap.com Thu Apr 15 08:18:21 2021 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Thu, 15 Apr 2021 08:18:21 +0000 Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Message-ID: Dear Community, I would happily accept reviews for this downport change. The change creates more meaningful output. ? Merge conflicts were only "technical". No complicated semantical considerations necessary. Original bug:??????????https://bugs.openjdk.java.net/browse/JDK-8261447 Downport webrev:???????https://cr.openjdk.java.net/~lucy/webrevs/8261447.11u.01/ Tests: SAP's internal build and test farm (all OpenJDK platforms (no 32-bit), and more). Tests include JCK, jtreg (hotspot and jdk), and SAP-private tests. No issues detected that are attributable to this change. Thanks, Lutz From dcubed at openjdk.java.net Thu Apr 15 17:05:58 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 15 Apr 2021 17:05:58 GMT Subject: RFR: 8257831: Suspend with handshakes [v9] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 07:14:09 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits: > > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - Obsolete unused flags > - Review fixes 3 > - Merge branch 'master' into SuspendInHandshake > - Review fixes 2 > - White space fixes > - Merge branch 'master' into SuspendInHandshake > - Review fixes > - ... and 3 more: https://git.openjdk.java.net/jdk/compare/b224b566...27bf041c Reviewed the v08 incremental. Still thumbs up. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From hohensee at amazon.com Thu Apr 15 20:45:34 2021 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 15 Apr 2021 20:45:34 +0000 Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Message-ID: Lgtm. Thanks, Paul ?-----Original Message----- From: serviceability-dev on behalf of "Schmidt, Lutz" Date: Thursday, April 15, 2021 at 1:19 AM To: "jdk-updates-dev at openjdk.java.net" Cc: "serviceability-dev at openjdk.java.net" Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Dear Community, I would happily accept reviews for this downport change. The change creates more meaningful output. Merge conflicts were only "technical". No complicated semantical considerations necessary. Original bug: https://bugs.openjdk.java.net/browse/JDK-8261447 Downport webrev: https://cr.openjdk.java.net/~lucy/webrevs/8261447.11u.01/ Tests: SAP's internal build and test farm (all OpenJDK platforms (no 32-bit), and more). Tests include JCK, jtreg (hotspot and jdk), and SAP-private tests. No issues detected that are attributable to this change. Thanks, Lutz From dholmes at openjdk.java.net Thu Apr 15 22:27:01 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 15 Apr 2021 22:27:01 GMT Subject: RFR: 8257831: Suspend with handshakes [v9] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 07:14:09 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits: > > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - Obsolete unused flags > - Review fixes 3 > - Merge branch 'master' into SuspendInHandshake > - Review fixes 2 > - White space fixes > - Merge branch 'master' into SuspendInHandshake > - Review fixes > - ... and 3 more: https://git.openjdk.java.net/jdk/compare/b224b566...27bf041c Only nits on the incremental. Like Dan I need to re-examine the complete set of changes to see if anything else sticks out (other than the non-encapsulation of the state changes around locking/unlocking when suspended). Thanks, David src/hotspot/share/prims/jvmtiEnv.cpp line 952: > 950: if (!JvmtiSuspendControl::suspend(java_thread)) { > 951: // Either the thread is already suspended or > 952: // it was in process of exiting. Nit: the previous comment was perfectly correct. It is okay to replace the second "the thread" with "it" but it should still read "in _the_ process of exiting". src/hotspot/share/prims/jvmtiEnv.cpp line 993: > 991: if (!JvmtiSuspendControl::suspend(java_thread)) { > 992: // Either the thread is already suspended or > 993: // it was in process of exiting. Ditto - "in the process ..." src/hotspot/share/prims/jvmtiEnv.cpp line 1006: > 1004: if (!JvmtiSuspendControl::suspend(current)) { > 1005: // Either the thread is already suspended or > 1006: // it was in process of exiting. Ditto - "in the process ..." ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From never at openjdk.java.net Thu Apr 15 23:45:37 2021 From: never at openjdk.java.net (Tom Rodriguez) Date: Thu, 15 Apr 2021 23:45:37 GMT Subject: Integrated: 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods In-Reply-To: References: Message-ID: <5ueJ4HEfOPZjJdNyiEAZgT9FnHXaoGC56m5ZgLK7UU0=.9db908f8-31e4-4ee5-acfb-45ef6a6ea49a@github.com> On Wed, 14 Apr 2021 00:35:55 GMT, Tom Rodriguez wrote: > 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods This pull request has now been integrated. Changeset: 3423f3e1 Author: Tom Rodriguez URL: https://git.openjdk.java.net/jdk/commit/3423f3e1 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod 8265180: JvmtiCompiledMethodLoadEvent should include the stub section of nmethods Reviewed-by: kvn, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/3481 From vromero at openjdk.java.net Fri Apr 16 01:15:58 2021 From: vromero at openjdk.java.net (Vicente Romero) Date: Fri, 16 Apr 2021 01:15:58 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature Message-ID: Please review this PR that intents to make sealed classes a final feature in Java. This PR contains compiler and VM changes. In line with similar PRs, which has made preview features final, this one is mostly removing preview related comments from APIs plus options in test cases. Thanks ------------- Commit messages: - Merge branch 'master' into JDK-8260517 - Merge branch 'master' into JDK-8260517 - fixing failing regression tests - JVM related changes - 8260517: Compiler implementation for Sealed Classes Changes: https://git.openjdk.java.net/jdk/pull/3526/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8260517 Stats: 450 lines in 56 files changed: 48 ins; 282 del; 120 mod Patch: https://git.openjdk.java.net/jdk/pull/3526.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3526/head:pull/3526 PR: https://git.openjdk.java.net/jdk/pull/3526 From dholmes at openjdk.java.net Fri Apr 16 02:14:02 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Fri, 16 Apr 2021 02:14:02 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature in Java [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 02:11:10 GMT, Vicente Romero wrote: >> Please review this PR that intents to make sealed classes a final feature in Java. This PR contains compiler and VM changes. In line with similar PRs, which has made preview features final, this one is mostly removing preview related comments from APIs plus options in test cases. Please also review the related [CSR](https://bugs.openjdk.java.net/browse/JDK-8265090) >> >> Thanks >> >> note: this PR is related to [PR-3528](https://github.com/openjdk/jdk/pull/3528) and must be integrated after it. > > Vicente Romero has updated the pull request incrementally with one additional commit since the last revision: > > removing javax.lang.model changes Hi Vincente, Hotspot and hotspot tests all look fine. One query: why was this test removed? test/hotspot/jtreg/runtime/sealedClasses/AbstractSealedTest.java is that functionality tested elsewhere? (The other deleted test seemed obviously trivial.) Thanks, David src/hotspot/share/classfile/classFileParser.cpp line 3916: > 3914: record_attribute_start = cfs->current(); > 3915: record_attribute_length = attribute_length; > 3916: } else if (_major_version >= JAVA_17_VERSION) { Can you update the comment at L3932 to say JAVA_17_VERSION please. src/hotspot/share/classfile/classFileParser.hpp line 345: > 343: > 344: bool supports_sealed_types(); > 345: bool supports_records(); Good catch! ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3526 From vromero at openjdk.java.net Fri Apr 16 02:14:01 2021 From: vromero at openjdk.java.net (Vicente Romero) Date: Fri, 16 Apr 2021 02:14:01 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature in Java [v2] In-Reply-To: References: Message-ID: > Please review this PR that intents to make sealed classes a final feature in Java. This PR contains compiler and VM changes. In line with similar PRs, which has made preview features final, this one is mostly removing preview related comments from APIs plus options in test cases. Please also review the related [CSR](https://bugs.openjdk.java.net/browse/JDK-8265090) > > Thanks > > note: this PR is related to [PR-3528](https://github.com/openjdk/jdk/pull/3528) and must be integrated after it. Vicente Romero has updated the pull request incrementally with one additional commit since the last revision: removing javax.lang.model changes ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3526/files - new: https://git.openjdk.java.net/jdk/pull/3526/files/6e2a99c6..5aef5108 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=00-01 Stats: 9 lines in 2 files changed: 6 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3526.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3526/head:pull/3526 PR: https://git.openjdk.java.net/jdk/pull/3526 From vromero at openjdk.java.net Fri Apr 16 03:26:34 2021 From: vromero at openjdk.java.net (Vicente Romero) Date: Fri, 16 Apr 2021 03:26:34 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature in Java [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 02:10:05 GMT, David Holmes wrote: > Hi Vicente, > > Hotspot and hotspot tests all look fine. One query: why was this test removed? > > test/hotspot/jtreg/runtime/sealedClasses/AbstractSealedTest.java > > is that functionality tested elsewhere? (The other deleted test seemed obviously trivial.) > > Thanks, > David Hi David, thanks for your comments, yes regarding `test test/hotspot/jtreg/runtime/sealedClasses/AbstractSealedTest.java`, it was removed because the functionality is tested in `test/langtools/tools/javac/sealed/SealedCompilationTests.java` > src/hotspot/share/classfile/classFileParser.cpp line 3916: > >> 3914: record_attribute_start = cfs->current(); >> 3915: record_attribute_length = attribute_length; >> 3916: } else if (_major_version >= JAVA_17_VERSION) { > > Can you update the comment at L3932 to say JAVA_17_VERSION please. sure ------------- PR: https://git.openjdk.java.net/jdk/pull/3526 From vromero at openjdk.java.net Fri Apr 16 03:35:06 2021 From: vromero at openjdk.java.net (Vicente Romero) Date: Fri, 16 Apr 2021 03:35:06 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature in Java [v3] In-Reply-To: References: Message-ID: > Please review this PR that intents to make sealed classes a final feature in Java. This PR contains compiler and VM changes. In line with similar PRs, which has made preview features final, this one is mostly removing preview related comments from APIs plus options in test cases. Please also review the related [CSR](https://bugs.openjdk.java.net/browse/JDK-8265090) > > Thanks > > note: this PR is related to [PR-3528](https://github.com/openjdk/jdk/pull/3528) and must be integrated after it. Vicente Romero has updated the pull request incrementally with one additional commit since the last revision: updating comment after review feedback ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3526/files - new: https://git.openjdk.java.net/jdk/pull/3526/files/5aef5108..8ebe56fd Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3526.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3526/head:pull/3526 PR: https://git.openjdk.java.net/jdk/pull/3526 From rehn at openjdk.java.net Fri Apr 16 06:44:06 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Fri, 16 Apr 2021 06:44:06 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: Fixed nits ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3191/files - new: https://git.openjdk.java.net/jdk/pull/3191/files/27bf041c..d6e091da Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=09 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=08-09 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Fri Apr 16 06:44:08 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Fri, 16 Apr 2021 06:44:08 GMT Subject: RFR: 8257831: Suspend with handshakes [v9] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 22:22:53 GMT, David Holmes wrote: > Only nits on the incremental. > > Like Dan I need to re-examine the complete set of changes to see if anything else sticks out (other than the non-encapsulation of the state changes around locking/unlocking when suspended). > > Thanks, > David Yes please. Robbin > src/hotspot/share/prims/jvmtiEnv.cpp line 952: > >> 950: if (!JvmtiSuspendControl::suspend(java_thread)) { >> 951: // Either the thread is already suspended or >> 952: // it was in process of exiting. > > Nit: the previous comment was perfectly correct. It is okay to replace the second "the thread" with "it" but it should still read "in _the_ process of exiting". Fixed > src/hotspot/share/prims/jvmtiEnv.cpp line 993: > >> 991: if (!JvmtiSuspendControl::suspend(java_thread)) { >> 992: // Either the thread is already suspended or >> 993: // it was in process of exiting. > > Ditto - "in the process ..." Fixed > src/hotspot/share/prims/jvmtiEnv.cpp line 1006: > >> 1004: if (!JvmtiSuspendControl::suspend(current)) { >> 1005: // Either the thread is already suspended or >> 1006: // it was in process of exiting. > > Ditto - "in the process ..." Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Fri Apr 16 06:44:08 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Fri, 16 Apr 2021 06:44:08 GMT Subject: RFR: 8257831: Suspend with handshakes [v9] In-Reply-To: References: Message-ID: <7_ieDC5PBhv0Xnc18l7N3ODNVvZcdPtAhq56wnTL5Xw=.9017a11d-96da-4354-bd64-0011a1d58659@github.com> On Thu, 15 Apr 2021 17:02:22 GMT, Daniel D. Daugherty wrote: > Reviewed the v08 incremental. Still thumbs up. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From lutz.schmidt at sap.com Fri Apr 16 13:09:42 2021 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Fri, 16 Apr 2021 13:09:42 +0000 Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow In-Reply-To: References: Message-ID: <1AC58C53-381C-4230-8B46-437608651107@sap.com> Hi Paul, thank you for the review. @All: It's annoying, I know. I need somebody to sponsor this change as well. Thanks, Lutz ?On 15.04.21, 22:45, "Hohensee, Paul" wrote: Lgtm. Thanks, Paul -----Original Message----- From: serviceability-dev on behalf of "Schmidt, Lutz" Date: Thursday, April 15, 2021 at 1:19 AM To: "jdk-updates-dev at openjdk.java.net" Cc: "serviceability-dev at openjdk.java.net" Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Dear Community, I would happily accept reviews for this downport change. The change creates more meaningful output. Merge conflicts were only "technical". No complicated semantical considerations necessary. Original bug: https://bugs.openjdk.java.net/browse/JDK-8261447 Downport webrev: https://cr.openjdk.java.net/~lucy/webrevs/8261447.11u.01/ Tests: SAP's internal build and test farm (all OpenJDK platforms (no 32-bit), and more). Tests include JCK, jtreg (hotspot and jdk), and SAP-private tests. No issues detected that are attributable to this change. Thanks, Lutz From winterhalter at openjdk.java.net Fri Apr 16 13:52:07 2021 From: winterhalter at openjdk.java.net (Rafael Winterhalter) Date: Fri, 16 Apr 2021 13:52:07 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes Message-ID: To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. ------------- Commit messages: - 8200559: Java agents doing instrumentation need a means to define auxiliary classes Changes: https://git.openjdk.java.net/jdk/pull/3546/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8200559 Stats: 185 lines in 4 files changed: 185 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3546.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3546/head:pull/3546 PR: https://git.openjdk.java.net/jdk/pull/3546 From forax at univ-mlv.fr Fri Apr 16 14:21:14 2021 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 16 Apr 2021 16:21:14 +0200 (CEST) Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: Message-ID: <1665771929.1527476.1618582874230.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Rafael Winterhalter" > ?: "core-libs-dev" , "serviceability-dev" > Envoy?: Vendredi 16 Avril 2021 15:52:07 > Objet: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes > To allow agents the definition of auxiliary classes, an API is needed to allow > this. Currently, this is often achieved by using `sun.misc.Unsafe` or > `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from > `sun.misc.Unsafe`. You can already use Lookup.defineClass() + privateLookupIn() + Instrumentation.redefineModule() for that ? R?mi > > ------------- > > Commit messages: > - 8200559: Java agents doing instrumentation need a means to define auxiliary > classes > > Changes: https://git.openjdk.java.net/jdk/pull/3546/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8200559 > Stats: 185 lines in 4 files changed: 185 ins; 0 del; 0 mod > Patch: https://git.openjdk.java.net/jdk/pull/3546.diff > Fetch: git fetch https://git.openjdk.java.net/jdk pull/3546/head:pull/3546 > > PR: https://git.openjdk.java.net/jdk/pull/3546 From alanb at openjdk.java.net Fri Apr 16 14:32:39 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 16 Apr 2021 14:32:39 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 13:44:16 GMT, Rafael Winterhalter wrote: > To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. JDK-8200559 is about defining auxiliary classes in the same runtime package at load-time whereas I think the proposal in this PR is adding an unrestricted defineClass to the Instrumentation class. I think this will require a lot of discussion as there are significant issues and concerns here. An unrestricted defineClass might be okay for tool/java agents when started from the command line with -javaagent but only if the Instrumentation object is never ever leaked to library or application code. It could potentially be part of a large effort to reduce the capabilities of agents loaded via the attach mechanism. More generally I think we need clearer separation of the requirements of tool agents from the requirement of framework/libraries that want to inject proxy and other classes at runtime. Separately, the proposal in JEP 410 is to terminally deprecate ProtectionDomain. ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From hohensee at amazon.com Fri Apr 16 15:38:11 2021 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 16 Apr 2021 15:38:11 +0000 Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Message-ID: <3B3457EF-81F9-4A90-B1A1-D018FB1A06CA@amazon.com> I'll sponsor. ?-----Original Message----- From: "Schmidt, Lutz" Date: Friday, April 16, 2021 at 6:10 AM To: "Hohensee, Paul" , "jdk-updates-dev at openjdk.java.net" Cc: "serviceability-dev at openjdk.java.net" Subject: RE: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Hi Paul, thank you for the review. @All: It's annoying, I know. I need somebody to sponsor this change as well. Thanks, Lutz On 15.04.21, 22:45, "Hohensee, Paul" wrote: Lgtm. Thanks, Paul -----Original Message----- From: serviceability-dev on behalf of "Schmidt, Lutz" Date: Thursday, April 15, 2021 at 1:19 AM To: "jdk-updates-dev at openjdk.java.net" Cc: "serviceability-dev at openjdk.java.net" Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Dear Community, I would happily accept reviews for this downport change. The change creates more meaningful output. Merge conflicts were only "technical". No complicated semantical considerations necessary. Original bug: https://bugs.openjdk.java.net/browse/JDK-8261447 Downport webrev: https://cr.openjdk.java.net/~lucy/webrevs/8261447.11u.01/ Tests: SAP's internal build and test farm (all OpenJDK platforms (no 32-bit), and more). Tests include JCK, jtreg (hotspot and jdk), and SAP-private tests. No issues detected that are attributable to this change. Thanks, Lutz From lutz.schmidt at sap.com Fri Apr 16 15:39:20 2021 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Fri, 16 Apr 2021 15:39:20 +0000 Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow In-Reply-To: <3B3457EF-81F9-4A90-B1A1-D018FB1A06CA@amazon.com> References: <3B3457EF-81F9-4A90-B1A1-D018FB1A06CA@amazon.com> Message-ID: <88BA86AA-A615-433B-8727-C1C62D945FF2@sap.com> Hey Paul, you are my hero! Thanks and have a great weekend, Lutz ?On 16.04.21, 17:38, "Hohensee, Paul" wrote: I'll sponsor. -----Original Message----- From: "Schmidt, Lutz" Date: Friday, April 16, 2021 at 6:10 AM To: "Hohensee, Paul" , "jdk-updates-dev at openjdk.java.net" Cc: "serviceability-dev at openjdk.java.net" Subject: RE: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Hi Paul, thank you for the review. @All: It's annoying, I know. I need somebody to sponsor this change as well. Thanks, Lutz On 15.04.21, 22:45, "Hohensee, Paul" wrote: Lgtm. Thanks, Paul -----Original Message----- From: serviceability-dev on behalf of "Schmidt, Lutz" Date: Thursday, April 15, 2021 at 1:19 AM To: "jdk-updates-dev at openjdk.java.net" Cc: "serviceability-dev at openjdk.java.net" Subject: [11u] RFR(M): 8261447 backport: MethodInvocationCounters frequently run into overflow Dear Community, I would happily accept reviews for this downport change. The change creates more meaningful output. Merge conflicts were only "technical". No complicated semantical considerations necessary. Original bug: https://bugs.openjdk.java.net/browse/JDK-8261447 Downport webrev: https://cr.openjdk.java.net/~lucy/webrevs/8261447.11u.01/ Tests: SAP's internal build and test farm (all OpenJDK platforms (no 32-bit), and more). Tests include JCK, jtreg (hotspot and jdk), and SAP-private tests. No issues detected that are attributable to this change. Thanks, Lutz From rafael.wth at gmail.com Fri Apr 16 16:27:46 2021 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 16 Apr 2021 18:27:46 +0200 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: <1665771929.1527476.1618582874230.JavaMail.zimbra@u-pem.fr> References: <1665771929.1527476.1618582874230.JavaMail.zimbra@u-pem.fr> Message-ID: Not by my understanding. A suitable lookup requires a loaded class for the package. A Java agent might however not provide a handle for a class that is not yet loaded. Or how would you suggest to approach this? Am Fr., 16. Apr. 2021 um 16:21 Uhr schrieb Remi Forax : > ----- Mail original ----- > > De: "Rafael Winterhalter" > > ?: "core-libs-dev" , > "serviceability-dev" > > Envoy?: Vendredi 16 Avril 2021 15:52:07 > > Objet: RFR: 8200559: Java agents doing instrumentation need a means to > define auxilary classes > > > To allow agents the definition of auxiliary classes, an API is needed to > allow > > this. Currently, this is often achieved by using `sun.misc.Unsafe` or > > `jdk.internal.misc.Unsafe` ever since the `defineClass` method was > removed from > > `sun.misc.Unsafe`. > > You can already use Lookup.defineClass() + privateLookupIn() + > Instrumentation.redefineModule() for that ? > > R?mi > > > > > ------------- > > > > Commit messages: > > - 8200559: Java agents doing instrumentation need a means to define > auxiliary > > classes > > > > Changes: https://git.openjdk.java.net/jdk/pull/3546/files > > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00 > > Issue: https://bugs.openjdk.java.net/browse/JDK-8200559 > > Stats: 185 lines in 4 files changed: 185 ins; 0 del; 0 mod > > Patch: https://git.openjdk.java.net/jdk/pull/3546.diff > > Fetch: git fetch https://git.openjdk.java.net/jdk > pull/3546/head:pull/3546 > > > > PR: https://git.openjdk.java.net/jdk/pull/3546 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Fri Apr 16 16:35:32 2021 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 16 Apr 2021 18:35:32 +0200 (CEST) Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: <1665771929.1527476.1618582874230.JavaMail.zimbra@u-pem.fr> Message-ID: <1924898916.1609700.1618590932170.JavaMail.zimbra@u-pem.fr> > De: "Rafael Winterhalter" > ?: "Remi Forax" > Cc: "Rafael Winterhalter" , "core-libs-dev" > , "serviceability-dev" > > Envoy?: Vendredi 16 Avril 2021 18:27:46 > Objet: Re: RFR: 8200559: Java agents doing instrumentation need a means to > define auxilary classes > Not by my understanding. A suitable lookup requires a loaded class for the > package. A Java agent might however not provide a handle for a class that is > not yet loaded. Or how would you suggest to approach this ? yes, you need a witness class in the package you want to define a new class. Apart if you load classes in the unamed module, you can not load a class in a non existing package anyway (apart if you generate your own module-info), so you need at least a dummy class to define a package, so you can use it to get a Lookup. R?mi > Am Fr., 16. Apr. 2021 um 16:21 Uhr schrieb Remi Forax < [ > mailto:forax at univ-mlv.fr | forax at univ-mlv.fr ] >: >> ----- Mail original ----- >>> De: "Rafael Winterhalter" < [ mailto:winterhalter at openjdk.java.net | >> > winterhalter at openjdk.java.net ] > >>> ?: "core-libs-dev" < [ mailto:core-libs-dev at openjdk.java.net | >>> core-libs-dev at openjdk.java.net ] >, "serviceability-dev" < [ >>> mailto:serviceability-dev at openjdk.java.net | >> > serviceability-dev at openjdk.java.net ] > >> > Envoy?: Vendredi 16 Avril 2021 15:52:07 >>> Objet: RFR: 8200559: Java agents doing instrumentation need a means to define >> > auxilary classes >> > To allow agents the definition of auxiliary classes, an API is needed to allow >> > this. Currently, this is often achieved by using `sun.misc.Unsafe` or >> > `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from >> > `sun.misc.Unsafe`. >> You can already use Lookup.defineClass() + privateLookupIn() + >> Instrumentation.redefineModule() for that ? >> R?mi >> > ------------- >> > Commit messages: >> > - 8200559: Java agents doing instrumentation need a means to define auxiliary >> > classes >>> Changes: [ https://git.openjdk.java.net/jdk/pull/3546/files | >> > https://git.openjdk.java.net/jdk/pull/3546/files ] >>> Webrev: [ https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00 | >> > https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00 ] >>> Issue: [ https://bugs.openjdk.java.net/browse/JDK-8200559 | >> > https://bugs.openjdk.java.net/browse/JDK-8200559 ] >> > Stats: 185 lines in 4 files changed: 185 ins; 0 del; 0 mod >>> Patch: [ https://git.openjdk.java.net/jdk/pull/3546.diff | >> > https://git.openjdk.java.net/jdk/pull/3546.diff ] >>> Fetch: git fetch [ https://git.openjdk.java.net/jdk | >> > https://git.openjdk.java.net/jdk ] pull/3546/head:pull/3546 >>> PR: [ https://git.openjdk.java.net/jdk/pull/3546 | >> > https://git.openjdk.java.net/jdk/pull/3546 ] -------------- next part -------------- An HTML attachment was scrubbed... URL: From winterhalter at openjdk.java.net Fri Apr 16 16:40:35 2021 From: winterhalter at openjdk.java.net (Rafael Winterhalter) Date: Fri, 16 Apr 2021 16:40:35 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 13:44:16 GMT, Rafael Winterhalter wrote: > To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. The ticket was created as a reaction to a [write-up I made in January 2018](http://mail.openjdk.java.net/pipermail/jdk-dev/2018-January/000405.html). I certainly did not intend to limit the scope to same-package class definitions for instrumented classes, and even for those I do not think a package-restricted API would be sufficient as I outlined in the comments to [JDK-8200559](https://bugs.openjdk.java.net/browse/JDK-8200559). I will try to make my case on the mailing list. I hoped this could get resolved within the release of Java 17 as this would make it possible to write agents without use of Unsafe API to support Java 17 and later. Since agents often are supplementary to a broad range of Java applications, the LTS release will likely be an important support boundary for years and years to come. You surely mean JEP 411 when mentioning `ProtectionDomain`? The JEP mentions > We will not deprecate some classes in the java.security package that are related to the Security Manager, for varying reasons: [...] ProtectionDomain ? Several significant APIs depend on ProtectionDomain, e.g., ClassLoader::defineClass and Class::getProtectionDomain. ProtectionDomain also has value independent of the Security Manager since it contains the CodeSource of a class. Also, since this is still a proposal, I would believe that APIs that are implemented before JEP 411 is realized should support `ProtectionDomain` for users still supporting the security manager in the transition time. ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From rafael.wth at gmail.com Fri Apr 16 16:41:26 2021 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 16 Apr 2021 18:41:26 +0200 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: <1924898916.1609700.1618590932170.JavaMail.zimbra@u-pem.fr> References: <1665771929.1527476.1618582874230.JavaMail.zimbra@u-pem.fr> <1924898916.1609700.1618590932170.JavaMail.zimbra@u-pem.fr> Message-ID: This would be a problem, however. Since there is currently no official way of defining a class, and since Java agents do not control the class loading order, if a class was loaded for the first time, you could for example not add a field with a type of an auxiliary class that you had planned to inject. A class being loaded is normally the first opportunity for a Java agent and if no witness class is available at this point, using a method handle is no option. Since it is difficult to know if such a witness class is available in the general case, it would also add quite a performance and managerial toll on agent authors. I would hope that an API equally convenient to today's unsafe options could be added. Am Fr., 16. Apr. 2021 um 18:35 Uhr schrieb : > > > ------------------------------ > > *De: *"Rafael Winterhalter" > *?: *"Remi Forax" > *Cc: *"Rafael Winterhalter" , > "core-libs-dev" , "serviceability-dev" < > serviceability-dev at openjdk.java.net> > *Envoy?: *Vendredi 16 Avril 2021 18:27:46 > *Objet: *Re: RFR: 8200559: Java agents doing instrumentation need a means > to define auxilary classes > > Not by my understanding. A suitable lookup requires a loaded class for the > package. A Java agent might however not provide a handle for a class that > is not yet loaded. Or how would you suggest to approach this ? > > > yes, you need a witness class in the package you want to define a new > class. > Apart if you load classes in the unamed module, you can not load a class > in a non existing package anyway (apart if you generate your own > module-info), > so you need at least a dummy class to define a package, so you can use it > to get a Lookup. > > R?mi > > > > Am Fr., 16. Apr. 2021 um 16:21 Uhr schrieb Remi Forax : > >> ----- Mail original ----- >> > De: "Rafael Winterhalter" >> > ?: "core-libs-dev" , >> "serviceability-dev" >> > Envoy?: Vendredi 16 Avril 2021 15:52:07 >> > Objet: RFR: 8200559: Java agents doing instrumentation need a means to >> define auxilary classes >> >> > To allow agents the definition of auxiliary classes, an API is needed >> to allow >> > this. Currently, this is often achieved by using `sun.misc.Unsafe` or >> > `jdk.internal.misc.Unsafe` ever since the `defineClass` method was >> removed from >> > `sun.misc.Unsafe`. >> >> You can already use Lookup.defineClass() + privateLookupIn() + >> Instrumentation.redefineModule() for that ? >> >> R?mi >> >> > >> > ------------- >> > >> > Commit messages: >> > - 8200559: Java agents doing instrumentation need a means to define >> auxiliary >> > classes >> > >> > Changes: https://git.openjdk.java.net/jdk/pull/3546/files >> > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00 >> > Issue: https://bugs.openjdk.java.net/browse/JDK-8200559 >> > Stats: 185 lines in 4 files changed: 185 ins; 0 del; 0 mod >> > Patch: https://git.openjdk.java.net/jdk/pull/3546.diff >> > Fetch: git fetch https://git.openjdk.java.net/jdk >> pull/3546/head:pull/3546 >> > >> > PR: https://git.openjdk.java.net/jdk/pull/3546 >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Apr 16 17:31:37 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 16 Apr 2021 18:31:37 +0100 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: Message-ID: On 16/04/2021 17:40, Rafael Winterhalter wrote: > : > I will try to make my case on the mailing list. I hoped this could get resolved within the release of Java 17 as this would make it possible to write agents without use of Unsafe API to support Java 17 and later. Since agents often are supplementary to a broad range of Java applications, the LTS release will likely be an important support boundary for years and years to come. > "are supplementary to a board range of Java applications" is part of the concern with the proposal. If possible, it would be good if the write-up could separate the requirements for injection/instrumentation by frameworks at runtime from the requirements of tool agents. If the requirements cover testing time and mocking then it would useful to separate those too. Just to add to R?mi's comment: For frameworks/libraries, the Lookup.defineClass and defineHiddenClass APIs are to define classes in the same run-time package as the Lookup class. There isn't any API for libraries/frameworks to define class in a "new run-time package". There's a chunky project there. Part of it is the Lookup API itself, part of is that there isn't an exposed way to extend the set of packages in a named module. Mandy has done some exploration on this topic and may be able to say a bit more about this. On Java agents, then I think the discussion will eventually lead into putting at least some restrictions on agents loaded into a running VM. Agents started on the command line with -javaagent are all-powerful but maybe agents loaded into a target VM get a restricted Instrumentation object that cannot redefine modules or retransform classes in named modules. The narrower requirement for agents doing load time instrumentation to define auxiliary classes in the same package as the class being loaded fits with the intent of the original API and I don't think is controversial. -Alan From forax at univ-mlv.fr Fri Apr 16 17:47:16 2021 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 16 Apr 2021 19:47:16 +0200 (CEST) Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: <1665771929.1527476.1618582874230.JavaMail.zimbra@u-pem.fr> <1924898916.1609700.1618590932170.JavaMail.zimbra@u-pem.fr> Message-ID: <259848735.1634666.1618595236467.JavaMail.zimbra@u-pem.fr> > De: "Rafael Winterhalter" > ?: "Remi Forax" > Cc: "Rafael Winterhalter" , "core-libs-dev" > , "serviceability-dev" > > Envoy?: Vendredi 16 Avril 2021 18:41:26 > Objet: Re: RFR: 8200559: Java agents doing instrumentation need a means to > define auxilary classes > This would be a problem, however. Since there is currently no official way of > defining a class, and since Java agents do not control the class loading order, > if a class was loaded for the first time, you could for example not add a field > with a type of an auxiliary class that you had planned to inject. A class being > loaded is normally the first opportunity for a Java agent and if no witness > class is available at this point, using a method handle is no option. Since it > is difficult to know if such a witness class is available in the general case, > it would also add quite a performance and managerial toll on agent authors. I > would hope that an API equally convenient to today's unsafe options could be > added. I was thinking about adding a dummy class in the package you want to load classes. Anyway, you can also call ClassLoader.defineClass directly. Put the code that calls defineClass in a module and use Instrumentation.redefineModule() to open java.base to this module. R?mi > Am Fr., 16. Apr. 2021 um 18:35 Uhr schrieb < [ mailto:forax at univ-mlv.fr | > forax at univ-mlv.fr ] >: >>> De: "Rafael Winterhalter" < [ mailto:rafael.wth at gmail.com | rafael.wth at gmail.com >>> ] > >>> ?: "Remi Forax" < [ mailto:forax at univ-mlv.fr | forax at univ-mlv.fr ] > >>> Cc: "Rafael Winterhalter" < [ mailto:winterhalter at openjdk.java.net | >>> winterhalter at openjdk.java.net ] >, "core-libs-dev" < [ >>> mailto:core-libs-dev at openjdk.java.net | core-libs-dev at openjdk.java.net ] >, >>> "serviceability-dev" < [ mailto:serviceability-dev at openjdk.java.net | >>> serviceability-dev at openjdk.java.net ] > >>> Envoy?: Vendredi 16 Avril 2021 18:27:46 >>> Objet: Re: RFR: 8200559: Java agents doing instrumentation need a means to >>> define auxilary classes >>> Not by my understanding. A suitable lookup requires a loaded class for the >>> package. A Java agent might however not provide a handle for a class that is >>> not yet loaded. Or how would you suggest to approach this ? >> yes, you need a witness class in the package you want to define a new class. >> Apart if you load classes in the unamed module, you can not load a class in a >> non existing package anyway (apart if you generate your own module-info), >> so you need at least a dummy class to define a package, so you can use it to get >> a Lookup. >> R?mi >>> Am Fr., 16. Apr. 2021 um 16:21 Uhr schrieb Remi Forax < [ >>> mailto:forax at univ-mlv.fr | forax at univ-mlv.fr ] >: >>>> ----- Mail original ----- >>>>> De: "Rafael Winterhalter" < [ mailto:winterhalter at openjdk.java.net | >>>> > winterhalter at openjdk.java.net ] > >>>>> ?: "core-libs-dev" < [ mailto:core-libs-dev at openjdk.java.net | >>>>> core-libs-dev at openjdk.java.net ] >, "serviceability-dev" < [ >>>>> mailto:serviceability-dev at openjdk.java.net | >>>> > serviceability-dev at openjdk.java.net ] > >>>> > Envoy?: Vendredi 16 Avril 2021 15:52:07 >>>>> Objet: RFR: 8200559: Java agents doing instrumentation need a means to define >>>> > auxilary classes >>>> > To allow agents the definition of auxiliary classes, an API is needed to allow >>>> > this. Currently, this is often achieved by using `sun.misc.Unsafe` or >>>> > `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from >>>> > `sun.misc.Unsafe`. >>>> You can already use Lookup.defineClass() + privateLookupIn() + >>>> Instrumentation.redefineModule() for that ? >>>> R?mi >>>> > ------------- >>>> > Commit messages: >>>> > - 8200559: Java agents doing instrumentation need a means to define auxiliary >>>> > classes >>>>> Changes: [ https://git.openjdk.java.net/jdk/pull/3546/files | >>>> > https://git.openjdk.java.net/jdk/pull/3546/files ] >>>>> Webrev: [ https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00 | >>>> > https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00 ] >>>>> Issue: [ https://bugs.openjdk.java.net/browse/JDK-8200559 | >>>> > https://bugs.openjdk.java.net/browse/JDK-8200559 ] >>>> > Stats: 185 lines in 4 files changed: 185 ins; 0 del; 0 mod >>>>> Patch: [ https://git.openjdk.java.net/jdk/pull/3546.diff | >>>> > https://git.openjdk.java.net/jdk/pull/3546.diff ] >>>>> Fetch: git fetch [ https://git.openjdk.java.net/jdk | >>>> > https://git.openjdk.java.net/jdk ] pull/3546/head:pull/3546 >>>>> PR: [ https://git.openjdk.java.net/jdk/pull/3546 | >>>> > https://git.openjdk.java.net/jdk/pull/3546 ] -------------- next part -------------- An HTML attachment was scrubbed... URL: From sspitsyn at openjdk.java.net Fri Apr 16 19:17:56 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 16 Apr 2021 19:17:56 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 06:44:06 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed nits Marked as reviewed by sspitsyn (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From winterhalter at openjdk.java.net Fri Apr 16 20:09:34 2021 From: winterhalter at openjdk.java.net (Rafael Winterhalter) Date: Fri, 16 Apr 2021 20:09:34 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 13:44:16 GMT, Rafael Winterhalter wrote: > To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. I have never seen a need for a non-agent to define a class in a non-existing package. Injection is typically required if you want to work with package-private types or methods which is really only relevant for the Spring framework, but even there I do not think it's such a big area that it cannot be addressed. Non-agent code generation is typically the use of proxies where the code generation framework is sharing a class loader with the user code and there is normally a way to weave it. Agents on the other hand have to deal with unknown class loader hierarchies and it can be necessary to inject code into a common class loader parent without instrumenting classes in this loader or even knowing any classes of this loader. It's really agents where this is required and therefore Instrumentation is a good target for such an addition. I have never heard about a 'discussion [that] will eventually lead into putting at least some restrictions on agents loaded into a running VM' and as a heavy user and someone who has helped to write a long row of commercial agents and followed them into their use in production and who has seen how helpful they are in reducing deployment complexity, I can only hope that you will change your mind on this. Dynamically attached agents must already run as the same user as the target process. If you are concerned about agents 'illegally intruding' your Java process, I'd say you have bigger security issues at hand and it is already possible to disable dynamic attachment if you want to avoid it. Dynamic attachment has become pretty much the default approach for a lot of Java tooling in production environments. It has proven to be very convenient if a for example a tracing tool is scanning Java processes to attach to them, rather than requiring the deployment operators to be explicitly set up command line arguments which are easily forgotten if they only need to be added in some environments. This reduction in complexity for operations has literally saved millions of dollars to Java users and Oracle customers. This makes Java a popular choice, especially compared to languages such as Go where this is naturally not possible. It's easy and there is no record of this feature doing any harm. I would not see any good reason to restrict this by default. That said, even if it was restricted in the future, this would mean that some of the Instrumentation API methods will throw exceptions in the future. There would not be much difference if an introduced defineClass method would do the same. Am Fr., 16. Apr. 2021 um 19:33 Uhr schrieb mlbridge[bot] < ***@***.***>: > *Mailing list message from Alan Bateman ***@***.***> on > core-libs-dev ***@***.***>:* > > On 16/04/2021 17:40, Rafael Winterhalter wrote: > > : > I will try to make my case on the mailing list. I hoped this could get > resolved within the release of Java 17 as this would make it possible to > write agents without use of Unsafe API to support Java 17 and later. Since > agents often are supplementary to a broad range of Java applications, the > LTS release will likely be an important support boundary for years and > years to come. > > "are supplementary to a board range of Java applications" is part of the > concern with the proposal. If possible, it would be good if the write-up > could separate the requirements for injection/instrumentation by > frameworks at runtime from the requirements of tool agents. If the > requirements cover testing time and mocking then it would useful to > separate those too. > > Just to add to R?mi's comment: For frameworks/libraries, the > Lookup.defineClass and defineHiddenClass APIs are to define classes in > the same run-time package as the Lookup class. There isn't any API for > libraries/frameworks to define class in a "new run-time package". > There's a chunky project there. Part of it is the Lookup API itself, > part of is that there isn't an exposed way to extend the set of packages > in a named module. Mandy has done some exploration on this topic and may > be able to say a bit more about this. > > On Java agents, then I think the discussion will eventually lead into > putting at least some restrictions on agents loaded into a running VM. > Agents started on the command line with -javaagent are all-powerful but > maybe agents loaded into a target VM get a restricted Instrumentation > object that cannot redefine modules or retransform classes in named > modules. The narrower requirement for agents doing load time > instrumentation to define auxiliary classes in the same package as the > class being loaded fits with the intent of the original API and I don't > think is controversial. > > -Alan > > ? > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From rafael.wth at gmail.com Fri Apr 16 20:20:03 2021 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 16 Apr 2021 22:20:03 +0200 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: Message-ID: I have never seen a need for a non-agent to define a class in a non-existing package. Injection is typically required if you want to work with package-private types or methods which is really only relevant for the Spring framework, but even there I do not think it's such a big area that it cannot be addressed. Non-agent code generation is typically the use of proxies where the code generation framework is sharing a class loader with the user code and there is normally a way to weave it. Agents on the other hand have to deal with unknown class loader hierarchies and it can be necessary to inject code into a common class loader parent without instrumenting classes in this loader or even knowing any classes of this loader. It's really agents where this is required and therefore Instrumentation is a good target for such an addition. I have never heard about a 'discussion [that] will eventually lead into putting at least some restrictions on agents loaded into a running VM' and as a heavy user and someone who has helped to write a long row of commercial agents and followed them into their use in production and who has seen how helpful they are in reducing deployment complexity, I can only hope that you will change your mind on this. Dynamically attached agents must already run as the same user as the target process. If you are concerned about agents 'illegally intruding' your Java process, I'd say you have bigger security issues at hand and it is already possible to disable dynamic attachment if you want to avoid it. Dynamic attachment has become pretty much the default approach for a lot of Java tooling in production environments. It has proven to be very convenient if a for example a tracing tool is scanning Java processes to attach to them, rather than requiring the deployment operators to be explicitly set up command line arguments which are easily forgotten if they only need to be added in some environments. This reduction in complexity for operations has literally saved millions of dollars to Java users and Oracle customers. This makes Java a popular choice, especially compared to languages such as Go where this is naturally not possible. It's easy and there is no record of this feature doing any harm. I would not see any good reason to restrict this by default. That said, even if it was restricted in the future, this would mean that some of the Instrumentation API methods will throw exceptions in the future. There would not be much difference if an introduced defineClass method would do the same. Am Fr., 16. Apr. 2021 um 19:31 Uhr schrieb Alan Bateman < Alan.Bateman at oracle.com>: > On 16/04/2021 17:40, Rafael Winterhalter wrote: > > : > > I will try to make my case on the mailing list. I hoped this could get > resolved within the release of Java 17 as this would make it possible to > write agents without use of Unsafe API to support Java 17 and later. Since > agents often are supplementary to a broad range of Java applications, the > LTS release will likely be an important support boundary for years and > years to come. > > > "are supplementary to a board range of Java applications" is part of the > concern with the proposal. If possible, it would be good if the write-up > could separate the requirements for injection/instrumentation by > frameworks at runtime from the requirements of tool agents. If the > requirements cover testing time and mocking then it would useful to > separate those too. > > Just to add to R?mi's comment: For frameworks/libraries, the > Lookup.defineClass and defineHiddenClass APIs are to define classes in > the same run-time package as the Lookup class. There isn't any API for > libraries/frameworks to define class in a "new run-time package". > There's a chunky project there. Part of it is the Lookup API itself, > part of is that there isn't an exposed way to extend the set of packages > in a named module. Mandy has done some exploration on this topic and may > be able to say a bit more about this. > > On Java agents, then I think the discussion will eventually lead into > putting at least some restrictions on agents loaded into a running VM. > Agents started on the command line with -javaagent are all-powerful but > maybe agents loaded into a target VM get a restricted Instrumentation > object that cannot redefine modules or retransform classes in named > modules. The narrower requirement for agents doing load time > instrumentation to define auxiliary classes in the same package as the > class being loaded fits with the intent of the original API and I don't > think is controversial. > > -Alan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From winterhalter at openjdk.java.net Fri Apr 16 20:30:15 2021 From: winterhalter at openjdk.java.net (Rafael Winterhalter) Date: Fri, 16 Apr 2021 20:30:15 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes [v2] In-Reply-To: References: Message-ID: > To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: 8200559: Java agents doing instrumentation need a means to define auxiliary classes ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3546/files - new: https://git.openjdk.java.net/jdk/pull/3546/files/362efc82..2ef21598 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3546&range=00-01 Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3546.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3546/head:pull/3546 PR: https://git.openjdk.java.net/jdk/pull/3546 From dcubed at openjdk.java.net Fri Apr 16 22:47:05 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 16 Apr 2021 22:47:05 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 06:44:06 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed nits Reviewed the v09 incremental. Still thumbs up. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From amenkov at openjdk.java.net Fri Apr 16 23:48:34 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 16 Apr 2021 23:48:34 GMT Subject: RFR: 8265028: JDWP debug agent thread lookup can be made faster In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 20:29:56 GMT, Chris Plummer wrote: > Improved thread lookup speeds in the debug agent, especially when there is a large number of threads. See CR for details. Marked as reviewed by amenkov (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3444 From ayang at openjdk.java.net Sun Apr 18 09:00:41 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Sun, 18 Apr 2021 09:00:41 GMT Subject: RFR: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 07:13:50 GMT, Per Liden wrote: > JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. > > Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. > > Testing: > * Tier1-7 with ZGC > * Manual testing with relevant jtreg tests Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3483 From Alan.Bateman at oracle.com Sun Apr 18 16:24:04 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 18 Apr 2021 17:24:04 +0100 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: References: Message-ID: <5eda4d71-362f-59f5-6053-1e176873dd91@oracle.com> On 16/04/2021 21:09, Rafael Winterhalter wrote: > I have never seen a need for a non-agent to define a class in a > non-existing package. Injection is typically required if you want to work > with package-private types or methods which is really only relevant for the > Spring framework, but even there I do not think it's such a big area that > it cannot be addressed. Non-agent code generation is typically the use of > proxies where the code generation framework is sharing a class loader with > the user code and there is normally a way to weave it. Agents on the other > hand have to deal with unknown class loader hierarchies and it can be > necessary to inject code into a common class loader parent without > instrumenting classes in this loader or even knowing any classes of this > loader. Yes, the injection of proxy classes and the like is mainly the same run-time package as an existing class and these are the use-cases that Lookup.defineClass and Lookup.defineHiddenClass are intended for. There has been a few requests for defining proxy classes into "new/empty packages" but has a few challenges, like extending the set of packages in a named module. In general I wouldn't expect Java agents, which was intending for tools, to be using Lookup objects. > I have never heard about a 'discussion [that] will eventually lead into > putting at least some restrictions on agents loaded into a running VM' and > as a heavy user and someone who has helped to write a long row of > commercial agents and followed them into their use in production and who > has seen how helpful they are in reducing deployment complexity, I can only > hope that you will change your mind on this. > : > > That said, even if it was restricted in the future, this would mean that > some of the Instrumentation API methods will throw exceptions in the > future. There would not be much difference if an introduced defineClass > method would do the same. There was fuss on this topic during JDK 9. I'll try to find the mails in the archive, it would have been on serviceability-dev in 2016 or 2017. The compromise at the time was to introduce the -XX:EnableDynamicAgentLoading option with an initial value of "true" and re-visit it later. Flipping the default would mean that JVMTI and Java agents could not be loaded into a target VM without opt-in on the command line. No impact to agents specified on the command line with -javaagent, no impact to other usages of the attach mechanism so jcmd and the other diagnostic tools would continue to work. To re-cap, the main concern is the Instrumentation object is all-powerful and is intended for tools and the instrumentation is intended to be benign to support use-cases such as monitoring and tracing. It was never intended to a back-door into JDK internals. Specifying an agent on the command line with -javaagent is the opt-in to trust that agent. A tool run by the same user that loads its agent into a target VM is the use-case we targeted when we added the attach mechanism and the late binding agent support in JDK 6. The issue with that (and it's a regret now) is that the mechanism doesn't distinguish usage by a tool from a library/application using the attach mechanism to load an agent into the current VM (directly or using an intermediate VM). Time has moved on and maybe a better approach is to not change the XX option but instead load the agents with reduced capabilities. It would require opt-in on the command line to give these agents the same capabilities as agents specified on the command line with -javaagent. It would require exploration but it might be that unrecognized agents would not be allowed to redefine java.base or other core modules or instrument classes in those modules. The APIs are already specified to allow Unmodified{Module,Class}Exception be thrown as there have been cases in the past where some classes could not be transformed. Project Panama is currently exploring how to restrict and grant access to native code and maybe it should be the same mechanism or at least be consistent. Hopefully this helps sets some context as to why we have to be cautious with this PR. If the proposal were a defineClass that was limited to agents specified on the command line then it might okay. This would serve use-cases with tool agents that are way more advanced that the use-cases that we envisaged back in Java 5/6. The previous exploration into allowing non-public auxiliary classes be defined in the same run-time package as the class being instrumented was also okay. -Alan From rafael.wth at gmail.com Sun Apr 18 21:05:43 2021 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Sun, 18 Apr 2021 23:05:43 +0200 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxilary classes In-Reply-To: <5eda4d71-362f-59f5-6053-1e176873dd91@oracle.com> References: <5eda4d71-362f-59f5-6053-1e176873dd91@oracle.com> Message-ID: As for the need to inject classes into 'new' packages, in Mockito we solve this easily by 'claiming' a package by defining a static dummy class within it which we then use as a hook for injection using a lookup. This works well for us, and I do of course not know the explicit requirements of others, but personally, I do not see a big need for this functionality otherwise. I remember the discussions about the dynamic agent attachment rather well, it cuts into my line of work. The main (!) reason why dynamic attachment is so popular is that it allows monitoring a JVM without additional configuration. In this light, it does not make much of a difference if a user has to set a special parameter to allow for dynamic attachment or if the user has to set up the agent parameter itself, both add about the same costs to operations. This is the cost you want to avoid by dynamic attachment in the first place. Imagine an enterprise with thousands of JVM deployments which are managed by different teams in different styles, with different technical understanding relying on long chains of communication. It can take years to roll out a tracing tool that requires explicit configuration in an org like this. Thanks to dynamic attachment, all you need is to drop an application per host which discovers all running JVMs and attaches to them. A tracing tool that is missing only a few VMs by misconfiguration will render a very incomplete picture of the tracing state, making it an expensive toy. Easily avoiding such simple misses is an important selling point for the JVM compared to other platforms where this is not possible. I know this problem does not sound like much, it's overcomeable, but orgs will not change, and reducing the capabilities of dynamic agents would take much more out of the JVM's attractiveness than the added security and consistency would bring to it. As for the proposed API, I understand that it needs thought, my hope is still that Java 17 would allow to write agents without Unsafe, since 17 will be a baseline for many years to come. I'd like to stress out that the proposed API only encapsulates something that is already implicitly possible today if one puts in the work. I think that the problem you want to solve is rather that JVMs should not attach to themselves to get hold of an instance of Instrumentation for their own JVM. In this context, one would need to restrict the use of JNI and prevent that a JVM starts new (Java) processes. I think it would be better to tackle the issue from this angle rather than taking away a feature that significantly adds to the JVMs attractiveness. Am So., 18. Apr. 2021 um 18:24 Uhr schrieb Alan Bateman < Alan.Bateman at oracle.com>: > On 16/04/2021 21:09, Rafael Winterhalter wrote: > > I have never seen a need for a non-agent to define a class in a > > non-existing package. Injection is typically required if you want to work > > with package-private types or methods which is really only relevant for > the > > Spring framework, but even there I do not think it's such a big area that > > it cannot be addressed. Non-agent code generation is typically the use of > > proxies where the code generation framework is sharing a class loader > with > > the user code and there is normally a way to weave it. Agents on the > other > > hand have to deal with unknown class loader hierarchies and it can be > > necessary to inject code into a common class loader parent without > > instrumenting classes in this loader or even knowing any classes of this > > loader. > Yes, the injection of proxy classes and the like is mainly the same > run-time package as an existing class and these are the use-cases that > Lookup.defineClass and Lookup.defineHiddenClass are intended for. There > has been a few requests for defining proxy classes into "new/empty > packages" but has a few challenges, like extending the set of packages > in a named module. In general I wouldn't expect Java agents, which was > intending for tools, to be using Lookup objects. > > > > I have never heard about a 'discussion [that] will eventually lead into > > putting at least some restrictions on agents loaded into a running VM' > and > > as a heavy user and someone who has helped to write a long row of > > commercial agents and followed them into their use in production and who > > has seen how helpful they are in reducing deployment complexity, I can > only > > hope that you will change your mind on this. > > : > > > > That said, even if it was restricted in the future, this would mean that > > some of the Instrumentation API methods will throw exceptions in the > > future. There would not be much difference if an introduced defineClass > > method would do the same. > There was fuss on this topic during JDK 9. I'll try to find the mails in > the archive, it would have been on serviceability-dev in 2016 or 2017. > The compromise at the time was to introduce the > -XX:EnableDynamicAgentLoading option with an initial value of "true" and > re-visit it later. Flipping the default would mean that JVMTI and Java > agents could not be loaded into a target VM without opt-in on the > command line. No impact to agents specified on the command line with > -javaagent, no impact to other usages of the attach mechanism so jcmd > and the other diagnostic tools would continue to work. > > To re-cap, the main concern is the Instrumentation object is > all-powerful and is intended for tools and the instrumentation is > intended to be benign to support use-cases such as monitoring and > tracing. It was never intended to a back-door into JDK internals. > Specifying an agent on the command line with -javaagent is the opt-in to > trust that agent. A tool run by the same user that loads its agent into > a target VM is the use-case we targeted when we added the attach > mechanism and the late binding agent support in JDK 6. The issue with > that (and it's a regret now) is that the mechanism doesn't distinguish > usage by a tool from a library/application using the attach mechanism to > load an agent into the current VM (directly or using an intermediate VM). > > Time has moved on and maybe a better approach is to not change the XX > option but instead load the agents with reduced capabilities. It would > require opt-in on the command line to give these agents the same > capabilities as agents specified on the command line with -javaagent. It > would require exploration but it might be that unrecognized agents would > not be allowed to redefine java.base or other core modules or instrument > classes in those modules. The APIs are already specified to allow > Unmodified{Module,Class}Exception be thrown as there have been cases in > the past where some classes could not be transformed. Project Panama is > currently exploring how to restrict and grant access to native code and > maybe it should be the same mechanism or at least be consistent. > > Hopefully this helps sets some context as to why we have to be cautious > with this PR. If the proposal were a defineClass that was limited to > agents specified on the command line then it might okay. This would > serve use-cases with tool agents that are way more advanced that the > use-cases that we envisaged back in Java 5/6. The previous exploration > into allowing non-public auxiliary classes be defined in the same > run-time package as the class being instrumented was also okay. > > -Alan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dholmes at openjdk.java.net Mon Apr 19 06:19:00 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Mon, 19 Apr 2021 06:19:00 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 06:44:06 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Fixed nits Hi Robbin, I have gone through everything again in detail. A few comments on comments and a couple of details I'm still not really clear on. A couple of small renaming requests, but otherwise this is good to go - with the caveat of a future RFE to clear up the suspend-while-locking implementations so things are more cleanly encapsulated. Thanks, David src/hotspot/share/runtime/handshake.cpp line 630: > 628: // This is the closure that prevents a suspended JavaThread from > 629: // escaping the suspend request. > 630: class ThreadSuspensionHandshake : public AsyncHandshakeClosure { I still find ThreadSuspensionHandShake versus SuspendThreadHandshake to be too similarly named and with no obvious way to remember which one is which. Perhaps this one could be called ThreadSelfSuspensionHandshake instead? src/hotspot/share/runtime/handshake.cpp line 636: > 634: JavaThread* target = thr->as_Java_thread(); > 635: target->handshake_state()->do_self_suspend(); > 636: } As this must be the current thread we are dealing with can we rename the variables to indicate that. src/hotspot/share/runtime/handshake.hpp line 128: > 126: // must take slow path, process_by_self(), if _lock is held. > 127: bool should_process() { > 128: // When doing thread suspension the holder of the _lock Potentially this could be any async handshake operation - right? It seems odd to talk specifically about thread suspension in the core of the handshake code. src/hotspot/share/runtime/handshake.hpp line 131: > 129: // can add an asynchronous handshake to queue. > 130: // To make sure it is seen by the handshakee, the handshakee must first > 131: // check the _lock, if held go to slow path. ..., _and_ if held ... src/hotspot/share/runtime/handshake.hpp line 133: > 131: // check the _lock, if held go to slow path. > 132: // Since the handshakee is unsafe if _lock gets locked after this check > 133: // we know another thread cannot process any handshakes. I can't quite understand this comment. I'm not sure what thread is calling this method and when, in relation to what the handshakee may be doing. src/hotspot/share/runtime/handshake.hpp line 134: > 132: // Since the handshakee is unsafe if _lock gets locked after this check > 133: // we know another thread cannot process any handshakes. > 134: // Now we can check queue if there is anything we should process. // Now we can check the queue to see if there is anything we should process. src/hotspot/share/runtime/handshake.hpp line 157: > 155: Thread* active_handshaker() const { return _active_handshaker; } > 156: > 157: // Suspend/resume support Taking a step back I can't help but think that this is entirely the wrong place to have the suspend/resume API and supporting code. It is only here because we re-use the HandshakeState _lock monitor right? If we introduced a new thread-suspension monitor instead then this code would all reside somewhere else - probably in the JavaThread class. src/hotspot/share/runtime/mutex.hpp line 70: > 68: tty = access + 2, > 69: special = tty + 3, > 70: oopstorage = special + 3, Why +3? I'm assuming to keep the same numeric value as before, but that doesn't seem necessary and just seems to add to the mystery of why these ranks are defined in such a strange way. src/hotspot/share/runtime/thread.cpp line 1444: > 1442: } > 1443: > 1444: // The careful dance between thread suspension and exit is handled here. :) It isn't really a "careful dance" any more, as it is very straight forward. exit() can't race with a suspend request as such because we won't hit a handshake polling point. src/hotspot/share/runtime/thread.cpp line 1793: > 1791: bool JavaThread::java_suspend() { > 1792: ThreadsListHandle tlh; > 1793: if (!tlh.includes(this)) { Pre-existing: this check bothers me. Surely we should just be asserting "this" thread is in a TL as we are supposed to have ensured that before we even got this far. Future RFE. src/hotspot/share/runtime/thread.cpp line 1814: > 1812: // state before returning are performed. The current thread is required to > 1813: // change to _thread_blocked in order to be seen to be safepoint/handshake safe > 1814: // whilst suspended and only after becoming handshake safe, the other thread can It is still not at all clear to me what suspended has to do with the execution of this method. The new code just ignores thread suspension. It seems to me that the old code could also have ignored suspension if the checks in handle_special_runtime_exit_condition had be reordered. src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/Thread.java line 83: > 81: } > 82: > 83: public boolean isExtSuspended() { A new comment to avoid the "resolved" conversation. I'm still not sure you can just delete this API from the SA. @sspitsyn do you know what the obligations are with respect to the S.A here? ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 19 07:18:48 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 19 Apr 2021 07:18:48 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 04:19:28 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed nits > > src/hotspot/share/runtime/handshake.cpp line 630: > >> 628: // This is the closure that prevents a suspended JavaThread from >> 629: // escaping the suspend request. >> 630: class ThreadSuspensionHandshake : public AsyncHandshakeClosure { > > I still find ThreadSuspensionHandShake versus SuspendThreadHandshake to be too similarly named and with no obvious way to remember which one is which. Perhaps this one could be called ThreadSelfSuspensionHandshake instead? Fixed > src/hotspot/share/runtime/handshake.cpp line 636: > >> 634: JavaThread* target = thr->as_Java_thread(); >> 635: target->handshake_state()->do_self_suspend(); >> 636: } > > As this must be the current thread we are dealing with can we rename the variables to indicate that. Fixed > src/hotspot/share/runtime/handshake.hpp line 128: > >> 126: // must take slow path, process_by_self(), if _lock is held. >> 127: bool should_process() { >> 128: // When doing thread suspension the holder of the _lock > > Potentially this could be any async handshake operation - right? It seems odd to talk specifically about thread suspension in the core of the handshake code. Changed > src/hotspot/share/runtime/handshake.hpp line 131: > >> 129: // can add an asynchronous handshake to queue. >> 130: // To make sure it is seen by the handshakee, the handshakee must first >> 131: // check the _lock, if held go to slow path. > > ..., _and_ if held ... Fixed > src/hotspot/share/runtime/handshake.hpp line 133: > >> 131: // check the _lock, if held go to slow path. >> 132: // Since the handshakee is unsafe if _lock gets locked after this check >> 133: // we know another thread cannot process any handshakes. > > I can't quite understand this comment. I'm not sure what thread is calling this method and when, in relation to what the handshakee may be doing. The handshakee is in a safe state, e.g. blocked and does: Point A: set_thread_state_fence(_thread_blocked_trans); ... Point B: _lock.is_locked() While the processor thread does: Point X: _lock.try_lock(); ... Point Z: thread->thread_state(); If point B is passed with a non-locked lock, point Z is guaranteed to see an unsafe thread and will not start to process ay handshakes. Is that make sense, maybe the comment make more sense ? :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 19 07:21:57 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 19 Apr 2021 07:21:57 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 04:34:00 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed nits > > src/hotspot/share/runtime/handshake.hpp line 134: > >> 132: // Since the handshakee is unsafe if _lock gets locked after this check >> 133: // we know another thread cannot process any handshakes. >> 134: // Now we can check queue if there is anything we should process. > > // Now we can check the queue to see if there is anything we should process. Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Mon Apr 19 07:30:00 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Mon, 19 Apr 2021 07:30:00 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: <_7hckHiny6Ljshr5-Bo1rJ4Z7nb5BHj1y364ODjnbdw=.dbd5eac3-3f5f-4d96-a46f-cd984a22c4eb@github.com> On Mon, 19 Apr 2021 05:48:44 GMT, David Holmes wrote: > It is still not at all clear to me what suspended has to do with the execution > of this method. The new code just ignores thread suspension. The caller enters a safe state. It can be suspended iff safe, so the old code checked for suspension. With the new code suspension is part of handshake processing, so it is sufficient to check for pending handshakes. > It seems to me that the old code could also have ignored suspension if the > checks in handle_special_runtime_exit_condition had be reordered. In that case JavaThread::java_suspend_self_with_safepoint_check() would have to be changed to check for `Thread::is_obj_deopt_suspend()` and handle it. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 19 07:29:59 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 19 Apr 2021 07:29:59 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 04:43:15 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed nits > > src/hotspot/share/runtime/handshake.hpp line 157: > >> 155: Thread* active_handshaker() const { return _active_handshaker; } >> 156: >> 157: // Suspend/resume support > > Taking a step back I can't help but think that this is entirely the wrong place to have the suspend/resume API and supporting code. It is only here because we re-use the HandshakeState _lock monitor right? If we introduced a new thread-suspension monitor instead then this code would all reside somewhere else - probably in the JavaThread class. When going to blocked inside the async handshake we **must** unlock the HandshakeState lock. Thus _lock.wait() unlocks and gives us something to notify. We could do: _lock.unlock(); { MutexLocker(SomeOtherLock) ml; SomeOtherLock.wait(); } _lock.lock(); Another alternative is to create a friend class which uses the HandshakeState lock and having the API there instead. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 19 07:32:59 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 19 Apr 2021 07:32:59 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 04:47:33 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed nits > > src/hotspot/share/runtime/mutex.hpp line 70: > >> 68: tty = access + 2, >> 69: special = tty + 3, >> 70: oopstorage = special + 3, > > Why +3? I'm assuming to keep the same numeric value as before, but that doesn't seem necessary and just seems to add to the mystery of why these ranks are defined in such a strange way. Yes I wanted to keep the numeric rank. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 19 07:38:10 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 19 Apr 2021 07:38:10 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 06:07:54 GMT, David Holmes wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixed nits > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/Thread.java line 83: > >> 81: } >> 82: >> 83: public boolean isExtSuspended() { > > A new comment to avoid the "resolved" conversation. I'm still not sure you can just delete this API from the SA. @sspitsyn do you know what the obligations are with respect to the S.A here? The SA do not have the HandshakeState class implemented, so there is a lot a work to make this flag work. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 19 08:08:33 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 19 Apr 2021 08:08:33 GMT Subject: RFR: 8257831: Suspend with handshakes [v11] In-Reply-To: References: Message-ID: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: Fixed DavidH second review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3191/files - new: https://git.openjdk.java.net/jdk/pull/3191/files/d6e091da..5a4d52ce Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=10 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=09-10 Stats: 13 lines in 2 files changed: 1 ins; 1 del; 11 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Mon Apr 19 08:22:35 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 19 Apr 2021 08:22:35 GMT Subject: RFR: 8257831: Suspend with handshakes [v12] In-Reply-To: References: Message-ID: <8SeoJUmAJV8_tx52dlAhwTVnlCYMrzqrT7yswrP07Fs=.908b8d5d-e60b-4c7d-8464-5c0c53a0cf3f@github.com> > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: - Merge branch 'master' into SuspendInHandshake - Fixed DavidH second review - Fixed nits - Merge branch 'master' into SuspendInHandshake - Review fixes 4 - Fixed flag undef dep + spelling error - Obsolete unused flags - Review fixes 3 - Merge branch 'master' into SuspendInHandshake - Review fixes 2 - ... and 6 more: https://git.openjdk.java.net/jdk/compare/e390e550...2c652f94 ------------- Changes: https://git.openjdk.java.net/jdk/pull/3191/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=11 Stats: 1351 lines in 40 files changed: 273 ins; 882 del; 196 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From winterhalter at openjdk.java.net Mon Apr 19 09:07:36 2021 From: winterhalter at openjdk.java.net (Rafael Winterhalter) Date: Mon, 19 Apr 2021 09:07:36 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter wrote: >> To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. > > Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8200559: Java agents doing instrumentation need a means to define auxiliary classes This won't work as agents are loaded by the system class loader, not the boot loader. Peter Levart ***@***.***> schrieb am Mo., 19. Apr. 2021, 11:02: > From: Alan Bateman > > JDK-8200559 is about defining auxiliary classes in the same runtime > package at load-time whereas I think the proposal in this PR is adding an > unrestricted defineClass to the Instrumentation class. I think this will > require a lot of discussion as there are significant issues and concerns > here. An unrestricted defineClass might be okay for tool/java agents when > started from the command line with -javaagent but only if the > Instrumentation object is never ever leaked to library or application code. > It could potentially be part of a large effort to reduce the capabilities > of agents loaded via the attach mechanism. More generally I think we need > clearer separation of the requirements of tool agents from the requirement > of framework/libraries that want to inject proxy and other classes at > runtime. > > I think it would be easy to limit the use of this Instrumentation method > to the agent code as agent classes are loaded by the bootstrap classloader. > Simply make the method implementation caller-sensitive and check the caller > is from bootstrap class loader. This way Instrumentation instance escaped > to application code would not give that code any ability to define > arbitrary classes. > > Good enough? > > Peter > > Separately, the proposal in JEP 410 is to terminally deprecate > ProtectionDomain. > > ? > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From plevart at openjdk.java.net Mon Apr 19 09:07:36 2021 From: plevart at openjdk.java.net (Peter Levart) Date: Mon, 19 Apr 2021 09:07:36 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter wrote: >> To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. > > Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8200559: Java agents doing instrumentation need a means to define auxiliary classes From: Alan Bateman > JDK-8200559 is about defining auxiliary classes in the same runtime package at load-time whereas I think the proposal in this PR is adding an unrestricted defineClass to the Instrumentation class. I think this will require a lot of discussion as there are significant issues and concerns here. An unrestricted defineClass might be okay for tool/java agents when started from the command line with -javaagent but only if the Instrumentation object is never ever leaked to library or application code. It could potentially be part of a large effort to reduce the capabilities of agents loaded via the attach mechanism. More generally I think we need clearer separation of the requirements of tool agents from the requirement of framework/libraries that want to inject proxy and other classes at runtime. I think it would be easy to limit the use of this Instrumentation method to the agent code as agent classes are loaded by the bootstrap classloader. Simply make the method implementation caller-sensitive and check the caller is from bootstrap class loader. This way Instrumentation instance escaped to application code would not give that code any ability to define arbitrary classes. Good enough? Peter > > Separately, the proposal in JEP 410 is to terminally deprecate ProtectionDomain. ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From plevart at openjdk.java.net Mon Apr 19 09:14:35 2021 From: plevart at openjdk.java.net (Peter Levart) Date: Mon, 19 Apr 2021 09:14:35 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter wrote: >> To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. > > Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8200559: Java agents doing instrumentation need a means to define auxiliary classes > This won't work as agents are loaded by the system class loader, not the boot loader. Peter Levart ***@***.***> schrieb am Mo., 19. Apr. 2021, 11:02: > [?](#) > From: Alan Bateman JDK-8200559 is about defining auxiliary classes in the same runtime package at load-time whereas I think the proposal in this PR is adding an unrestricted defineClass to the Instrumentation class. I think this will require a lot of discussion as there are significant issues and concerns here. An unrestricted defineClass might be okay for tool/java agents when started from the command line with -javaagent but only if the Instrumentation object is never ever leaked to library or application code. It could potentially be part of a large effort to reduce the capabilities of agents loaded via the attach mechanism. More generally I think we need clearer separation of the requirements of tool agents from the requirement of framework/libraries that want to inject proxy and other classes at runtime. I think it would be easy to limit the use of this Instrumentation method to the agent code as agent classes are loaded by the bootstrap classloader. Simply make the method imp lementation caller-sensitive and check the caller is from bootstrap class loader. This way Instrumentation instance escaped to application code would not give that code any ability to define arbitrary classes. Good enough? Peter Separately, the proposal in JEP 410 is to terminally deprecate ProtectionDomain. ? You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <[#3546 (comment)](https://github.com/openjdk/jdk/pull/3546#issuecomment-822302347)>, or unsubscribe . Ah sorry, I didn't know that. So what about the following: agent is able to define (or load) a class from bootstrap loader. So it would be able to instantiate an intermediary class, loaded by bootstrap loader which would serve as an intermediary to call into this limited Instrumentation API point... Or better: add a Lookup parameter to the Instrumentation method. The implementation would check that the Lookup is an unrestricted Lookup from a class loaded by bootstrap loader. Agent would just have to take the pain of obtaining such Lookup instance... ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From rehn at openjdk.java.net Mon Apr 19 09:26:04 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Mon, 19 Apr 2021 09:26:04 GMT Subject: RFR: 8257831: Suspend with handshakes [v12] In-Reply-To: <8SeoJUmAJV8_tx52dlAhwTVnlCYMrzqrT7yswrP07Fs=.908b8d5d-e60b-4c7d-8464-5c0c53a0cf3f@github.com> References: <8SeoJUmAJV8_tx52dlAhwTVnlCYMrzqrT7yswrP07Fs=.908b8d5d-e60b-4c7d-8464-5c0c53a0cf3f@github.com> Message-ID: On Mon, 19 Apr 2021 08:22:35 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: > > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - Obsolete unused flags > - Review fixes 3 > - Merge branch 'master' into SuspendInHandshake > - Review fixes 2 > - ... and 6 more: https://git.openjdk.java.net/jdk/compare/e390e550...2c652f94 Hi David, > I have gone through everything again in detail. A few comments on comments and a couple of details I'm still not really clear on. > I hope I made things more clear. > A couple of small renaming requests, but otherwise this is good to go - with the caveat of a future RFE to clear up the suspend-while-locking implementations so things are more cleanly encapsulated. > Yes, good. Fixed what I could in this commit, also fixed merged conflict. Thanks, Robbin > Thanks, > David ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From alanb at openjdk.java.net Mon Apr 19 09:37:38 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 19 Apr 2021 09:37:38 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 09:11:34 GMT, Peter Levart wrote: > I think it would be easy to limit the use of this Instrumentation method to the agent code as agent classes are loaded by the bootstrap classloader. Simply make the method implementation caller-sensitive and check the caller is from bootstrap class loader. This way Instrumentation instance escaped to application code would not give that code any ability to define arbitrary classes. The agent JAR file is added to application class path and is loaded using the system class loader. So almost always the defining loader will be the application class loader. In general it's a hard problem to try to balance the integrity and security of the platform with the needs of agents that do arbitrary injection and instrumentation. Specifying an agent on the command line with -javaagent is the opt-in to trust that agent and a defineClass that allows arbitrary injection is plausible for that deployment. As Rafael's mentioned in one of the messages, there is enough power in the existing Instrumentation API to do that in a round about way already. We don't have anything equivalent for agents that are loaded by tools into a target VM. I added the attach mechanism and the dynamic agent loading back in JDK 6 and regret not putting more restrictions around this. As it stands, it is open to mis-use in that an application or library can use the attach mechanism (directly or via the attach API in a child VM) to load an agent and leak the Instrumentation object. This is the genie that somehow needs to be put back in its bottle. One approach that I mentioned here to create is a less powerful Instrumentation object for untrusted agents. Trusted agents would continue to the full-power Instrumentation object. A less powerful Instrumentation object would not be able to redefine java.base or other core modules and would not be able to retransform classes in those modules. The option on the table during JDK 9 was to disable dynamic loading of agents by default but that was kicked down the road. I don't particularly like that option and I th ink we can do better. ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From egahlin at openjdk.java.net Mon Apr 19 10:33:01 2021 From: egahlin at openjdk.java.net (Erik Gahlin) Date: Mon, 19 Apr 2021 10:33:01 GMT Subject: RFR: 8265036: JFR: Remove use of -XX:StartFlightRecording= and -XX:FlightRecorderOptions= Message-ID: Hi, Could I have a review of fix that removes the use of "=" together with -XX:StartFlightRecording and -XX:FlightRecorderOptions. It's been possible to use "-XX:StartFlightRecording:" and "-XX:FlightRecorderOption:" since JFR was introduced into OpenJDK (JDK 11), so this is not a change of the specification, just an update to make the use consistent in tests, comments, documentation etc. I also removed the use of -XX:+FlightRecorder, which is not needed, and has been deprecated since JDK 13. Testing: jdk/jdk/jfr, tier 1-4. Thanks Erik ------------- Commit messages: - Initial Changes: https://git.openjdk.java.net/jdk/pull/3561/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3561&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265036 Stats: 97 lines in 39 files changed: 0 ins; 0 del; 97 mod Patch: https://git.openjdk.java.net/jdk/pull/3561.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3561/head:pull/3561 PR: https://git.openjdk.java.net/jdk/pull/3561 From plevart at openjdk.java.net Mon Apr 19 14:10:34 2021 From: plevart at openjdk.java.net (Peter Levart) Date: Mon, 19 Apr 2021 14:10:34 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 09:34:56 GMT, Alan Bateman wrote: > an application or library can use the attach mechanism (directly or via the attach API in a child VM) to load an agent and leak the Instrumentation object. This is the genie that somehow needs to be put back in its bottle. One approach that I mentioned here to create is a less powerful Instrumentation object for untrusted agents. Trusted agents would continue to the full-power I hear Rafael that dynamic attach is important to support monitoring and instrumenting large numbers of JVMs with no preparations (i.e. without issueing special command-line options to enable it). As I understand, current attach mechanism is designed to allow a process running under the same UID as the JVM or under root to attach to the JVM. What if this dynamic attach mechanism was modified so that only a process running under root could dynamically attach to the JVM? Old behavior would be enabled by special command line option, so by default, dynamic attach would be limited to tools running under root. Rafael mentions discovery, monitoring and instrumenting large number of JVMs running on hosts, so if one such tool has to attach to different JVMs running under different UIDs, it has to run as root now anyway. With such default "secure" dynamic attach and when the JVM is not running as root (which is a recommended security practice anyway), a library in such JVM could not attach back to the same JVM even through spawning sub-processes. How to achieve such "secure" dynamic attach? One way that comes to mind is a modified handshake. Currently, I think at least on Linux, the tool that wishes to attach to the JVM searches for a special UNIX socket (`$PWD/.java_pid`, `/tmp/.java_pid`) and if not found, creates a special attach file (`$PWD/.attach_pid`, `/tmp/.attach_pid`) to signal the JVM to create a listening UNIX socket under mentioned special path, then it connects to the socket. The UNIX socket file has UID:GID set to effective UID:GID of the JVM process and permissions to 0600, so only a tool running under same UID or root can connect to such socket. In modified handshake, JVM not running as root could not create a UNIX socket file with permissions to allow only root user to connect to it, but a tool running under root could create a listening UNIX socket with permission to allow JVM to connect to it in a way that the JVM connecting to such socket would know that the listening process is running as root. Simply by checking the owner of the listening UNIX socket file. Such socket file would have to have permission 0666 in order to allow JVMs running under any UID to connect to it, but otherwise be "hidden". This can be achieved by the tool creating a special directory and a UNIX socket in this directory, say: `/tmp/.attach_dir/`, The directory UID:GID would be 0:0 and have permission 0711. This means, any user could connect to the socket as long as it knows the ``, but no user but root can list the content of the directory to discover the name of the socket file. The last piece of the puzzle is how to signal to the JVM about the name of the socket file. Well, creating a file with the content holding the name of the socket file would be OK, as long as only target JVM could read it. File permissions could be set such that any process running under the same UID as the JVM could read the file. This would give a rouge library a chance to connect to the tool and pretend to be the monitoring JVM, but it could not connect to back to the JVM though... WDYT? ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From Alan.Bateman at oracle.com Mon Apr 19 15:18:15 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 19 Apr 2021 16:18:15 +0100 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: <9988c09f-2c0f-87e9-a936-32c878daf778@oracle.com> On 19/04/2021 15:10, Peter Levart wrote: > : > I hear Rafael that dynamic attach is important to support monitoring and instrumenting large numbers of JVMs with no preparations (i.e. without issueing special command-line options to enable it). As I understand, current attach mechanism is designed to allow a process running under the same UID as the JVM or under root to attach to the JVM. > > What if this dynamic attach mechanism was modified so that only a process running under root could dynamically attach to the JVM? Old behavior would be enabled by special command line option, so by default, dynamic attach would be limited to tools running under root. Rafael mentions discovery, monitoring and instrumenting large number of JVMs running on hosts, so if one such tool has to attach to different JVMs running under different UIDs, it has to run as root now anyway. > > With such default "secure" dynamic attach and when the JVM is not running as root (which is a recommended security practice anyway), a library in such JVM could not attach back to the same JVM even through spawning sub-processes. > > How to achieve such "secure" dynamic attach? One way that comes to mind is a modified handshake. Currently, I think at least on Linux, the tool that wishes to attach to the JVM searches for a special UNIX socket (`$PWD/.java_pid`, `/tmp/.java_pid`) and if not found, creates a special attach file (`$PWD/.attach_pid`, `/tmp/.attach_pid`) to signal the JVM to create a listening UNIX socket under mentioned special path, then it connects to the socket. The UNIX socket file has UID:GID set to effective UID:GID of the JVM process and permissions to 0600, so only a tool running under same UID or root can connect to such socket. > > In modified handshake, JVM not running as root could not create a UNIX socket file with permissions to allow only root user to connect to it, but a tool running under root could create a listening UNIX socket with permission to allow JVM to connect to it in a way that the JVM connecting to such socket would know that the listening process is running as root. Simply by checking the owner of the listening UNIX socket file. Such socket file would have to have permission 0666 in order to allow JVMs running under any UID to connect to it, but otherwise be "hidden". This can be achieved by the tool creating a special directory and a UNIX socket in this directory, say: `/tmp/.attach_dir/`, The directory UID:GID would be 0:0 and have permission 0711. This means, any user could connect to the socket as long as it knows the ``, but no user but root can list the content of the directory to discover the name of the socket file. The last piece of the puzzle > is how to signal to the JVM about the name of the socket file. Well, creating a file with the content holding the name of the socket file would be OK, as long as only target JVM could read it. File permissions could be set such that any process running under the same UID as the JVM could read the file. This would give a rouge library a chance to connect to the tool and pretend to be the monitoring JVM, but it could not connect to back to the JVM though... > One initial comment is that the attach mechanism is also used for the troubleshooting tools (jcmd, jstack, jmap, ...). This is how tools start the JMX agent in the target VM, or request a stack trace or heap dump, to name a few. It's just the loading of JVMTI agents that would need to be restricted (Java agents use the JPLIS JVMTI agent). A second comment is that the attach mechanism on Linux and macOS is based on Unix domain sockets and the effective uid/gid of the peer is available. So if you are exploring here then you should be able to just use that rather than changing the handshake. The Windows implementation injects a stub into the target VM to queue the command so there isn't the equivalent. -Alan From pchilanomate at openjdk.java.net Mon Apr 19 18:22:30 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Mon, 19 Apr 2021 18:22:30 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 15:25:16 GMT, Daniel D. Daugherty wrote: >> If we process the async suspension handshake we can go to safepoint. >> And before safepoint we must drop the tty lock. > > Is this worth a comment above the `break_tty_lock_for_safepoint()` call? I also thought we could remove this since we are already releasing it in the ThreadInVMForHandshake constructor above. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dcubed at openjdk.java.net Mon Apr 19 19:31:34 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 19 Apr 2021 19:31:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v12] In-Reply-To: <8SeoJUmAJV8_tx52dlAhwTVnlCYMrzqrT7yswrP07Fs=.908b8d5d-e60b-4c7d-8464-5c0c53a0cf3f@github.com> References: <8SeoJUmAJV8_tx52dlAhwTVnlCYMrzqrT7yswrP07Fs=.908b8d5d-e60b-4c7d-8464-5c0c53a0cf3f@github.com> Message-ID: <7s-mPFHx9T7eKI6WH0lZbE3eA6MsEUoGcBNPcjLfkDI=.066b8585-1dd0-4c9f-ba4c-c6736eea8b8a@github.com> On Mon, 19 Apr 2021 08:22:35 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: > > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - Obsolete unused flags > - Review fixes 3 > - Merge branch 'master' into SuspendInHandshake > - Review fixes 2 > - ... and 6 more: https://git.openjdk.java.net/jdk/compare/e390e550...2c652f94 One minor nit from a review of incremental v10. Still thumbs up. src/hotspot/share/runtime/handshake.cpp line 636: > 634: JavaThread* thread_current = thr->as_Java_thread(); > 635: assert(thread_current == Thread::current(), "Must be self executed."); > 636: thread_current->handshake_state()->do_self_suspend(); nit - When you know it is the "current" thread, @dholmes-ora has been using just `current` and not `current_thread` or `thread_current`. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From cjplummer at openjdk.java.net Mon Apr 19 20:05:05 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 19 Apr 2021 20:05:05 GMT Subject: RFR: 8265028: JDWP debug agent thread lookup can be made faster In-Reply-To: References: Message-ID: <8XGHt6PSmqSO5n6M9lBf6ecpH9OY6jB6wifQd59mgFY=.78bf1775-594f-459b-b285-3c52f877710a@github.com> On Mon, 12 Apr 2021 20:29:56 GMT, Chris Plummer wrote: > Improved thread lookup speeds in the debug agent, especially when there is a large number of threads. See CR for details. Thank you Alex and Serguei! ------------- PR: https://git.openjdk.java.net/jdk/pull/3444 From cjplummer at openjdk.java.net Mon Apr 19 20:20:07 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 19 Apr 2021 20:20:07 GMT Subject: RFR: 8265036: JFR: Remove use of -XX:StartFlightRecording= and -XX:FlightRecorderOptions= In-Reply-To: References: Message-ID: On Sun, 18 Apr 2021 15:17:35 GMT, Erik Gahlin wrote: > Hi, > > Could I have a review of fix that removes the use of "=" together with -XX:StartFlightRecording and -XX:FlightRecorderOptions. It's been possible to use "-XX:StartFlightRecording:" and "-XX:FlightRecorderOption:" since JFR was introduced into OpenJDK (JDK 11), so this is not a change of the specification, just an update to make the use consistent in tests, comments, documentation etc. > > I also removed the use of -XX:+FlightRecorder, which is not needed, and has been deprecated since JDK 13. > > Testing: jdk/jdk/jfr, tier 1-4. > > Thanks > Erik The changes look good. Have you considered doing a test run where the use of `=` and `XX:+FlightRecorder` are disallowed just to make sure you caught them all? ------------- Marked as reviewed by cjplummer (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3561 From dcubed at openjdk.java.net Mon Apr 19 20:38:03 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 19 Apr 2021 20:38:03 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 00:08:52 GMT, Daniel D. Daugherty wrote: > The synopsis pretty much says it all. There's a more detailed history in the RFE itself. > > Currently running the new test thru Mach5 Tier[1-7]. > I've run the test thru several 12 hour runs on my MBP13 and > on my Linux-X64 server. Ping! Any takers for the new test review? ------------- PR: https://git.openjdk.java.net/jdk/pull/3478 From cjplummer at openjdk.java.net Mon Apr 19 21:16:08 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 19 Apr 2021 21:16:08 GMT Subject: RFR: 8265036: JFR: Remove use of -XX:StartFlightRecording= and -XX:FlightRecorderOptions= In-Reply-To: References: Message-ID: <2-NewUlYrdJ7H67-xrLMo71CB-m2YPTrjJmYVPR7Hi4=.134d7e91-2d75-49f6-b3df-dfaec043fef7@github.com> On Sun, 18 Apr 2021 15:17:35 GMT, Erik Gahlin wrote: > Hi, > > Could I have a review of fix that removes the use of "=" together with -XX:StartFlightRecording and -XX:FlightRecorderOptions. It's been possible to use "-XX:StartFlightRecording:" and "-XX:FlightRecorderOption:" since JFR was introduced into OpenJDK (JDK 11), so this is not a change of the specification, just an update to make the use consistent in tests, comments, documentation etc. > > I also removed the use of -XX:+FlightRecorder, which is not needed, and has been deprecated since JDK 13. > > Testing: jdk/jdk/jfr, tier 1-4. > > Thanks > Erik Sorry, I accidentally stuck an "integrate" label in this PR (was meant for my own PR). I immediately deleted it. Hopefully no harm done. ------------- PR: https://git.openjdk.java.net/jdk/pull/3561 From cjplummer at openjdk.java.net Mon Apr 19 21:18:03 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Mon, 19 Apr 2021 21:18:03 GMT Subject: Integrated: 8265028: JDWP debug agent thread lookup can be made faster In-Reply-To: References: Message-ID: <4M1MH0xiydyq-yoAkmkTfxF-PiIRup4ZPTT4uRGHEnk=.835e20f4-b81a-48f5-b299-38567bea8a8b@github.com> On Mon, 12 Apr 2021 20:29:56 GMT, Chris Plummer wrote: > Improved thread lookup speeds in the debug agent, especially when there is a large number of threads. See CR for details. This pull request has now been integrated. Changeset: e0fd5fc0 Author: Chris Plummer URL: https://git.openjdk.java.net/jdk/commit/e0fd5fc0 Stats: 30 lines in 1 file changed: 8 ins; 10 del; 12 mod 8265028: JDWP debug agent thread lookup can be made faster Reviewed-by: sspitsyn, amenkov ------------- PR: https://git.openjdk.java.net/jdk/pull/3444 From winterhalter at openjdk.java.net Mon Apr 19 21:20:06 2021 From: winterhalter at openjdk.java.net (Rafael Winterhalter) Date: Mon, 19 Apr 2021 21:20:06 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter wrote: >> To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. > > Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8200559: Java agents doing instrumentation need a means to define auxiliary classes At the moment, it is required for root to switch to the user that owns the JVM process as the domain socket is only accessible to that user to avoid that users without access to the JVM can inject themselves into a JVM. I am not sure if operations teams would be thrilled to have a monitoring agent required to run as root, even in these times of Kubernetes. I mainly have two comments: 1. The problem is the possibility of self-attach. I think this is the problem to solve, a library getting agent privileges without being an agent. I think this should be prevented while dynamic attach should continue to be possible in today's format. It has proven to be so useful, it would be a shame if the current tooling convenience would disappear from the JVM. As it's my understanding, JNI is supposed to be restricted in the future, in line with Panama. Without this restriction, JNI already allows for random class definition, for example, which similarly to an agent offers surpassing the majority of JVM restrictions. The second restriction would be a control to restrict how a JVM process starts new processes. I think both are reasonable restrictions for a library to face which require explicit enabling. Especially with the security manager on it's way out, certain capabilities should be rethought to begin with. If both are no longer freely available, self-attachment is no longer possible anyways and dynamic agents could retain their capabilities. 2. The question of introducing an Instrumentation::defineClass method is fully independent of that first question. If a dynamic agent was to be restricted, the method could reject classloader/package combinations for dynamically loaded agents the same way that Instrumentation::retransformClasses would need to. At the same time, introducing the method would allow agents to move to an official API with a Java 17 baseline which will be the next long-standing base line. I fully understand it needs a thorough discussion but it is a less complicated problem then (1) and could therefore be decided prior to having found a satisfactory solution for it. Am Mo., 19. Apr. 2021 um 16:07 Uhr schrieb Peter Levart < ***@***.***>: > an application or library can use the attach mechanism (directly or via > the attach API in a child VM) to load an agent and leak the Instrumentation > object. This is the genie that somehow needs to be put back in its bottle. > One approach that I mentioned here to create is a less powerful > Instrumentation object for untrusted agents. Trusted agents would continue > to the full-power > > I hear Rafael that dynamic attach is important to support monitoring and > instrumenting large numbers of JVMs with no preparations (i.e. without > issueing special command-line options to enable it). As I understand, > current attach mechanism is designed to allow a process running under the > same UID as the JVM or under root to attach to the JVM. > > What if this dynamic attach mechanism was modified so that only a process > running under root could dynamically attach to the JVM? Old behavior would > be enabled by special command line option, so by default, dynamic attach > would be limited to tools running under root. Rafael mentions discovery, > monitoring and instrumenting large number of JVMs running on hosts, so if > one such tool has to attach to different JVMs running under different UIDs, > it has to run as root now anyway. > > With such default "secure" dynamic attach and when the JVM is not running > as root (which is a recommended security practice anyway), a library in > such JVM could not attach back to the same JVM even through spawning > sub-processes. > > How to achieve such "secure" dynamic attach? One way that comes to mind is > a modified handshake. Currently, I think at least on Linux, the tool that > wishes to attach to the JVM searches for a special UNIX socket ( > $PWD/.java_pid, /tmp/.java_pid) and if not found, creates a > special attach file ($PWD/.attach_pid, /tmp/.attach_pid) to > signal the JVM to create a listening UNIX socket under mentioned special > path, then it connects to the socket. The UNIX socket file has UID:GID set > to effective UID:GID of the JVM process and permissions to 0600, so only a > tool running under same UID or root can connect to such socket. > > In modified handshake, JVM not running as root could not create a UNIX > socket file with permissions to allow only root user to connect to it, but > a tool running under root could create a listening UNIX socket with > permission to allow JVM to connect to it in a way that the JVM connecting > to such socket would know that the listening process is running as root. > Simply by checking the owner of the listening UNIX socket file. Such socket > file would have to have permission 0666 in order to allow JVMs running > under any UID to connect to it, but otherwise be "hidden". This can be > achieved by the tool creating a special directory and a UNIX socket in this > directory, say: /tmp/.attach_dir/, The directory > UID:GID would be 0:0 and have permission 0711. This means, any user could > connect to the socket as long as it knows the , but no > user but root can list the content of the directory to discover the name of > the socket file. The last piece of the puzzle is how to signal to the JVM > about the name of the socket file. Well, creating a file with the content > holding the name of the socket file would be OK, as long as only target JVM > could read it. File permissions could be set such that any process running > under the same UID as the JVM could read the file. This would give a rouge > library a chance to connect to the tool and pretend to be the monitoring > JVM, but it could not connect to back to the JVM though... > > WDYT? > > ? > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From rehn at openjdk.java.net Tue Apr 20 06:48:47 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 20 Apr 2021 06:48:47 GMT Subject: RFR: 8257831: Suspend with handshakes [v13] In-Reply-To: References: Message-ID: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 19 commits: - Merge branch 'master' into SuspendInHandshake - Removed double tty lock unlocks - Fixed comment from Dan and fixed name of handshake - Merge branch 'master' into SuspendInHandshake - Fixed DavidH second review - Fixed nits - Merge branch 'master' into SuspendInHandshake - Review fixes 4 - Fixed flag undef dep + spelling error - Obsolete unused flags - ... and 9 more: https://git.openjdk.java.net/jdk/compare/f1d4ae6c...56dc3df0 ------------- Changes: https://git.openjdk.java.net/jdk/pull/3191/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=12 Stats: 1349 lines in 40 files changed: 271 ins; 882 del; 196 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 20 06:48:50 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 20 Apr 2021 06:48:50 GMT Subject: RFR: 8257831: Suspend with handshakes [v12] In-Reply-To: <8SeoJUmAJV8_tx52dlAhwTVnlCYMrzqrT7yswrP07Fs=.908b8d5d-e60b-4c7d-8464-5c0c53a0cf3f@github.com> References: <8SeoJUmAJV8_tx52dlAhwTVnlCYMrzqrT7yswrP07Fs=.908b8d5d-e60b-4c7d-8464-5c0c53a0cf3f@github.com> Message-ID: <37QVdM3ZPotAqRfKd2LYzGq9mb2kj__txXi2hBuvynA=.f1f55dbd-267a-4b15-b08f-5886653607b6@github.com> On Mon, 19 Apr 2021 08:22:35 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: > > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - Obsolete unused flags > - Review fixes 3 > - Merge branch 'master' into SuspendInHandshake > - Review fixes 2 > - ... and 6 more: https://git.openjdk.java.net/jdk/compare/e390e550...2c652f94 Thanks for all input, done done !? :) Re-running test. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Tue Apr 20 06:48:51 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 20 Apr 2021 06:48:51 GMT Subject: RFR: 8257831: Suspend with handshakes [v2] In-Reply-To: References: Message-ID: <0zxUFM8s5JR-XKxLezw1Rht7-J15iIb2VqNIsv2OQBU=.abdf6bd4-6c6c-43fe-8155-92d908f76240@github.com> On Mon, 19 Apr 2021 18:19:05 GMT, Patricio Chilano Mateo wrote: >> Is this worth a comment above the `break_tty_lock_for_safepoint()` call? > > I also thought we could remove this since we are already releasing it in the ThreadInVMForHandshake constructor above. Yes, sorry, removed. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From egahlin at openjdk.java.net Tue Apr 20 07:43:36 2021 From: egahlin at openjdk.java.net (Erik Gahlin) Date: Tue, 20 Apr 2021 07:43:36 GMT Subject: RFR: 8265036: JFR: Remove use of -XX:StartFlightRecording= and -XX:FlightRecorderOptions= [v2] In-Reply-To: References: Message-ID: > Hi, > > Could I have a review of fix that removes the use of "=" together with -XX:StartFlightRecording and -XX:FlightRecorderOptions. It's been possible to use "-XX:StartFlightRecording:" and "-XX:FlightRecorderOption:" since JFR was introduced into OpenJDK (JDK 11), so this is not a change of the specification, just an update to make the use consistent in tests, comments, documentation etc. > > I also removed the use of -XX:+FlightRecorder, which is not needed, and has been deprecated since JDK 13. > > Testing: jdk/jdk/jfr, tier 1-4. > > Thanks > Erik Erik Gahlin has updated the pull request incrementally with one additional commit since the last revision: Replace '=' in jfrOptionsSet.cpp ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3561/files - new: https://git.openjdk.java.net/jdk/pull/3561/files/4ce08939..138bac16 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3561&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3561&range=00-01 Stats: 7 lines in 1 file changed: 0 ins; 0 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3561.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3561/head:pull/3561 PR: https://git.openjdk.java.net/jdk/pull/3561 From egahlin at openjdk.java.net Tue Apr 20 07:55:08 2021 From: egahlin at openjdk.java.net (Erik Gahlin) Date: Tue, 20 Apr 2021 07:55:08 GMT Subject: RFR: 8265036: JFR: Remove use of -XX:StartFlightRecording= and -XX:FlightRecorderOptions= [v2] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 20:16:59 GMT, Chris Plummer wrote: > The changes look good. Have you considered doing a test run where the use of `=` and `XX:+FlightRecorder` are disallowed just to make sure you caught them all? It's a good idea, but it requires changes outside OpenJDK which I rather not do now. I'm sure somebody will reintroduce '=', so this is not so much about getting rid of them all, but to avoid them being propagated, when code is copy pasted for new tests etc. ------------- PR: https://git.openjdk.java.net/jdk/pull/3561 From pliden at openjdk.java.net Tue Apr 20 07:57:07 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 20 Apr 2021 07:57:07 GMT Subject: Integrated: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles In-Reply-To: References: Message-ID: <_xjkRtU8YzJs9Medo7heL5IBS4816UdsaKXdfbuqPn4=.c180d053-ef84-454c-99a0-641067f491db@github.com> On Wed, 14 Apr 2021 07:13:50 GMT, Per Liden wrote: > JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. > > Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. > > Testing: > * Tier1-7 with ZGC > * Manual testing with relevant jtreg tests This pull request has now been integrated. Changeset: 79798c65 Author: Per Liden URL: https://git.openjdk.java.net/jdk/commit/79798c65 Stats: 179 lines in 9 files changed: 107 ins; 25 del; 47 mod 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles Reviewed-by: sspitsyn, ayang ------------- PR: https://git.openjdk.java.net/jdk/pull/3483 From pliden at openjdk.java.net Tue Apr 20 07:57:06 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 20 Apr 2021 07:57:06 GMT Subject: RFR: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles In-Reply-To: References: Message-ID: <4DvtADYdaqAdqNMUObnhpI5j7O_7EfVoerLh7Aih_s0=.10899634-2901-4334-94e4-07c56d627383@github.com> On Thu, 15 Apr 2021 04:20:28 GMT, Serguei Spitsyn wrote: >> JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. >> >> Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. >> >> Testing: >> * Tier1-7 with ZGC >> * Manual testing with relevant jtreg tests > > Hi Per, > It looks good to me. > Thanks, > Serguei Thanks for reviewing @sspitsyn and @albertnetymk! ------------- PR: https://git.openjdk.java.net/jdk/pull/3483 From ysuenaga at openjdk.java.net Tue Apr 20 13:03:29 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 20 Apr 2021 13:03:29 GMT Subject: RFR: 8265505: findsym does not work on remote debug server Message-ID: We can see following error when we run `findsym` on CLHSDB which connects to remote debug server. hsdb> verbose true hsdb> findsym gHotSpotVMTypes 0x00007f913d4a45b0Error: java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor$7.doit(CommandProcessor.java:618) at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2116) at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2086) at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.run(CommandProcessor.java:1957) at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:112) at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:44) at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:282) at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:493) The cause of this NPE is that CDebugger is null. It happens when the debugger is running on debugd server. Command line debugger like CLHSDB can delegate the command to debugd, like `pmap` and `pstack`. `findsym` can also use this scheme. This PR has passed serviceability/sa tests on Linux x64. ------------- Commit messages: - 8265505: findsym does not work on remote debug server Changes: https://git.openjdk.java.net/jdk/pull/3582/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3582&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265505 Stats: 92 lines in 6 files changed: 44 ins; 28 del; 20 mod Patch: https://git.openjdk.java.net/jdk/pull/3582.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3582/head:pull/3582 PR: https://git.openjdk.java.net/jdk/pull/3582 From Alan.Bateman at oracle.com Tue Apr 20 13:35:52 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 20 Apr 2021 14:35:52 +0100 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On 19/04/2021 22:20, Rafael Winterhalter wrote: > : > At the moment, it is required for root to switch to the user that owns the > JVM process as the domain socket is only accessible to that user to avoid > that users without access to the JVM can inject themselves into a JVM. I am > not sure if operations teams would be thrilled to have a monitoring agent > required to run as root, even in these times of Kubernetes. > > I mainly have two comments: > > 1. The problem is the possibility of self-attach. I think this is the > problem to solve, a library getting agent privileges without being an > agent. I think this should be prevented while dynamic attach should > continue to be possible in today's format. It has proven to be so useful, > it would be a shame if the current tooling convenience would disappear from > the JVM. As it's my understanding, JNI is supposed to be restricted in the > future, in line with Panama. Without this restriction, JNI already allows > for random class definition, for example, which similarly to an agent > offers surpassing the majority of JVM restrictions. The second restriction > would be a control to restrict how a JVM process starts new processes. I > think both are reasonable restrictions for a library to face which require > explicit enabling. Especially with the security manager on it's way out, > certain capabilities should be rethought to begin with. If both are no > longer freely available, self-attachment is no longer possible anyways and > dynamic agents could retain their capabilities. > > 2. The question of introducing an Instrumentation::defineClass method is > fully independent of that first question. If a dynamic agent was to be > restricted, the method could reject classloader/package combinations for > dynamically loaded agents the same way that > Instrumentation::retransformClasses would need to. At the same time, > introducing the method would allow agents to move to an official API with a > Java 17 baseline which will be the next long-standing base line. I fully > understand it needs a thorough discussion but it is a less complicated > problem then (1) and could therefore be decided prior to having found a > satisfactory solution for it. I should have been clearer, it's the combination of the two that creates the attractive nuisance. I don't think there are any objections to a defineClass for agents specified on the command line with -javaagent. However we have to be cautious about extending that capability to agents that are loaded into a running VM with the attach mechanism. ByteBuddy looks great for code generation and transforming classes but ByteBuddyAgent makes me nervous. It looks like I can deploy byte-buddy-agent-.jar on my class path and invoke the public static ByteBuddyAgent.install() method to get the Instrumentation object for the current VM. That may be convenient for some but this is the all-powerful Instrumentation object that shouldn't be leaked to library or application code. Now combine this with the proposed defineClass and it means that any code on the class path could inject a class into java.lang or any run-time package without any agent voodoo or opt-in via the command line. That would be difficult genie to re-bottle if it were to get traction. You mentioned restricting JNI in the future. I'm not aware of a definite plan or time-frame. Project Panama is pioneering restricting access to native operations as a bug or mis-use with the linker API can easily crash the VM or breakage in other ways. Extending this to JNI would be a logical next step but I could imagine it taking a long time and many releases to get there. As regards this PR then I would be happy to work with you on a revised proposed that would limit it to agents specified with -javaagent. That would not preclude extending the capability, maybe in a more restricted form, to agents loaded into a running VM in the future. -Alan. From github.com+70650887+skodanda at openjdk.java.net Tue Apr 20 15:02:28 2021 From: github.com+70650887+skodanda at openjdk.java.net (K Suman Rajkumaar) Date: Tue, 20 Apr 2021 15:02:28 GMT Subject: RFR: JDK-8260690: JConsole User Guide Link from the Help menu is not accessible by keyboard Message-ID: Hi All, Could you please review this JConsole fix for JDK-17? Problem: JConsole User Guide Link from the Help menu is not accessible by keyboard. Fix: The JConsole User Guide Link can be highlighted and can now be accessed using keyboard. Using the TAB key, the link can be highlighted and ENTER or SPACE key will browse the link. ------------- Commit messages: - Update AboutDialog.java - Deleted tabs - Fix - JDK-8260690 JConsole User Guide Link from the Help menu is not accessible by keyboard. Changes: https://git.openjdk.java.net/jdk/pull/3587/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3587&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8260690 Stats: 122 lines in 1 file changed: 108 ins; 0 del; 14 mod Patch: https://git.openjdk.java.net/jdk/pull/3587.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3587/head:pull/3587 PR: https://git.openjdk.java.net/jdk/pull/3587 From dcubed at openjdk.java.net Tue Apr 20 15:49:31 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 20 Apr 2021 15:49:31 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating Message-ID: I'm updating the runtime/Thread/SuspendAtExit.java test: - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() - switch from counter-based to time-based testing - improve error checking since we're now using an API with error codes! I've used this test to stress @robehn's fix for JDK-8257831 using both invocation styles for 9 hours each in {fastdebug, release, slowdebug} configs without any issues. I've run the updated test thru Mach5 Tier[134567] testing; one timeout was observed in a single Tier6 run on Win-X64. I believe this is a case of a lost Thread.interrupt() call. ------------- Commit messages: - 8265240: runtime/Thread/SuspendAtExit.java needs updating Changes: https://git.openjdk.java.net/jdk/pull/3576/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3576&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265240 Stats: 213 lines in 2 files changed: 165 ins; 21 del; 27 mod Patch: https://git.openjdk.java.net/jdk/pull/3576.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3576/head:pull/3576 PR: https://git.openjdk.java.net/jdk/pull/3576 From rehn at openjdk.java.net Tue Apr 20 15:49:31 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Tue, 20 Apr 2021 15:49:31 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 19:15:18 GMT, Daniel D. Daugherty wrote: > I'm updating the runtime/Thread/SuspendAtExit.java test: > > - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() > - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() > - switch from counter-based to time-based testing > - improve error checking since we're now using an API with error codes! > > I've used this test to stress @robehn's fix for JDK-8257831 using both > invocation styles for 9 hours each in {fastdebug, release, slowdebug} > configs without any issues. > > I've run the updated test thru Mach5 Tier[134567] testing; one timeout > was observed in a single Tier6 run on Win-X64. I believe this is a case of > a lost Thread.interrupt() call. Thanks for fixing. Oh only in draft state ;) ------------- Marked as reviewed by rehn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3576 From dcubed at openjdk.java.net Tue Apr 20 15:49:31 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 20 Apr 2021 15:49:31 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 19:15:18 GMT, Daniel D. Daugherty wrote: > I'm updating the runtime/Thread/SuspendAtExit.java test: > > - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() > - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() > - switch from counter-based to time-based testing > - improve error checking since we're now using an API with error codes! > > I've used this test to stress @robehn's fix for JDK-8257831 using both > invocation styles for 9 hours each in {fastdebug, release, slowdebug} > configs without any issues. > > I've run the updated test thru Mach5 Tier[134567] testing; one timeout > was observed in a single Tier6 run on Win-X64. I believe this is a case of > a lost Thread.interrupt() call. Oops. Forgot to mark it ready for review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3576 From egahlin at openjdk.java.net Tue Apr 20 15:58:06 2021 From: egahlin at openjdk.java.net (Erik Gahlin) Date: Tue, 20 Apr 2021 15:58:06 GMT Subject: Integrated: 8265036: JFR: Remove use of -XX:StartFlightRecording= and -XX:FlightRecorderOptions= In-Reply-To: References: Message-ID: <-SX_kKhJgQjALbNUDSfjyWa5IETaK5rUHwewSUm7zhg=.9c50555e-5afc-4499-a39a-d6ff860cfd6c@github.com> On Sun, 18 Apr 2021 15:17:35 GMT, Erik Gahlin wrote: > Hi, > > Could I have a review of fix that removes the use of "=" together with -XX:StartFlightRecording and -XX:FlightRecorderOptions. It's been possible to use "-XX:StartFlightRecording:" and "-XX:FlightRecorderOption:" since JFR was introduced into OpenJDK (JDK 11), so this is not a change of the specification, just an update to make the use consistent in tests, comments, documentation etc. > > I also removed the use of -XX:+FlightRecorder, which is not needed, and has been deprecated since JDK 13. > > Testing: jdk/jdk/jfr, tier 1-4. > > Thanks > Erik This pull request has now been integrated. Changeset: 4dcaac1f Author: Erik Gahlin URL: https://git.openjdk.java.net/jdk/commit/4dcaac1f Stats: 104 lines in 40 files changed: 0 ins; 0 del; 104 mod 8265036: JFR: Remove use of -XX:StartFlightRecording= and -XX:FlightRecorderOptions= Reviewed-by: cjplummer ------------- PR: https://git.openjdk.java.net/jdk/pull/3561 From pchilanomate at openjdk.java.net Tue Apr 20 18:12:31 2021 From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo) Date: Tue, 20 Apr 2021 18:12:31 GMT Subject: RFR: 8257831: Suspend with handshakes [v13] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 06:48:47 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 19 commits: > > - Merge branch 'master' into SuspendInHandshake > - Removed double tty lock unlocks > - Fixed comment from Dan and fixed name of handshake > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - Obsolete unused flags > - ... and 9 more: https://git.openjdk.java.net/jdk/compare/f1d4ae6c...56dc3df0 Latest version LGTM! Thanks, Patricio ------------- Marked as reviewed by pchilanomate (Committer). PR: https://git.openjdk.java.net/jdk/pull/3191 From cjplummer at openjdk.java.net Tue Apr 20 18:17:04 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 20 Apr 2021 18:17:04 GMT Subject: RFR: 8265505: findsym does not work on remote debug server In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 07:31:43 GMT, Yasumasa Suenaga wrote: > We can see following error when we run `findsym` on CLHSDB which connects to remote debug server. > > > hsdb> verbose true > hsdb> findsym gHotSpotVMTypes > 0x00007f913d4a45b0Error: java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null > java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null > at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor$7.doit(CommandProcessor.java:618) > at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2116) > at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2086) > at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.run(CommandProcessor.java:1957) > at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:112) > at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:44) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:282) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:493) > > > The cause of this NPE is that CDebugger is null. It happens when the debugger is running on debugd server. > Command line debugger like CLHSDB can delegate the command to debugd, like `pmap` and `pstack`. `findsym` can also use this scheme. > > This PR has passed serviceability/sa tests on Linux x64. src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/CommandProcessor.java line 603: > 601: } else { > 602: String result = VM.getVM().getDebugger().findSymbol(t.nextToken()); > 603: out.println(result == null ? "Symbol not found" : result); The import of sun.jvm.hotspot.debugger.cdbg.CDebugger is no longer needed in this file. I think the more traditional approach would be to add remote support for LoadObject. That provides the building blocks needed here. It's harder to do, but might have some benefit down the road, although it's hard to say if there would ever be other users of it. ------------- PR: https://git.openjdk.java.net/jdk/pull/3582 From winterhalter at openjdk.java.net Tue Apr 20 20:26:04 2021 From: winterhalter at openjdk.java.net (Rafael Winterhalter) Date: Tue, 20 Apr 2021 20:26:04 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: <-fco_my21rYwE8RlEaFlsc-g0WL_rPzbLBWllNqY2rw=.9782b940-c827-4dad-bba8-6901f75fdcf8@github.com> On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter wrote: >> To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. > > Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8200559: Java agents doing instrumentation need a means to define auxiliary classes I fully understand your concerns about ByteBuddyAgent.install(). It is simply a convenience for something that can be meaningful in some contexts where I prefer offering a simple API. I use it mainly for two purposes: a) For testing Java agents and integrations against Instrumentation within the current VM when tests are triggered by tools that do not support javaagents, also because builds do not bundle jars until after tests are executed. b) For purposefully "hacky" test libraries like Mockito that need agent capabilities without this being meant to be used in production environments. I have earlier proposed to offer a "jdk.test" module that offers the Instrumentation instance via a simple API similar to Byte Buddy's. The JVM would not load this module unless requested on the command line. Build tools like Maven's surefire or Gradle's testrunner could then standardize on loading this module as a convention to give access to this test module by default such that libraries like Mockito could continue to function out of the box without the libraries functioning on a standard VM without extra configuration. As far as I know, mainly test libraries need this API. This would also emphasise that Mockito and others are meant for testing and fewer people would abuse it for production applications. People would also have an explicit means of running a JVM for a production application or for executing a test. As for adding the API, my thought is that if the Instrumentation API were to throw exceptions on some methods/arguments for dynamic agents in the future, for example for retransformClasses(Object.class), this breaking change would then simply extend to the proposed "defineClass" method. In this sense, the Instrumentation API already assumes full power, I find it not problematic to add the missing bit to this API even if it was restricted in the future in the same spirit as other methods of the API would be. I mentioned JNI as it is a well-known approach to defining a class today, using a minimal native binding to an interface that directly calls down to JNI's: jclass DefineClass(JNIEnv *env, const char *name, jobject loader, const jbyte *buf, jsize bufLen); This interface can then simply be used to define any class just as I propse, even when not writing an agent or attaching. This method makes class definitions also already trivial for JVMTI agents compared to Java agents. Unless restricting JNI, the defineClass method is already a low hanging fruit, but at the cost of having to maintain a tiny bit of native code. I'd rather see this avoided and a standard API being offered to agents up to the time that Panama is in place and a JNI restriction is possibly also included. As a bonus: Once JNI is restricted, Byte Buddy's "install" would no longer work unless self-attachment (or JNI) is explicitly allowed. The emulation already requires to run native code while the Virtual Machine API explicitly checks for the process id of the current VM against the one that is targeted. With both disabled, self-attachment would no longer be practically be possible without needing to prune the capabilities of dynamic agents which is what I understand would be the desired effect. >From this viewpoint, I think that adding Instrumentation::defineClass method does no harm compared to the status quo. And on the upside, it gives agents an API to migrate to, avoiding the last need of using unsafe. To make the JVM a safe platform, binding native code would anyways need restriction and this would then also solve the problem of dynamic agents attaching from the same VM being used in libraries. This would in my eyes be the cleanest solution to the self-attachment problem without disturbing the existing landscape of dynamic agents. To run Mockito, one would then instead configure Maven surefire or Gradle to run the JVM with -Djdk.attach.allowAttachSelf=true. Ideally, some "jdk.test" module would be added at some point, to avoid the overhead of self-attachment, but I think this better fits into separate debate. Am Di., 20. Apr. 2021 um 15:38 Uhr schrieb mlbridge[bot] < ***@***.***>: > *Mailing list message from Alan Bateman ***@***.***> on > core-libs-dev ***@***.***>:* > > On 19/04/2021 22:20, Rafael Winterhalter wrote: > > : > At the moment, it is required for root to switch to the user that owns the > JVM process as the domain socket is only accessible to that user to avoid > that users without access to the JVM can inject themselves into a JVM. I am > not sure if operations teams would be thrilled to have a monitoring agent > required to run as root, even in these times of Kubernetes. > > I mainly have two comments: > > 1. The problem is the possibility of self-attach. I think this is the > problem to solve, a library getting agent privileges without being an > agent. I think this should be prevented while dynamic attach should > continue to be possible in today's format. It has proven to be so useful, > it would be a shame if the current tooling convenience would disappear from > the JVM. As it's my understanding, JNI is supposed to be restricted in the > future, in line with Panama. Without this restriction, JNI already allows > for random class definition, for example, which similarly to an agent > offers surpassing the majority of JVM restrictions. The second restriction > would be a control to restrict how a JVM process starts new processes. I > think both are reasonable restrictions for a library to face which require > explicit enabling. Especially with the security manager on it's way out, > certain capabilities should be rethought to begin with. If both are no > longer freely available, self-attachment is no longer possible anyways and > dynamic agents could retain their capabilities. > > 2. The question of introducing an Instrumentation::defineClass method is > fully independent of that first question. If a dynamic agent was to be > restricted, the method could reject classloader/package combinations for > dynamically loaded agents the same way that > Instrumentation::retransformClasses would need to. At the same time, > introducing the method would allow agents to move to an official API with a > Java 17 baseline which will be the next long-standing base line. I fully > understand it needs a thorough discussion but it is a less complicated > problem then (1) and could therefore be decided prior to having found a > satisfactory solution for it. > > I should have been clearer, it's the combination of the two that creates > the attractive nuisance. I don't think there are any objections to a > defineClass for agents specified on the command line with -javaagent. > However we have to be cautious about extending that capability to agents > that are loaded into a running VM with the attach mechanism. > > ByteBuddy looks great for code generation and transforming classes but > ByteBuddyAgent makes me nervous. It looks like I can deploy > byte-buddy-agent-.jar on my class path and invoke the public > static ByteBuddyAgent.install() method to get the Instrumentation object > for the current VM. That may be convenient for some but this is the > all-powerful Instrumentation object that shouldn't be leaked to library > or application code. Now combine this with the proposed defineClass and > it means that any code on the class path could inject a class into > java.lang or any run-time package without any agent voodoo or opt-in via > the command line. That would be difficult genie to re-bottle if it were > to get traction. > > You mentioned restricting JNI in the future. I'm not aware of a definite > plan or time-frame. Project Panama is pioneering restricting access to > native operations as a bug or mis-use with the linker API can easily > crash the VM or breakage in other ways. Extending this to JNI would be a > logical next step but I could imagine it taking a long time and many > releases to get there. > > As regards this PR then I would be happy to work with you on a revised > proposed that would limit it to agents specified with -javaagent. That > would not preclude extending the capability, maybe in a more restricted > form, to agents loaded into a running VM in the future. > > -Alan. > > ? > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From cjplummer at openjdk.java.net Tue Apr 20 21:54:13 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 20 Apr 2021 21:54:13 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 12:35:27 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 25 commits: > > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - code clean > - fix trailing white space issue > - ... and 15 more: https://git.openjdk.java.net/jdk/compare/e2106d5a...997784b1 > Hi Chris, > I am going to close the CSR https://bugs.openjdk.java.net/browse/JDK-8260424. > But it seems this PR also revised the `parallel` option's help message for `jmap -histo`, Do you think it is ok to keep the change in the PR? > > FYI, here is the commits [6789ba2](https://github.com/openjdk/jdk/commit/6789ba291a97a01fdc70258fb25dbf0d6464dba8) > > Thanks, > Lin I think it would be best to create a separate PR for that change. No CSR is needed for it. ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From cjplummer at openjdk.java.net Tue Apr 20 21:58:26 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 20 Apr 2021 21:58:26 GMT Subject: RFR: 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out [v8] In-Reply-To: References: <_4pksW9FB4VgM4sosjHs7q9lbPZtyGGy_wc_G53zLVg=.918f3888-1f52-4d30-bf1c-38e344af9e20@github.com> Message-ID: On Wed, 14 Apr 2021 12:35:33 GMT, Lin Zang wrote: >> 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision: > > - Merge branch 'master' > - Merge branch 'master' into sf > - rename writeThrough to unbufferedMode and code refine > - fix typo in comments > - Merge branch 'master' into sf > - Revert "reduce memory consumption" > > This reverts commit 70e43ddd453724ce36bf729fa6489c0027957b8e. > - reduce memory consumption > - code refine > - 8262386: resourcehogs/serviceability/sa/TestHeapDumpForLargeArray.java timed out I think it's fine to fix this issue first with something simpler, and worry about the rewrite later. ------------- PR: https://git.openjdk.java.net/jdk/pull/2803 From ysuenaga at openjdk.java.net Tue Apr 20 23:18:43 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 20 Apr 2021 23:18:43 GMT Subject: RFR: 8265505: findsym does not work on remote debug server [v2] In-Reply-To: References: Message-ID: > We can see following error when we run `findsym` on CLHSDB which connects to remote debug server. > > > hsdb> verbose true > hsdb> findsym gHotSpotVMTypes > 0x00007f913d4a45b0Error: java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null > java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null > at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor$7.doit(CommandProcessor.java:618) > at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2116) > at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2086) > at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.run(CommandProcessor.java:1957) > at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:112) > at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:44) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:282) > at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:493) > > > The cause of this NPE is that CDebugger is null. It happens when the debugger is running on debugd server. > Command line debugger like CLHSDB can delegate the command to debugd, like `pmap` and `pstack`. `findsym` can also use this scheme. > > This PR has passed serviceability/sa tests on Linux x64. Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Remove unused imports ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3582/files - new: https://git.openjdk.java.net/jdk/pull/3582/files/b6bbadac..6ef93d7d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3582&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3582&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3582.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3582/head:pull/3582 PR: https://git.openjdk.java.net/jdk/pull/3582 From ysuenaga at openjdk.java.net Tue Apr 20 23:36:05 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 20 Apr 2021 23:36:05 GMT Subject: RFR: 8265505: findsym does not work on remote debug server [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 18:14:14 GMT, Chris Plummer wrote: >> Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove unused imports > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/CommandProcessor.java line 603: > >> 601: } else { >> 602: String result = VM.getVM().getDebugger().findSymbol(t.nextToken()); >> 603: out.println(result == null ? "Symbol not found" : result); > > The import of sun.jvm.hotspot.debugger.cdbg.CDebugger is no longer needed in this file. > > I think the more traditional approach would be to add remote support for LoadObject. That provides the building blocks needed here. It's harder to do, but might have some benefit down the road, although it's hard to say if there would ever be other users of it. I removed both CDebugger and LoadObject from CommandProcessor.java in new commit. > I think the more traditional approach would be to add remote support for LoadObject. That provides the building blocks needed here. It's harder to do, but might have some benefit down the road, although it's hard to say if there would ever be other users of it. Agree. I saw similar issue in `dis`, `disassemble`, `dumpcodecache`, and `livenmethods`. I hope they will be fixed, and I believe remote support for LoadObject helps it. (They can be fixed in Disassembler.java) I think LoadObject should be Serializable if we do so, but it might make big impact for the classes which implements LoadObject (e.g. DSO.java). It might be harder and we might have no more users for it as you said, so I decided to delegate the process to remote debug server. ------------- PR: https://git.openjdk.java.net/jdk/pull/3582 From cjplummer at openjdk.java.net Tue Apr 20 23:55:14 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 20 Apr 2021 23:55:14 GMT Subject: RFR: 8265505: findsym does not work on remote debug server [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 23:18:43 GMT, Yasumasa Suenaga wrote: >> We can see following error when we run `findsym` on CLHSDB which connects to remote debug server. >> >> >> hsdb> verbose true >> hsdb> findsym gHotSpotVMTypes >> 0x00007f913d4a45b0Error: java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null >> java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null >> at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor$7.doit(CommandProcessor.java:618) >> at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2116) >> at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2086) >> at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.run(CommandProcessor.java:1957) >> at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:112) >> at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:44) >> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:282) >> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:493) >> >> >> The cause of this NPE is that CDebugger is null. It happens when the debugger is running on debugd server. >> Command line debugger like CLHSDB can delegate the command to debugd, like `pmap` and `pstack`. `findsym` can also use this scheme. >> >> This PR has passed serviceability/sa tests on Linux x64. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Remove unused imports Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3582 From cjplummer at openjdk.java.net Wed Apr 21 02:01:17 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 21 Apr 2021 02:01:17 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 12:35:27 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 25 commits: > > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - code clean > - fix trailing white space issue > - ... and 15 more: https://git.openjdk.java.net/jdk/compare/e2106d5a...997784b1 Ok made a closer pass this time. Overall looks good except for some suggested edits, but I couldn't review all of it in detail. What type of testing are you doing? src/hotspot/share/services/attachListener.cpp line 252: > 250: // parallel thread number for heap dump, initialize based on active processor count. > 251: // Note the real number of threads used is also determined by active workers and compression > 252: // backend thread number, see heapDumper.cpp. The commas should really be periods to start new sentences. src/hotspot/share/services/heapDumper.cpp line 579: > 577: // this is already the correct length, since we don't add more sub-records. > 578: write_u4(len); > 579: assert (Bytes::get_Java_u4((address)(buffer() + 5)) == len, "Inconsitent size!"); Remove space. src/hotspot/share/services/heapDumper.cpp line 701: > 699: size_t _internal_buffer_used; > 700: char* _buffer_base; > 701: bool _splited_data; `_split_data`. Split is one of those english verbs that doesn't follow the rules. "I split the log yesterday" is correct, not "splited". src/hotspot/share/services/heapDumper.cpp line 721: > 719: > 720: public: > 721: ParDumpWriter(DumpWriter* dw) : The DumpWriter constructor has the following comment, which you should add here also: `// Check for error after constructing the object and destroy it in case of an error.` src/hotspot/share/services/heapDumper.cpp line 737: > 735: if (_buffer_base != NULL) { > 736: os::free(_buffer_base); > 737: _buffer_base = NULL; remove extra indentation space src/hotspot/share/services/heapDumper.cpp line 853: > 851: > 852: void flush_to_backend(bool force) { > 853: // Guarantee there is only one writer update backend buffers. "updating the backend buffers"? src/hotspot/share/services/heapDumper.cpp line 862: > 860: entry = NULL; > 861: } > 862: assert(_pos == 0, "available buffer must be clean before flush"); "empty" instead of "clean"? src/hotspot/share/services/heapDumper.cpp line 1665: > 1663: // To avoid memory consumption, when dumping large objects such as huge array and > 1664: // large objects whose size are larger than LARGE_OBJECT_DUMP_THRESHOLD, the scanned > 1665: // partial object/array data will be send to the backend directly instead of caching "sent" src/hotspot/share/services/heapDumper.cpp line 1668: > 1666: // the whole object/array in the internal buffer. > 1667: // The HeapDumpLargeObjectList is used to save the large object when dumper scans > 1668: // the heap. The large objects could be added (push) parallelly by multiple dumpers. "dumpers," src/hotspot/share/services/heapDumper.cpp line 1669: > 1667: // The HeapDumpLargeObjectList is used to save the large object when dumper scans > 1668: // the heap. The large objects could be added (push) parallelly by multiple dumpers. > 1669: // But they will be removed (pop) serially only by the VM thread. "popped" src/hotspot/share/services/heapDumper.cpp line 1690: > 1688: HeapDumpLargeObjectListElem* entry = new HeapDumpLargeObjectListElem(obj); > 1689: if (entry == NULL) { > 1690: warning("Fail to allocate element for large object list"); "failed" src/hotspot/share/services/heapDumper.cpp line 1774: > 1772: } > 1773: > 1774: // If large object list exist and it is large object/array, "exists" src/hotspot/share/services/heapDumper.cpp line 1775: > 1773: > 1774: // If large object list exist and it is large object/array, > 1775: // add oop into the list and skip scan, VM thread will process it later. "...scan. VM thread..." src/hotspot/share/services/heapDumper.cpp line 1842: > 1840: ml.wait(); > 1841: } > 1842: assert(_started == true, "dumper is waken up with wrong state"); "dumper work up" src/hotspot/share/services/heapDumper.cpp line 1918: > 1916: // Calculate dumper and writer threads number. > 1917: _num_writer_threads = num_total - _num_dumper_threads; > 1918: // If dumper threads number is zero, there is only VMThread work as a dumper. "If dumper threads number is zero, only the VMThread works as a dumper." src/hotspot/share/services/heapDumper.cpp line 1919: > 1917: _num_writer_threads = num_total - _num_dumper_threads; > 1918: // If dumper threads number is zero, there is only VMThread work as a dumper. > 1919: // If dumper threads number is equal to active workers, need at lest one thread work as writer. "...at least one worker thread..."? src/hotspot/share/services/heapDumperCompression.cpp line 240: > 238: } > 239: > 240: void CompressionBackend::flush_buffer() { This method looks to be indentical to `CompressionBackend::deactivate()` except `CompressionBackend::deactivate()` does a couple more tasks at the end. Perhaps `CompressionBackend::deactivate()` should be calling this now. src/hotspot/share/services/heapDumperCompression.cpp line 404: > 402: assert(buffer != NULL && used != 0 && max != 0, "Invalid data send to compression backend"); > 403: assert(_active == true, "Backend must be active when flushing external buffer"); > 404: // force flush buffered data and get new buffer. This is an odd location for this comment. I think i should go before the method and also match the (modified) comment found in the .hpp file. src/hotspot/share/services/heapDumperCompression.cpp line 410: > 408: > 409: MonitorLocker ml(_lock, Mutex::_no_safepoint_check_flag); > 410: // first try current work. if it is clean Do you mean "First try current work buffer. Use it if empty."? src/hotspot/share/services/heapDumperCompression.cpp line 417: > 415: get_new_buffer(&buf, &tmp_used, &tmp_max, true); > 416: } > 417: assert (_current != NULL && _current->_in != NULL && _current->_in_max >= max && `_current` can't possibly be NULL here since you referenced `_current->_in_used` a few lines above. src/hotspot/share/services/heapDumperCompression.hpp line 221: > 219: char const* error() const { return _err; } > 220: > 221: // sets up a internal buffer, fill with external buffer and send to compressor // Sets up a internal buffer, fills with external buffer, and sends to compressor. src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java line 232: > 230: if (compress_level.length() == 0) { > 231: System.err.println("Fail: no number provided in option: '" + subopt + "'"); > 232: usage(1); I think lines 231 and 232 have an extra indentation of 1 space. Since this change is unrelated to the CR being fixed, perhaps it would be best to do these changes with the new CR you are going to create for the help output changes below. ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From lzang at openjdk.java.net Wed Apr 21 02:24:25 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 21 Apr 2021 02:24:25 GMT Subject: RFR: 8265612: revise the help info for jmap histo command Message-ID: This PR revise the help message of the `parallel=[N]` option of command `jmap -histo`. It mainly comes from the discussion at https://github.com/openjdk/jdk/pull/2379#issuecomment-782609582 and https://github.com/openjdk/jdk/pull/2261#issuecomment-823623186. ------------- Commit messages: - 8265612: revise the help info for jmap histo command Changes: https://git.openjdk.java.net/jdk/pull/3598/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3598&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265612 Stats: 7 lines in 1 file changed: 0 ins; 0 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3598.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3598/head:pull/3598 PR: https://git.openjdk.java.net/jdk/pull/3598 From lzang at openjdk.java.net Wed Apr 21 02:28:13 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 21 Apr 2021 02:28:13 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 12:35:27 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 25 commits: > > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - code clean > - fix trailing white space issue > - ... and 15 more: https://git.openjdk.java.net/jdk/compare/e2106d5a...997784b1 Dear Chris, Thanks a lot for your help, I have created a new PR for the `jmap -histo:parallel=n` help message at https://github.com/openjdk/jdk/pull/3598. And I will revert it in this PR and also revise the patch based on your latest comments. The tests I have conducted are quite similar with what Ralf have done and suggested: > > ``` > jmap.exe -dump:file=ser.hprof,all 30600 > Dumping heap to ser.hprof ... > Heap dump file created [32009882362 bytes in 59.303 secs] > > jmap.exe -dump:file=par1.hprof,parallel=1,all 30600 > Dumping heap to par1.hprof ... > Heap dump file created [32009885809 bytes in 72.719 secs] > > jmap.exe -dump:file=par2.hprof,parallel=2,all 30600 > Dumping heap to par2.hprof ... > Heap dump file created [32009881876 bytes in 57.546 secs] > > jmap.exe -dump:file=par4.hprof,parallel=4,all 30600 > Dumping heap to par4.hprof ... > Heap dump file created [32009882956 bytes in 44.301 secs] > > jmap.exe -dump:file=par5.hprof,parallel=5,all 30600 > Dumping heap to par5.hprof ... > Heap dump file created [32009882164 bytes in 40.282 secs] > > jmap.exe -dump:file=par6.hprof,parallel=6,all 30600 > Dumping heap to par6.hprof ... > Heap dump file created [32009881156 bytes in 45.988 secs] > ``` > > And here for the compressed case: > > ``` > jmap.exe -dump:file=par1.hprof.gz,parallel=1,all,gz=1 52372 > Dumping heap to par1.hprof.gz ... > Heap dump file created [8076994216 bytes in 54.057 secs] > > jmap.exe -dump:file=par2.hprof.gz,parallel=2,all,gz=1 52372 > Dumping heap to par2.hprof.gz ... > Heap dump file created [8075859421 bytes in 43.442 secs] > > jmap.exe -dump:file=par4.hprof.gz,parallel=4,all,gz=1 52372 > Dumping heap to par4.hprof.gz ... > Heap dump file created [8075886152 bytes in 28.710 secs] > > jmap.exe -dump:file=par6.hprof.gz,parallel=6,all,gz=1 52372 > Dumping heap to par6.hprof.gz ... > Heap dump file created [8075758374 bytes in 25.730 secs] > > jmap.exe -dump:file=par8.hprof.gz,parallel=8,all,gz=1 52372 > Dumping heap to par8.hprof.gz ... > Heap dump file created [8075652558 bytes in 26.039 secs] > > jmap.exe -dump:file=par16.hprof.gz,parallel=16,all,gz=1 52372 > Dumping heap to par16.hprof.gz ... > Heap dump file created [8075644423 bytes in 31.977 secs] > > jmap.exe -dump:file=par24.hprof.gz,parallel=24,all,gz=1 52372 > Dumping heap to par24.hprof.gz ... > Heap dump file created [8075579546 bytes in 41.094 secs] > ``` And I will conduct it again since now there is no `parallel=` option. BRs, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From dholmes at openjdk.java.net Wed Apr 21 04:35:29 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 21 Apr 2021 04:35:29 GMT Subject: RFR: 8257831: Suspend with handshakes [v13] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 06:48:47 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 19 commits: > > - Merge branch 'master' into SuspendInHandshake > - Removed double tty lock unlocks > - Fixed comment from Dan and fixed name of handshake > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - Obsolete unused flags > - ... and 9 more: https://git.openjdk.java.net/jdk/compare/f1d4ae6c...56dc3df0 Looks good. src/hotspot/share/runtime/handshake.cpp line 633: > 631: void do_thread(Thread* thr) { > 632: JavaThread* current = thr->as_Java_thread(); > 633: assert(current == Thread::current(), "Must be self executed."); Possibly overkill given do_self_suspend() has a similar assertion. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dholmes at openjdk.java.net Wed Apr 21 04:39:23 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 21 Apr 2021 04:39:23 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 07:13:57 GMT, Robbin Ehn wrote: >> src/hotspot/share/runtime/handshake.hpp line 133: >> >>> 131: // check the _lock, if held go to slow path. >>> 132: // Since the handshakee is unsafe if _lock gets locked after this check >>> 133: // we know another thread cannot process any handshakes. >> >> I can't quite understand this comment. I'm not sure what thread is calling this method and when, in relation to what the handshakee may be doing. > > The handshakee is in a safe state, e.g. blocked and does: > Point A: set_thread_state_fence(_thread_blocked_trans); > ... > Point B: _lock.is_locked() > > While the processor thread does: > Point X: _lock.try_lock(); > ... > Point Z: thread->thread_state(); > > If point B is passed with a non-locked lock, point Z is guaranteed to see an unsafe thread and will not start to process ay handshakes. > > Is that make sense, maybe the comment make more sense ? :) The comment makes more sense now :) >> src/hotspot/share/runtime/handshake.hpp line 157: >> >>> 155: Thread* active_handshaker() const { return _active_handshaker; } >>> 156: >>> 157: // Suspend/resume support >> >> Taking a step back I can't help but think that this is entirely the wrong place to have the suspend/resume API and supporting code. It is only here because we re-use the HandshakeState _lock monitor right? If we introduced a new thread-suspension monitor instead then this code would all reside somewhere else - probably in the JavaThread class. > > When going to blocked inside the async handshake we **must** unlock the HandshakeState lock. > Thus _lock.wait() unlocks and gives us something to notify. > We could do: > > _lock.unlock(); > { > MutexLocker(SomeOtherLock) ml; > SomeOtherLock.wait(); > } > _lock.lock(); > > > Another alternative is to create a friend class which uses the HandshakeState lock and having the API there instead. Borrowing the HandshakeState lock does create an artificial coupling here. I'd prefer to see this API in a more natural place with friendship used to access the mechanism as needed. Future cleanup though. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From dholmes at openjdk.java.net Wed Apr 21 04:42:27 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 21 Apr 2021 04:42:27 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 04:36:07 GMT, David Holmes wrote: >> When going to blocked inside the async handshake we **must** unlock the HandshakeState lock. >> Thus _lock.wait() unlocks and gives us something to notify. >> We could do: >> >> _lock.unlock(); >> { >> MutexLocker(SomeOtherLock) ml; >> SomeOtherLock.wait(); >> } >> _lock.lock(); >> >> >> Another alternative is to create a friend class which uses the HandshakeState lock and having the API there instead. > > Borrowing the HandshakeState lock does create an artificial coupling here. I'd prefer to see this API in a more natural place with friendship used to access the mechanism as needed. Future cleanup though. Also I think you'd have a lost-wakeup problem if two locks were involved. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From cjplummer at openjdk.java.net Wed Apr 21 05:22:04 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 21 Apr 2021 05:22:04 GMT Subject: RFR: 8265612: revise the help info for jmap histo command In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 02:18:12 GMT, Lin Zang wrote: > This PR revise the help message of the `parallel=[N]` option of command `jmap -histo`. It mainly comes from the discussion at https://github.com/openjdk/jdk/pull/2379#issuecomment-782609582 and https://github.com/openjdk/jdk/pull/2261#issuecomment-823623186. The changes look good, but in the review for PR 2261 I suggested you also move some other minor edits in JMap.java from that PR to this one: https://github.com/openjdk/jdk/pull/2261/files#r617063125 ------------- Marked as reviewed by cjplummer (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3598 From cjplummer at openjdk.java.net Wed Apr 21 05:25:12 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 21 Apr 2021 05:25:12 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: References: Message-ID: <9Q69ICl9o5uaRsoDDEa43b9Vl21lzaZIHe2f8mvQy7c=.b1b36b58-787a-4950-9afe-31ca4c79165b@github.com> On Wed, 14 Apr 2021 12:35:27 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 25 commits: > > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - code clean > - fix trailing white space issue > - ... and 15 more: https://git.openjdk.java.net/jdk/compare/e2106d5a...997784b1 My concerns with your proposed testing is that it always targets the same application with the same heap, and does not read/process the hprof file to make sure it is not corrupt. ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From lzang at openjdk.java.net Wed Apr 21 05:47:35 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 21 Apr 2021 05:47:35 GMT Subject: RFR: 8265612: revise the help info for jmap histo command [v2] In-Reply-To: References: Message-ID: > This PR revise the help message of the `parallel=[N]` option of command `jmap -histo`. It mainly comes from the discussion at https://github.com/openjdk/jdk/pull/2379#issuecomment-782609582 and https://github.com/openjdk/jdk/pull/2261#issuecomment-823623186. Lin Zang has updated the pull request incrementally with one additional commit since the last revision: fix identation issue ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3598/files - new: https://git.openjdk.java.net/jdk/pull/3598/files/af11b5ad..b14bc8d5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3598&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3598&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3598.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3598/head:pull/3598 PR: https://git.openjdk.java.net/jdk/pull/3598 From lzang at openjdk.java.net Wed Apr 21 05:59:10 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Wed, 21 Apr 2021 05:59:10 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: <9Q69ICl9o5uaRsoDDEa43b9Vl21lzaZIHe2f8mvQy7c=.b1b36b58-787a-4950-9afe-31ca4c79165b@github.com> References: <9Q69ICl9o5uaRsoDDEa43b9Vl21lzaZIHe2f8mvQy7c=.b1b36b58-787a-4950-9afe-31ca4c79165b@github.com> Message-ID: On Wed, 21 Apr 2021 05:22:26 GMT, Chris Plummer wrote: > My concerns with your proposed testing is that it always targets the same application with the same heap, and does not read/process the hprof file to make sure it is not corrupt. Hmm, I agree, I will do more test and update here. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From dholmes at openjdk.java.net Wed Apr 21 06:07:06 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 21 Apr 2021 06:07:06 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 19:15:18 GMT, Daniel D. Daugherty wrote: > I'm updating the runtime/Thread/SuspendAtExit.java test: > > - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() > - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() > - switch from counter-based to time-based testing > - improve error checking since we're now using an API with error codes! > > I've used this test to stress @robehn's fix for JDK-8257831 using both > invocation styles for 9 hours each in {fastdebug, release, slowdebug} > configs without any issues. > > I've run the updated test thru Mach5 Tier[134567] testing; one timeout > was observed in a single Tier6 run on Win-X64. I believe this is a case of > a lost Thread.interrupt() call. Hi Dan, A few nits (mostly pre-existing) but otherwise the conversion to JVMTI looks good. Thanks, David test/hotspot/jtreg/runtime/Thread/SuspendAtExit.java line 90: > 88: while (System.currentTimeMillis() < start_time + (timeMax * 1000)) { > 89: count++; > 90: SuspendAtExit threads[] = new SuspendAtExit[N_THREADS]; Style nit pre-existing: The [] go on the type not the variable. test/hotspot/jtreg/runtime/Thread/SuspendAtExit.java line 103: > 101: // the exitSyncObj.await() call and the SuspendThread() > 102: // calls will come in during thread exit. > 103: threads[i].interrupt(); Pre-existing: why use interrupt() instead of simply calling countDown() on the latch ?? test/hotspot/jtreg/runtime/Thread/SuspendAtExit.java line 159: > 157: "after thread #" + i + > 158: " has been join()'ed"); > 159: } This is unnecessary, you aren't testing the correctness of Thread.join() here. join() can't return normally if the thread is alive. test/hotspot/jtreg/runtime/Thread/libSuspendAtExit.cpp line 2: > 1: /* > 2: * Copyright (c) 2001, 2021, Oracle and/or its affiliates. All rights reserved. Copyright year should just be 2021. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3576 From rehn at openjdk.java.net Wed Apr 21 07:32:28 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 21 Apr 2021 07:32:28 GMT Subject: RFR: 8257831: Suspend with handshakes [v10] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 04:39:35 GMT, David Holmes wrote: >> Borrowing the HandshakeState lock does create an artificial coupling here. I'd prefer to see this API in a more natural place with friendship used to access the mechanism as needed. Future cleanup though. > > Also I think you'd have a lost-wakeup problem if two locks were involved. You are correct, above those not work as is. We would need to protect the suspend flag with SomeOtherLock, thus also needing to lock SomeOtherLock when suspending. ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 21 07:42:34 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 21 Apr 2021 07:42:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v13] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 04:32:36 GMT, David Holmes wrote: > Looks good. Thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 21 07:42:34 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 21 Apr 2021 07:42:34 GMT Subject: RFR: 8257831: Suspend with handshakes [v13] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 18:09:07 GMT, Patricio Chilano Mateo wrote: > Latest version LGTM! > > Thanks, > Patricio Thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 21 07:59:09 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 21 Apr 2021 07:59:09 GMT Subject: RFR: 8257831: Suspend with handshakes [v14] In-Reply-To: References: Message-ID: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits: - Merge branch 'master' into SuspendInHandshake - Merge branch 'master' into SuspendInHandshake - Removed double tty lock unlocks - Fixed comment from Dan and fixed name of handshake - Merge branch 'master' into SuspendInHandshake - Fixed DavidH second review - Fixed nits - Merge branch 'master' into SuspendInHandshake - Review fixes 4 - Fixed flag undef dep + spelling error - ... and 10 more: https://git.openjdk.java.net/jdk/compare/7146104f...ec344661 ------------- Changes: https://git.openjdk.java.net/jdk/pull/3191/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3191&range=13 Stats: 1349 lines in 40 files changed: 271 ins; 882 del; 196 mod Patch: https://git.openjdk.java.net/jdk/pull/3191.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3191/head:pull/3191 PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 21 09:18:14 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 21 Apr 2021 09:18:14 GMT Subject: RFR: 8257831: Suspend with handshakes [v14] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 07:59:09 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits: > > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - Removed double tty lock unlocks > - Fixed comment from Dan and fixed name of handshake > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - ... and 10 more: https://git.openjdk.java.net/jdk/compare/7146104f...ec344661 I'll push tomorrow morning and fill RFE to fix the manual transitions in OM/rawmonitor. Please object if there is some major that cannot be fixed as a follow-up. Thanks for all the feedback! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From duke at openjdk.java.net Wed Apr 21 13:21:39 2021 From: duke at openjdk.java.net (duke) Date: Wed, 21 Apr 2021 13:21:39 GMT Subject: Withdrawn: 8259383: AsyncGetCallTrace() crashes sporadically In-Reply-To: <55VhodtLlLULjpxGm3PXOn2iFgrVIdqcZwysOSEK8rg=.d1571446-fc26-4b7d-9450-29ecacb31dda@github.com> References: <55VhodtLlLULjpxGm3PXOn2iFgrVIdqcZwysOSEK8rg=.d1571446-fc26-4b7d-9450-29ecacb31dda@github.com> Message-ID: <6hOp3Z-00yD5XS_sJ0LKViyBLSg8-F0AsxZuNep9TkY=.820f279d-3fe0-4df9-a236-67fb5ed17e9e@github.com> On Mon, 11 Jan 2021 18:39:41 GMT, Lutz Schmidt wrote: > Hi, > may I please ask the community to review this small fix? It closes another hole in AsyncGetCallTrace(). > Thanks a lot! > Lutz This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/2032 From dcubed at openjdk.java.net Wed Apr 21 13:22:01 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 21 Apr 2021 13:22:01 GMT Subject: RFR: 8257831: Suspend with handshakes [v14] In-Reply-To: References: Message-ID: <39QMhQ-RWdnbK6EAn-fhGZyhjFXL7JcIK-Cu7pPSNOI=.d3cc6af7-792e-467b-aa03-af69ccdcfd5f@github.com> On Wed, 21 Apr 2021 07:59:09 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits: > > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - Removed double tty lock unlocks > - Fixed comment from Dan and fixed name of handshake > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - ... and 10 more: https://git.openjdk.java.net/jdk/compare/7146104f...ec344661 Reviewed the v11 and v12 incrementals. Still a thumbs up. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 21 13:22:01 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 21 Apr 2021 13:22:01 GMT Subject: RFR: 8257831: Suspend with handshakes [v14] In-Reply-To: <39QMhQ-RWdnbK6EAn-fhGZyhjFXL7JcIK-Cu7pPSNOI=.d3cc6af7-792e-467b-aa03-af69ccdcfd5f@github.com> References: <39QMhQ-RWdnbK6EAn-fhGZyhjFXL7JcIK-Cu7pPSNOI=.d3cc6af7-792e-467b-aa03-af69ccdcfd5f@github.com> Message-ID: On Wed, 21 Apr 2021 13:17:02 GMT, Daniel D. Daugherty wrote: > Reviewed the v11 and v12 incrementals. > Still a thumbs up. Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rrich at openjdk.java.net Wed Apr 21 13:29:00 2021 From: rrich at openjdk.java.net (Richard Reingruber) Date: Wed, 21 Apr 2021 13:29:00 GMT Subject: RFR: 8257831: Suspend with handshakes [v14] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 07:59:09 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits: > > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - Removed double tty lock unlocks > - Fixed comment from Dan and fixed name of handshake > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - ... and 10 more: https://git.openjdk.java.net/jdk/compare/7146104f...ec344661 I've followed the discussion and the increments. Still looks very good to me ?? ------------- Marked as reviewed by rrich (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Wed Apr 21 13:49:54 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Wed, 21 Apr 2021 13:49:54 GMT Subject: RFR: 8257831: Suspend with handshakes [v14] In-Reply-To: References: Message-ID: <1XFsfMxTH_U2FAlEBehu-t27WpqjLOmH0CpEWxrOptg=.77cd0dae-d594-4113-bd9b-4ce30cc58d53@github.com> On Wed, 21 Apr 2021 13:25:53 GMT, Richard Reingruber wrote: > I've followed the discussion and the increments. Still looks very good to me Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From mcimadamore at openjdk.java.net Wed Apr 21 14:45:34 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 21 Apr 2021 14:45:34 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature in Java [v3] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 03:35:06 GMT, Vicente Romero wrote: >> Please review this PR that intents to make sealed classes a final feature in Java. This PR contains compiler and VM changes. In line with similar PRs, which has made preview features final, this one is mostly removing preview related comments from APIs plus options in test cases. Please also review the related [CSR](https://bugs.openjdk.java.net/browse/JDK-8265090) >> >> Thanks >> >> note: this PR is related to [PR-3528](https://github.com/openjdk/jdk/pull/3528) and must be integrated after it. > > Vicente Romero has updated the pull request incrementally with one additional commit since the last revision: > > updating comment after review feedback Compiler changes look good (I have not checked SymbolGenerator). Why were some tests removed? ------------- Marked as reviewed by mcimadamore (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3526 From vromero at openjdk.java.net Wed Apr 21 15:13:33 2021 From: vromero at openjdk.java.net (Vicente Romero) Date: Wed, 21 Apr 2021 15:13:33 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature in Java [v3] In-Reply-To: References: Message-ID: <2gJQiuoDTZDgr62xRlnZLCoL1-EFjUCSHsKBrSpPV2U=.1918b494-34bb-4f1a-939b-9a931b583117@github.com> On Wed, 21 Apr 2021 14:42:39 GMT, Maurizio Cimadamore wrote: > Compiler changes look good (I have not checked SymbolGenerator). > > Why were some tests removed? thanks for the review. The removed tests were already covered in langtools regression tests, so I only removed duplicated tests ------------- PR: https://git.openjdk.java.net/jdk/pull/3526 From ron.pressler at oracle.com Wed Apr 21 15:53:43 2021 From: ron.pressler at oracle.com (Ron Pressler) Date: Wed, 21 Apr 2021 15:53:43 +0000 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: <-fco_my21rYwE8RlEaFlsc-g0WL_rPzbLBWllNqY2rw=.9782b940-c827-4dad-bba8-6901f75fdcf8@github.com> References: <-fco_my21rYwE8RlEaFlsc-g0WL_rPzbLBWllNqY2rw=.9782b940-c827-4dad-bba8-6901f75fdcf8@github.com> Message-ID: <50F9CB87-C727-440B-82BE-D282B308B7A2@oracle.com> I think you?re coming to this discussion with a very different perspective from us, which makes understanding what is or isn?t on the table unclear, and also steers the ideas in directions that are different from the one we?re going. Our goal is not to maintain some status quo until such time we decide to restrict it. We?ve started on a long process of strengthening the platform?s integrity, both to increase security and to help with the ecosystem?s maintenance. Making some things that are possible today less ?convenient? is intentional. Locked doors are less convenient than unlocked ones, but sometimes you can only strive to increase convenience under the hard constraint that doors must be locked. This is where we are: we?ve decided doors will be locked unless explicitly unlocked. But this is a process. As long as there are gaps in the garden fence ? Unsafe, self-attach, JNI ? the locks can be circumvented. Rather than fixing all those loopholes at once, we?re doing it gradually one at a time. This does mean that a motivated developer can circumvent the locks up until the point the last loophole is closed, but the hope is that, knowing where we?re headed, library developers will gradually accept the reality of this direction (or not prepare their users for the coming changes, keep using those gaps that are still open to them, and then surprise their users when the last of them is closed). Suggestions should, therefore, focus on ideas compatible with this vision. To be more constructive and less frustrating, the discussion should proceed under this assumption, even if it means accepting that some things will be less convenient than they are today. For example, self-attach is not the only issue. Leaking powerful Instrumentation objects to libraries circumvents encapsulation barriers without there being a key placed in the lock through the command line. That this can be circumvented today is irrelevant, as these workarounds will be *gradually* removed. I hope the operations people will be thrilled with the platform?s increased security and reduced maintenance pain when upgrading JDK versions, but either way, they should start preparing for the new reality sooner rather than later. Our goal, then, should be to make people?s life easier, but only under these constraints, that, at this point, should be taken as an axiom for the purpose of discussion. ? Ron > On 20 Apr 2021, at 21:26, Rafael Winterhalter wrote: > > On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter wrote: > >>> To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. >> >> Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: >> >> 8200559: Java agents doing instrumentation need a means to define auxiliary classes > > I fully understand your concerns about ByteBuddyAgent.install(). It is > simply a convenience for something that can be meaningful in some contexts > where I prefer offering a simple API. I use it mainly for two purposes: > > a) For testing Java agents and integrations against Instrumentation within > the current VM when tests are triggered by tools that do not support > javaagents, also because builds do not bundle jars until after tests are > executed. > > b) For purposefully "hacky" test libraries like Mockito that need agent > capabilities without this being meant to be used in production > environments. I have earlier proposed to offer a "jdk.test" module that > offers the Instrumentation instance via a simple API similar to Byte > Buddy's. The JVM would not load this module unless requested on the command > line. Build tools like Maven's surefire or Gradle's testrunner could then > standardize on loading this module as a convention to give access to this > test module by default such that libraries like Mockito could continue to > function out of the box without the libraries functioning on a standard VM > without extra configuration. As far as I know, mainly test libraries need > this API. This would also emphasise that Mockito and others are meant for > testing and fewer people would abuse it for production applications. People > would also have an explicit means of running a JVM for a production > application or for executing a test. > > As for adding the API, my thought is that if the Instrumentation API were > to throw exceptions on some methods/arguments for dynamic agents in the > future, for example for retransformClasses(Object.class), this breaking > change would then simply extend to the proposed "defineClass" method. In > this sense, the Instrumentation API already assumes full power, I find it > not problematic to add the missing bit to this API even if it was > restricted in the future in the same spirit as other methods of the API > would be. > > I mentioned JNI as it is a well-known approach to defining a class today, > using a minimal native binding to an interface that directly calls down to > JNI's: > > jclass DefineClass(JNIEnv *env, const char *name, jobject loader, const > jbyte *buf, jsize bufLen); > > This interface can then simply be used to define any class just as I > propse, even when not writing an agent or attaching. This method makes > class definitions also already trivial for JVMTI agents compared to Java > agents. Unless restricting JNI, the defineClass method is already a low > hanging fruit, but at the cost of having to maintain a tiny bit of native > code. I'd rather see this avoided and a standard API being offered to > agents up to the time that Panama is in place and a JNI restriction is > possibly also included. As a bonus: Once JNI is restricted, Byte Buddy's > "install" would no longer work unless self-attachment (or JNI) is > explicitly allowed. The emulation already requires to run native code while > the Virtual Machine API explicitly checks for the process id of the current > VM against the one that is targeted. With both disabled, self-attachment > would no longer be practically be possible without needing to prune the > capabilities of dynamic agents which is what I understand would be the > desired effect. > > From this viewpoint, I think that adding Instrumentation::defineClass > method does no harm compared to the status quo. And on the upside, it gives > agents an API to migrate to, avoiding the last need of using unsafe. To > make the JVM a safe platform, binding native code would anyways need > restriction and this would then also solve the problem of dynamic agents > attaching from the same VM being used in libraries. This would in my eyes > be the cleanest solution to the self-attachment problem without disturbing > the existing landscape of dynamic agents. To run Mockito, one would then > instead configure Maven surefire or Gradle to run the JVM with > -Djdk.attach.allowAttachSelf=true. Ideally, some "jdk.test" module would be > added at some point, to avoid the overhead of self-attachment, but I think > this better fits into separate debate. > > Am Di., 20. Apr. 2021 um 15:38 Uhr schrieb mlbridge[bot] < > ***@***.***>: > >> *Mailing list message from Alan Bateman ***@***.***> on >> core-libs-dev ***@***.***>:* >> >> On 19/04/2021 22:20, Rafael Winterhalter wrote: >> >> : >> At the moment, it is required for root to switch to the user that owns the >> JVM process as the domain socket is only accessible to that user to avoid >> that users without access to the JVM can inject themselves into a JVM. I am >> not sure if operations teams would be thrilled to have a monitoring agent >> required to run as root, even in these times of Kubernetes. >> >> I mainly have two comments: >> >> 1. The problem is the possibility of self-attach. I think this is the >> problem to solve, a library getting agent privileges without being an >> agent. I think this should be prevented while dynamic attach should >> continue to be possible in today's format. It has proven to be so useful, >> it would be a shame if the current tooling convenience would disappear from >> the JVM. As it's my understanding, JNI is supposed to be restricted in the >> future, in line with Panama. Without this restriction, JNI already allows >> for random class definition, for example, which similarly to an agent >> offers surpassing the majority of JVM restrictions. The second restriction >> would be a control to restrict how a JVM process starts new processes. I >> think both are reasonable restrictions for a library to face which require >> explicit enabling. Especially with the security manager on it's way out, >> certain capabilities should be rethought to begin with. If both are no >> longer freely available, self-attachment is no longer possible anyways and >> dynamic agents could retain their capabilities. >> >> 2. The question of introducing an Instrumentation::defineClass method is >> fully independent of that first question. If a dynamic agent was to be >> restricted, the method could reject classloader/package combinations for >> dynamically loaded agents the same way that >> Instrumentation::retransformClasses would need to. At the same time, >> introducing the method would allow agents to move to an official API with a >> Java 17 baseline which will be the next long-standing base line. I fully >> understand it needs a thorough discussion but it is a less complicated >> problem then (1) and could therefore be decided prior to having found a >> satisfactory solution for it. >> >> I should have been clearer, it's the combination of the two that creates >> the attractive nuisance. I don't think there are any objections to a >> defineClass for agents specified on the command line with -javaagent. >> However we have to be cautious about extending that capability to agents >> that are loaded into a running VM with the attach mechanism. >> >> ByteBuddy looks great for code generation and transforming classes but >> ByteBuddyAgent makes me nervous. It looks like I can deploy >> byte-buddy-agent-.jar on my class path and invoke the public >> static ByteBuddyAgent.install() method to get the Instrumentation object >> for the current VM. That may be convenient for some but this is the >> all-powerful Instrumentation object that shouldn't be leaked to library >> or application code. Now combine this with the proposed defineClass and >> it means that any code on the class path could inject a class into >> java.lang or any run-time package without any agent voodoo or opt-in via >> the command line. That would be difficult genie to re-bottle if it were >> to get traction. >> >> You mentioned restricting JNI in the future. I'm not aware of a definite >> plan or time-frame. Project Panama is pioneering restricting access to >> native operations as a bug or mis-use with the linker API can easily >> crash the VM or breakage in other ways. Extending this to JNI would be a >> logical next step but I could imagine it taking a long time and many >> releases to get there. >> >> As regards this PR then I would be happy to work with you on a revised >> proposed that would limit it to agents specified with -javaagent. That >> would not preclude extending the capability, maybe in a more restricted >> form, to agents loaded into a running VM in the future. >> >> -Alan. >> >> ? >> You are receiving this because you were mentioned. >> Reply to this email directly, view it on GitHub >> , or >> unsubscribe >> >> . >> > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3546 From rafael.wth at gmail.com Wed Apr 21 18:38:32 2021 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Wed, 21 Apr 2021 20:38:32 +0200 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: <50F9CB87-C727-440B-82BE-D282B308B7A2@oracle.com> References: <-fco_my21rYwE8RlEaFlsc-g0WL_rPzbLBWllNqY2rw=.9782b940-c827-4dad-bba8-6901f75fdcf8@github.com> <50F9CB87-C727-440B-82BE-D282B308B7A2@oracle.com> Message-ID: Hi Ron, we certainly do come from different backgrounds and therefore perspectives, I find this exchange of perspective to be the main advantage of this open mailing list. I do believe that I have an understanding of your angle, I tried to give my feedback since before Java 9 introduced the module system and so long I feel like it always helped to find a good compromise. In the end, if a library like Mockito or the large set of agents would no longer work, this would be significantly more impactful to the JVM ecosystem then any minor API change that gets carefully reviewed for good reason. That's why I am trying to preserve them. The lack of a class definition utility for Java agents is, in my opinion, one of the last bits within "unsafe migration" that has not been addressed. And it seems to me that this view is agreed upon, Oracle already proposed an API to potentially tackle this issue, I just disagree with the scope, given my extensive experience in agent development, and try to revive the hibernated debate at the same time. I am very much aware that this metaphoric fence between Java agents and regular libraries pretending to be one is being built. I also think this is necessary, it's a good step forward. And as a matter of fact, expecting this is at the core of my argument. With the fence being completed at some point, the Instrumentation instance would be shielded off from regular libraries, therefore the restriction to emulate being an agent would imply a restriction to define classes using this instance. At the same time, Java agents could use a facility that they certainly require, I hopefully lined out why this is needed. Today most Java agents are using Unsafe, but I would hope that they could migrate to an official API for Java 17. This way agents could disregard this fence and continue to function when building the fence is completed, even if a user upgrades the JVM. My second argument is that restricting dynamic agents would require to stub some API of Instrumentation for some arguments already. I do not see any (additional) harm in stubbing the defineClass API in a similar manner as it would require stubbing retransformClasses. I think it would therefore be best to keep this argument out of the consideration of what the best API would be for "favored agents", where arguments in favor of my proposal were made by not only me. Therefore, I hope this API can be considered. Agent developers would certainly want to migrate to stable, supported API. But as of today no such API is offered. Since Java agents are often supplementary and need to support unmaintained software, their baseline moves slower then that of other Java code which is why I hoped that Java 17 could be the first release to offer such API as it gets the LTS label attached. Best regards, Rafael Am Mi., 21. Apr. 2021 um 19:00 Uhr schrieb Ron Pressler < ron.pressler at oracle.com>: > I think you?re coming to this discussion with a very different perspective > from us, which > makes understanding what is or isn?t on the table unclear, and also steers > the ideas in > directions that are different from the one we?re going. > > Our goal is not to maintain some status quo until such time we decide to > restrict it. We?ve > started on a long process of strengthening the platform?s integrity, both > to increase security > and to help with the ecosystem?s maintenance. Making some things that are > possible today > less ?convenient? is intentional. Locked doors are less convenient than > unlocked ones, but > sometimes you can only strive to increase convenience under the hard > constraint that doors > must be locked. This is where we are: we?ve decided doors will be locked > unless explicitly > unlocked. > > But this is a process. As long as there are gaps in the garden fence ? > Unsafe, self-attach, JNI ? > the locks can be circumvented. Rather than fixing all those loopholes at > once, we?re doing it > gradually one at a time. This does mean that a motivated developer can > circumvent the locks up > until the point the last loophole is closed, but the hope is that, knowing > where we?re headed, library > developers will gradually accept the reality of this direction (or not > prepare their users for the coming > changes, keep using those gaps that are still open to them, and then > surprise their users when the > last of them is closed). > > Suggestions should, therefore, focus on ideas compatible with this vision. > To be more constructive > and less frustrating, the discussion should proceed under this > assumption, even if it means > accepting that some things will be less convenient than they are today. > > For example, self-attach is not the only issue. Leaking powerful > Instrumentation objects to libraries > circumvents encapsulation barriers without there being a key placed in the > lock through the command > line. That this can be circumvented today is irrelevant, as these > workarounds will be *gradually* > removed. I hope the operations people will be thrilled with the platform?s > increased security and > reduced maintenance pain when upgrading JDK versions, but either way, they > should start > preparing for the new reality sooner rather than later. Our goal, then, > should be to make people?s life > easier, but only under these constraints, that, at this point, should be > taken as an axiom for the purpose > of discussion. > > ? Ron > > > On 20 Apr 2021, at 21:26, Rafael Winterhalter < > winterhalter at openjdk.java.net> wrote: > > > > On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter < > winterhalter at openjdk.org> wrote: > > > >>> To allow agents the definition of auxiliary classes, an API is needed > to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` > or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was > removed from `sun.misc.Unsafe`. > >> > >> Rafael Winterhalter has refreshed the contents of this pull request, > and previous commits have been removed. The incremental views will show > differences compared to the previous content of the PR. The pull request > contains one new commit since the last revision: > >> > >> 8200559: Java agents doing instrumentation need a means to define > auxiliary classes > > > > I fully understand your concerns about ByteBuddyAgent.install(). It is > > simply a convenience for something that can be meaningful in some > contexts > > where I prefer offering a simple API. I use it mainly for two purposes: > > > > a) For testing Java agents and integrations against Instrumentation > within > > the current VM when tests are triggered by tools that do not support > > javaagents, also because builds do not bundle jars until after tests are > > executed. > > > > b) For purposefully "hacky" test libraries like Mockito that need agent > > capabilities without this being meant to be used in production > > environments. I have earlier proposed to offer a "jdk.test" module that > > offers the Instrumentation instance via a simple API similar to Byte > > Buddy's. The JVM would not load this module unless requested on the > command > > line. Build tools like Maven's surefire or Gradle's testrunner could then > > standardize on loading this module as a convention to give access to this > > test module by default such that libraries like Mockito could continue to > > function out of the box without the libraries functioning on a standard > VM > > without extra configuration. As far as I know, mainly test libraries need > > this API. This would also emphasise that Mockito and others are meant for > > testing and fewer people would abuse it for production applications. > People > > would also have an explicit means of running a JVM for a production > > application or for executing a test. > > > > As for adding the API, my thought is that if the Instrumentation API were > > to throw exceptions on some methods/arguments for dynamic agents in the > > future, for example for retransformClasses(Object.class), this breaking > > change would then simply extend to the proposed "defineClass" method. In > > this sense, the Instrumentation API already assumes full power, I find it > > not problematic to add the missing bit to this API even if it was > > restricted in the future in the same spirit as other methods of the API > > would be. > > > > I mentioned JNI as it is a well-known approach to defining a class today, > > using a minimal native binding to an interface that directly calls down > to > > JNI's: > > > > jclass DefineClass(JNIEnv *env, const char *name, jobject loader, const > > jbyte *buf, jsize bufLen); > > > > This interface can then simply be used to define any class just as I > > propse, even when not writing an agent or attaching. This method makes > > class definitions also already trivial for JVMTI agents compared to Java > > agents. Unless restricting JNI, the defineClass method is already a low > > hanging fruit, but at the cost of having to maintain a tiny bit of native > > code. I'd rather see this avoided and a standard API being offered to > > agents up to the time that Panama is in place and a JNI restriction is > > possibly also included. As a bonus: Once JNI is restricted, Byte Buddy's > > "install" would no longer work unless self-attachment (or JNI) is > > explicitly allowed. The emulation already requires to run native code > while > > the Virtual Machine API explicitly checks for the process id of the > current > > VM against the one that is targeted. With both disabled, self-attachment > > would no longer be practically be possible without needing to prune the > > capabilities of dynamic agents which is what I understand would be the > > desired effect. > > > > From this viewpoint, I think that adding Instrumentation::defineClass > > method does no harm compared to the status quo. And on the upside, it > gives > > agents an API to migrate to, avoiding the last need of using unsafe. To > > make the JVM a safe platform, binding native code would anyways need > > restriction and this would then also solve the problem of dynamic agents > > attaching from the same VM being used in libraries. This would in my eyes > > be the cleanest solution to the self-attachment problem without > disturbing > > the existing landscape of dynamic agents. To run Mockito, one would then > > instead configure Maven surefire or Gradle to run the JVM with > > -Djdk.attach.allowAttachSelf=true. Ideally, some "jdk.test" module would > be > > added at some point, to avoid the overhead of self-attachment, but I > think > > this better fits into separate debate. > > > > Am Di., 20. Apr. 2021 um 15:38 Uhr schrieb mlbridge[bot] < > > ***@***.***>: > > > >> *Mailing list message from Alan Bateman ***@***.***> on > >> core-libs-dev ***@***.***>:* > >> > >> On 19/04/2021 22:20, Rafael Winterhalter wrote: > >> > >> : > >> At the moment, it is required for root to switch to the user that owns > the > >> JVM process as the domain socket is only accessible to that user to > avoid > >> that users without access to the JVM can inject themselves into a JVM. > I am > >> not sure if operations teams would be thrilled to have a monitoring > agent > >> required to run as root, even in these times of Kubernetes. > >> > >> I mainly have two comments: > >> > >> 1. The problem is the possibility of self-attach. I think this is the > >> problem to solve, a library getting agent privileges without being an > >> agent. I think this should be prevented while dynamic attach should > >> continue to be possible in today's format. It has proven to be so > useful, > >> it would be a shame if the current tooling convenience would disappear > from > >> the JVM. As it's my understanding, JNI is supposed to be restricted in > the > >> future, in line with Panama. Without this restriction, JNI already > allows > >> for random class definition, for example, which similarly to an agent > >> offers surpassing the majority of JVM restrictions. The second > restriction > >> would be a control to restrict how a JVM process starts new processes. I > >> think both are reasonable restrictions for a library to face which > require > >> explicit enabling. Especially with the security manager on it's way out, > >> certain capabilities should be rethought to begin with. If both are no > >> longer freely available, self-attachment is no longer possible anyways > and > >> dynamic agents could retain their capabilities. > >> > >> 2. The question of introducing an Instrumentation::defineClass method is > >> fully independent of that first question. If a dynamic agent was to be > >> restricted, the method could reject classloader/package combinations for > >> dynamically loaded agents the same way that > >> Instrumentation::retransformClasses would need to. At the same time, > >> introducing the method would allow agents to move to an official API > with a > >> Java 17 baseline which will be the next long-standing base line. I fully > >> understand it needs a thorough discussion but it is a less complicated > >> problem then (1) and could therefore be decided prior to having found a > >> satisfactory solution for it. > >> > >> I should have been clearer, it's the combination of the two that creates > >> the attractive nuisance. I don't think there are any objections to a > >> defineClass for agents specified on the command line with -javaagent. > >> However we have to be cautious about extending that capability to agents > >> that are loaded into a running VM with the attach mechanism. > >> > >> ByteBuddy looks great for code generation and transforming classes but > >> ByteBuddyAgent makes me nervous. It looks like I can deploy > >> byte-buddy-agent-.jar on my class path and invoke the public > >> static ByteBuddyAgent.install() method to get the Instrumentation object > >> for the current VM. That may be convenient for some but this is the > >> all-powerful Instrumentation object that shouldn't be leaked to library > >> or application code. Now combine this with the proposed defineClass and > >> it means that any code on the class path could inject a class into > >> java.lang or any run-time package without any agent voodoo or opt-in via > >> the command line. That would be difficult genie to re-bottle if it were > >> to get traction. > >> > >> You mentioned restricting JNI in the future. I'm not aware of a definite > >> plan or time-frame. Project Panama is pioneering restricting access to > >> native operations as a bug or mis-use with the linker API can easily > >> crash the VM or breakage in other ways. Extending this to JNI would be a > >> logical next step but I could imagine it taking a long time and many > >> releases to get there. > >> > >> As regards this PR then I would be happy to work with you on a revised > >> proposed that would limit it to agents specified with -javaagent. That > >> would not preclude extending the capability, maybe in a more restricted > >> form, to agents loaded into a running VM in the future. > >> > >> -Alan. > >> > >> ? > >> You are receiving this because you were mentioned. > >> Reply to this email directly, view it on GitHub > >> , or > >> unsubscribe > >> < > https://github.com/notifications/unsubscribe-auth/ABCIA4FE2B4DGBZS4QO6SM3TJV7T5ANCNFSM43BSDEGQ > > > >> . > >> > > > > ------------- > > > > PR: https://git.openjdk.java.net/jdk/pull/3546 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Apr 21 18:40:44 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Apr 2021 19:40:44 +0100 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: <-fco_my21rYwE8RlEaFlsc-g0WL_rPzbLBWllNqY2rw=.9782b940-c827-4dad-bba8-6901f75fdcf8@github.com> References: <-fco_my21rYwE8RlEaFlsc-g0WL_rPzbLBWllNqY2rw=.9782b940-c827-4dad-bba8-6901f75fdcf8@github.com> Message-ID: On 20/04/2021 21:26, Rafael Winterhalter wrote: > I have earlier proposed to offer a "jdk.test" module that > offers the Instrumentation instance via a simple API similar to Byte > Buddy's. The JVM would not load this module unless requested on the command > line. Build tools like Maven's surefire or Gradle's testrunner could then > standardize on loading this module as a convention to give access to this > test module by default such that libraries like Mockito could continue to > function out of the box without the libraries functioning on a standard VM > without extra configuration. As far as I know, mainly test libraries need > this API. This would also emphasise that Mockito and others are meant for > testing and fewer people would abuse it for production applications. People > would also have an explicit means of running a JVM for a production > application or for executing a test. Helping testing is good but a jdk.test module that hands out the Instrumentation object could be problematic. Is there a reason why the runner for Mockito and other mocking frameworks can't specify -javaagent when launching? I would expect at least some mocking scenarios to require load time transformations (to drop the final modifier from some API elements for example) so important to have the transformer set before classes are loaded. > As for adding the API, my thought is that if the Instrumentation API were > to throw exceptions on some methods/arguments for dynamic agents in the > future, for example for retransformClasses(Object.class), this breaking > change would then simply extend to the proposed "defineClass" method. In > this sense, the Instrumentation API already assumes full power, I find it > not problematic to add the missing bit to this API even if it was > restricted in the future in the same spirit as other methods of the API > would be. I think it would really hard to put this genie back into a bottle. It's way more attractive to use that than the very agent oriented redefineModule and retransformClasses. > > I mentioned JNI as it is a well-known approach to defining a class today, > using a minimal native binding to an interface that directly calls down to > JNI's: > > jclass DefineClass(JNIEnv *env, const char *name, jobject loader, const > jbyte *buf, jsize bufLen); > > This interface can then simply be used to define any class just as I > propse, even when not writing an agent or attaching. This method makes > class definitions also already trivial for JVMTI agents compared to Java > agents. Unless restricting JNI, the defineClass method is already a low > hanging fruit, but at the cost of having to maintain a tiny bit of native > code. Sure, if you are using native code then you have the full power of JVM TI and JNI available. Project Panama is exploring how to restrict access to native code, I think too early to say how this might extend to JNI. -Alan From cjplummer at openjdk.java.net Wed Apr 21 18:43:40 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 21 Apr 2021 18:43:40 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: References: <9Q69ICl9o5uaRsoDDEa43b9Vl21lzaZIHe2f8mvQy7c=.b1b36b58-787a-4950-9afe-31ca4c79165b@github.com> Message-ID: <-qf3V7rOYbOMN9Wf0Q4Nnuz8seoJwA6VeUBQgqLVnak=.ace9f2ac-4669-4c2e-990a-1547533094cc@github.com> On Wed, 21 Apr 2021 05:56:21 GMT, Lin Zang wrote: > > My concerns with your proposed testing is that it always targets the same application with the same heap, and does not read/process the hprof file to make sure it is not corrupt. > > Hmm, I agree, I will do more test and update here. Thanks! As for jtreg testing, we have a few heap dumping related tests. I'm not sure how good they are. It would be good to understand what testing we currently do. In addition to jtreg testing, you might want to try just launching some large java apps (netbeans and intellij come to mind), dump their heaps, and then process them with some existing tool. I'm not suggesting this be part of regular testing, but just a sanity check you do on your own before pushing the changes. ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From cjplummer at openjdk.java.net Wed Apr 21 18:46:45 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 21 Apr 2021 18:46:45 GMT Subject: RFR: 8265612: revise the help info for jmap histo command [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 05:47:35 GMT, Lin Zang wrote: >> This PR revise the help message of the `parallel=[N]` option of command `jmap -histo`. It mainly comes from the discussion at https://github.com/openjdk/jdk/pull/2379#issuecomment-782609582 and https://github.com/openjdk/jdk/pull/2261#issuecomment-823623186. > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > fix identation issue src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java line 232: > 230: if (compress_level.length() == 0) { > 231: System.err.println("Fail: no number provided in option: '" + subopt + "'"); > 232: usage(1); Actually lines 231 and 232 have the correct indentation. It's 229, 230, and 233 that are wrong. They need to be indented one more space. ------------- PR: https://git.openjdk.java.net/jdk/pull/3598 From dcubed at openjdk.java.net Wed Apr 21 19:40:34 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 21 Apr 2021 19:40:34 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating In-Reply-To: References: Message-ID: <7_z9JKf0wDbryySASsNAGbS1laveYYxTbAUNCOxmNwI=.35e24f78-0ebb-4964-ac0a-4b5da5585880@github.com> On Wed, 21 Apr 2021 05:50:01 GMT, David Holmes wrote: >> I'm updating the runtime/Thread/SuspendAtExit.java test: >> >> - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() >> - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() >> - switch from counter-based to time-based testing >> - improve error checking since we're now using an API with error codes! >> >> I've used this test to stress @robehn's fix for JDK-8257831 using both >> invocation styles for 9 hours each in {fastdebug, release, slowdebug} >> configs without any issues. >> >> I've run the updated test thru Mach5 Tier[134567] testing; one timeout >> was observed in a single Tier6 run on Win-X64. I believe this is a case of >> a lost Thread.interrupt() call. > > test/hotspot/jtreg/runtime/Thread/SuspendAtExit.java line 90: > >> 88: while (System.currentTimeMillis() < start_time + (timeMax * 1000)) { >> 89: count++; >> 90: SuspendAtExit threads[] = new SuspendAtExit[N_THREADS]; > > Style nit pre-existing: The [] go on the type not the variable. Good catch! As I'm re-reading this code, it occurs to me that I really don't need to create the array of `N_THREADS` any more since we are now time based and we'll create a new Thread in each loop and run it through the gauntlet until times runs out. What do you think about changing `SuspendAtExit[N_THREADS]` into just a single `SuspendAtExit`? > test/hotspot/jtreg/runtime/Thread/SuspendAtExit.java line 103: > >> 101: // the exitSyncObj.await() call and the SuspendThread() >> 102: // calls will come in during thread exit. >> 103: threads[i].interrupt(); > > Pre-existing: why use interrupt() instead of simply calling countDown() on the latch ?? Hmmm.... I don't remember why I chose to use Thread.interrupt() instead of countDown() when I created these tests for the Thread-SMR project. I'll try switching. > test/hotspot/jtreg/runtime/Thread/SuspendAtExit.java line 159: > >> 157: "after thread #" + i + >> 158: " has been join()'ed"); >> 159: } > > This is unnecessary, you aren't testing the correctness of Thread.join() here. join() can't return normally if the thread is alive. Okay. Will delete the "late" isAlive() check. > test/hotspot/jtreg/runtime/Thread/libSuspendAtExit.cpp line 2: > >> 1: /* >> 2: * Copyright (c) 2001, 2021, Oracle and/or its affiliates. All rights reserved. > > Copyright year should just be 2021. I copied this code from another file that is "Copyright (c) 2001, 2021" and renamed just the prefix for the suspendThread() and resumeThread() functions. Agent_OnLoad() is a straight copy. So I think I should keep the copyright as is. ------------- PR: https://git.openjdk.java.net/jdk/pull/3576 From amenkov at openjdk.java.net Wed Apr 21 20:51:49 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 21 Apr 2021 20:51:49 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" [v2] In-Reply-To: References: Message-ID: > The test actually failed starting from jdk13, but the error is masked by JDK-8264667 (and shell part of the test didn't verify result of the java execution) > The fix: > - updates JvmtiClassFileReconstituter to add attributes in the same order as javac does > - makes the test java instead of shell > - added workaround for JDK-8264667 > - test code is ready to ignore order of attributes Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: Updated the fix accordingly feedback ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3426/files - new: https://git.openjdk.java.net/jdk/pull/3426/files/62b5b474..67ce2ea0 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3426&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3426&range=00-01 Stats: 30 lines in 2 files changed: 1 ins; 6 del; 23 mod Patch: https://git.openjdk.java.net/jdk/pull/3426.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3426/head:pull/3426 PR: https://git.openjdk.java.net/jdk/pull/3426 From amenkov at openjdk.java.net Wed Apr 21 20:51:51 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 21 Apr 2021 20:51:51 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" [v2] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 03:14:34 GMT, Serguei Spitsyn wrote: >> Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: >> >> Updated the fix accordingly feedback > > test/jdk/java/lang/instrument/ATransformerManagementTestCase.java line 125: > >> 123: manager.addTransformer(transformer, canRetransform); >> 124: // verbosePrint("Added transformer " + transformer >> 125: // + " with canRetransform=" + canRetransform); > > I'd suggest to remove the commented out lines. fixed > test/jdk/java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.java line 152: > >> 150: // directory so there is no conflict with the file >> 151: // in the test classes directory. >> 152: String resourceName = fTargetClassName + ".class"; > > This name can be defined out of methods `originalTargetClassFile` and `transformClassFile`. The method name `transformClassFile` is confusing. I'd suggest to rename it to `transformedClassFile` or `modifiedClassFile`. Also, the name `originalTargetClassFile` can be shortened to `originalClassFile`. done > test/jdk/java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.java line 186: > >> 184: int lineNum = 0; >> 185: for (String line: out1) { >> 186: if (expectedDiffenent(line)) { > > Is it possible to use the `expectedDiffenent()` as a filter to get only needed lines from `disassembleClassFile()`? It's implemented this way to keep lineNum correct. But it was error here. Fixed now ------------- PR: https://git.openjdk.java.net/jdk/pull/3426 From amenkov at openjdk.java.net Wed Apr 21 20:51:52 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 21 Apr 2021 20:51:52 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 20:46:43 GMT, Alex Menkov wrote: >> test/jdk/java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.java line 186: >> >>> 184: int lineNum = 0; >>> 185: for (String line: out1) { >>> 186: if (expectedDiffenent(line)) { >> >> Is it possible to use the `expectedDiffenent()` as a filter to get only needed lines from `disassembleClassFile()`? > > It's implemented this way to keep lineNum correct. But it was error here. Fixed now And fixed typo in the method name ------------- PR: https://git.openjdk.java.net/jdk/pull/3426 From dcubed at openjdk.java.net Wed Apr 21 21:22:50 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 21 Apr 2021 21:22:50 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating [v2] In-Reply-To: References: Message-ID: > I'm updating the runtime/Thread/SuspendAtExit.java test: > > - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() > - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() > - switch from counter-based to time-based testing > - improve error checking since we're now using an API with error codes! > > I've used this test to stress @robehn's fix for JDK-8257831 using both > invocation styles for 9 hours each in {fastdebug, release, slowdebug} > configs without any issues. > > I've run the updated test thru Mach5 Tier[134567] testing; one timeout > was observed in a single Tier6 run on Win-X64. I believe this is a case of > a lost Thread.interrupt() call. Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: dholmes CR changes. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3576/files - new: https://git.openjdk.java.net/jdk/pull/3576/files/ed352ea3..8235e5f2 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3576&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3576&range=00-01 Stats: 90 lines in 1 file changed: 22 ins; 33 del; 35 mod Patch: https://git.openjdk.java.net/jdk/pull/3576.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3576/head:pull/3576 PR: https://git.openjdk.java.net/jdk/pull/3576 From ysuenaga at openjdk.java.net Thu Apr 22 00:08:26 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Thu, 22 Apr 2021 00:08:26 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: <3pXwhVt-X7NBs0DL9kXhsBLCst7qKHD8y3JwJ1ojmag=.8bfd1a86-6c92-458d-baff-ec5938f19194@github.com> On Wed, 7 Apr 2021 00:12:11 GMT, Yasumasa Suenaga wrote: >> `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. >> >> This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Rename --disableregistry to --disable-registry Gentle ping: Could I please have review? CSR has been approved. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From dholmes at openjdk.java.net Thu Apr 22 02:42:23 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 22 Apr 2021 02:42:23 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating [v2] In-Reply-To: References: Message-ID: <3ea1rkVUQcFqT8YPeHQvmB0SgCcSYPPwkMab3gajohE=.d3a6c134-68a2-4a43-8ef3-511ed68d3e98@github.com> On Wed, 21 Apr 2021 21:22:50 GMT, Daniel D. Daugherty wrote: >> I'm updating the runtime/Thread/SuspendAtExit.java test: >> >> - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() >> - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() >> - switch from counter-based to time-based testing >> - improve error checking since we're now using an API with error codes! >> >> I've used this test to stress @robehn's fix for JDK-8257831 using both >> invocation styles for 9 hours each in {fastdebug, release, slowdebug} >> configs without any issues. >> >> I've run the updated test thru Mach5 Tier[134567] testing; one timeout >> was observed in a single Tier6 run on Win-X64. I believe this is a case of >> a lost Thread.interrupt() call. > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes CR changes. Marked as reviewed by dholmes (Reviewer). I managed to lose my review comment when switching between commits :) so I'll add it here. Updates look good. I agree there is no need for the Thread[] any more. Thanks, David test/hotspot/jtreg/runtime/Thread/SuspendAtExit.java line 38: > 36: private final static String AGENT_LIB = "SuspendAtExit"; > 37: private final static int DEF_TIME_MAX = 30; // default max # secs to test > 38: private final static int N_THREADS = 32; N_THREADS is unused now. ------------- PR: https://git.openjdk.java.net/jdk/pull/3576 From dholmes at openjdk.java.net Thu Apr 22 02:42:24 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 22 Apr 2021 02:42:24 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating [v2] In-Reply-To: <7_z9JKf0wDbryySASsNAGbS1laveYYxTbAUNCOxmNwI=.35e24f78-0ebb-4964-ac0a-4b5da5585880@github.com> References: <7_z9JKf0wDbryySASsNAGbS1laveYYxTbAUNCOxmNwI=.35e24f78-0ebb-4964-ac0a-4b5da5585880@github.com> Message-ID: On Wed, 21 Apr 2021 19:37:28 GMT, Daniel D. Daugherty wrote: >> test/hotspot/jtreg/runtime/Thread/libSuspendAtExit.cpp line 2: >> >>> 1: /* >>> 2: * Copyright (c) 2001, 2021, Oracle and/or its affiliates. All rights reserved. >> >> Copyright year should just be 2021. > > I copied this code from another file that is "Copyright (c) 2001, 2021" > and renamed just the prefix for the suspendThread() and resumeThread() > functions. Agent_OnLoad() is a straight copy. > > So I think I should keep the copyright as is. Okay. I have no idea what is supposed to happen in such cases. ------------- PR: https://git.openjdk.java.net/jdk/pull/3576 From lzang at openjdk.java.net Thu Apr 22 05:53:35 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Thu, 22 Apr 2021 05:53:35 GMT Subject: RFR: 8265612: revise the help info for jmap histo command [v3] In-Reply-To: References: Message-ID: > This PR revise the help message of the `parallel=[N]` option of command `jmap -histo`. It mainly comes from the discussion at https://github.com/openjdk/jdk/pull/2379#issuecomment-782609582 and https://github.com/openjdk/jdk/pull/2261#issuecomment-823623186. Lin Zang has updated the pull request incrementally with one additional commit since the last revision: fix indentation issue ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3598/files - new: https://git.openjdk.java.net/jdk/pull/3598/files/b14bc8d5..f8da3659 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3598&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3598&range=01-02 Stats: 5 lines in 1 file changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/3598.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3598/head:pull/3598 PR: https://git.openjdk.java.net/jdk/pull/3598 From lzang at openjdk.java.net Thu Apr 22 05:57:21 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Thu, 22 Apr 2021 05:57:21 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: <-qf3V7rOYbOMN9Wf0Q4Nnuz8seoJwA6VeUBQgqLVnak=.ace9f2ac-4669-4c2e-990a-1547533094cc@github.com> References: <9Q69ICl9o5uaRsoDDEa43b9Vl21lzaZIHe2f8mvQy7c=.b1b36b58-787a-4950-9afe-31ca4c79165b@github.com> <-qf3V7rOYbOMN9Wf0Q4Nnuz8seoJwA6VeUBQgqLVnak=.ace9f2ac-4669-4c2e-990a-1547533094cc@github.com> Message-ID: On Wed, 21 Apr 2021 18:40:16 GMT, Chris Plummer wrote: > > > My concerns with your proposed testing is that it always targets the same application with the same heap, and does not read/process the hprof file to make sure it is not corrupt. > > > > > > Hmm, I agree, I will do more test and update here. Thanks! > > As for jtreg testing, we have a few heap dumping related tests. I'm not sure how good they are. It would be good to understand what testing we currently do. > > In addition to jtreg testing, you might want to try just launching some large java apps (netbeans and intellij come to mind), dump their heaps, and then process them with some existing tool. I'm not suggesting this be part of regular testing, but just a sanity check you do on your own before pushing the changes. Thanks a lot for suggestion! I will do these tests and update here. ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From dholmes at openjdk.java.net Thu Apr 22 09:39:52 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 22 Apr 2021 09:39:52 GMT Subject: RFR: 8257831: Suspend with handshakes [v14] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 07:59:09 GMT, Robbin Ehn wrote: >> A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. >> >> Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). >> Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. >> >> Suspend: >> Requesting thread -> synchronous handshake -> target thread >> Inside synchronus handshake (HandshakeState lock is locked while >> executing any handshake): >> - Set suspended flag >> - Install asynchronous handshake >> >> Target thread -> tries to leave dormant state -> Executes handshakes >> Target only executes asynchronous handshake: >> - While suspended >> - Go to blocked >> - Wait on HandshakeState lock >> >> Resume: >> Resuming thread: >> - Lock HandshakeState lock >> - Clear suspended flag >> - Notify HandshakeState lock >> - Unlock HandshakeState lock >> >> The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. >> >> ---- >> Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. >> >> ---- >> Regarding the changed test, the documentation says: >> "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> But the code: >> LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); >> err = jvmti->SuspendThreadList(threads_count, threads, results); >> ... >> // Allow the Main thread to inspect the result of tested threads suspension >> agent_unlock(jni); >> >> The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). >> Thus main thread is stuck forever on: >> // Block until the suspender thread competes the tested threads suspension >> agent_lock(jni); >> >> And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. >> >> ---- >> >> This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. >> (Pre-review comments here: >> https://github.com/openjdk/jdk/pull/2625) >> >> ---- >> Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and >> combinations like running with -XX:ZCollectionInterval=0.01 - >> XX:ZFragmentationLimit=0. >> Running above some of above concurrently (load ~240), slow debug, >> etc... > > Robbin Ehn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits: > > - Merge branch 'master' into SuspendInHandshake > - Merge branch 'master' into SuspendInHandshake > - Removed double tty lock unlocks > - Fixed comment from Dan and fixed name of handshake > - Merge branch 'master' into SuspendInHandshake > - Fixed DavidH second review > - Fixed nits > - Merge branch 'master' into SuspendInHandshake > - Review fixes 4 > - Fixed flag undef dep + spelling error > - ... and 10 more: https://git.openjdk.java.net/jdk/compare/7146104f...ec344661 Approving again for good measure. :) ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3191 From kevinw at openjdk.java.net Thu Apr 22 09:52:35 2021 From: kevinw at openjdk.java.net (Kevin Walls) Date: Thu, 22 Apr 2021 09:52:35 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 00:12:11 GMT, Yasumasa Suenaga wrote: >> `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. >> >> This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Rename --disableregistry to --disable-registry Yes, I think this looks good. If the test ever causes problems with port availability, it could retry on new ports or use a random port, but this looks good for now. ------------- Marked as reviewed by kevinw (Committer). PR: https://git.openjdk.java.net/jdk/pull/3233 From rehn at openjdk.java.net Thu Apr 22 10:21:49 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 22 Apr 2021 10:21:49 GMT Subject: RFR: 8257831: Suspend with handshakes [v14] In-Reply-To: References: Message-ID: <-BXdMPWTk0hjRi1OEycWrIdp5w4N8pyNSxCR1Wq74xo=.901d9044-270a-41ae-946e-486c6ccb2589@github.com> On Thu, 22 Apr 2021 09:36:26 GMT, David Holmes wrote: > Approving again for good measure. :) Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From rehn at openjdk.java.net Thu Apr 22 10:35:47 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 22 Apr 2021 10:35:47 GMT Subject: Integrated: 8257831: Suspend with handshakes In-Reply-To: References: Message-ID: On Thu, 25 Mar 2021 10:56:23 GMT, Robbin Ehn wrote: > A suspend request is done by handshaking thread target thread(s). When executing the handshake operation we know the target mutator thread is in a dormant state (as in safepoint safe state). We have a guarantee that it will check it's poll before leaving the dormant state. To stop the thread from leaving the the dormant state we install a second asynchronous handshake to be executed by the targeted thread. The asynchronous handshake will wait on a monitor while the thread is suspended. The target thread cannot not leave the dormant state without a resume request. > > Per thread suspend requests are naturally serialized by the per thread HandshakeState lock (we can only execute one handshake at a time per thread). > Instead of having a separate lock we use this to our advantage and use HandshakeState lock for serializing access to the suspend flag and for wait/notify. > > Suspend: > Requesting thread -> synchronous handshake -> target thread > Inside synchronus handshake (HandshakeState lock is locked while > executing any handshake): > - Set suspended flag > - Install asynchronous handshake > > Target thread -> tries to leave dormant state -> Executes handshakes > Target only executes asynchronous handshake: > - While suspended > - Go to blocked > - Wait on HandshakeState lock > > Resume: > Resuming thread: > - Lock HandshakeState lock > - Clear suspended flag > - Notify HandshakeState lock > - Unlock HandshakeState lock > > The "suspend requested" flag is an optimization, without it a dormant thread could be suspended and resumed many times and each would add a new asynchronous handshake. Suspend requested flag means there is already an asynchronous suspend handshake in queue which can be re-used, only the suspend flag needs to be set. > > ---- > Some code can be simplified or done in a smarter way but I refrained from doing such changes instead tried to keep existing code as is as far as possible. This concerns especially raw monitors. > > ---- > Regarding the changed test, the documentation says: > "If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > But the code: > LOG("suspendTestedThreads: before JVMTI SuspendThreadList"); > err = jvmti->SuspendThreadList(threads_count, threads, results); > ... > // Allow the Main thread to inspect the result of tested threads suspension > agent_unlock(jni); > > The thread will never return from SuspendThreadList until resumed, so it cannot unlock with agent_unlock(). > Thus main thread is stuck forever on: > // Block until the suspender thread competes the tested threads suspension > agent_lock(jni); > > And never checks and resumes the threads. So I removed that lock instead just sleep and check until all thread have the expected suspended state. > > ---- > > This version already contains updates after pre-review comments from @dcubed-ojdk, @pchilano, @coleenp. > (Pre-review comments here: > https://github.com/openjdk/jdk/pull/2625) > > ---- > Testing t1-t8, nsk_jdi/nsk_jvmti/jdk_jdi/tck, KS24, RunThese and > combinations like running with -XX:ZCollectionInterval=0.01 - > XX:ZFragmentationLimit=0. > Running above some of above concurrently (load ~240), slow debug, > etc... This pull request has now been integrated. Changeset: 86bd44fe Author: Robbin Ehn URL: https://git.openjdk.java.net/jdk/commit/86bd44fe Stats: 1349 lines in 40 files changed: 271 ins; 882 del; 196 mod 8257831: Suspend with handshakes Reviewed-by: dcubed, rrich, dholmes, pchilanomate, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/3191 From ysuenaga at openjdk.java.net Thu Apr 22 13:25:22 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Thu, 22 Apr 2021 13:25:22 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: On Thu, 22 Apr 2021 09:49:56 GMT, Kevin Walls wrote: >> Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: >> >> Rename --disableregistry to --disable-registry > > Yes, I think this looks good. > If the test ever causes problems with port availability, it could retry on new ports or use a random port, but this looks good for now. Thanks @kevinjwalls for the review! > If the test ever causes problems with port availability, it could retry on new ports or use a random port, but this looks good for now. Yeah, it is the point what I want to improve. I want to use random port for it, but I do not yet have good idea to implement it. I will try to work for it in future. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From dcubed at openjdk.java.net Thu Apr 22 14:17:23 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 14:17:23 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating [v2] In-Reply-To: <3ea1rkVUQcFqT8YPeHQvmB0SgCcSYPPwkMab3gajohE=.d3a6c134-68a2-4a43-8ef3-511ed68d3e98@github.com> References: <3ea1rkVUQcFqT8YPeHQvmB0SgCcSYPPwkMab3gajohE=.d3a6c134-68a2-4a43-8ef3-511ed68d3e98@github.com> Message-ID: On Thu, 22 Apr 2021 02:35:26 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes CR changes. > > test/hotspot/jtreg/runtime/Thread/SuspendAtExit.java line 38: > >> 36: private final static String AGENT_LIB = "SuspendAtExit"; >> 37: private final static int DEF_TIME_MAX = 30; // default max # secs to test >> 38: private final static int N_THREADS = 32; > > N_THREADS is unused now. Meant to delete that! Will fix it. ------------- PR: https://git.openjdk.java.net/jdk/pull/3576 From rehn at openjdk.java.net Thu Apr 22 14:17:22 2021 From: rehn at openjdk.java.net (Robbin Ehn) Date: Thu, 22 Apr 2021 14:17:22 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 21:22:50 GMT, Daniel D. Daugherty wrote: >> I'm updating the runtime/Thread/SuspendAtExit.java test: >> >> - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() >> - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() >> - switch from counter-based to time-based testing >> - improve error checking since we're now using an API with error codes! >> >> I've used this test to stress @robehn's fix for JDK-8257831 using both >> invocation styles for 9 hours each in {fastdebug, release, slowdebug} >> configs without any issues. >> >> I've run the updated test thru Mach5 Tier[134567] testing; one timeout >> was observed in a single Tier6 run on Win-X64. I believe this is a case of >> a lost Thread.interrupt() call. > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes CR changes. Marked as reviewed by rehn (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3576 From ysuenaga at openjdk.java.net Thu Apr 22 15:06:25 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Thu, 22 Apr 2021 15:06:25 GMT Subject: Integrated: 8263636: Add --disable-registry option to jhsdb debugd In-Reply-To: References: Message-ID: <4XGoEYqG_sf4VktyA1spk9r2TUvREcy3MrjbnrdMQ3Y=.ea633a8e-142a-4f83-9c73-65f7a5d6b390@github.com> On Sun, 28 Mar 2021 07:53:31 GMT, Yasumasa Suenaga wrote: > `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. > > This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. This pull request has now been integrated. Changeset: 159f5e1e Author: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/159f5e1e Stats: 173 lines in 4 files changed: 158 ins; 3 del; 12 mod 8263636: Add --disable-registry option to jhsdb debugd Reviewed-by: cjplummer, kevinw ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From adinn at openjdk.java.net Thu Apr 22 16:04:27 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Thu, 22 Apr 2021 16:04:27 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter wrote: >> To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. > > Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. Sorry to be late to this party. I've been wanting to read this thread for a while but have bene too busy up to now. I have just a few comments I too was party to the discussions about agent capabilities and recall well the decision to gradually impose restrictions, the first one being to control dynamic agent loading. I was happy to accept that general decision and the specific one to limit the opportunity for an agent self-hoisting into the current JVM. However, a key part of the plan to move forward on restrictions was to provide an override switch. I'd very much like to see that retained as an option. I know that in some uses self-hoisting is much preferable to having to install the agent on the command line and I'd expect he same to be true for any other capabilities for which restrictions were adopted. Although it is true -- as Ron said - -that configuring -javaagent on the command line is always /possible/ there are actually many scenarios for agent use where deployment of an agent after startup is pragmatically much more desirable. An obvious use is trouble shooting, where you only want an agent in place when something goes wrong. That turns out to be critical to solving some seriously difficult support cases. The interesting use cases also fall under testing where self-hoisting of a test agent by a test framework can result in an enormous simplification of test configuration. Usage of Byteman for testing went through the roof with this capability in place. Never underestimate the degree to which even the most minimal configuration complexity puts off Enterprise java developers when it is multiplied by the test suite size of a large project. So, likewise with other restrictions on behaviour, I'm very happy to see them put in place for dynamically hoisted agents so long as there is still a command line override along the lines of the agent attach property that allows a dynamic agent to do all that a command line agent can. One other thing I'd like to correct is a point made in the discussion about agent code residing in the system loader. It is true that the main agent class gets loaded by the system loader. However, it is perfectly possible to ensure that all the rest of the agent code is loaded by the bootstrap loader. A Main class can add the agent jar to the bootstrap path and then load and use reflection to invoke an effective main routine on a bootstrap loaded SubMain class. Byteman uses this trick on request in order to allow it to instrument bootstrap classes. Because all the Byteman classes except for the original Main shell class are loaded by the bootstrap loader Byteman can safely inject references to the Byteman rule engine and Byteman exception classes into bootstrap code. ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From dcubed at openjdk.java.net Thu Apr 22 17:20:43 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 17:20:43 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating [v3] In-Reply-To: References: Message-ID: > I'm updating the runtime/Thread/SuspendAtExit.java test: > > - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() > - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() > - switch from counter-based to time-based testing > - improve error checking since we're now using an API with error codes! > > I've used this test to stress @robehn's fix for JDK-8257831 using both > invocation styles for 9 hours each in {fastdebug, release, slowdebug} > configs without any issues. > > I've run the updated test thru Mach5 Tier[134567] testing; one timeout > was observed in a single Tier6 run on Win-X64. I believe this is a case of > a lost Thread.interrupt() call. Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: dholmes CR - delete unused N_THREADS. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3576/files - new: https://git.openjdk.java.net/jdk/pull/3576/files/8235e5f2..a4aac0d2 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3576&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3576&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3576.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3576/head:pull/3576 PR: https://git.openjdk.java.net/jdk/pull/3576 From cjplummer at openjdk.java.net Thu Apr 22 17:35:37 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 22 Apr 2021 17:35:37 GMT Subject: RFR: 8265612: revise the help info for jmap histo command [v3] In-Reply-To: References: Message-ID: On Thu, 22 Apr 2021 05:53:35 GMT, Lin Zang wrote: >> This PR revise the help message of the `parallel=[N]` option of command `jmap -histo`. It mainly comes from the discussion at https://github.com/openjdk/jdk/pull/2379#issuecomment-782609582 and https://github.com/openjdk/jdk/pull/2261#issuecomment-823623186. > > Lin Zang has updated the pull request incrementally with one additional commit since the last revision: > > fix indentation issue Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3598 From cjplummer at openjdk.java.net Thu Apr 22 18:13:39 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 22 Apr 2021 18:13:39 GMT Subject: RFR: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" Message-ID: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> This bug was introduced by my recent changes for [JDK-8265028](https://bugs.openjdk.java.net/browse/JDK-8265028), which attempted to speed up ThreadNode lookups by not looking in the runningThreads list if the TLS lookup failed. At the time it was thought that the thread could not possibly be on the list, but it turns out sometimes it can. For now I'm just doing a quick fix to replace the assert being triggered with a lookup instead, which is pretty much how it worked before JDK-8265028. However, I eventually want to get back to not having to do the lookup, but first I need to better understand why this is happening in the first place, and the tests are failing too often to wait for that, thus the quick fix. ------------- Commit messages: - Lookup node in runningThreads list rather than assert. Changes: https://git.openjdk.java.net/jdk/pull/3634/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3634&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265683 Stats: 9 lines in 1 file changed: 7 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3634.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3634/head:pull/3634 PR: https://git.openjdk.java.net/jdk/pull/3634 From amenkov at openjdk.java.net Thu Apr 22 20:07:24 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Thu, 22 Apr 2021 20:07:24 GMT Subject: RFR: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" In-Reply-To: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> References: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> Message-ID: On Thu, 22 Apr 2021 15:59:09 GMT, Chris Plummer wrote: > This bug was introduced by my recent changes for [JDK-8265028](https://bugs.openjdk.java.net/browse/JDK-8265028), which attempted to speed up ThreadNode lookups by not looking in the runningThreads list if the TLS lookup failed. At the time it was thought that the thread could not possibly be on the list, but it turns out sometimes it can. > > For now I'm just doing a quick fix to replace the assert being triggered with a lookup instead, which is pretty much how it worked before JDK-8265028. However, I eventually want to get back to not having to do the lookup, but first I need to better understand why this is happening in the first place, and the tests are failing too often to wait for that, thus the quick fix. Marked as reviewed by amenkov (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3634 From dcubed at openjdk.java.net Thu Apr 22 20:14:26 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 20:14:26 GMT Subject: RFR: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" In-Reply-To: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> References: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> Message-ID: On Thu, 22 Apr 2021 15:59:09 GMT, Chris Plummer wrote: > This bug was introduced by my recent changes for [JDK-8265028](https://bugs.openjdk.java.net/browse/JDK-8265028), which attempted to speed up ThreadNode lookups by not looking in the runningThreads list if the TLS lookup failed. At the time it was thought that the thread could not possibly be on the list, but it turns out sometimes it can. > > For now I'm just doing a quick fix to replace the assert being triggered with a lookup instead, which is pretty much how it worked before JDK-8265028. However, I eventually want to get back to not having to do the lookup, but first I need to better understand why this is happening in the first place, and the tests are failing too often to wait for that, thus the quick fix. Thumbs up for the quick and dirty fix. Do you plan to another bug to continue your investigation? src/jdk.jdwp.agent/share/native/libjdwp/threadControl.c line 260: > 258: * thread has terminated, but the ThreadNode may still be present. > 259: */ > 260: if ( node == NULL ) { nit - s/( /(/ and s/ )/)/ src/jdk.jdwp.agent/share/native/libjdwp/threadControl.c line 261: > 259: */ > 260: if ( node == NULL ) { > 261: if (list == NULL || list == &runningThreads ) { nit - s/ )/)/ ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3634 From dcubed at openjdk.java.net Thu Apr 22 20:41:31 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 20:41:31 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 00:12:11 GMT, Yasumasa Suenaga wrote: >> `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. >> >> This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Rename --disableregistry to --disable-registry Serviceability Reviewers! When a new SA test is added, please ask if the new test has been run with ZGC. If it doesn't work with ZGC, then it probably needs to be added to ProblemList-zgc.txt with an entry referring to this bug: JDK-8220624 SA: Out of sync with ZGC's internal data structures ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From dcubed at openjdk.java.net Thu Apr 22 20:41:42 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 20:41:42 GMT Subject: RFR: 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC Message-ID: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> A trivial fix to ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC. ------------- Commit messages: - 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC Changes: https://git.openjdk.java.net/jdk/pull/3641/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3641&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265786 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3641.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3641/head:pull/3641 PR: https://git.openjdk.java.net/jdk/pull/3641 From darcy at openjdk.java.net Thu Apr 22 20:41:42 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Thu, 22 Apr 2021 20:41:42 GMT Subject: RFR: 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC In-Reply-To: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> References: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> Message-ID: <5xqsUIn2UVr-3jNeRsyY9-ccGqVTc1q_HutE4--55NM=.338b58dd-4725-46b0-bb0c-862910e660be@github.com> On Thu, 22 Apr 2021 20:32:33 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC. Marked as reviewed by darcy (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3641 From dcubed at openjdk.java.net Thu Apr 22 20:48:25 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 20:48:25 GMT Subject: Integrated: 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC In-Reply-To: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> References: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> Message-ID: <6M9SoNqyIVD--eY8uEVNetiDuW2-UzQr7kH9hUIyPIM=.b9be38d6-7302-4423-a6a6-ee3d7be15687@github.com> On Thu, 22 Apr 2021 20:32:33 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC. This pull request has now been integrated. Changeset: e81baead Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/e81baead Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC Reviewed-by: darcy ------------- PR: https://git.openjdk.java.net/jdk/pull/3641 From dcubed at openjdk.java.net Thu Apr 22 20:48:24 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 20:48:24 GMT Subject: RFR: 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC In-Reply-To: <5xqsUIn2UVr-3jNeRsyY9-ccGqVTc1q_HutE4--55NM=.338b58dd-4725-46b0-bb0c-862910e660be@github.com> References: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> <5xqsUIn2UVr-3jNeRsyY9-ccGqVTc1q_HutE4--55NM=.338b58dd-4725-46b0-bb0c-862910e660be@github.com> Message-ID: On Thu, 22 Apr 2021 20:34:48 GMT, Joe Darcy wrote: >> A trivial fix to ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC. > > Marked as reviewed by darcy (Reviewer). @jddarcy - Thanks for the fast review! ------------- PR: https://git.openjdk.java.net/jdk/pull/3641 From cjplummer at openjdk.java.net Thu Apr 22 21:47:28 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 22 Apr 2021 21:47:28 GMT Subject: RFR: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" In-Reply-To: References: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> Message-ID: On Thu, 22 Apr 2021 20:11:56 GMT, Daniel D. Daugherty wrote: > Do you plan to another bug to continue your investigation? Yes. The search of runningThreads becomes a performance issue in Loom when the list could have a million items. > src/jdk.jdwp.agent/share/native/libjdwp/threadControl.c line 260: > >> 258: * thread has terminated, but the ThreadNode may still be present. >> 259: */ >> 260: if ( node == NULL ) { > > nit - s/( /(/ and s/ )/)/ The extra spaces are intentional to remain consistent with the surrounding code. ------------- PR: https://git.openjdk.java.net/jdk/pull/3634 From winterhalter at openjdk.java.net Thu Apr 22 21:53:25 2021 From: winterhalter at openjdk.java.net (Rafael Winterhalter) Date: Thu, 22 Apr 2021 21:53:25 GMT Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 20:30:15 GMT, Rafael Winterhalter wrote: >> To allow agents the definition of auxiliary classes, an API is needed to allow this. Currently, this is often achieved by using `sun.misc.Unsafe` or `jdk.internal.misc.Unsafe` ever since the `defineClass` method was removed from `sun.misc.Unsafe`. > > Rafael Winterhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. Setting '-javaagent' is mainly an operations problem. Many build tools do not allow to declare a test dependency this way as the life cycles are not laid out for it, the internal repository location might be machine dependent, for example, and it's difficult to point to a specific folder and file reliably. In this case, I'd say it would be easier to specify a parameter in the sense of '-Djdk.attach.allowAttachSelf=true' as it is a static value. This would however only work for build tools that initiate a new VM for running tests which is not overly attractive for simple builds due to the overhead. Of course, Maven, Gradle and similar tools could set this parameter by default for their own JVM, that part is likely overcomeable but it will certainly create confusion about how to run libraries that are omnipresent today and make the JVM ecosystem less approachable. What bothers me more is the tooling perspective for non-self-attached agents. For example, Aaeron offers a Java agent that adds plenty of debug logging to relevant lines of code. This affects method size and so forth, with Aaeron as a high-performance tool for banking and finance which is written very consciously with regards to the JIT, adding it directly was not feasible. Normally this logging is therefore thrown into a VM in retrospect, once an application starts failing and is already taken off the load balancer. For such a post-mortem, it would be rather annoying to realize that a JVM cannot be attached to with full capabilities if you forgot to set some flag. And often you did of course not consider the VM instance to fail, sometimes it takes months to get a JVM into this buggy state. This would be fairly inconvenient to face. Therefore, I really hope that the dynamic attach from 'outside' the VM will survive without imposing limits and that rather the self-attachment problem will be targeted as such. When I mention a 'jdk.test' module in the Mockito context, I am also rather hoping to improve performance compared to convenience. The problem with '-Djdk.attach.allowAttachSelf=true' is that you still need to start a new VM etc. Since Java 9, running single tests with Mockito has for example become much slower compared to Java 8. Despite the startup performance improvements in the JVM. If one could avoid the location-bound '-javaagent:...', but get the Instrumentation instance directly, I think this would render a real performance improvement in actual execution scenarios. Am Mi., 21. Apr. 2021 um 20:42 Uhr schrieb mlbridge[bot] < ***@***.***>: > *Mailing list message from Alan Bateman ***@***.***> on > core-libs-dev ***@***.***>:* > > On 20/04/2021 21:26, Rafael Winterhalter wrote: > > I have earlier proposed to offer a "jdk.test" module that > offers the Instrumentation instance via a simple API similar to Byte > Buddy's. The JVM would not load this module unless requested on the command > line. Build tools like Maven's surefire or Gradle's testrunner could then > standardize on loading this module as a convention to give access to this > test module by default such that libraries like Mockito could continue to > function out of the box without the libraries functioning on a standard VM > without extra configuration. As far as I know, mainly test libraries need > this API. This would also emphasise that Mockito and others are meant for > testing and fewer people would abuse it for production applications. People > would also have an explicit means of running a JVM for a production > application or for executing a test. > > Helping testing is good but a jdk.test module that hands out the > Instrumentation object could be problematic. Is there a reason why the > runner for Mockito and other mocking frameworks can't specify -javaagent > when launching? I would expect at least some mocking scenarios to > require load time transformations (to drop the final modifier from some > API elements for example) so important to have the transformer set > before classes are loaded. > > As for adding the API, my thought is that if the Instrumentation API were > to throw exceptions on some methods/arguments for dynamic agents in the > future, for example for retransformClasses(Object.class), this breaking > change would then simply extend to the proposed "defineClass" method. In > this sense, the Instrumentation API already assumes full power, I find it > not problematic to add the missing bit to this API even if it was > restricted in the future in the same spirit as other methods of the API > would be. > > I think it would really hard to put this genie back into a bottle. It's > way more attractive to use that than the very agent oriented > redefineModule and retransformClasses. > > I mentioned JNI as it is a well-known approach to defining a class today, > using a minimal native binding to an interface that directly calls down to > JNI's: > > jclass DefineClass(JNIEnv *env, const char *name, jobject loader, const > jbyte *buf, jsize bufLen); > > This interface can then simply be used to define any class just as I > propse, even when not writing an agent or attaching. This method makes > class definitions also already trivial for JVMTI agents compared to Java > agents. Unless restricting JNI, the defineClass method is already a low > hanging fruit, but at the cost of having to maintain a tiny bit of native > code. > > Sure, if you are using native code then you have the full power of JVM > TI and JNI available. Project Panama is exploring how to restrict access > to native code, I think too early to say how this might extend to JNI. > > -Alan > > ? > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ------------- PR: https://git.openjdk.java.net/jdk/pull/3546 From dcubed at openjdk.java.net Thu Apr 22 21:55:26 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 21:55:26 GMT Subject: RFR: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" In-Reply-To: References: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> Message-ID: <19xj8DwowqH9s-5Y4R3AfT0E7VWGqYExyMn-edNZ0tI=.3f9c6397-dc52-42cb-9197-3e0146b774ee@github.com> On Thu, 22 Apr 2021 20:10:43 GMT, Daniel D. Daugherty wrote: >> This bug was introduced by my recent changes for [JDK-8265028](https://bugs.openjdk.java.net/browse/JDK-8265028), which attempted to speed up ThreadNode lookups by not looking in the runningThreads list if the TLS lookup failed. At the time it was thought that the thread could not possibly be on the list, but it turns out sometimes it can. >> >> For now I'm just doing a quick fix to replace the assert being triggered with a lookup instead, which is pretty much how it worked before JDK-8265028. However, I eventually want to get back to not having to do the lookup, but first I need to better understand why this is happening in the first place, and the tests are failing too often to wait for that, thus the quick fix. > > src/jdk.jdwp.agent/share/native/libjdwp/threadControl.c line 261: > >> 259: */ >> 260: if ( node == NULL ) { >> 261: if (list == NULL || list == &runningThreads ) { > > nit - s/ )/)/ In that case, this one is inconsistent. ------------- PR: https://git.openjdk.java.net/jdk/pull/3634 From dcubed at openjdk.java.net Thu Apr 22 21:55:25 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 21:55:25 GMT Subject: RFR: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" In-Reply-To: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> References: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> Message-ID: On Thu, 22 Apr 2021 15:59:09 GMT, Chris Plummer wrote: > This bug was introduced by my recent changes for [JDK-8265028](https://bugs.openjdk.java.net/browse/JDK-8265028), which attempted to speed up ThreadNode lookups by not looking in the runningThreads list if the TLS lookup failed. At the time it was thought that the thread could not possibly be on the list, but it turns out sometimes it can. > > For now I'm just doing a quick fix to replace the assert being triggered with a lookup instead, which is pretty much how it worked before JDK-8265028. However, I eventually want to get back to not having to do the lookup, but first I need to better understand why this is happening in the first place, and the tests are failing too often to wait for that, thus the quick fix. Marked as reviewed by dcubed (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3634 From cjplummer at openjdk.java.net Thu Apr 22 23:14:55 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 22 Apr 2021 23:14:55 GMT Subject: RFR: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" [v2] In-Reply-To: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> References: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> Message-ID: > This bug was introduced by my recent changes for [JDK-8265028](https://bugs.openjdk.java.net/browse/JDK-8265028), which attempted to speed up ThreadNode lookups by not looking in the runningThreads list if the TLS lookup failed. At the time it was thought that the thread could not possibly be on the list, but it turns out sometimes it can. > > For now I'm just doing a quick fix to replace the assert being triggered with a lookup instead, which is pretty much how it worked before JDK-8265028. However, I eventually want to get back to not having to do the lookup, but first I need to better understand why this is happening in the first place, and the tests are failing too often to wait for that, thus the quick fix. Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: Minor formatting fix. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3634/files - new: https://git.openjdk.java.net/jdk/pull/3634/files/784962da..b65a64af Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3634&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3634&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3634.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3634/head:pull/3634 PR: https://git.openjdk.java.net/jdk/pull/3634 From cjplummer at openjdk.java.net Thu Apr 22 23:14:56 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 22 Apr 2021 23:14:56 GMT Subject: RFR: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" [v2] In-Reply-To: <19xj8DwowqH9s-5Y4R3AfT0E7VWGqYExyMn-edNZ0tI=.3f9c6397-dc52-42cb-9197-3e0146b774ee@github.com> References: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> <19xj8DwowqH9s-5Y4R3AfT0E7VWGqYExyMn-edNZ0tI=.3f9c6397-dc52-42cb-9197-3e0146b774ee@github.com> Message-ID: <6BCmFPUV0NSuoPvNNOhoMpd3mEzBrgwYgMfOLdhdZCw=.c71e6d6f-8767-435a-a085-1d30e20c6cdd@github.com> On Thu, 22 Apr 2021 21:52:20 GMT, Daniel D. Daugherty wrote: >> src/jdk.jdwp.agent/share/native/libjdwp/threadControl.c line 261: >> >>> 259: */ >>> 260: if ( node == NULL ) { >>> 261: if (list == NULL || list == &runningThreads ) { >> >> nit - s/ )/)/ > > In that case, this one is inconsistent. Ok. Fixed. ------------- PR: https://git.openjdk.java.net/jdk/pull/3634 From dholmes at openjdk.java.net Thu Apr 22 23:18:24 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 22 Apr 2021 23:18:24 GMT Subject: RFR: 8265240: runtime/Thread/SuspendAtExit.java needs updating [v3] In-Reply-To: References: Message-ID: On Thu, 22 Apr 2021 17:20:43 GMT, Daniel D. Daugherty wrote: >> I'm updating the runtime/Thread/SuspendAtExit.java test: >> >> - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() >> - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() >> - switch from counter-based to time-based testing >> - improve error checking since we're now using an API with error codes! >> >> I've used this test to stress @robehn's fix for JDK-8257831 using both >> invocation styles for 9 hours each in {fastdebug, release, slowdebug} >> configs without any issues. >> >> I've run the updated test thru Mach5 Tier[134567] testing; one timeout >> was observed in a single Tier6 run on Win-X64. I believe this is a case of >> a lost Thread.interrupt() call. > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes CR - delete unused N_THREADS. Marked as reviewed by dholmes (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3576 From cjplummer at openjdk.java.net Thu Apr 22 23:30:27 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 22 Apr 2021 23:30:27 GMT Subject: Integrated: 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" In-Reply-To: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> References: <4UO32CyE6wtb7LZaEMRPZUiXX2ksH7JE7bVcfLZ2QM8=.cd117364-833b-40b5-ad2b-c9d3b016ce33@github.com> Message-ID: On Thu, 22 Apr 2021 15:59:09 GMT, Chris Plummer wrote: > This bug was introduced by my recent changes for [JDK-8265028](https://bugs.openjdk.java.net/browse/JDK-8265028), which attempted to speed up ThreadNode lookups by not looking in the runningThreads list if the TLS lookup failed. At the time it was thought that the thread could not possibly be on the list, but it turns out sometimes it can. > > For now I'm just doing a quick fix to replace the assert being triggered with a lookup instead, which is pretty much how it worked before JDK-8265028. However, I eventually want to get back to not having to do the lookup, but first I need to better understand why this is happening in the first place, and the tests are failing too often to wait for that, thus the quick fix. This pull request has now been integrated. Changeset: a8ddbd15 Author: Chris Plummer URL: https://git.openjdk.java.net/jdk/commit/a8ddbd15 Stats: 9 lines in 1 file changed: 7 ins; 0 del; 2 mod 8265683: vmTestbase/nsk/jdb tests failed with "JDWP exit error AGENT_ERROR_INTERNAL(181)" Reviewed-by: amenkov, dcubed ------------- PR: https://git.openjdk.java.net/jdk/pull/3634 From cjplummer at openjdk.java.net Fri Apr 23 00:39:27 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 23 Apr 2021 00:39:27 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: On Thu, 22 Apr 2021 20:38:50 GMT, Daniel D. Daugherty wrote: > Serviceability Reviewers! > When a new SA test is added, please ask if the new test has been run with ZGC. > If it doesn't work with ZGC, then it probably needs to be added to ProblemList-zgc.txt > with an entry referring to this bug: > > JDK-8220624 SA: Out of sync with ZGC's internal data structures Yes, I've been guilty of this many times both as a committer and as a reviewer. In fact I don't think a new SA test has ever been added and proactively problem listed for ZGC. Will try to do better. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From ysuenaga at openjdk.java.net Fri Apr 23 01:12:28 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Fri, 23 Apr 2021 01:12:28 GMT Subject: RFR: 8263636: Add --disable-registry option to jhsdb debugd [v5] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 00:12:11 GMT, Yasumasa Suenaga wrote: >> `jhsdb debugd` will start RMI registry by default, but we want to prevent it due to port confliction in some cases. We can control it with `sun.jvm.hotspot.rmi.startRegistry` system property. However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. >> >> This PR introduces `--disableregistry` to `jhsdb debugd`. Please review [CSR](https://bugs.openjdk.java.net/browse/JDK-8264021) too. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Rename --disableregistry to --disable-registry Sorry for my change breaks ZGC tests. I keep it in my mind. ------------- PR: https://git.openjdk.java.net/jdk/pull/3233 From vromero at openjdk.java.net Fri Apr 23 02:16:04 2021 From: vromero at openjdk.java.net (Vicente Romero) Date: Fri, 23 Apr 2021 02:16:04 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature in Java [v4] In-Reply-To: References: Message-ID: > Please review this PR that intents to make sealed classes a final feature in Java. This PR contains compiler and VM changes. In line with similar PRs, which has made preview features final, this one is mostly removing preview related comments from APIs plus options in test cases. Please also review the related [CSR](https://bugs.openjdk.java.net/browse/JDK-8265090) > > Thanks > > note: this PR is related to [PR-3528](https://github.com/openjdk/jdk/pull/3528) and must be integrated after it. Vicente Romero has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: - Merge branch 'master' into JDK-8260517 - updating comment after review feedback - removing javax.lang.model changes - Merge branch 'master' into JDK-8260517 - Merge branch 'master' into JDK-8260517 - fixing failing regression tests - JVM related changes - 8260517: Compiler implementation for Sealed Classes ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3526/files - new: https://git.openjdk.java.net/jdk/pull/3526/files/8ebe56fd..43e9cb5f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=02-03 Stats: 40790 lines in 1391 files changed: 8730 ins; 27464 del; 4596 mod Patch: https://git.openjdk.java.net/jdk/pull/3526.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3526/head:pull/3526 PR: https://git.openjdk.java.net/jdk/pull/3526 From dcubed at openjdk.java.net Fri Apr 23 14:23:30 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 23 Apr 2021 14:23:30 GMT Subject: Integrated: 8265240: runtime/Thread/SuspendAtExit.java needs updating In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 19:15:18 GMT, Daniel D. Daugherty wrote: > I'm updating the runtime/Thread/SuspendAtExit.java test: > > - switch from java.lang.Thread.suspend() to JVM/TI SuspendThread() > - switch from java.lang.Thread.resume() to JVM/TI ResumeThread() > - switch from counter-based to time-based testing > - improve error checking since we're now using an API with error codes! > > I've used this test to stress @robehn's fix for JDK-8257831 using both > invocation styles for 9 hours each in {fastdebug, release, slowdebug} > configs without any issues. > > I've run the updated test thru Mach5 Tier[134567] testing; one timeout > was observed in a single Tier6 run on Win-X64. I believe this is a case of > a lost Thread.interrupt() call. This pull request has now been integrated. Changeset: c9b70c80 Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/c9b70c80 Stats: 186 lines in 2 files changed: 144 ins; 12 del; 30 mod 8265240: runtime/Thread/SuspendAtExit.java needs updating Reviewed-by: rehn, dholmes ------------- PR: https://git.openjdk.java.net/jdk/pull/3576 From fmatte at openjdk.java.net Fri Apr 23 15:09:41 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Fri, 23 Apr 2021 15:09:41 GMT Subject: RFR: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][] Message-ID: findComponentType() logic is wrong. In findComponentType() method, We always get vm.classesByName() retruns empty list list = vm.classesByName(parser.typeName()); We have "parser.typeName()" retruns " double[][]" vm.classesByName("") is expecting the fully qualified name example "java.lang.Double" This always returns empty list, resulting into ClassNotLoadedException as it assumes the Component class has not yet been loaded, hence the test case fails. There was a suggested fix from Egor Ushakov from JetBrains, I am proposing the same to get this fix. I have verified the patch with required testing it works fine. ------------- Commit messages: - 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][] Changes: https://git.openjdk.java.net/jdk/pull/3658/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3658&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8221503 Stats: 25 lines in 2 files changed: 0 ins; 23 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3658.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3658/head:pull/3658 PR: https://git.openjdk.java.net/jdk/pull/3658 From javalists at cbfiddle.com Fri Apr 23 19:57:53 2021 From: javalists at cbfiddle.com (Alan Snyder) Date: Fri, 23 Apr 2021 12:57:53 -0700 Subject: RFR: 8200559: Java agents doing instrumentation need a means to define auxiliary classes [v2] In-Reply-To: References: <-fco_my21rYwE8RlEaFlsc-g0WL_rPzbLBWllNqY2rw=.9782b940-c827-4dad-bba8-6901f75fdcf8@github.com> Message-ID: <85FD6F38-71A6-4055-AB02-76DF7BC08A5B@cbfiddle.com> On Apr 21, 2021, at 11:40 AM, Alan Bateman wrote: > > Sure, if you are using native code then you have the full power of JVM TI and JNI available. Project Panama is exploring how to restrict access to native code, I think too early to say how this might extend to JNI. I looked at some of the Panama documents and saw no hint that it might be extended to JNI. It seems to be positioned as an (partial) alternative to JNI. What I do see is that native code has no direct access to the JDK via Panama, only the ability to invoke provided upcalls, which can only access objects and methods that the caller has access to. That is indeed much more restricted than JNI. Even though it is ?too early?, can you explain why you think Panama?s restrictions might apply to JNI? As a library developer, I have often made use of JNI to work around limitations in current and older JDKs. I would hate to lose that ability. Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lmesnik at openjdk.java.net Fri Apr 23 21:14:31 2021 From: lmesnik at openjdk.java.net (Leonid Mesnik) Date: Fri, 23 Apr 2021 21:14:31 GMT Subject: RFR: 8264663: Update test SuspendWithCurrentThread.java to verify that suspend doesn't exit until resumed Message-ID: Test test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java doesn't check that thread stops in SuspendThreadList(...). Actually, before https://bugs.openjdk.java.net/browse/JDK-8257831 the thread didn't suspend itself but only get a request to be suspended. So it continued to execute and stopped a little bit later. Such behavior is a violation of spec which says " If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." While the bug is fixed it is still useful to verify correct behavior. If the fix is backported without JDK-8257831 test should start failing. ------------- Commit messages: - 8264663: Update serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java to verify that suspend don't exit until resumed Changes: https://git.openjdk.java.net/jdk/pull/3665/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3665&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264663 Stats: 8 lines in 1 file changed: 7 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3665.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3665/head:pull/3665 PR: https://git.openjdk.java.net/jdk/pull/3665 From dcubed at openjdk.java.net Fri Apr 23 21:20:35 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 23 Apr 2021 21:20:35 GMT Subject: RFR: 8265880: ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64 Message-ID: A trivial fix to ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64. ------------- Commit messages: - 8265880: ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64 Changes: https://git.openjdk.java.net/jdk/pull/3666/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3666&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265880 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3666.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3666/head:pull/3666 PR: https://git.openjdk.java.net/jdk/pull/3666 From dcubed at openjdk.java.net Fri Apr 23 21:29:24 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 23 Apr 2021 21:29:24 GMT Subject: RFR: 8264663: Update test SuspendWithCurrentThread.java to verify that suspend doesn't exit until resumed In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 21:07:56 GMT, Leonid Mesnik wrote: > Test test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java > doesn't check that thread stops in SuspendThreadList(...). > > Actually, before https://bugs.openjdk.java.net/browse/JDK-8257831 the thread didn't suspend itself but only get a request to be suspended. So it continued to execute and stopped a little bit later. > > Such behavior is a violation of spec which says " If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > While the bug is fixed it is still useful to verify correct behavior. If the fix is backported without JDK-8257831 test should start failing. Thumbs up. I'm good with the changes. I just a question about the use of `std::atomic`. test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/libSuspendWithCurrentThread.cpp line 33: > 31: static jthread* threads = NULL; > 32: static jsize threads_count = 0; > 33: static std::atomic is_exited_from_suspend; Is use of `std::atomic` permitted in tests? ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3665 From rriggs at openjdk.java.net Fri Apr 23 21:31:24 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Fri, 23 Apr 2021 21:31:24 GMT Subject: RFR: 8265880: ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64 In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 21:14:17 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64. Marked as reviewed by rriggs (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3666 From dcubed at openjdk.java.net Fri Apr 23 21:35:23 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 23 Apr 2021 21:35:23 GMT Subject: RFR: 8265880: ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64 In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 21:28:21 GMT, Roger Riggs wrote: >> A trivial fix to ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64. > > Marked as reviewed by rriggs (Reviewer). @RogerRiggs - Thanks for the fast review! ------------- PR: https://git.openjdk.java.net/jdk/pull/3666 From dcubed at openjdk.java.net Fri Apr 23 21:42:23 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 23 Apr 2021 21:42:23 GMT Subject: Integrated: 8265880: ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64 In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 21:14:17 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64. This pull request has now been integrated. Changeset: 6803ab2b Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/6803ab2b Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod 8265880: ProblemList serviceability/dcmd/gc/RunFinalizationTest.java on Linux-X64 Reviewed-by: rriggs ------------- PR: https://git.openjdk.java.net/jdk/pull/3666 From sspitsyn at openjdk.java.net Fri Apr 23 22:08:21 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 23 Apr 2021 22:08:21 GMT Subject: RFR: 8264663: Update test SuspendWithCurrentThread.java to verify that suspend doesn't exit until resumed In-Reply-To: References: Message-ID: <_Q5U0alMK4Jp6eT0JXktyar0kEKd5iZJDl5GsceNg7s=.f6016953-abf9-4cfa-8dc0-846b03918405@github.com> On Fri, 23 Apr 2021 21:07:56 GMT, Leonid Mesnik wrote: > Test test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java > doesn't check that thread stops in SuspendThreadList(...). > > Actually, before https://bugs.openjdk.java.net/browse/JDK-8257831 the thread didn't suspend itself but only get a request to be suspended. So it continued to execute and stopped a little bit later. > > Such behavior is a violation of spec which says " If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > While the bug is fixed it is still useful to verify correct behavior. If the fix is backported without JDK-8257831 test should start failing. Hi Leonid, It looks good to me. I just wonder if it works on all platforms including Windows. Thanks, serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3665 From sspitsyn at openjdk.java.net Fri Apr 23 22:27:36 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Fri, 23 Apr 2021 22:27:36 GMT Subject: RFR: 8264663: Update test SuspendWithCurrentThread.java to verify that suspend doesn't exit until resumed In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 21:26:09 GMT, Daniel D. Daugherty wrote: >> Test test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java >> doesn't check that thread stops in SuspendThreadList(...). >> >> Actually, before https://bugs.openjdk.java.net/browse/JDK-8257831 the thread didn't suspend itself but only get a request to be suspended. So it continued to execute and stopped a little bit later. >> >> Such behavior is a violation of spec which says " If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." >> >> While the bug is fixed it is still useful to verify correct behavior. If the fix is backported without JDK-8257831 test should start failing. > > test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/libSuspendWithCurrentThread.cpp line 33: > >> 31: static jthread* threads = NULL; >> 32: static jsize threads_count = 0; >> 33: static std::atomic is_exited_from_suspend; > > Is use of `std::atomic` permitted in tests? I've never seen any such restrictions in our development guidelines. In general, there are cases where using atomic vs raw monitors is desirable to avoid over sync in tests when there is a need to stress multi-threading. ------------- PR: https://git.openjdk.java.net/jdk/pull/3665 From lmesnik at openjdk.java.net Sat Apr 24 01:47:24 2021 From: lmesnik at openjdk.java.net (Leonid Mesnik) Date: Sat, 24 Apr 2021 01:47:24 GMT Subject: RFR: 8264663: Update test SuspendWithCurrentThread.java to verify that suspend doesn't exit until resumed In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 22:25:07 GMT, Serguei Spitsyn wrote: >> test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/libSuspendWithCurrentThread.cpp line 33: >> >>> 31: static jthread* threads = NULL; >>> 32: static jsize threads_count = 0; >>> 33: static std::atomic is_exited_from_suspend; >> >> Is use of `std::atomic` permitted in tests? > > I've never seen any such restrictions in our development guidelines. > In general, there are cases where using atomic vs raw monitors is desirable to avoid over sync in tests when there is a need to stress multi-threading. I don't know anything about such restrictions also. The atomics are required in this test because the thread actually managed to process the suspend request during RawMonitorEnter if it is used and just suspend right before entering the critical section. So RawMonitorEnter/RawMonitorExitt might not show the problem. Also, as Serguei mentioned atomics synchronization might help us in stress tests. ------------- PR: https://git.openjdk.java.net/jdk/pull/3665 From lmesnik at openjdk.java.net Sat Apr 24 01:59:28 2021 From: lmesnik at openjdk.java.net (Leonid Mesnik) Date: Sat, 24 Apr 2021 01:59:28 GMT Subject: RFR: 8264663: Update test SuspendWithCurrentThread.java to verify that suspend doesn't exit until resumed In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 21:07:56 GMT, Leonid Mesnik wrote: > Test test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java > doesn't check that thread stops in SuspendThreadList(...). > > Actually, before https://bugs.openjdk.java.net/browse/JDK-8257831 the thread didn't suspend itself but only get a request to be suspended. So it continued to execute and stopped a little bit later. > > Such behavior is a violation of spec which says " If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > While the bug is fixed it is still useful to verify correct behavior. If the fix is backported without JDK-8257831 test should start failing. Dan, Serguei. Thank you for review. I've verified that test build and passed on linux/win/mac x64 and linux-aarch64. ------------- PR: https://git.openjdk.java.net/jdk/pull/3665 From ysuenaga at openjdk.java.net Sun Apr 25 02:08:31 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Sun, 25 Apr 2021 02:08:31 GMT Subject: RFR: 8265505: findsym does not work on remote debug server [v2] In-Reply-To: References: Message-ID: <-GGoLAFTdcm55ZYO7pkOwclHH2BUPaXeFoQviRUnQWk=.4fd5d698-99bd-42e6-874f-b59c50c65193@github.com> On Tue, 20 Apr 2021 23:18:43 GMT, Yasumasa Suenaga wrote: >> We can see following error when we run `findsym` on CLHSDB which connects to remote debug server. >> >> >> hsdb> verbose true >> hsdb> findsym gHotSpotVMTypes >> 0x00007f913d4a45b0Error: java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null >> java.lang.NullPointerException: Cannot invoke "sun.jvm.hotspot.debugger.cdbg.CDebugger.loadObjectContainingPC(sun.jvm.hotspot.debugger.Address)" because "cdbg" is null >> at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor$7.doit(CommandProcessor.java:618) >> at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2116) >> at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:2086) >> at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.run(CommandProcessor.java:1957) >> at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:112) >> at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:44) >> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:282) >> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:493) >> >> >> The cause of this NPE is that CDebugger is null. It happens when the debugger is running on debugd server. >> Command line debugger like CLHSDB can delegate the command to debugd, like `pmap` and `pstack`. `findsym` can also use this scheme. >> >> This PR has passed serviceability/sa tests on Linux x64. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Remove unused imports This PR adds a new test [RunCommandOnServerTest.java](https://github.com/YaSuenag/jdk/blob/JDK-8265505/test/hotspot/jtreg/serviceability/sa/sadebugd/RunCommandOnServerTest.java). I confirmed it can be passed with ZGC on Linux x64. I'm waiting for second reviewer. ------------- PR: https://git.openjdk.java.net/jdk/pull/3582 From ysuenaga at openjdk.java.net Sun Apr 25 07:35:10 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Sun, 25 Apr 2021 07:35:10 GMT Subject: RFR: 8263635: Add --prefix option to jhsdb debugd Message-ID: `jhsdb debugd` supports server name prefix with `sun.jvm.hotspot.rmi.serverNamePrefix` system property. It will be used as remote name for SA remote object. It is "SARemoteDebugger" by default. As a result, remote name will be constructed as following: //host[:port]/['_'] However we have no way to set it excepting system property. jhsdb should provide the way to set it as a command line option. And also we should add the way to specify the prefix when we connect to debug server. I will add it like `--connect id at server:1234/prefix`. I've also filed [CSR for this PR](https://bugs.openjdk.java.net/browse/JDK-8265897). Please review it. This PR modifies DisableRegistryTest.java, but it has been addressed in ProblemList-zgc.txt now. So this PR does not affect ZGC. ------------- Commit messages: - 8263635: Add --prefix option to jhsdb debugd Changes: https://git.openjdk.java.net/jdk/pull/3669/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3669&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8263635 Stats: 59 lines in 5 files changed: 27 ins; 14 del; 18 mod Patch: https://git.openjdk.java.net/jdk/pull/3669.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3669/head:pull/3669 PR: https://git.openjdk.java.net/jdk/pull/3669 From shade at openjdk.java.net Sun Apr 25 08:14:58 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Sun, 25 Apr 2021 08:14:58 GMT Subject: RFR: 8264028: Typo in javax.management.relation.RelationService::purgeRelations In-Reply-To: <9LcMa_awvTUUQUrc8y_D3kQl2wFTraWLYfK_prqwsvI=.9a10ef23-f23d-4e1d-8833-c25af1067deb@github.com> References: <9LcMa_awvTUUQUrc8y_D3kQl2wFTraWLYfK_prqwsvI=.9a10ef23-f23d-4e1d-8833-c25af1067deb@github.com> Message-ID: On Sun, 28 Feb 2021 10:48:55 GMT, ?? wrote: > 8264028: Typo in javax.management.relation.RelationService::purgeRelations Marked as reviewed by shade (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/2765 From github.com+22524871+horizonzy at openjdk.java.net Sun Apr 25 08:17:36 2021 From: github.com+22524871+horizonzy at openjdk.java.net (=?UTF-8?B?6LW15bu2?=) Date: Sun, 25 Apr 2021 08:17:36 GMT Subject: Integrated: 8264028: Typo in javax.management.relation.RelationService::purgeRelations In-Reply-To: <9LcMa_awvTUUQUrc8y_D3kQl2wFTraWLYfK_prqwsvI=.9a10ef23-f23d-4e1d-8833-c25af1067deb@github.com> References: <9LcMa_awvTUUQUrc8y_D3kQl2wFTraWLYfK_prqwsvI=.9a10ef23-f23d-4e1d-8833-c25af1067deb@github.com> Message-ID: On Sun, 28 Feb 2021 10:48:55 GMT, ?? wrote: > 8264028: Typo in javax.management.relation.RelationService::purgeRelations This pull request has now been integrated. Changeset: f1f2afda Author: horizonzy <1060026287 at qq.com> Committer: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/f1f2afda Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod 8264028: Typo in javax.management.relation.RelationService::purgeRelations Reviewed-by: sspitsyn, shade ------------- PR: https://git.openjdk.java.net/jdk/pull/2765 From mli at openjdk.java.net Sun Apr 25 12:47:28 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Sun, 25 Apr 2021 12:47:28 GMT Subject: RFR: JDK-8261034: improve jcmd GC.class_histogram to support parallel [v7] In-Reply-To: References: Message-ID: On Sun, 7 Feb 2021 12:38:55 GMT, Hamlin Li wrote: >> parallel -histo of jmap was added by JDK-8214535, it's better to support parallel for jcmd GC.class_histogram too. > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > improve jcmd GC.class_histogram to support parallel still waiting for pr #2261 ------------- PR: https://git.openjdk.java.net/jdk/pull/2379 From lzang at openjdk.java.net Sun Apr 25 15:01:50 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Sun, 25 Apr 2021 15:01:50 GMT Subject: RFR: JDK-8261034: improve jcmd GC.class_histogram to support parallel [v7] In-Reply-To: References: Message-ID: On Sun, 25 Apr 2021 12:44:09 GMT, Hamlin Li wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> improve jcmd GC.class_histogram to support parallel > > still waiting for pr #2261 Hi @Hamlin-Li @plummercj, @sspitsyn I think maybe this PR can be pushed now because in #2261, we finally decided to not expose `parallel=` option for `jmap -dump`. Do you think it is reasonable? Thanks, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/2379 From mli at openjdk.java.net Mon Apr 26 02:22:47 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 02:22:47 GMT Subject: RFR: JDK-8261034: improve jcmd GC.class_histogram to support parallel [v8] In-Reply-To: References: Message-ID: > parallel -histo of jmap was added by JDK-8214535, it's better to support parallel for jcmd GC.class_histogram too. Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: refine help message. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2379/files - new: https://git.openjdk.java.net/jdk/pull/2379/files/ec2324ec..4584c3a5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2379&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2379&range=06-07 Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/2379.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2379/head:pull/2379 PR: https://git.openjdk.java.net/jdk/pull/2379 From mli at openjdk.java.net Mon Apr 26 02:40:30 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 02:40:30 GMT Subject: RFR: JDK-8261034: improve jcmd GC.class_histogram to support parallel [v8] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 02:22:47 GMT, Hamlin Li wrote: >> parallel -histo of jmap was added by JDK-8214535, it's better to support parallel for jcmd GC.class_histogram too. > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > refine help message. Thanks Lin for reminding. ------------- PR: https://git.openjdk.java.net/jdk/pull/2379 From lzang at openjdk.java.net Mon Apr 26 12:31:34 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Mon, 26 Apr 2021 12:31:34 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 22:20:17 GMT, Chris Plummer wrote: >> Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 25 commits: >> >> - Merge branch 'master' into par-dump >> - Merge branch 'master' into par-dump >> - Typo fix >> - remove parallel option and dumpheapext command >> - Revert "hide the dumpheapext error message" >> >> This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. >> - resize large object threshold >> - Merge branch 'master' into par-dump >> - Merge branch 'master' into par-dump >> - code clean >> - fix trailing white space issue >> - ... and 15 more: https://git.openjdk.java.net/jdk/compare/e2106d5a...997784b1 > > src/hotspot/share/services/heapDumperCompression.cpp line 240: > >> 238: } >> 239: >> 240: void CompressionBackend::flush_buffer() { > > This method looks to be indentical to `CompressionBackend::deactivate()` except `CompressionBackend::deactivate()` does a couple more tasks at the end. Perhaps `CompressionBackend::deactivate()` should be calling this now. The reason for dup of the code is that both method use the MonitorLocker, but with different scope. Do you think that it is better if I extract those common codes as a method that accept a MonitorLocker as an argument? Such as `flush_buffer_without_lock(MonitorLocker* lock)`. ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From lzang at openjdk.java.net Mon Apr 26 12:45:56 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Mon, 26 Apr 2021 12:45:56 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v21] In-Reply-To: References: Message-ID: > 8252842: Extend jmap to support parallel heap dump Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 27 commits: - Merge branch 'master' into par-dump - refine with several fix - Merge branch 'master' into par-dump - Merge branch 'master' into par-dump - Typo fix - remove parallel option and dumpheapext command - Revert "hide the dumpheapext error message" This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. - resize large object threshold - Merge branch 'master' into par-dump - Merge branch 'master' into par-dump - ... and 17 more: https://git.openjdk.java.net/jdk/compare/c3ac6900...6d14790a ------------- Changes: https://git.openjdk.java.net/jdk/pull/2261/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2261&range=20 Stats: 861 lines in 6 files changed: 653 ins; 63 del; 145 mod Patch: https://git.openjdk.java.net/jdk/pull/2261.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2261/head:pull/2261 PR: https://git.openjdk.java.net/jdk/pull/2261 From lzang at openjdk.java.net Mon Apr 26 12:48:46 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Mon, 26 Apr 2021 12:48:46 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 12:35:27 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 25 commits: > > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - code clean > - fix trailing white space issue > - ... and 15 more: https://git.openjdk.java.net/jdk/compare/e2106d5a...997784b1 Update the PR that refined based on comments from Chris. @plummercj , thanks very much for your patience reviewing this pr. Testing is still WIP. I will update the result asap. BRs, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From lmesnik at openjdk.java.net Mon Apr 26 19:52:37 2021 From: lmesnik at openjdk.java.net (Leonid Mesnik) Date: Mon, 26 Apr 2021 19:52:37 GMT Subject: Integrated: 8264663: Update test SuspendWithCurrentThread.java to verify that suspend doesn't exit until resumed In-Reply-To: References: Message-ID: <6IjORIMxH9GR2wV35viKUkrH-14xeV-qPWQPsI1GOLQ=.3a6532bf-7f80-448e-b775-8ff7b8bbe106@github.com> On Fri, 23 Apr 2021 21:07:56 GMT, Leonid Mesnik wrote: > Test test/hotspot/jtreg/serviceability/jvmti/SuspendWithCurrentThread/SuspendWithCurrentThread.java > doesn't check that thread stops in SuspendThreadList(...). > > Actually, before https://bugs.openjdk.java.net/browse/JDK-8257831 the thread didn't suspend itself but only get a request to be suspended. So it continued to execute and stopped a little bit later. > > Such behavior is a violation of spec which says " If the calling thread is specified in the request_list array, this function will not return until some other thread resumes it." > > While the bug is fixed it is still useful to verify correct behavior. If the fix is backported without JDK-8257831 test should start failing. This pull request has now been integrated. Changeset: b5c63513 Author: Leonid Mesnik URL: https://git.openjdk.java.net/jdk/commit/b5c63513 Stats: 8 lines in 1 file changed: 7 ins; 0 del; 1 mod 8264663: Update test SuspendWithCurrentThread.java to verify that suspend doesn't exit until resumed Reviewed-by: dcubed, sspitsyn ------------- PR: https://git.openjdk.java.net/jdk/pull/3665 From kvn at openjdk.java.net Mon Apr 26 22:45:25 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 26 Apr 2021 22:45:25 GMT Subject: RFR: 8264805: Remove the experimental Ahead-of-Time Compiler [v6] In-Reply-To: References: Message-ID: > As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: > > - `jdk.aot` module ? the `jaotc` tool > - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution > - related HotSpot code guarded by `#if INCLUDE_AOT` > > Additionally, remove tests as well as code in makefiles related to AoT compilation. > > Tested hs-tier1-4 Vladimir Kozlov has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: - Merge branch 'master' into JDK-8264805 - Address review comments - Remove exports from Graal module to jdk.aot - Merge branch 'master' into JDK-8264805 - Merge branch 'master' into JDK-8264805 - 8264805: Remove the experimental Ahead-of-Time Compiler ------------- Changes: https://git.openjdk.java.net/jdk/pull/3398/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3398&range=05 Stats: 26972 lines in 378 files changed: 2 ins; 26772 del; 198 mod Patch: https://git.openjdk.java.net/jdk/pull/3398.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3398/head:pull/3398 PR: https://git.openjdk.java.net/jdk/pull/3398 From kvn at openjdk.java.net Tue Apr 27 01:15:37 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Tue, 27 Apr 2021 01:15:37 GMT Subject: Integrated: 8264805: Remove the experimental Ahead-of-Time Compiler In-Reply-To: References: Message-ID: <3MIAR4n0RJot9rerXkZf7hB7NlYhJbPI6ri5q9IP3SM=.c6fecb4a-0d84-4c78-bf41-b750ee03d6df@github.com> On Thu, 8 Apr 2021 15:23:52 GMT, Vladimir Kozlov wrote: > As part of [JEP 410](http://openjdk.java.net/jeps/410) remove code related to Ahead-of-Time Compiler from JDK: > > - `jdk.aot` module ? the `jaotc` tool > - `src/hotspot/share/aot` ? loads AoT compiled code into VM for execution > - related HotSpot code guarded by `#if INCLUDE_AOT` > > Additionally, remove tests as well as code in makefiles related to AoT compilation. > > Tested hs-tier1-4 This pull request has now been integrated. Changeset: 694acedf Author: Vladimir Kozlov URL: https://git.openjdk.java.net/jdk/commit/694acedf Stats: 26972 lines in 378 files changed: 2 ins; 26772 del; 198 mod 8264805: Remove the experimental Ahead-of-Time Compiler Reviewed-by: coleenp, erikj, stefank, iignatyev, dholmes, aph, shade, iklam, mchung, iveresov ------------- PR: https://git.openjdk.java.net/jdk/pull/3398 From mli at openjdk.java.net Tue Apr 27 01:16:42 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Tue, 27 Apr 2021 01:16:42 GMT Subject: RFR: JDK-8261034: improve jcmd GC.class_histogram to support parallel [v8] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 02:22:47 GMT, Hamlin Li wrote: >> parallel -histo of jmap was added by JDK-8214535, it's better to support parallel for jcmd GC.class_histogram too. > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > refine help message. Seems the build failures on windows are not related to this change. So, I think this change is ready for a reviewing. ------------- PR: https://git.openjdk.java.net/jdk/pull/2379 From yyang at openjdk.java.net Tue Apr 27 09:50:45 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Tue, 27 Apr 2021 09:50:45 GMT Subject: RFR: 8265914: Duplicated NotANode and not_a_node Message-ID: NotANode(node.hpp) and not_a_node(compile.cpp) are completely repeated guys. ------------- Commit messages: - not_a_node Changes: https://git.openjdk.java.net/jdk/pull/3715/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3715&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265914 Stats: 36 lines in 5 files changed: 0 ins; 14 del; 22 mod Patch: https://git.openjdk.java.net/jdk/pull/3715.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3715/head:pull/3715 PR: https://git.openjdk.java.net/jdk/pull/3715 From ysuenaga at openjdk.java.net Tue Apr 27 11:53:54 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 27 Apr 2021 11:53:54 GMT Subject: RFR: 8266038: Move newAddress() to JVMDebugger Message-ID: SA has `newAddress()` to create `Address` instance, however it is declared in each debugger classes (e.g. `LinuxDebugger`). So we can't access it directly from `VM` class. Before SA improvement for ZGC in [JDK-8220624](https://bugs.openjdk.java.net/browse/JDK-8220624), we need to move `newAddress()` to super class (`JVMDebugger`). ------------- Commit messages: - 8266038: Move newAddress() to JVMDebugger Changes: https://git.openjdk.java.net/jdk/pull/3714/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3714&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266038 Stats: 10 lines in 5 files changed: 1 ins; 4 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/3714.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3714/head:pull/3714 PR: https://git.openjdk.java.net/jdk/pull/3714 From kevinw at openjdk.java.net Tue Apr 27 12:32:33 2021 From: kevinw at openjdk.java.net (Kevin Walls) Date: Tue, 27 Apr 2021 12:32:33 GMT Subject: RFR: 8266038: Move newAddress() to JVMDebugger In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 09:19:34 GMT, Yasumasa Suenaga wrote: > SA has `newAddress()` to create `Address` instance, however it is declared in each debugger classes (e.g. `LinuxDebugger`). So we can't access it directly from `VM` class. > > Before SA improvement for ZGC in [JDK-8220624](https://bugs.openjdk.java.net/browse/JDK-8220624), we need to move `newAddress()` to super class (`JVMDebugger`). Marked as reviewed by kevinw (Committer). Moving this common method up the hierarchy looks good to me. ------------- PR: https://git.openjdk.java.net/jdk/pull/3714 From dcubed at openjdk.java.net Tue Apr 27 19:36:19 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 27 Apr 2021 19:36:19 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 00:08:52 GMT, Daniel D. Daugherty wrote: > The synopsis pretty much says it all. There's a more detailed history in the RFE itself. > > Currently running the new test thru Mach5 Tier[1-7]. > I've run the test thru several 12 hour runs on my MBP13 and > on my Linux-X64 server. Tap, tap, tap... is this thing working? :-) ------------- PR: https://git.openjdk.java.net/jdk/pull/3478 From amenkov at openjdk.java.net Tue Apr 27 21:38:27 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Tue, 27 Apr 2021 21:38:27 GMT Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes Message-ID: Class loading may cause loading of some other system/internal classes (for example loading of java.util.concurrent classes when an other thread loads some classes concurrently). The fix updates ClassPrepare test so it skip events for "unexpected" classes. Also it adds a testcase to verify that classes which are explicitly loaded on other thread do not generate ClassPrepare event on the test thread. ------------- Commit messages: - ClassPrepare test fix Changes: https://git.openjdk.java.net/jdk/pull/3732/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3732&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266002 Stats: 81 lines in 2 files changed: 47 ins; 16 del; 18 mod Patch: https://git.openjdk.java.net/jdk/pull/3732.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3732/head:pull/3732 PR: https://git.openjdk.java.net/jdk/pull/3732 From amenkov at openjdk.java.net Tue Apr 27 21:47:20 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Tue, 27 Apr 2021 21:47:20 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 20:51:49 GMT, Alex Menkov wrote: >> The test actually failed starting from jdk13, but the error is masked by JDK-8264667 (and shell part of the test didn't verify result of the java execution) >> The fix: >> - updates JvmtiClassFileReconstituter to add attributes in the same order as javac does >> - makes the test java instead of shell >> - added workaround for JDK-8264667 >> - test code is ready to ignore order of attributes > > Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: > > Updated the fix accordingly feedback Ping ------------- PR: https://git.openjdk.java.net/jdk/pull/3426 From coleenp at openjdk.java.net Tue Apr 27 22:29:08 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 27 Apr 2021 22:29:08 GMT Subject: RFR: 8262002: java/lang/instrument/VerifyLocalVariableTableOnRetransformTest.sh failed with "TestCaseScaffoldException: DummyClassWithLVT did not match .class file" [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 20:51:49 GMT, Alex Menkov wrote: >> The test actually failed starting from jdk13, but the error is masked by JDK-8264667 (and shell part of the test didn't verify result of the java execution) >> The fix: >> - updates JvmtiClassFileReconstituter to add attributes in the same order as javac does >> - makes the test java instead of shell >> - added workaround for JDK-8264667 >> - test code is ready to ignore order of attributes > > Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: > > Updated the fix accordingly feedback Looks good and great to replace the shell file in the process. ------------- Marked as reviewed by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3426 From cjplummer at openjdk.java.net Tue Apr 27 23:21:07 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 27 Apr 2021 23:21:07 GMT Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 21:30:09 GMT, Alex Menkov wrote: > Class loading may cause loading of some other system/internal classes (for example loading of java.util.concurrent classes when an other thread loads some classes concurrently). > The fix updates ClassPrepare test so it skip events for "unexpected" classes. > Also it adds a testcase to verify that classes which are explicitly loaded on other thread do not generate ClassPrepare event on the test thread. I don't see any mention in the CR of the test failing due to this issue. How was it discovered? How is the fix being verified? test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassPrepare/classprep001/classprep001.cpp line 65: > 63: { "Lnsk/jvmti/ClassPrepare/classprep001$TestClass;", EXP_STATUS, 3, 2, 1 } > 64: }; > 65: // This classes are loaded on a different thread. "These classes..." test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassPrepare/classprep001/classprep001.cpp line 292: > 290: > 291: JNIEXPORT void JNICALL > 292: Java_nsk_jvmti_ClassPrepare_classprep001_getReady(JNIEnv *env, jclass cls, jthread thread) { It's not clear to me why you now pass in the thread instead of calling GetCurrentThread, when the thread passed in is still always the current thread. Same thing for the `check()` method. ------------- Changes requested by cjplummer (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3732 From cjplummer at openjdk.java.net Tue Apr 27 23:25:23 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 27 Apr 2021 23:25:23 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 00:08:52 GMT, Daniel D. Daugherty wrote: > The synopsis pretty much says it all. There's a more detailed history in the RFE itself. > > Currently running the new test thru Mach5 Tier[1-7]. > I've run the test thru several 12 hour runs on my MBP13 and > on my Linux-X64 server. When reviewing this new test, is it worth comparing it with counter based tests that it was based on, or is it best just to view it as a completely new test. ------------- PR: https://git.openjdk.java.net/jdk/pull/3478 From cjplummer at openjdk.java.net Tue Apr 27 23:50:53 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Tue, 27 Apr 2021 23:50:53 GMT Subject: RFR: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][] In-Reply-To: References: Message-ID: <_WRGPHgrZJ-7Vc7aX_2dnVYwSKwaehGfow0u2fIDBds=.3210bf41-161f-49c3-a3f7-a1ecbce238eb@github.com> On Fri, 23 Apr 2021 15:03:42 GMT, Fairoz Matte wrote: > findComponentType() logic is wrong. In findComponentType() method, We always get vm.classesByName() retruns empty list > list = vm.classesByName(parser.typeName()); > We have "parser.typeName()" retruns " double[][]" > vm.classesByName("") is expecting the fully qualified name example "java.lang.Double" > This always returns empty list, resulting into ClassNotLoadedException as it assumes the Component class has not yet been loaded, hence the test case fails. > > There was a suggested fix from Egor Ushakov from JetBrains, I am proposing the same to get this fix. I have verified the patch with required testing it works fine. src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java line 94: > 92: */ > 93: Type findComponentType(String signature) throws ClassNotLoadedException { > 94: return findType(signature); Do we even need `findComponentType()` any more? Isn't `ReferenceTypeImpl.findType()` sufficient. The comment above `findComponentType()` is kind of explicit as to why it was needed. Are you sure none of that still applies, and there isn't some edge case that `findType()` is not covering? ------------- PR: https://git.openjdk.java.net/jdk/pull/3658 From dcubed at openjdk.java.net Tue Apr 27 23:55:53 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 27 Apr 2021 23:55:53 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 00:08:52 GMT, Daniel D. Daugherty wrote: > The synopsis pretty much says it all. There's a more detailed history in the RFE itself. > > Currently running the new test thru Mach5 Tier[1-7]. > I've run the test thru several 12 hour runs on my MBP13 and > on my Linux-X64 server. It is not really useful to compare with the counter based M&M tests because this test is exercising/stressing a different API. It could be useful to compare this test to runtime/Thread/SuspendAtExit.java which is a test that was also updated to be time based instead of counter based. I am reworking the remaining existing counter based Thread-SMR tests with a different bug ID (https://bugs.openjdk.java.net/browse/JDK-8266130), but that is a work-in-progress. ------------- PR: https://git.openjdk.java.net/jdk/pull/3478 From whuang at openjdk.java.net Wed Apr 28 06:53:18 2021 From: whuang at openjdk.java.net (Wang Huang) Date: Wed, 28 Apr 2021 06:53:18 GMT Subject: RFR: 8266187: Memory leak in appendBootClassPath() Message-ID: Dear All, I find a memory leak in `appendBootClassPath()` https://github.com/openjdk/jdk/blob/75a2354dc276e107d64516d20fc72bc7ef3d5f86/src/java.instrument/share/native/libinstrument/InvocationAdapter.c#L950 we malloc `resolved` from resolve(parent, path) * we use `resolved` in line 951 * we don't free() this memory after using. I think we can fix this bug by adding a free() after line 951 as my commit. Thank you for your review. Any suggestion is welcome. Yours , Wang Huang ------------- Commit messages: - 8266187: Memory leak in appendBootClassPath() Changes: https://git.openjdk.java.net/jdk/pull/3751/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3751&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266187 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3751.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3751/head:pull/3751 PR: https://git.openjdk.java.net/jdk/pull/3751 From fmatte at openjdk.java.net Wed Apr 28 08:01:52 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Wed, 28 Apr 2021 08:01:52 GMT Subject: RFR: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][] In-Reply-To: <_WRGPHgrZJ-7Vc7aX_2dnVYwSKwaehGfow0u2fIDBds=.3210bf41-161f-49c3-a3f7-a1ecbce238eb@github.com> References: <_WRGPHgrZJ-7Vc7aX_2dnVYwSKwaehGfow0u2fIDBds=.3210bf41-161f-49c3-a3f7-a1ecbce238eb@github.com> Message-ID: On Tue, 27 Apr 2021 23:47:26 GMT, Chris Plummer wrote: >> findComponentType() logic is wrong. In findComponentType() method, We always get vm.classesByName() retruns empty list >> list = vm.classesByName(parser.typeName()); >> We have "parser.typeName()" retruns " double[][]" >> vm.classesByName("") is expecting the fully qualified name example "java.lang.Double" >> This always returns empty list, resulting into ClassNotLoadedException as it assumes the Component class has not yet been loaded, hence the test case fails. >> >> There was a suggested fix from Egor Ushakov from JetBrains, I am proposing the same to get this fix. I have verified the patch with required testing it works fine. > > src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java line 94: > >> 92: */ >> 93: Type findComponentType(String signature) throws ClassNotLoadedException { >> 94: return findType(signature); > > Do we even need `findComponentType()` any more? Isn't `ReferenceTypeImpl.findType()` sufficient. > > The comment above `findComponentType()` is kind of explicit as to why it was needed. Are you sure none of that still applies, and there isn't some edge case that `findType()` is not covering? Hi Chris, Thanks for looking into this, >Do we even need findComponentType() any more? Isn't ReferenceTypeImpl.findType() sufficient. We still need findComponentType(), Difference between findType() and findComponentType() is that, findComponentType() tries to get the list of ReferenceType from the "vm.classesByName". In case list is empty, it explicitly throws ClassNotLoadedException. This exception check is required in validateAssignment(ValueContainer destination) call from ObjectReferenceImpl.java. https://github.com/openjdk/jdk/blob/master/src/jdk.jdi/share/classes/com/sun/tools/jdi/ObjectReferenceImpl.java#L579 diff --git a/src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java b/src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java index e544c81ae3e..54deba43894 100644 --- a/src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java +++ b/src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java @@ -95,20 +95,13 @@ public class ArrayTypeImpl extends ReferenceTypeImpl JNITypeParser sig = new JNITypeParser(signature); if (sig.isReference()) { // It's a reference type - JNITypeParser parser = new JNITypeParser(componentSignature()); - List list = vm.classesByName(parser.typeName()); - Iterator iter = list.iterator(); - while (iter.hasNext()) { - ReferenceType type = iter.next(); - ClassLoaderReference cl = type.classLoader(); - if ((cl == null)? - (classLoader() == null) : - (cl.equals(classLoader()))) { - return type; - } + try { + Type componentType = findType(componentSignature()); + return componentType; + // Component class has not yet been loaded + } catch (ClassNotLoadedException ex) { + throw new ClassNotLoadedException(componentTypeName()); } - // Component class has not yet been loaded - throw new ClassNotLoadedException(componentTypeName()); } else { // It's a primitive type return vm.primitiveTypeMirror(sig.jdwpTag()); Thanks, ------------- PR: https://git.openjdk.java.net/jdk/pull/3658 From ayang at openjdk.java.net Wed Apr 28 09:30:10 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Wed, 28 Apr 2021 09:30:10 GMT Subject: RFR: 8234532: Remove ThreadLocalAllocBuffer::_fast_refill_waste since it is never set Message-ID: It has some cascading effect; two performance variables, `fastWaste` and `maxFastWaste`, can be removed also. ------------- Commit messages: - refill Changes: https://git.openjdk.java.net/jdk/pull/3756/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3756&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8234532 Stats: 47 lines in 6 files changed: 0 ins; 40 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3756.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3756/head:pull/3756 PR: https://git.openjdk.java.net/jdk/pull/3756 From ayang at openjdk.java.net Wed Apr 28 09:41:14 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Wed, 28 Apr 2021 09:41:14 GMT Subject: RFR: 8234532: Remove ThreadLocalAllocBuffer::_fast_refill_waste since it is never set [v2] In-Reply-To: References: Message-ID: > It has some cascading effect; two performance variables, `fastWaste` and `maxFastWaste`, can be removed also. Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - merge - refill ------------- Changes: https://git.openjdk.java.net/jdk/pull/3756/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3756&range=01 Stats: 42 lines in 5 files changed: 0 ins; 35 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3756.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3756/head:pull/3756 PR: https://git.openjdk.java.net/jdk/pull/3756 From kevinw at openjdk.java.net Wed Apr 28 12:19:59 2021 From: kevinw at openjdk.java.net (Kevin Walls) Date: Wed, 28 Apr 2021 12:19:59 GMT Subject: RFR: 8266187: Memory leak in appendBootClassPath() In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 06:38:57 GMT, Wang Huang wrote: > Dear All, > I find a memory leak in `appendBootClassPath()` > https://github.com/openjdk/jdk/blob/75a2354dc276e107d64516d20fc72bc7ef3d5f86/src/java.instrument/share/native/libinstrument/InvocationAdapter.c#L950 > * we malloc `resolved` from resolve(parent, path) > * we use `resolved` in line 951 > * we don't free() this memory after using. > > I think we can fix this bug by adding a free() after line 951 as my commit. > Thank you for your review. Any suggestion is welcome. > > Yours , > Wang Huang Hi, I didn't find it immediately obvious that this was safe, but I followed things and think that it is correct: The malloc'd pointer gets passed to... JvmtiEnv::AddToBootstrapClassLoaderSearch(const char* segment) { which calls ClassPathZipEntry* ClassLoader::create_class_path_zip_entry(const char *path, bool is_boot_append) { ..which calls char* ClassLoader::get_canonical_path(const char* orig, Thread* thread) { ...which makes a copy of the string: char* orig_copy = NEW_RESOURCE_ARRAY_IN_THREAD(thread, char, strlen(orig)+1); strcpy(orig_copy, orig); ...and doesn't apear to keep the pointer. So yes I think we should free it. 8-) ------------- PR: https://git.openjdk.java.net/jdk/pull/3751 From kevinw at openjdk.java.net Wed Apr 28 12:32:53 2021 From: kevinw at openjdk.java.net (Kevin Walls) Date: Wed, 28 Apr 2021 12:32:53 GMT Subject: RFR: 8266187: Memory leak in appendBootClassPath() In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 06:38:57 GMT, Wang Huang wrote: > Dear All, > I find a memory leak in `appendBootClassPath()` > https://github.com/openjdk/jdk/blob/75a2354dc276e107d64516d20fc72bc7ef3d5f86/src/java.instrument/share/native/libinstrument/InvocationAdapter.c#L950 > * we malloc `resolved` from resolve(parent, path) > * we use `resolved` in line 951 > * we don't free() this memory after using. > > I think we can fix this bug by adding a free() after line 951 as my commit. > Thank you for your review. Any suggestion is welcome. > > Yours , > Wang Huang Marked as reviewed by kevinw (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3751 From dholmes at openjdk.java.net Wed Apr 28 13:05:50 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 28 Apr 2021 13:05:50 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() In-Reply-To: References: Message-ID: <8vtwdpSZT5T3xeXvBMLFtnkPAcTRnqf8KziTzEim43Q=.e791053c-0907-4a32-a111-739f590352da@github.com> On Wed, 14 Apr 2021 00:08:52 GMT, Daniel D. Daugherty wrote: > The synopsis pretty much says it all. There's a more detailed history in the RFE itself. > > Currently running the new test thru Mach5 Tier[1-7]. > I've run the test thru several 12 hour runs on my MBP13 and > on my Linux-X64 server. HI Dan, Some minor comments, but to be frank I have no idea what this test is actually doing - sorry. Cheers, David test/hotspot/jtreg/serviceability/monitoring/ThreadInfo/getLockOwnerName/getLockOwnerName.java line 30: > 28: * non-null string for a blocked thread and then makes repeated calls > 29: * to getThreadInfo() and ThreadInfo.getLockOwnerName() until the thread > 30: * has exited. Extra space before has. test/hotspot/jtreg/serviceability/monitoring/ThreadInfo/getLockOwnerName/getLockOwnerName.java line 33: > 31: * @requires vm.jvmti > 32: * @library /test/lib > 33: * @compile getLockOwnerName.java You don't need a @compile statement for the current test file test/hotspot/jtreg/serviceability/monitoring/ThreadInfo/getLockOwnerName/getLockOwnerName.java line 137: > 135: } > 136: > 137: System.exit(run(timeMax, System.out) + exit_delta); jtreg tests don't use System.exit! test/hotspot/jtreg/serviceability/monitoring/ThreadInfo/getLockOwnerName/getLockOwnerName.java line 189: > 187: while (testState != TS_BLOCKER_RUNNING) { > 188: try { > 189: barrierLaunch.wait(0); // wait until it is running Nit: we use wait() rather than wait(0) ------------- PR: https://git.openjdk.java.net/jdk/pull/3478 From github.com+2217224+kariya-mitsuru at openjdk.java.net Wed Apr 28 13:12:07 2021 From: github.com+2217224+kariya-mitsuru at openjdk.java.net (Mitsuru kariya) Date: Wed, 28 Apr 2021 13:12:07 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation Message-ID: ?tation ------------- Commit messages: - 8264734: SA's Address subclasses could use better hashCode() implementation Changes: https://git.openjdk.java.net/jdk/pull/3522/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3522&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264734 Stats: 12 lines in 6 files changed: 0 ins; 6 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3522.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3522/head:pull/3522 PR: https://git.openjdk.java.net/jdk/pull/3522 From amenkov at openjdk.java.net Wed Apr 28 19:21:56 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 28 Apr 2021 19:21:56 GMT Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 23:15:35 GMT, Chris Plummer wrote: > I don't see any mention in the CR of the test failing due to this issue. How was it discovered? How is the fix being verified? I discovered it researching JCK failure on a platform with UsageTracker enabled. JCK test does exactly the same as this test, but I was not able to reproduce the failure for this test (but this is a race, so that's normal) I ensured the fix works during development when created otherTread before getReady and had printdump on. It produced a lot of ClassPrepare event for Thread and lambda-related classes. > test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassPrepare/classprep001/classprep001.cpp line 292: > >> 290: >> 291: JNIEXPORT void JNICALL >> 292: Java_nsk_jvmti_ClassPrepare_classprep001_getReady(JNIEnv *env, jclass cls, jthread thread) { > > It's not clear to me why you now pass in the thread instead of calling GetCurrentThread, when the thread passed in is still always the current thread. Same thing for the `check()` method. I think it makes the code more flexible and easier to extend ------------- PR: https://git.openjdk.java.net/jdk/pull/3732 From amenkov at openjdk.java.net Wed Apr 28 19:32:12 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 28 Apr 2021 19:32:12 GMT Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes [v2] In-Reply-To: References: Message-ID: > Class loading may cause loading of some other system/internal classes (for example loading of java.util.concurrent classes when an other thread loads some classes concurrently). > The fix updates ClassPrepare test so it skip events for "unexpected" classes. > Also it adds a testcase to verify that classes which are explicitly loaded on other thread do not generate ClassPrepare event on the test thread. Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: Updated comment ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3732/files - new: https://git.openjdk.java.net/jdk/pull/3732/files/23d5ec66..4cebb5d6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3732&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3732&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3732.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3732/head:pull/3732 PR: https://git.openjdk.java.net/jdk/pull/3732 From amenkov at openjdk.java.net Wed Apr 28 19:32:13 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 28 Apr 2021 19:32:13 GMT Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes [v2] In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 23:12:36 GMT, Chris Plummer wrote: >> Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: >> >> Updated comment > > test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassPrepare/classprep001/classprep001.cpp line 65: > >> 63: { "Lnsk/jvmti/ClassPrepare/classprep001$TestClass;", EXP_STATUS, 3, 2, 1 } >> 64: }; >> 65: // This classes are loaded on a different thread. > > "These classes..." Fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3732 From david.holmes at oracle.com Wed Apr 28 20:45:14 2021 From: david.holmes at oracle.com (David Holmes) Date: Thu, 29 Apr 2021 06:45:14 +1000 Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes In-Reply-To: References: Message-ID: On 29/04/2021 5:21 am, Alex Menkov wrote: > On Tue, 27 Apr 2021 23:15:35 GMT, Chris Plummer wrote: > >> I don't see any mention in the CR of the test failing due to this issue. How was it discovered? How is the fix being verified? > > I discovered it researching JCK failure on a platform with UsageTracker enabled. > JCK test does exactly the same as this test, but I was not able to reproduce the failure for this test (but this is a race, so that's normal) > > I ensured the fix works during development when created otherTread before getReady and had printdump on. otherTread -> otherThread ? Cheers, David ----- > It produced a lot of ClassPrepare event for Thread and lambda-related classes. > >> test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassPrepare/classprep001/classprep001.cpp line 292: >> >>> 290: >>> 291: JNIEXPORT void JNICALL >>> 292: Java_nsk_jvmti_ClassPrepare_classprep001_getReady(JNIEnv *env, jclass cls, jthread thread) { >> >> It's not clear to me why you now pass in the thread instead of calling GetCurrentThread, when the thread passed in is still always the current thread. Same thing for the `check()` method. > > I think it makes the code more flexible and easier to extend > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3732 > From amenkov at openjdk.java.net Wed Apr 28 21:48:15 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 28 Apr 2021 21:48:15 GMT Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes [v3] In-Reply-To: References: Message-ID: > Class loading may cause loading of some other system/internal classes (for example loading of java.util.concurrent classes when an other thread loads some classes concurrently). > The fix updates ClassPrepare test so it skip events for "unexpected" classes. > Also it adds a testcase to verify that classes which are explicitly loaded on other thread do not generate ClassPrepare event on the test thread. Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: fixed type in otherThread variable name ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3732/files - new: https://git.openjdk.java.net/jdk/pull/3732/files/4cebb5d6..9929298f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3732&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3732&range=01-02 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3732.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3732/head:pull/3732 PR: https://git.openjdk.java.net/jdk/pull/3732 From amenkov at openjdk.java.net Wed Apr 28 21:48:15 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Wed, 28 Apr 2021 21:48:15 GMT Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 20:47:29 GMT, David Holmes wrote: > > otherTread -> otherThread ? > fixed ------------- PR: https://git.openjdk.java.net/jdk/pull/3732 From cjplummer at openjdk.java.net Wed Apr 28 22:10:50 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 28 Apr 2021 22:10:50 GMT Subject: RFR: 8266002: vmTestbase/nsk/jvmti/ClassPrepare/classprep001 should skip events for unexpected classes [v3] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 21:48:15 GMT, Alex Menkov wrote: >> Class loading may cause loading of some other system/internal classes (for example loading of java.util.concurrent classes when an other thread loads some classes concurrently). >> The fix updates ClassPrepare test so it skip events for "unexpected" classes. >> Also it adds a testcase to verify that classes which are explicitly loaded on other thread do not generate ClassPrepare event on the test thread. > > Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: > > fixed type in otherThread variable name Marked as reviewed by cjplummer (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3732 From cjplummer at openjdk.java.net Wed Apr 28 22:25:51 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Wed, 28 Apr 2021 22:25:51 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 15:44:15 GMT, Mitsuru kariya wrote: > ?tation Please update all copyright dates to 2021. Only the latest year need to be updated. For example: ` * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights reserved.` becomes ` * Copyright (c) 2002, 2021, Oracle and/or its affiliates. All rights reserved.` Also, please give a proper description of the PR. Right now it just says "?tation" ------------- Changes requested by cjplummer (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3522 From whuang at openjdk.java.net Thu Apr 29 01:33:58 2021 From: whuang at openjdk.java.net (Wang Huang) Date: Thu, 29 Apr 2021 01:33:58 GMT Subject: RFR: 8266187: Memory leak in appendBootClassPath() In-Reply-To: References: Message-ID: <3Cp7SuEjfiLSlJOk08KERBEcnLwvUETLlExenJFrytc=.da60764b-2c83-4867-b3d5-edbe23f78d42@github.com> On Wed, 28 Apr 2021 12:17:18 GMT, Kevin Walls wrote: > Hi, > I didn't find it immediately obvious that this was safe, but I followed things and think that it is correct: > > The malloc'd pointer gets passed to... > JvmtiEnv::AddToBootstrapClassLoaderSearch(const char* segment) { > > which calls > > ClassPathZipEntry* ClassLoader::create_class_path_zip_entry(const char *path, bool is_boot_append) { > > ..which calls > > char* ClassLoader::get_canonical_path(const char* orig, Thread* thread) { > > ...which makes a copy of the string: > char* orig_copy = NEW_RESOURCE_ARRAY_IN_THREAD(thread, char, strlen(orig)+1); > strcpy(orig_copy, orig); > ...and doesn't apear to keep the pointer. So yes I think we should free it. 8-) Thank you for your review and explanation. ------------- PR: https://git.openjdk.java.net/jdk/pull/3751 From cjplummer at openjdk.java.net Thu Apr 29 04:34:52 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 29 Apr 2021 04:34:52 GMT Subject: RFR: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][] In-Reply-To: References: <_WRGPHgrZJ-7Vc7aX_2dnVYwSKwaehGfow0u2fIDBds=.3210bf41-161f-49c3-a3f7-a1ecbce238eb@github.com> Message-ID: On Wed, 28 Apr 2021 07:57:03 GMT, Fairoz Matte wrote: > > Do we even need findComponentType() any more? Isn't ReferenceTypeImpl.findType() sufficient. > > We still need findComponentType(), > Difference between findType() and findComponentType() is that, findComponentType() tries to get the list of ReferenceType from the "vm.classesByName". In case list is empty, it explicitly throws ClassNotLoadedException. > This exception check is required in validateAssignment(ValueContainer destination) call from ObjectReferenceImpl.java. I'm not sure what you mean by this. After your changes, this is all `findComponentType()` does: Type findComponentType(String signature) throws ClassNotLoadedException { return findType(signature); } And `findType()` has the exact same signature, including the `throws`: ` Type findType(String signature) throws ClassNotLoadedException {` So my suggestion is to get rid of `findComponentType()` and just have current users call `findType()` instead. ------------- PR: https://git.openjdk.java.net/jdk/pull/3658 From cjplummer at openjdk.java.net Thu Apr 29 04:43:52 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Thu, 29 Apr 2021 04:43:52 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 15:44:15 GMT, Mitsuru kariya wrote: > ?tation @kariya-mitsuru Please enable github actions on your jdk personal fork to enable the pre-submit testing. Start with the following link: https://github.com/kariya-mitsuru/jdk/actions Click on "Settings" tab and then click on "Actions" from the list of settings (not the "Actions" tab). Then enable "Allow all Actions". ------------- PR: https://git.openjdk.java.net/jdk/pull/3522 From github.com+2217224+kariya-mitsuru at openjdk.java.net Thu Apr 29 07:05:18 2021 From: github.com+2217224+kariya-mitsuru at openjdk.java.net (Mitsuru kariya) Date: Thu, 29 Apr 2021 07:05:18 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation [v2] In-Reply-To: References: Message-ID: > ?tation Mitsuru kariya has updated the pull request incrementally with one additional commit since the last revision: Update copyright ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3522/files - new: https://git.openjdk.java.net/jdk/pull/3522/files/5f671f95..d1e30f77 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3522&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3522&range=00-01 Stats: 6 lines in 6 files changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3522.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3522/head:pull/3522 PR: https://git.openjdk.java.net/jdk/pull/3522 From github.com+2217224+kariya-mitsuru at openjdk.java.net Thu Apr 29 07:07:50 2021 From: github.com+2217224+kariya-mitsuru at openjdk.java.net (Mitsuru kariya) Date: Thu, 29 Apr 2021 07:07:50 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 22:22:38 GMT, Chris Plummer wrote: > Please update all copyright dates to 2021. Thank you for your advice. I just updated the copyright. ------------- PR: https://git.openjdk.java.net/jdk/pull/3522 From tschatzl at openjdk.java.net Thu Apr 29 07:08:54 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 07:08:54 GMT Subject: RFR: 8234532: Remove ThreadLocalAllocBuffer::_fast_refill_waste since it is never set [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 09:41:14 GMT, Albert Mingkun Yang wrote: >> It has some cascading effect; two performance variables, `fastWaste` and `maxFastWaste`, can be removed also. > > Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: > > - merge > - refill Lgtm apart from that minor formatting nit. src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp line 109: > 107: _slow_refill_waste); > 108: } else { > 109: assert(_number_of_refills == 0 && Please fill back the clause from the next line. I think the assert condition now fits a single line, the text may be on the next line ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3756 From github.com+2217224+kariya-mitsuru at openjdk.java.net Thu Apr 29 07:28:50 2021 From: github.com+2217224+kariya-mitsuru at openjdk.java.net (Mitsuru kariya) Date: Thu, 29 Apr 2021 07:28:50 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation In-Reply-To: References: Message-ID: <4jlbe1kavQc0cn5OgLg95k4G3uVs3crXvM_d-OonSO0=.c8685d3a-ce2e-4af6-b5a8-a6f147b4a9e3@github.com> On Wed, 28 Apr 2021 22:23:17 GMT, Chris Plummer wrote: > Also, please give a proper description of the PR. Right now it just says "?tation" Oh, I'm embarrassed to say that I completely overlooked this. I just updated the description. ------------- PR: https://git.openjdk.java.net/jdk/pull/3522 From github.com+2217224+kariya-mitsuru at openjdk.java.net Thu Apr 29 07:43:49 2021 From: github.com+2217224+kariya-mitsuru at openjdk.java.net (Mitsuru kariya) Date: Thu, 29 Apr 2021 07:43:49 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 04:41:20 GMT, Chris Plummer wrote: > Please enable github actions on your jdk personal fork to enable the pre-submit testing. I've changed the settings. ------------- PR: https://git.openjdk.java.net/jdk/pull/3522 From fmatte at openjdk.java.net Thu Apr 29 07:58:26 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Thu, 29 Apr 2021 07:58:26 GMT Subject: RFR: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][] [v2] In-Reply-To: References: Message-ID: > findComponentType() logic is wrong. In findComponentType() method, We always get vm.classesByName() retruns empty list > list = vm.classesByName(parser.typeName()); > We have "parser.typeName()" retruns " double[][]" > vm.classesByName("") is expecting the fully qualified name example "java.lang.Double" > This always returns empty list, resulting into ClassNotLoadedException as it assumes the Component class has not yet been loaded, hence the test case fails. > > There was a suggested fix from Egor Ushakov from JetBrains, I am proposing the same to get this fix. I have verified the patch with required testing it works fine. Fairoz Matte has updated the pull request incrementally with two additional commits since the last revision: - Update ArrayReferenceImpl.java - Update ArrayTypeImpl.java ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3658/files - new: https://git.openjdk.java.net/jdk/pull/3658/files/17fe300c..37170d21 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3658&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3658&range=00-01 Stats: 17 lines in 2 files changed: 0 ins; 13 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3658.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3658/head:pull/3658 PR: https://git.openjdk.java.net/jdk/pull/3658 From fmatte at openjdk.java.net Thu Apr 29 07:58:26 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Thu, 29 Apr 2021 07:58:26 GMT Subject: RFR: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][] [v2] In-Reply-To: References: <_WRGPHgrZJ-7Vc7aX_2dnVYwSKwaehGfow0u2fIDBds=.3210bf41-161f-49c3-a3f7-a1ecbce238eb@github.com> Message-ID: On Thu, 29 Apr 2021 04:31:35 GMT, Chris Plummer wrote: >> Hi Chris, >> >> Thanks for looking into this, >> >>>Do we even need findComponentType() any more? Isn't ReferenceTypeImpl.findType() sufficient. >> >> We still need findComponentType(), >> Difference between findType() and findComponentType() is that, findComponentType() tries to get the list of ReferenceType from the "vm.classesByName". In case list is empty, it explicitly throws ClassNotLoadedException. >> This exception check is required in validateAssignment(ValueContainer destination) call from ObjectReferenceImpl.java. >> >> https://github.com/openjdk/jdk/blob/master/src/jdk.jdi/share/classes/com/sun/tools/jdi/ObjectReferenceImpl.java#L579 >> >> >> >> I have modified the method a bit to handle that scenario >> >> >> diff --git a/src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java b/src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java >> index e544c81ae3e..54deba43894 100644 >> --- a/src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java >> +++ b/src/jdk.jdi/share/classes/com/sun/tools/jdi/ArrayTypeImpl.java >> @@ -95,20 +95,13 @@ public class ArrayTypeImpl extends ReferenceTypeImpl >> JNITypeParser sig = new JNITypeParser(signature); >> if (sig.isReference()) { >> // It's a reference type >> - JNITypeParser parser = new JNITypeParser(componentSignature()); >> - List list = vm.classesByName(parser.typeName()); >> - Iterator iter = list.iterator(); >> - while (iter.hasNext()) { >> - ReferenceType type = iter.next(); >> - ClassLoaderReference cl = type.classLoader(); >> - if ((cl == null)? >> - (classLoader() == null) : >> - (cl.equals(classLoader()))) { >> - return type; >> - } >> + try { >> + Type componentType = findType(componentSignature()); >> + return componentType; >> + // Component class has not yet been loaded >> + } catch (ClassNotLoadedException ex) { >> + throw new ClassNotLoadedException(componentTypeName()); >> } >> - // Component class has not yet been loaded >> - throw new ClassNotLoadedException(componentTypeName()); >> } else { >> // It's a primitive type >> return vm.primitiveTypeMirror(sig.jdwpTag());``` >> >> Thanks, > >> > Do we even need findComponentType() any more? Isn't ReferenceTypeImpl.findType() sufficient. >> >> We still need findComponentType(), >> Difference between findType() and findComponentType() is that, findComponentType() tries to get the list of ReferenceType from the "vm.classesByName". In case list is empty, it explicitly throws ClassNotLoadedException. >> This exception check is required in validateAssignment(ValueContainer destination) call from ObjectReferenceImpl.java. > > I'm not sure what you mean by this. After your changes, this is all `findComponentType()` does: > > > Type findComponentType(String signature) throws ClassNotLoadedException { > return findType(signature); > } > > > And `findType()` has the exact same signature, including the `throws`: > > ` Type findType(String signature) throws ClassNotLoadedException {` > > > So my suggestion is to get rid of `findComponentType()` and just have current users call `findType()` instead. Thanks Chris, for the review comments. I have modified the suggested changes, kindly review. Thanks, ------------- PR: https://git.openjdk.java.net/jdk/pull/3658 From ayang at openjdk.java.net Thu Apr 29 08:11:12 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Thu, 29 Apr 2021 08:11:12 GMT Subject: RFR: 8234532: Remove ThreadLocalAllocBuffer::_fast_refill_waste since it is never set [v3] In-Reply-To: References: Message-ID: > It has some cascading effect; two performance variables, `fastWaste` and `maxFastWaste`, can be removed also. Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3756/files - new: https://git.openjdk.java.net/jdk/pull/3756/files/867bbe14..dcaff0bb Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3756&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3756&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3756.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3756/head:pull/3756 PR: https://git.openjdk.java.net/jdk/pull/3756 From sjohanss at openjdk.java.net Thu Apr 29 08:40:51 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 29 Apr 2021 08:40:51 GMT Subject: RFR: 8234532: Remove ThreadLocalAllocBuffer::_fast_refill_waste since it is never set [v3] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 08:11:12 GMT, Albert Mingkun Yang wrote: >> It has some cascading effect; two performance variables, `fastWaste` and `maxFastWaste`, can be removed also. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > review Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3756 From pliden at openjdk.java.net Thu Apr 29 08:49:54 2021 From: pliden at openjdk.java.net (Per Liden) Date: Thu, 29 Apr 2021 08:49:54 GMT Subject: RFR: 8234532: Remove ThreadLocalAllocBuffer::_fast_refill_waste since it is never set [v3] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 08:11:12 GMT, Albert Mingkun Yang wrote: >> It has some cascading effect; two performance variables, `fastWaste` and `maxFastWaste`, can be removed also. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > review Looks good. But we should also probably rename `_slow_refill_waste` to `_refill_waste`, since "slow" doesn't really make sense anymore. Of course, that would also affect the `slowWaste` PerfCounter. Maybe this could be a follow up refinement? ------------- Marked as reviewed by pliden (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3756 From ayang at openjdk.java.net Thu Apr 29 09:06:51 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Thu, 29 Apr 2021 09:06:51 GMT Subject: RFR: 8234532: Remove ThreadLocalAllocBuffer::_fast_refill_waste since it is never set [v3] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 08:46:57 GMT, Per Liden wrote: > Maybe this could be a follow up refinement? Sure, will do so after this PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/3756 From kevinw at openjdk.java.net Thu Apr 29 10:39:57 2021 From: kevinw at openjdk.java.net (Kevin Walls) Date: Thu, 29 Apr 2021 10:39:57 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation [v2] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 07:05:18 GMT, Mitsuru kariya wrote: >> The current `hashCode` implementation of SA's Address subclasses ignores the upper 32 bits of the long value. >> This PR changes to use `Long.hashCode` instead. > > Mitsuru kariya has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright Hi - There are a few more places where a long is cast to an int and used as the hashCode: open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/windbg/amd64/WindbgAMD64Thread.java public int hashCode() { return (int) getThreadID(); } open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/windbg/x86/WindbgX86Thread.java open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeapRegion.java open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/remote/RemoteDebuggerClient.java getThreadHashCode(long id) returns a truncated long if it caught a RemoteException. If you have time, these could all use the same change? ------------- PR: https://git.openjdk.java.net/jdk/pull/3522 From github.com+2217224+kariya-mitsuru at openjdk.java.net Thu Apr 29 16:33:00 2021 From: github.com+2217224+kariya-mitsuru at openjdk.java.net (Mitsuru kariya) Date: Thu, 29 Apr 2021 16:33:00 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation [v2] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 10:36:59 GMT, Kevin Walls wrote: > If you have time, these could all use the same change? I have enough time to change them. May I mix that fix into this pull request? ------------- PR: https://git.openjdk.java.net/jdk/pull/3522 From mli at openjdk.java.net Fri Apr 30 03:11:56 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 30 Apr 2021 03:11:56 GMT Subject: RFR: 8266187: Memory leak in appendBootClassPath() In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 06:38:57 GMT, Wang Huang wrote: > Dear All, > I find a memory leak in `appendBootClassPath()` > https://github.com/openjdk/jdk/blob/75a2354dc276e107d64516d20fc72bc7ef3d5f86/src/java.instrument/share/native/libinstrument/InvocationAdapter.c#L950 > * we malloc `resolved` from resolve(parent, path) > * we use `resolved` in line 951 > * we don't free() this memory after using. > > I think we can fix this bug by adding a free() after line 951 as my commit. > Thank you for your review. Any suggestion is welcome. > > Yours , > Wang Huang looks good. ------------- Marked as reviewed by mli (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3751 From kevinw at openjdk.java.net Fri Apr 30 08:53:52 2021 From: kevinw at openjdk.java.net (Kevin Walls) Date: Fri, 30 Apr 2021 08:53:52 GMT Subject: RFR: 8264734: SA's Address subclasses could use better hashCode() implementation [v2] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 07:05:18 GMT, Mitsuru kariya wrote: >> The current `hashCode` implementation of SA's Address subclasses ignores the upper 32 bits of the long value. >> This PR changes to use `Long.hashCode` instead. > > Mitsuru kariya has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright Thanks, yes it can definitely be an update to this change. We can update the bug title, as I notice that is specific to Address classes, but as there are only a few more methods doing the same truncation, it makes sense to include them. e.g. "Some SA classes could use better hashCode() implementation." ------------- PR: https://git.openjdk.java.net/jdk/pull/3522 From lzang at openjdk.java.net Fri Apr 30 09:18:57 2021 From: lzang at openjdk.java.net (Lin Zang) Date: Fri, 30 Apr 2021 09:18:57 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v20] In-Reply-To: <-qf3V7rOYbOMN9Wf0Q4Nnuz8seoJwA6VeUBQgqLVnak=.ace9f2ac-4669-4c2e-990a-1547533094cc@github.com> References: <9Q69ICl9o5uaRsoDDEa43b9Vl21lzaZIHe2f8mvQy7c=.b1b36b58-787a-4950-9afe-31ca4c79165b@github.com> <-qf3V7rOYbOMN9Wf0Q4Nnuz8seoJwA6VeUBQgqLVnak=.ace9f2ac-4669-4c2e-990a-1547533094cc@github.com> Message-ID: On Wed, 21 Apr 2021 18:40:16 GMT, Chris Plummer wrote: >>> My concerns with your proposed testing is that it always targets the same application with the same heap, and does not read/process the hprof file to make sure it is not corrupt. >> >> Hmm, I agree, I will do more test and update here. Thanks! > >> > My concerns with your proposed testing is that it always targets the same application with the same heap, and does not read/process the hprof file to make sure it is not corrupt. >> >> Hmm, I agree, I will do more test and update here. Thanks! > > As for jtreg testing, we have a few heap dumping related tests. I'm not sure how good they are. It would be good to understand what testing we currently do. > > In addition to jtreg testing, you might want to try just launching some large java apps (netbeans and intellij come to mind), dump their heaps, and then process them with some existing tool. I'm not suggesting this be part of regular testing, but just a sanity check you do on your own before pushing the changes. Hi Chris, @plummercj Here are the tests I have conducted: - tier1, tier2,tier3 all passed - netbeans with heap usage of 500mb, 1000mb, plus "randomly do some operation and dump the heap". tested 30 times and all passed. - a workload that generate different object to fill heap, I have tested with heap usage from 1GB to 8GB. All passed. All the scenario are tested with/without the gz option. Do you think these tests are ok? BRs, Lin ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From kevinw at openjdk.java.net Fri Apr 30 10:20:49 2021 From: kevinw at openjdk.java.net (Kevin Walls) Date: Fri, 30 Apr 2021 10:20:49 GMT Subject: RFR: 8266187: Memory leak in appendBootClassPath() In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 06:38:57 GMT, Wang Huang wrote: > Dear All, > I find a memory leak in `appendBootClassPath()` > https://github.com/openjdk/jdk/blob/75a2354dc276e107d64516d20fc72bc7ef3d5f86/src/java.instrument/share/native/libinstrument/InvocationAdapter.c#L950 > * we malloc `resolved` from resolve(parent, path) > * we use `resolved` in line 951 > * we don't free() this memory after using. > > I think we can fix this bug by adding a free() after line 951 as my commit. > Thank you for your review. Any suggestion is welcome. > > Yours , > Wang Huang Oops - please update the copyright year, thanks! (second year in the header is currently 2019, should become 2021). ------------- PR: https://git.openjdk.java.net/jdk/pull/3751 From alanb at openjdk.java.net Fri Apr 30 16:39:50 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 30 Apr 2021 16:39:50 GMT Subject: RFR: 8266187: Memory leak in appendBootClassPath() In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 06:38:57 GMT, Wang Huang wrote: > Dear All, > I find a memory leak in `appendBootClassPath()` > https://github.com/openjdk/jdk/blob/75a2354dc276e107d64516d20fc72bc7ef3d5f86/src/java.instrument/share/native/libinstrument/InvocationAdapter.c#L950 > * we malloc `resolved` from resolve(parent, path) > * we use `resolved` in line 951 > * we don't free() this memory after using. > > I think we can fix this bug by adding a free() after line 951 as my commit. > Thank you for your review. Any suggestion is welcome. > > Yours , > Wang Huang Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3751 From vromero at openjdk.java.net Fri Apr 30 17:40:47 2021 From: vromero at openjdk.java.net (Vicente Romero) Date: Fri, 30 Apr 2021 17:40:47 GMT Subject: RFR: 8260517: implement Sealed Classes as a standard feature in Java [v5] In-Reply-To: References: Message-ID: > Please review this PR that intents to make sealed classes a final feature in Java. This PR contains compiler and VM changes. In line with similar PRs, which has made preview features final, this one is mostly removing preview related comments from APIs plus options in test cases. Please also review the related [CSR](https://bugs.openjdk.java.net/browse/JDK-8265090) > > Thanks > > note: this PR is related to [PR-3528](https://github.com/openjdk/jdk/pull/3528) and must be integrated after it. Vicente Romero has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision: - Merge branch 'master' into JDK-8260517 - Merge branch 'master' into JDK-8260517 - updating comment after review feedback - removing javax.lang.model changes - Merge branch 'master' into JDK-8260517 - Merge branch 'master' into JDK-8260517 - fixing failing regression tests - JVM related changes - 8260517: Compiler implementation for Sealed Classes ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3526/files - new: https://git.openjdk.java.net/jdk/pull/3526/files/43e9cb5f..2744f615 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3526&range=03-04 Stats: 506473 lines in 3844 files changed: 20558 ins; 483521 del; 2394 mod Patch: https://git.openjdk.java.net/jdk/pull/3526.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3526/head:pull/3526 PR: https://git.openjdk.java.net/jdk/pull/3526 From cjplummer at openjdk.java.net Fri Apr 30 19:24:03 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 30 Apr 2021 19:24:03 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v21] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 12:45:56 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 27 commits: > > - Merge branch 'master' into par-dump > - refine with several fix > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - ... and 17 more: https://git.openjdk.java.net/jdk/compare/c3ac6900...6d14790a Hi Lin, Yes, that seems like sufficient testing. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From cjplummer at openjdk.java.net Fri Apr 30 19:30:00 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 30 Apr 2021 19:30:00 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v21] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 12:45:56 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 27 commits: > > - Merge branch 'master' into par-dump > - refine with several fix > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - ... and 17 more: https://git.openjdk.java.net/jdk/compare/c3ac6900...6d14790a src/hotspot/share/services/heapDumperCompression.hpp line 222: > 220: char const* error() const { return _err; } > 221: > 222: // Sets up an internal buffer, fill with external buffer, and sends to compressor. "fills" ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From cjplummer at openjdk.java.net Fri Apr 30 19:34:07 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 30 Apr 2021 19:34:07 GMT Subject: RFR: 8252842: Extend jmap to support parallel heap dump [v21] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 12:45:56 GMT, Lin Zang wrote: >> 8252842: Extend jmap to support parallel heap dump > > Lin Zang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 27 commits: > > - Merge branch 'master' into par-dump > - refine with several fix > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - Typo fix > - remove parallel option and dumpheapext command > - Revert "hide the dumpheapext error message" > > This reverts commit 1af0e1e2bfab4f5d079c03ff0cb25067acacdac4. > - resize large object threshold > - Merge branch 'master' into par-dump > - Merge branch 'master' into par-dump > - ... and 17 more: https://git.openjdk.java.net/jdk/compare/c3ac6900...6d14790a src/hotspot/share/services/heapDumperCompression.cpp line 240: > 238: } > 239: > 240: void CompressionBackend::flush_buffer_without_lock(MonitorLocker* ml) { I'm trying to think of a better name since "without_lock" makes it sound like there is no locking, but there is. The difference is the lock is passed in. How about just calling it `flush_buffer(MonitorLocker* ml)` and rely on C++ method overloading to disambiguate it from `flush_buffer()`. ------------- PR: https://git.openjdk.java.net/jdk/pull/2261 From cjplummer at openjdk.java.net Fri Apr 30 19:39:54 2021 From: cjplummer at openjdk.java.net (Chris Plummer) Date: Fri, 30 Apr 2021 19:39:54 GMT Subject: RFR: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][] [v2] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 07:58:26 GMT, Fairoz Matte wrote: >> findComponentType() logic is wrong. In findComponentType() method, We always get vm.classesByName() retruns empty list >> list = vm.classesByName(parser.typeName()); >> We have "parser.typeName()" retruns " double[][]" >> vm.classesByName("") is expecting the fully qualified name example "java.lang.Double" >> This always returns empty list, resulting into ClassNotLoadedException as it assumes the Component class has not yet been loaded, hence the test case fails. >> >> There was a suggested fix from Egor Ushakov from JetBrains, I am proposing the same to get this fix. I have verified the patch with required testing it works fine. > > Fairoz Matte has updated the pull request incrementally with two additional commits since the last revision: > > - Update ArrayReferenceImpl.java > - Update ArrayTypeImpl.java Looks good. Please make sure you have run all of the following tests: test/hotspot/jtreg/vmTestbase/nsk/jdb test/hotspot/jtreg/serviceability/jdwp test/hotspot/jtreg/vmTestbase/nsk/jdwp test/hotspot/jtreg/vmTestbase/nsk/jdi test/jdk/com/sun/jdi ------------- Marked as reviewed by cjplummer (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3658 From dcubed at openjdk.java.net Fri Apr 30 20:57:53 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 30 Apr 2021 20:57:53 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() In-Reply-To: <8vtwdpSZT5T3xeXvBMLFtnkPAcTRnqf8KziTzEim43Q=.e791053c-0907-4a32-a111-739f590352da@github.com> References: <8vtwdpSZT5T3xeXvBMLFtnkPAcTRnqf8KziTzEim43Q=.e791053c-0907-4a32-a111-739f590352da@github.com> Message-ID: <4VOXQ7X3vPyTIQJQ38PdExpGKj3f_ABeABa2ZDquBSg=.813d4dba-f14e-421f-811e-f2b65420fdaa@github.com> On Wed, 28 Apr 2021 12:55:18 GMT, David Holmes wrote: >> The synopsis pretty much says it all. There's a more detailed history in the RFE itself. >> >> Currently running the new test thru Mach5 Tier[1-7]. >> I've run the test thru several 12 hour runs on my MBP13 and >> on my Linux-X64 server. > > test/hotspot/jtreg/serviceability/monitoring/ThreadInfo/getLockOwnerName/getLockOwnerName.java line 30: > >> 28: * non-null string for a blocked thread and then makes repeated calls >> 29: * to getThreadInfo() and ThreadInfo.getLockOwnerName() until the thread >> 30: * has exited. > > Extra space before has. Fixed. > test/hotspot/jtreg/serviceability/monitoring/ThreadInfo/getLockOwnerName/getLockOwnerName.java line 33: > >> 31: * @requires vm.jvmti >> 32: * @library /test/lib >> 33: * @compile getLockOwnerName.java > > You don't need a @compile statement for the current test file Deleted. > test/hotspot/jtreg/serviceability/monitoring/ThreadInfo/getLockOwnerName/getLockOwnerName.java line 137: > >> 135: } >> 136: >> 137: System.exit(run(timeMax, System.out) + exit_delta); > > jtreg tests don't use System.exit! Hmmm... that's generally true, but this is a test that must be run as "othervm" so this style of exit with the "+ exit_delta" logic has been used for these kinds of stress tests. I think this style came from the VMTestbase tests and I've used it with other stress tests. > test/hotspot/jtreg/serviceability/monitoring/ThreadInfo/getLockOwnerName/getLockOwnerName.java line 189: > >> 187: while (testState != TS_BLOCKER_RUNNING) { >> 188: try { >> 189: barrierLaunch.wait(0); // wait until it is running > > Nit: we use wait() rather than wait(0) I fixed it in 5 places. ------------- PR: https://git.openjdk.java.net/jdk/pull/3478 From dcubed at openjdk.java.net Fri Apr 30 20:57:52 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 30 Apr 2021 20:57:52 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 00:08:52 GMT, Daniel D. Daugherty wrote: > The synopsis pretty much says it all. There's a more detailed history in the RFE itself. > > Currently running the new test thru Mach5 Tier[1-7]. > I've run the test thru several 12 hour runs on my MBP13 and > on my Linux-X64 server. @dholmes-ora - Thanks for the review. Sorry for the delay in responding. I've been working on other issues. Please see my embedded replies. ------------- PR: https://git.openjdk.java.net/jdk/pull/3478 From dcubed at openjdk.java.net Fri Apr 30 21:50:25 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 30 Apr 2021 21:50:25 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() [v2] In-Reply-To: References: Message-ID: > The synopsis pretty much says it all. There's a more detailed history in the RFE itself. > > Currently running the new test thru Mach5 Tier[1-7]. > I've run the test thru several 12 hour runs on my MBP13 and > on my Linux-X64 server. Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: dholmes CR changes. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3478/files - new: https://git.openjdk.java.net/jdk/pull/3478/files/f6fbf715..682a7c9d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3478&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3478&range=00-01 Stats: 7 lines in 1 file changed: 0 ins; 1 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3478.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3478/head:pull/3478 PR: https://git.openjdk.java.net/jdk/pull/3478 From dcubed at openjdk.java.net Fri Apr 30 23:57:37 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 30 Apr 2021 23:57:37 GMT Subject: RFR: 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() [v3] In-Reply-To: References: Message-ID: <5C70XkosbvCKiIcgWrlbTY0elIateAXiSG5oSROGmm4=.0ca72fb3-460b-4e35-be8d-49a73472babe@github.com> > The synopsis pretty much says it all. There's a more detailed history in the RFE itself. > > Currently running the new test thru Mach5 Tier[1-7]. > I've run the test thru several 12 hour runs on my MBP13 and > on my Linux-X64 server. Daniel D. Daugherty has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - Merge branch 'master' into JDK-8265153 - dholmes CR changes. - 8265153: add time based test for ThreadMXBean.getThreadInfo() and ThreadInfo.getLockOwnerName() ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3478/files - new: https://git.openjdk.java.net/jdk/pull/3478/files/682a7c9d..3b4004d8 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3478&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3478&range=01-02 Stats: 568378 lines in 5345 files changed: 48342 ins; 511803 del; 8233 mod Patch: https://git.openjdk.java.net/jdk/pull/3478.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3478/head:pull/3478 PR: https://git.openjdk.java.net/jdk/pull/3478