From amenkov at openjdk.org Thu Jun 1 00:57:10 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 1 Jun 2023 00:57:10 GMT Subject: RFR: 8308978: regression with a deadlock involving FollowReferences [v3] In-Reply-To: References: <2J1qItzUgmfjRPS0xUbHgXZQ-b12JBxe8XPRftU2GyA=.025e7855-5df4-413a-bea7-585a53832025@github.com> <6xcKqU3mLr9TocEUpoXXzcNWSnKijSlhcyxIfXXrFD0=.2987b8de-827e-43d0-907b-7ef2016ddab4@github.com> Message-ID: On Wed, 31 May 2023 22:57:08 GMT, Serguei Spitsyn wrote: > > Something went wrong after 1st merge, testing failed with OOMEInAQS (which is problem-listed) > > How do you run tests? Do you run tiers 1-5 or something else as well? Please, remember that the tier-5 runs the needed SVC tests with `main.wrapper=virtual`. tier1-tier5 as per David's request initial fix (before the test were problem-listed) was tested with JTREG_TEST_THREAD_FACTORY=Virtual ------------- PR Comment: https://git.openjdk.org/jdk/pull/14233#issuecomment-1571154252 From david.holmes at oracle.com Thu Jun 1 01:56:00 2023 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Jun 2023 11:56:00 +1000 Subject: Clarifying jmethodID Usage and Potential JVM Crashes In-Reply-To: References: Message-ID: <31e606ae-f54e-c8e6-c00b-720ef192d279@oracle.com> Hi Jaroslav, On 31/05/2023 9:12 pm, Jaroslav Bachor?k wrote: > Dear Team, > > I've been investigating the unusual JVM crashes occurring in JVMTI calls > on a J9 JVM. During my investigation, I scrutinized the `jmethodID` > definition closely, available here: [jmethodID > definition](https://docs.oracle.com/en/java/javase/17/docs/specs/jvmti.html#jmethodID ). > > To paraphrase, the definition suggests that `jmethodID` identifies a > Java method, initializer, or constructor. These identifiers, once > returned by JVM TI functions and events, can be safely stored. However, > when the class is unloaded, they become invalid, rendering them > unsuitable for use. > > My interpretation is that the JVMTI user should verify the validity of a > `jmethodID` value before using it to prevent potential crashes. Would > you agree with this interpretation? Not quite - as you note you can't verify the jmethodID validity. What the user needs to do, in line with what Dan was saying, is ensure that they keep track of the classes to which the methods belong and keep them alive if necessary. Now that may be easier said than done, but that is the gist of it. This comes from the JNI spec: "A field or method ID does not prevent the VM from unloading the class from which the ID has been derived. After the class is unloaded, the method or field ID becomes invalid and may not be passed to any function taking such an ID. The native code, therefore, must make sure to: keep a live reference to the underlying class, or recompute the method or field ID if it intends to use a method or field ID for an extended period of time." > This sounds like a sensible requirement, but its practical application > remains unclear. As far as I know, methods can be unloaded concurrently > to the native code executing JVMTI functions. This introduces a > potential race condition where the JVM unloads the methods during the > check->use flow, making it only a partial solution. To complicate > matters further, no method exists to confirm whether a `jmethodID` is valid. > > Theoretically, we could monitor the `CompiledMethodUnload` event to > track the validity state, creating a constantly expanding set of > unloaded `jmethodID` values or a bloom filter, if one does not care > about few potential false positives. This strategy, however, doesn't > address the potential race condition, and it could even exacerbate it > due to possible event delays. This delay might mistakenly validate a > `jmethodID` value that has already been unloaded, but for which the > event hasn't been delivered yet. > > Honestly, I don't see a way to use `jmethodID` safely unless the code > using it suspends the entire JVM and doesn't resume until it's finished > with that `jmethodID`. Any other approach might lead to JVM crashes, as > we've observed with J9. > > Lastly, it's noteworthy that Hotspot takes meticulous measures to ensure > that using jmethodIDs for unloaded methods doesn't crash the JVM and > even provides useful information. This observation has led me to > question whether the documentation aligns with the Hotspot > implementation, especially given that following closely the > documentation appears to increase the risk associated with the use of > `jmethodID` values. There have been folk who wanted to make this area more user-friendly but that shouldn't be mistaken for moving towards a world where jmethodIDs are always safe to use. Cheers, David > I welcome your thoughts and perspectives on this matter. > > Best regards, > > Jaroslav From dholmes at openjdk.org Thu Jun 1 02:12:05 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 1 Jun 2023 02:12:05 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v3] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Wed, 31 May 2023 23:39:19 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > move check for EnableDynamicAgentLoading to invoke_Agent_OnAttach A change in behaviour like this requires a CSR request - the code has been doing the wrong thing for a long time now! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14244#issuecomment-1571218267 From dholmes at openjdk.org Thu Jun 1 02:19:08 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 1 Jun 2023 02:19:08 GMT Subject: RFR: 8308090: Add container tests for on-the-fly resource quota updates [v2] In-Reply-To: References: Message-ID: On Wed, 31 May 2023 12:07:49 GMT, Severin Gehwolf wrote: >> Severin Gehwolf has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - Merge branch 'master' into jdk-8308090-tests-container-on-fly-updates >> - Fix whitespace >> - 8308090: Add container tests for on-the-fly resource quota updates > > Anyone willing to review this? @jerboaa I can't really review the tests themselves but will run through our CI to see if they cause any problems. If not then they should be okay to add. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14090#issuecomment-1571223360 From sspitsyn at openjdk.org Thu Jun 1 02:38:11 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 1 Jun 2023 02:38:11 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v7] In-Reply-To: References: Message-ID: <0qOfnlOvT1jPGl7Z3pYdXX4rf7tweXmPdlpxbt3hsso=.caaa8df4-4529-4b4a-b66b-7babc078793c@github.com> On Wed, 31 May 2023 20:04:04 GMT, Chris Plummer wrote: >> Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 17 additional commits since the last revision: >> >> - Add impl note to document the XX option >> - Cleanup >> - Merge >> - Allow for warning to be skipped when same agent loaded a second/subsequent time >> - Merge >> - Tweak javadoc, update test to use more test infra >> - Merge >> - Merge >> - Refresh package description >> - Merge >> - ... and 7 more: https://git.openjdk.org/jdk/compare/09807c2a...2d9d5922 > > src/hotspot/share/prims/jvmtiAgent.cpp line 512: > >> 510: >> 511: // Print warning if EnableDynamicAgentLoading not enabled on the command line >> 512: if (!FLAG_IS_CMDLINE(EnableDynamicAgentLoading) && !agent->is_instrument_lib() && !JvmtiAgentList::is_loaded(library)) { > > While looking at some code related to this, I noticed a couple of typos in the pre-existing load_agent_from_executable() comment. See lines 265 ("cant't") and 268 (".&&"). Maybe you could clean them up. Nit: I'd suggest to add dot at the end of comment at 511. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1212499408 From sspitsyn at openjdk.org Thu Jun 1 03:16:04 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 1 Jun 2023 03:16:04 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v3] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Thu, 1 Jun 2023 02:09:42 GMT, David Holmes wrote: > A change in behaviour like this requires a CSR request - the code has been doing the wrong thing for a long time now! Thank you for the comment. It was not fully clear if a CSR is needed in this case. Created CSR (it is listed in the PR description). ------------- PR Comment: https://git.openjdk.org/jdk/pull/14244#issuecomment-1571263870 From sspitsyn at openjdk.org Thu Jun 1 05:23:10 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 1 Jun 2023 05:23:10 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v7] In-Reply-To: References: Message-ID: On Wed, 31 May 2023 20:21:15 GMT, Chris Plummer wrote: >> Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 17 additional commits since the last revision: >> >> - Add impl note to document the XX option >> - Cleanup >> - Merge >> - Allow for warning to be skipped when same agent loaded a second/subsequent time >> - Merge >> - Tweak javadoc, update test to use more test infra >> - Merge >> - Merge >> - Refresh package description >> - Merge >> - ... and 7 more: https://git.openjdk.org/jdk/compare/a3c18c96...2d9d5922 > > src/hotspot/share/prims/jvmtiAgent.cpp line 512: > >> 510: >> 511: // Print warning if EnableDynamicAgentLoading not enabled on the command line >> 512: if (!FLAG_IS_CMDLINE(EnableDynamicAgentLoading) && !agent->is_instrument_lib() && !JvmtiAgentList::is_loaded(library)) { > > The use of `!JvmtiAgentList::is_loaded(library)` here is a bit confusing. Isn't the library always already loaded by the time we get here (the assert below seems to imply that)? If so, wouldn't it already be in the list? If it's not in the list yet, perhaps a comment explaining why would be helpful here. It looks like you are right. There is also and assert at line 519: `519 assert(agent->is_loaded(), "invariant");` So, the agent has to be loaded if we got to the line 512. Also, there is a statement at line 507 (before line 512): `507 agent->set_os_lib(library);` ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1212584288 From thartmann at openjdk.org Thu Jun 1 05:26:21 2023 From: thartmann at openjdk.org (Tobias Hartmann) Date: Thu, 1 Jun 2023 05:26:21 GMT Subject: RFR: 8309044: Replace NULL with nullptr, final sweep of hotspot code [v2] In-Reply-To: References: <3FoMnGeBp8DqkpVb6YGXKxdPsgGz6ej-jrf2U2stVfU=.56a11e19-38dd-420a-a07d-3b025120f194@github.com> Message-ID: <9RnqD_bbEcutUNcfw9elCwYbPS_7-JiHMn_9cVAmDxQ=.830877a3-eac8-4af9-9ac0-5244c37989fb@github.com> On Tue, 30 May 2023 19:15:38 GMT, Johan Sj?len wrote: >> A final sweep of Hotspot to remove all re-added NULLs. With only 110 changes I'd appreciate if this was considered trivial. > > Johan Sj?len has updated the pull request incrementally with two additional commits since the last revision: > > - Align > - Suggestions What's the plan now to prevent re-introducing `NULL`? ------------- PR Comment: https://git.openjdk.org/jdk/pull/14198#issuecomment-1571360076 From alanb at openjdk.org Thu Jun 1 05:55:08 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 1 Jun 2023 05:55:08 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v7] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 05:20:31 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiAgent.cpp line 512: >> >>> 510: >>> 511: // Print warning if EnableDynamicAgentLoading not enabled on the command line >>> 512: if (!FLAG_IS_CMDLINE(EnableDynamicAgentLoading) && !agent->is_instrument_lib() && !JvmtiAgentList::is_loaded(library)) { >> >> The use of `!JvmtiAgentList::is_loaded(library)` here is a bit confusing. Isn't the library always already loaded by the time we get here (the assert below seems to imply that)? If so, wouldn't it already be in the list? If it's not in the list yet, perhaps a comment explaining why would be helpful here. > > It looks like you are right. > There is also and assert at line 519: > `519 assert(agent->is_loaded(), "invariant");` > > So, the agent has to be loaded if we got to the line 512. > Also, there is a statement at line 507 (before line 512): > `507 agent->set_os_lib(library);` > The use of `!JvmtiAgentList::is_loaded(library)` here is a bit confusing. Isn't the library always already loaded by the time we get here (the assert below seems to imply that)? JvmtiAgentList::load_agent creates the JvmtiAgent, attempts to load it, and adds it to the agent list if is succeeds. The test that you are looking is checking the list of already loaded agents. Maybe the function name "is_loaded" is the confusion here, maybe a better name is needed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1212610098 From alanb at openjdk.org Thu Jun 1 05:58:11 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 1 Jun 2023 05:58:11 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v6] In-Reply-To: References: Message-ID: On Wed, 31 May 2023 20:37:23 GMT, Chris Plummer wrote: >> Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 14 additional commits since the last revision: >> >> - Allow for warning to be skipped when same agent loaded a second/subsequent time >> - Merge >> - Tweak javadoc, update test to use more test infra >> - Merge >> - Merge >> - Refresh package description >> - Merge >> - Tweak docs >> - Merge >> - Draft docs changes >> - ... and 4 more: https://git.openjdk.org/jdk/compare/5a6c8363...a6d3c23c > > src/hotspot/share/prims/jvmtiAgentList.cpp line 231: > >> 229: if (agent->is_static_lib() && agent->is_loaded()) { >> 230: return true; >> 231: } > > This doesn't make sense to me. If you pass in `null` for `os_lib`, then we return true if any loaded static lib is found. Is this an attempt to limit the warning to just the first static lib that is loaded? Also, why would `null` ever be passed in if there wasn't at least one static lib. Some clarify comments would be useful. load_agent_from_executable has a comment to explain how statically linked agents are started, that's why it needs to use agent->is_static_lib().&& agent->is_loaded() here. There isn't currently a way to test this but there is other work going to support static builds so it might be possible to write some automated tests at that point. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1212613094 From alanb at openjdk.org Thu Jun 1 06:10:11 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 1 Jun 2023 06:10:11 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v4] In-Reply-To: References: <53KwBsPMnp0w_Xz-YomWCF8XdmvtuJV_m_9k8gkPzZo=.5a8ffaca-156d-4010-97f9-690b19f7ec29@github.com> Message-ID: On Wed, 31 May 2023 22:13:08 GMT, Chris Plummer wrote: >> We can't currently test `jcmd JVMTI.agent_load` with `-XX:-EnableDynamicAgentLoading` due to JDK-8304438. But this test is for warnings, not that dynamic loading is disallowed. > > @AlanBateman @sspitsyn Can this be revisited now that JDK-8304438 is being worked on? Perhaps the changes to JDK-8304438 can include an update to this test, although it all depends on which is being pushed first. The tests have different concerns. The test here is checking that warnings are printed or can be suppressed. Tests for -EnableDynamicAgentLoading are a different concern. I see Serguei is expanding test coverage as part of JDK-8304438 so I think we are good. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1212624593 From dholmes at openjdk.org Thu Jun 1 06:23:07 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 1 Jun 2023 06:23:07 GMT Subject: RFR: 8308090: Add container tests for on-the-fly resource quota updates [v2] In-Reply-To: References: Message-ID: On Tue, 23 May 2023 09:04:11 GMT, Severin Gehwolf wrote: >> Please review these test changes which implement automatic testing of container resource updates without JVM restart. Note that this merely tests container detection code handling this case. It doesn't do anything special for the JVM itself, though it might make sense to add some sanity checks should we detect certain limits changing. In another PR, though. >> >> As to the test design, it works similar to the shared temp tests: Interact between the two containers by virtue of a shared filesystem `/tmp` and creating marker files there in order to make them cooperate. Note that the new test needs `podman` version `4.3.0` and better (`4.5` is current). >> >> Testing: >> - [x] GHA >> - [x] Linux x86_64 container tests on cg v1 and cg v2 system >> - [x] Newly added tests on Linux x86_64 cg v1 and cg v2 (`podman` and `docker`) > > Severin Gehwolf has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - Merge branch 'master' into jdk-8308090-tests-container-on-fly-updates > - Fix whitespace > - 8308090: Add container tests for on-the-fly resource quota updates These tests didn't fail but I couldn't properly validate the output as we don't seem to save the process stdout file. But I guess these are okay. Thanks. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14090#pullrequestreview-1454663420 From sspitsyn at openjdk.org Thu Jun 1 06:50:11 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 1 Jun 2023 06:50:11 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v7] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 05:51:51 GMT, Alan Bateman wrote: >> It looks like you are right. >> There is also and assert at line 519: >> `519 assert(agent->is_loaded(), "invariant");` >> >> So, the agent has to be loaded if we got to the line 512. >> Also, there is a statement at line 507 (before line 512): >> `507 agent->set_os_lib(library);` > >> The use of `!JvmtiAgentList::is_loaded(library)` here is a bit confusing. Isn't the library always already loaded by the time we get here (the assert below seems to imply that)? > > JvmtiAgentList::load_agent creates the JvmtiAgent, attempts to load it, and adds it to the agent list if is succeeds. The test that you are looking is checking the list of already loaded agents. Maybe the function name "is_loaded" is the confusion here, maybe a better name is needed. I see the source of my confusion. The function `invoke_Agent_OnAttach` is called from the `JvmtiAgent::load`. But at the time of `invoke_Agent_OnAttach` the agent has not been added to the list yet. It is added after the `JvmtiAgent::load` with the function `JvmtiAgentList::add`: jint JvmtiAgentList::load_agent(const char* agent_name, const char* absParam, const char* options, outputStream* st) { // The abs parameter should be "true" or "false" const bool is_absolute_path = (absParam != nullptr) && (strcmp(absParam, "true") == 0); JvmtiAgent* const agent = new JvmtiAgent(agent_name, options, is_absolute_path, /* dynamic agent */ true); if (agent->load(st)) { add(agent); . . . Now I see the code is correct. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1212665902 From jaroslav.bachorik at datadoghq.com Thu Jun 1 07:16:38 2023 From: jaroslav.bachorik at datadoghq.com (=?UTF-8?Q?Jaroslav_Bachor=C3=ADk?=) Date: Thu, 1 Jun 2023 09:16:38 +0200 Subject: Clarifying jmethodID Usage and Potential JVM Crashes In-Reply-To: <31e606ae-f54e-c8e6-c00b-720ef192d279@oracle.com> References: <31e606ae-f54e-c8e6-c00b-720ef192d279@oracle.com> Message-ID: Hi David, On Thu, Jun 1, 2023 at 3:56?AM David Holmes wrote: > Hi Jaroslav, > > On 31/05/2023 9:12 pm, Jaroslav Bachor?k wrote: > > Dear Team, > > > > I've been investigating the unusual JVM crashes occurring in JVMTI calls > > on a J9 JVM. During my investigation, I scrutinized the `jmethodID` > > definition closely, available here: [jmethodID > > definition]( > https://docs.oracle.com/en/java/javase/17/docs/specs/jvmti.html#jmethodID > >). > > > > To paraphrase, the definition suggests that `jmethodID` identifies a > > Java method, initializer, or constructor. These identifiers, once > > returned by JVM TI functions and events, can be safely stored. However, > > when the class is unloaded, they become invalid, rendering them > > unsuitable for use. > > > > My interpretation is that the JVMTI user should verify the validity of a > > `jmethodID` value before using it to prevent potential crashes. Would > > you agree with this interpretation? > > Not quite - as you note you can't verify the jmethodID validity. What > the user needs to do, in line with what Dan was saying, is ensure that > they keep track of the classes to which the methods belong and keep them > alive if necessary. Now that may be easier said than done, but that is > the gist of it. This comes from the JNI spec: > > "A field or method ID does not prevent the VM from unloading the class > from which the ID has been derived. After the class is unloaded, the > method or field ID becomes invalid and may not be passed to any function > taking such an ID. The native code, therefore, must make sure to: > > keep a live reference to the underlying class, or > recompute the method or field ID > > if it intends to use a method or field ID for an extended period of time." > > > This sounds like a sensible requirement, but its practical application > > remains unclear. As far as I know, methods can be unloaded concurrently > > to the native code executing JVMTI functions. This introduces a > > potential race condition where the JVM unloads the methods during the > > check->use flow, making it only a partial solution. To complicate > > matters further, no method exists to confirm whether a `jmethodID` is > valid. > > > > Theoretically, we could monitor the `CompiledMethodUnload` event to > > track the validity state, creating a constantly expanding set of > > unloaded `jmethodID` values or a bloom filter, if one does not care > > about few potential false positives. This strategy, however, doesn't > > address the potential race condition, and it could even exacerbate it > > due to possible event delays. This delay might mistakenly validate a > > `jmethodID` value that has already been unloaded, but for which the > > event hasn't been delivered yet. > > > > Honestly, I don't see a way to use `jmethodID` safely unless the code > > using it suspends the entire JVM and doesn't resume until it's finished > > with that `jmethodID`. Any other approach might lead to JVM crashes, as > > we've observed with J9. > > > > Lastly, it's noteworthy that Hotspot takes meticulous measures to ensure > > that using jmethodIDs for unloaded methods doesn't crash the JVM and > > even provides useful information. This observation has led me to > > question whether the documentation aligns with the Hotspot > > implementation, especially given that following closely the > > documentation appears to increase the risk associated with the use of > > `jmethodID` values. > > There have been folk who wanted to make this area more user-friendly but > that shouldn't be mistaken for moving towards a world where jmethodIDs > are always safe to use. > Yes, I see your point. Unfortunately, this confirms my worries that using AsyncGetCallTrace (ASGCT) on a system strictly adhering to the JVMTI spec of jmethoID is not really possible without risking random and quite frequent crashes on systems with concurrent class unloading enabled. FTR, ASGCT will record the stack trace as a list of frames, each one containing the corresponding jmethodID value. Considering that the most common usage of ASGCT is in a signal handler it makes it impossible to use JVMTI calls to resolve the holder class and create a strong reference to prevent it from being unloaded. And even if this would be possible we would need to figure out when to release the class reference when it is no more needed - and it is not really clear how we could do that reliably, leaving us with the option of holding the class references indefinitely or risking crashing JVM. I want to emphasize that not being able to resolve additional details for a jmethodID pointing to a method of an unloaded class is not an issue, as long as the JVMTI call does not crash. I think that https://bugs.openjdk.org/browse/JDK-8268364 did address exactly the problem of concurrent class unloading causing races in the code that is checking for validity of jmethodID and then using it. Can this be summarize in a way that the user is not guaranteed to get any additional information for an invalid jmethodID but it would be really nice for JVM not to crash when jmethodID becomes invalid as there is no way for the user to check for its validity in an atomic manner - and yes, even calling GetMethodDeclaringClass in order to obtain the class one could create a strong reference is a subject to racy behaviour so it really can not be used as a workaround. Cheers, -JB- > > Cheers, > David > > > I welcome your thoughts and perspectives on this matter. > > > > Best regards, > > > > Jaroslav > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sspitsyn at openjdk.org Thu Jun 1 07:38:15 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 1 Jun 2023 07:38:15 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v7] In-Reply-To: References: Message-ID: On Wed, 31 May 2023 15:01:37 GMT, Alan Bateman wrote: >> This is the implementation for JEP 451. There are two parts to this: >> >> 1. A multi-line warning is printed when a JVM TI or Java agent is loaded into a running VM. For JVM TI, the message is printed to stderr from JvmtiAgent::load. For Java agents, it is printed to System.err (as that may be redirected) in the JPLIS (j.l.instrumentation) implementation. This part includes an update to the JVM TI spec and API docs to require the warning. >> >> 2. If running with -Djdk.instrument.traceUsage or -Djdk.instrument.traceUsage=true, the calls to the Instrumentation API print a trace message and stack trace. >> >> Testing: tier1-6 > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 17 additional commits since the last revision: > > - Add impl note to document the XX option > - Cleanup > - Merge > - Allow for warning to be skipped when same agent loaded a second/subsequent time > - Merge > - Tweak javadoc, update test to use more test infra > - Merge > - Merge > - Refresh package description > - Merge > - ... and 7 more: https://git.openjdk.org/jdk/compare/381973e4...2d9d5922 The update looks good to me. Posted one nit though. Thanks, Serguei test/jdk/com/sun/tools/attach/warnings/DynamicLoadWarningTest.java line 67: > 65: class DynamicLoadWarningTest { > 66: private static final String JVMTI_AGENT_WARNING = "WARNING: A JVM TI agent has been dynamically loaded"; > 67: private static final String JAVA_AGENT_WARNING = "WARNING: A Java agent has been loaded dynamically"; Nit: The warnings can be unified to say "loaded dynamically" or "dynamically loaded" in the same order. Of course, it also impacts the files: `src/hotspot/share/prims/jvmtiAgent.cpp` and `src/java.instrument/share/classes/sun/instrument/InstrumentationImpl.java`. ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/13899#pullrequestreview-1454785048 PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1212717874 From thomas.stuefe at gmail.com Thu Jun 1 08:03:39 2023 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 1 Jun 2023 10:03:39 +0200 Subject: Clarifying jmethodID Usage and Potential JVM Crashes In-Reply-To: References: <31e606ae-f54e-c8e6-c00b-720ef192d279@oracle.com> Message-ID: On Thu, Jun 1, 2023 at 9:17?AM Jaroslav Bachor?k < jaroslav.bachorik at datadoghq.com> wrote: > Hi David, > > On Thu, Jun 1, 2023 at 3:56?AM David Holmes > wrote: > >> Hi Jaroslav, >> >> On 31/05/2023 9:12 pm, Jaroslav Bachor?k wrote: >> > Dear Team, >> > >> > I've been investigating the unusual JVM crashes occurring in JVMTI >> calls >> > on a J9 JVM. During my investigation, I scrutinized the `jmethodID` >> > definition closely, available here: [jmethodID >> > definition]( >> https://docs.oracle.com/en/java/javase/17/docs/specs/jvmti.html#jmethodID >> < >> https://docs.oracle.com/en/java/javase/17/docs/specs/jvmti.html#jmethodID >> >). >> > >> > To paraphrase, the definition suggests that `jmethodID` identifies a >> > Java method, initializer, or constructor. These identifiers, once >> > returned by JVM TI functions and events, can be safely stored. However, >> > when the class is unloaded, they become invalid, rendering them >> > unsuitable for use. >> > >> > My interpretation is that the JVMTI user should verify the validity of >> a >> > `jmethodID` value before using it to prevent potential crashes. Would >> > you agree with this interpretation? >> >> Not quite - as you note you can't verify the jmethodID validity. What >> the user needs to do, in line with what Dan was saying, is ensure that >> they keep track of the classes to which the methods belong and keep them >> alive if necessary. Now that may be easier said than done, but that is >> the gist of it. This comes from the JNI spec: >> >> "A field or method ID does not prevent the VM from unloading the class >> from which the ID has been derived. After the class is unloaded, the >> method or field ID becomes invalid and may not be passed to any function >> taking such an ID. The native code, therefore, must make sure to: >> >> keep a live reference to the underlying class, or >> recompute the method or field ID >> >> if it intends to use a method or field ID for an extended period of time." >> >> > This sounds like a sensible requirement, but its practical application >> > remains unclear. As far as I know, methods can be unloaded concurrently >> > to the native code executing JVMTI functions. This introduces a >> > potential race condition where the JVM unloads the methods during the >> > check->use flow, making it only a partial solution. To complicate >> > matters further, no method exists to confirm whether a `jmethodID` is >> valid. >> > >> > Theoretically, we could monitor the `CompiledMethodUnload` event to >> > track the validity state, creating a constantly expanding set of >> > unloaded `jmethodID` values or a bloom filter, if one does not care >> > about few potential false positives. This strategy, however, doesn't >> > address the potential race condition, and it could even exacerbate it >> > due to possible event delays. This delay might mistakenly validate a >> > `jmethodID` value that has already been unloaded, but for which the >> > event hasn't been delivered yet. >> > >> > Honestly, I don't see a way to use `jmethodID` safely unless the code >> > using it suspends the entire JVM and doesn't resume until it's finished >> > with that `jmethodID`. Any other approach might lead to JVM crashes, as >> > we've observed with J9. >> > >> > Lastly, it's noteworthy that Hotspot takes meticulous measures to >> ensure >> > that using jmethodIDs for unloaded methods doesn't crash the JVM and >> > even provides useful information. This observation has led me to >> > question whether the documentation aligns with the Hotspot >> > implementation, especially given that following closely the >> > documentation appears to increase the risk associated with the use of >> > `jmethodID` values. >> >> There have been folk who wanted to make this area more user-friendly but >> that shouldn't be mistaken for moving towards a world where jmethodIDs >> are always safe to use. >> > > Yes, I see your point. Unfortunately, this confirms my worries that using > AsyncGetCallTrace (ASGCT) on a system strictly adhering to the JVMTI spec > of jmethoID is not really possible without risking random and quite > frequent crashes on systems with concurrent class unloading enabled. > FTR, ASGCT will record the stack trace as a list of frames, each one > containing the corresponding jmethodID value. Considering that the most > common usage of ASGCT is in a signal handler it makes it impossible to use > JVMTI calls to resolve the holder class and create a strong reference to > prevent it from being unloaded. > And even if this would be possible we would need to figure out when to > release the class reference when it is no more needed - and it is not > really clear how we could do that reliably, leaving us with the option of > holding the class references indefinitely or risking crashing JVM. > > I want to emphasize that not being able to resolve additional details for > a jmethodID pointing to a method of an unloaded class is not an issue, as > long as the JVMTI call does not crash. I think that > https://bugs.openjdk.org/browse/JDK-8268364 did address exactly the > problem of concurrent class unloading causing races in the code that is > checking for validity of jmethodID and then using it. > > > Can this be summarize in a way that the user is not guaranteed to get any > additional information for an invalid jmethodID but it would be really nice > for JVM not to crash when jmethodID becomes invalid as there is no way for > the user to check for its validity in an atomic manner > - and yes, even calling GetMethodDeclaringClass in order to obtain the > class one could create a strong reference is a subject to racy behaviour so > it really can not be used as a workaround. > > I wonder about this. Looking at https://bugs.openjdk.org/browse/JDK-8268364 , we zero out the JNI handle table for the CLD earlier during class unloading. IIUC ClassLoaderData::unload() can run concurrently to Method::checked_resolve_jmethod_id, so could we still not end up in the same situation (resolving Method when the JNI table slot had still been occupied, but dereferencing Method after it was freed? Admittedly more rarely? Freeing method would return it to metaspace, and it may or may not be uncommitted so crashes would be intermittent and random. This is a Coleen question :) Cheers, > > -JB- > > >> >> Cheers, >> David >> >> > I welcome your thoughts and perspectives on this matter. >> > >> > Best regards, >> > >> > Jaroslav >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgehwolf at openjdk.org Thu Jun 1 08:24:06 2023 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 1 Jun 2023 08:24:06 GMT Subject: RFR: 8308090: Add container tests for on-the-fly resource quota updates [v2] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 02:16:12 GMT, David Holmes wrote: >> Anyone willing to review this? > > @jerboaa I can't really review the tests themselves but will run through our CI to see if they cause any problems. If not then they should be okay to add. Thanks @dholmes-ora for running them through your CI. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14090#issuecomment-1571584032 From david.holmes at oracle.com Thu Jun 1 08:28:50 2023 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Jun 2023 18:28:50 +1000 Subject: Clarifying jmethodID Usage and Potential JVM Crashes In-Reply-To: References: <31e606ae-f54e-c8e6-c00b-720ef192d279@oracle.com> Message-ID: On 1/06/2023 5:16 pm, Jaroslav Bachor?k wrote: > Hi David, > > On Thu, Jun 1, 2023 at 3:56?AM David Holmes > wrote: > > Hi Jaroslav, > > On 31/05/2023 9:12 pm, Jaroslav Bachor?k wrote: > > Dear Team, > > > > I've been investigating the unusual JVM crashes occurring in > JVMTI calls > > on a J9 JVM. During my investigation, I scrutinized the `jmethodID` > > definition closely, available here: [jmethodID > > > definition](https://docs.oracle.com/en/java/javase/17/docs/specs/jvmti.html#jmethodID >). > > > > To paraphrase, the definition suggests that `jmethodID` identifies a > > Java method, initializer, or constructor. These identifiers, once > > returned by JVM TI functions and events, can be safely stored. > However, > > when the class is unloaded, they become invalid, rendering them > > unsuitable for use. > > > > My interpretation is that the JVMTI user should verify the > validity of a > > `jmethodID` value before using it to prevent potential crashes. > Would > > you agree with this interpretation? > > Not quite - as you note you can't verify the jmethodID validity. What > the user needs to do, in line with what Dan was saying, is ensure that > they keep track of the classes to which the methods belong and keep > them > alive if necessary. Now that may be easier said than done, but that is > the gist of it. This comes from the JNI spec: > > "A field or method ID does not prevent the VM from unloading the class > from which the ID has been derived. After the class is unloaded, the > method or field ID becomes invalid and may not be passed to any > function > taking such an ID. The native code, therefore, must make sure to: > > ? ? ?keep a live reference to the underlying class, or > ? ? ?recompute the method or field ID > > if it intends to use a method or field ID for an extended period of > time." > > > This sounds like a sensible requirement, but its practical > application > > remains unclear. As far as I know, methods can be unloaded > concurrently > > to the native code executing JVMTI functions. This introduces a > > potential race condition where the JVM unloads the methods during > the > > check->use flow, making it only a partial solution. To complicate > > matters further, no method exists to confirm whether a > `jmethodID` is valid. > > > > Theoretically, we could monitor the `CompiledMethodUnload` event to > > track the validity state, creating a constantly expanding set of > > unloaded `jmethodID` values or a bloom filter, if one does not care > > about few potential false positives. This strategy, however, doesn't > > address the potential race condition, and it could even > exacerbate it > > due to possible event delays. This delay might mistakenly validate a > > `jmethodID` value that has already been unloaded, but for which the > > event hasn't been delivered yet. > > > > Honestly, I don't see a way to use `jmethodID` safely unless the > code > > using it suspends the entire JVM and doesn't resume until it's > finished > > with that `jmethodID`. Any other approach might lead to JVM > crashes, as > > we've observed with J9. > > > > Lastly, it's noteworthy that Hotspot takes meticulous measures to > ensure > > that using jmethodIDs for unloaded methods doesn't crash the JVM and > > even provides useful information. This observation has led me to > > question whether the documentation aligns with the Hotspot > > implementation, especially given that following closely the > > documentation appears to increase the risk associated with the > use of > > `jmethodID` values. > > There have been folk who wanted to make this area more user-friendly > but > that shouldn't be mistaken for moving towards a world where jmethodIDs > are always safe to use. > > > Yes, I see your point. Unfortunately, this confirms my worries that > using AsyncGetCallTrace (ASGCT) on a system strictly adhering to the > JVMTI spec of jmethoID is not really possible without risking random and > quite frequent crashes on systems with concurrent class unloading enabled. > FTR, ASGCT will record the stack trace as a list of frames, each one > containing?the corresponding jmethodID value. Considering that the most > common usage of ASGCT is in a signal handler it makes it impossible to > use JVMTI calls to resolve the holder class and create a strong > reference to prevent it from being unloaded. > And even if this would be possible we would need to figure out when to > release the class reference when it is no more needed - and it is not > really clear how we could do that reliably, leaving us with the option > of holding the class references indefinitely or risking crashing JVM. I would have to agree you cannot use jmethodID's for that purpose, not without also recording (and keeping-alive) the associated classes. As to when you would clear such references I can't say as I don't know how the ASGCT stack record would be used. > I want to emphasize that not being able to resolve additional details > for a jmethodID pointing to a method of an unloaded class is not an > issue, as long as the JVMTI call does not crash. I think that > https://bugs.openjdk.org/browse/JDK-8268364 > did address exactly the > problem of concurrent class unloading causing races in the code that is > checking for validity of jmethodID and then using it. Yes but internal to the JVMTI implementation. The basic null check that was used was found to be insufficient when used with ZGC and so we would crash more often than was considered reasonable. There is a quoted comment in the code in https://bugs.openjdk.org/browse/JDK-8268088: // These aren't deallocated and are going to look like a leak, but that's // needed because we can't really get rid of jmethodIDs because we don't // know when native code is going to stop using them. The spec says that // they're "invalid" but existing programs likely rely on their being // NULL after class unloading. that kind of sums up the position of trying to accommodate "bad code" in a reasonable way. > Can this be summarize in a way that the user is not guaranteed to get > any additional information for an invalid jmethodID but it would be > really nice for JVM not to crash when jmethodID becomes invalid as there > is no way for the user to check for its validity in an atomic manner > - and yes, even calling GetMethodDeclaringClass in order to obtain the > class one could create a strong reference is a subject to racy behaviour > so it really can not be used as a workaround. The only real solution IMO would be to encode jmethodIDs in a way that validity can be tracked and then have all JVMTI methods be able to check that validity and guarantee atomicity of the method such that the id can't become invalid whilst in use. Whether that is at all feasible/practical I do not know - but it sounds like a major effort to me. Cheers, David > Cheers, > > -JB- > > > Cheers, > David > > > I welcome your thoughts and perspectives on this matter. > > > > Best regards, > > > > Jaroslav > From dchuyko at openjdk.org Thu Jun 1 09:30:41 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Thu, 1 Jun 2023 09:30:41 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives Message-ID: Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives. Methods in general are often inlined, and this information is hard to track down. Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Obviously there is a performance penalty, so it should be applied with care. Hot code will most likely be recompiled soon, as nothing happens to its hotness. A new flag '`-d`' has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and marks for deoptimization those methods that have any active non-default matching compiler directives. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be deoptimized. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be deoptimized, but this can be achieved by having rules for them. In addition, a new diagnistic command `Compiler.replace_directives`, has been added for convenience. It's like a combination of `Compiler.clear_directives` and `Compiler.add_directives`. It supports the same new optional '-d' flag that marks both cleared and added methods for deoptimization. The behavior of the '-d' flag is implemented in the new `CodeCache::mark_for_deoptimization_directives_matches` and `DirectivesStack::hasMatchingDirectives` methods. `CompilerDirectivesDCMDTest` now checks add, remove and replace commands in two modes (default and '-d') and checks that '-d' flag causes deoptimization. An alternative approach to the '-d' flag could be to have a special diagnostic command for deoptimization. It will get a list of method patterns and reuse the matcher, however this is not so trivial. Overall usage and effects will be similar but this is one more file format. The user will also need to monitor or query active directives in advance, e.g. to deoptimize all mentioned methods after clearing all directives. An alternative approach for selection of deoptimized methods could be to track down all inlining dependencies. This may be similar to searching references to old methods, but it requires scanning all code blobs, which looks too expensive. An alternative naming for the flag is welcome. The obvious '-f' ('force') unfortunately has a conflict. Other verbs can be 'update', 'refresh' or 'apply'. Deoptimization is just what's done to reconcile the state. It could be something else, like first compiling with a different compiler and then switching to that version. Although in the latter case, triggered compilation would be an essential detail. ------------- Commit messages: - Merge branch 'openjdk:master' into compiler-directives-force-update - Formatting - Formatting - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Correct arguments info for new commands - Update through de-optimization Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309271 Stats: 214 lines in 9 files changed: 194 ins; 0 del; 20 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From aph at openjdk.org Thu Jun 1 09:46:25 2023 From: aph at openjdk.org (Andrew Haley) Date: Thu, 1 Jun 2023 09:46:25 GMT Subject: RFR: 8309044: Replace NULL with nullptr, final sweep of hotspot code [v2] In-Reply-To: References: <3FoMnGeBp8DqkpVb6YGXKxdPsgGz6ej-jrf2U2stVfU=.56a11e19-38dd-420a-a07d-3b025120f194@github.com> Message-ID: On Tue, 30 May 2023 00:57:40 GMT, David Holmes wrote: > Can we now poison NULL so it can't get reintroduced? Or would that potentially break standard headers? I'm sure it would. Maybe some changes to Skara? ------------- PR Comment: https://git.openjdk.org/jdk/pull/14198#issuecomment-1571706648 From jsjolen at openjdk.org Thu Jun 1 09:54:26 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Thu, 1 Jun 2023 09:54:26 GMT Subject: RFR: 8309044: Replace NULL with nullptr, final sweep of hotspot code [v2] In-Reply-To: <9RnqD_bbEcutUNcfw9elCwYbPS_7-JiHMn_9cVAmDxQ=.830877a3-eac8-4af9-9ac0-5244c37989fb@github.com> References: <3FoMnGeBp8DqkpVb6YGXKxdPsgGz6ej-jrf2U2stVfU=.56a11e19-38dd-420a-a07d-3b025120f194@github.com> <9RnqD_bbEcutUNcfw9elCwYbPS_7-JiHMn_9cVAmDxQ=.830877a3-eac8-4af9-9ac0-5244c37989fb@github.com> Message-ID: <2WGxhOHFo3u8TGyhklFPc-Ipml5HL8PO9aakXX9cgFA=.d8e6a4c6-2da5-4173-ac96-fa4088f25502@github.com> On Thu, 1 Jun 2023 05:23:25 GMT, Tobias Hartmann wrote: > What's the plan now to prevent re-introducing `NULL`? Hi Tobias. The only plan in place is social, the reviewers have to look out for it. I am however researching how to do this through machine. I'm currently researching ways of preventing any re-introductions by machine. These include poisoning the NULL macro by re-defining it and finding a tool which is capable of parsing C++ code which is yet to go through the pre-processor. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14198#issuecomment-1571722147 From dnsimon at openjdk.org Thu Jun 1 10:16:24 2023 From: dnsimon at openjdk.org (Doug Simon) Date: Thu, 1 Jun 2023 10:16:24 GMT Subject: RFR: 8309044: Replace NULL with nullptr, final sweep of hotspot code [v2] In-Reply-To: References: <3FoMnGeBp8DqkpVb6YGXKxdPsgGz6ej-jrf2U2stVfU=.56a11e19-38dd-420a-a07d-3b025120f194@github.com> Message-ID: On Tue, 30 May 2023 19:15:38 GMT, Johan Sj?len wrote: >> A final sweep of Hotspot to remove all re-added NULLs. With only 110 changes I'd appreciate if this was considered trivial. > > Johan Sj?len has updated the pull request incrementally with two additional commits since the last revision: > > - Align > - Suggestions It may be simpler to use simple grepping + an allow list. For example using [`ag`](https://github.com/ggreer/the_silver_searcher) and `grep` seems to catch the few remaining offenders: > ag NULL src/hotspot/ --cpp | grep -v _NULL | grep -v NULL_ | grep -v -E '[A-Z]NULL' | grep -v -E '//.*NULL' | grep -v '"NULL"' src/hotspot/cpu/ppc/macroAssembler_ppc.hpp:735: void load_klass_check_null(Register dst, Register src, Label* is_null = NULL); src/hotspot/cpu/ppc/stubGenerator_ppc.cpp:4700: if (UnsafeCopyMemory::_table == NULL) { src/hotspot/cpu/x86/jvmciCodeInstaller_x86.cpp:191: if (nop == NULL) { src/hotspot/cpu/riscv/codeBuffer_riscv.cpp:74: if (cb->stubs()->maybe_expand_to_ensure_remaining(total_requested_size) && cb->blob() == NULL) { src/hotspot/cpu/riscv/stubGenerator_riscv.cpp:4019: if (UnsafeCopyMemory::_table == NULL) { src/hotspot/cpu/riscv/stubGenerator_riscv.cpp:4077: if (bs_nm != NULL) { src/hotspot/cpu/aarch64/jvmciCodeInstaller_aarch64.cpp:125: NativeCall* call = NULL; src/hotspot/cpu/aarch64/jvmciCodeInstaller_aarch64.cpp:158: if (nop == NULL) { src/hotspot/share/jfr/dcmd/jfrDcmds.hpp:162: JavaPermission p = {"java.lang.management.ManagementPermission", "monitor", NULL}; src/hotspot/share/jfr/dcmd/jfrDcmds.hpp:187: JavaPermission p = {"java.lang.management.ManagementPermission", "monitor", NULL}; src/hotspot/share/include/jvm.h:423: * Find a class from a boot class loader. Returns NULL if class not found. src/hotspot/share/prims/jvmtiAgent.cpp:375: const jint err = (*on_load_entry)(&main_vm, const_cast(agent->options()), NULL); src/hotspot/share/prims/whitebox.cpp:1885: if (cp->cache() == NULL) { src/hotspot/share/prims/whitebox.cpp:1894: if (cp->cache() == NULL) { src/hotspot/share/classfile/stringTable.hpp:150: static oop init_shared_table(const DumpedInternedStrings* dumped_interned_strings) NOT_CDS_JAVA_HEAP_RETURN_(NULL); src/hotspot/share/utilities/globalDefinitions_xlc.hpp:95: #undef NULL src/hotspot/share/utilities/globalDefinitions_xlc.hpp:96: #define NULL 0L src/hotspot/share/utilities/globalDefinitions_xlc.hpp:98: #ifndef NULL src/hotspot/share/utilities/globalDefinitions_xlc.hpp:99: #define NULL 0 src/hotspot/share/cds/filemap.cpp:363: assert(ent != NULL, "sanity"); src/hotspot/share/utilities/globalDefinitions_visCPP.hpp:65:#undef NULL src/hotspot/share/utilities/globalDefinitions_visCPP.hpp:69:#define NULL 0LL src/hotspot/share/utilities/globalDefinitions_visCPP.hpp:71:#ifndef NULL src/hotspot/share/utilities/globalDefinitions_visCPP.hpp:72:#define NULL 0 src/hotspot/share/utilities/globalDefinitions.cpp:162: static_assert(sizeof(NULL) == sizeof(char*), "NULL must be same size as pointer"); src/hotspot/share/adlc/output_c.cpp:279: for (pipeline->_reslist.reset(); (resource = pipeline->_reslist.iter()) != NULL;) { src/hotspot/share/adlc/output_c.cpp:305: for (pipeline->_reslist.reset(); (resource = pipeline->_reslist.iter()) != NULL;) { src/hotspot/share/adlc/output_c.cpp:368: for (pipeline->_reslist.reset(); (resource = pipeline->_reslist.iter()) != NULL;) { src/hotspot/share/adlc/output_c.cpp:393: for (pipeline->_reslist.reset(); (resource = pipeline->_reslist.iter()) != NULL;) { src/hotspot/share/adlc/output_c.cpp:1009: for (_pipeline->_reslist.reset(); (resource = _pipeline->_reslist.iter()) != NULL;) { src/hotspot/share/gc/x/xBarrierSet.inline.hpp:187: return Raw::oop_arraycopy_in_heap(nullptr, 0, src, NULL, 0, dst, length); src/hotspot/share/gc/x/xPageTable.inline.hpp:43: if (entry != NULL && entry != _prev) { src/hotspot/share/gc/x/xBarrier.cpp:242: return NULL; src/hotspot/share/oops/cpCache.cpp:888: LogStream* log_stream = NULL; src/hotspot/share/oops/cpCache.cpp:906: assert(resolved_references->obj_at(appendix_index) == NULL, "init just once"); src/hotspot/share/oops/cpCache.cpp:914: if (log_stream != NULL) { src/hotspot/share/opto/runtime.cpp:491: fields[TypeFunc::Parms+0] = NULL; // void src/hotspot/share/jvmci/jvmciEnv.cpp:366: if (ex != NULL) { ------------- PR Comment: https://git.openjdk.org/jdk/pull/14198#issuecomment-1571756690 From aph at openjdk.org Thu Jun 1 10:17:19 2023 From: aph at openjdk.org (Andrew Haley) Date: Thu, 1 Jun 2023 10:17:19 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) In-Reply-To: References: Message-ID: On Fri, 26 May 2023 20:46:29 GMT, Kelvin Nilsen wrote: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 147: > 145: } > 146: #endif > 147: } This logic is so gnarly that it's very hard to review and maintain, and IMO it's dangerous. The problem is that its correctness depends on exactly how registers are allocated in its caller. This needs restructuring so that the register allocation is defined in a single place then passed down to everyone who needs it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1212922771 From stefank at openjdk.org Thu Jun 1 10:25:21 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 1 Jun 2023 10:25:21 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) In-Reply-To: References: Message-ID: On Fri, 26 May 2023 20:46:29 GMT, Kelvin Nilsen wrote: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. I've looked at the shared code and it's really nice that you've managed to keep them to a minimum. I have one tiny nit that would be nice to fix. src/hotspot/share/gc/shared/gcConfiguration.cpp line 88: > 86: } > 87: #endif > 88: return NA; You moved the order between Shenandoah and ZGC in `young_collector()`, so you should probably do the same here. ------------- PR Review: https://git.openjdk.org/jdk/pull/14185#pullrequestreview-1455087651 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1212919666 From mdoerr at openjdk.org Thu Jun 1 12:18:19 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 1 Jun 2023 12:18:19 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) In-Reply-To: References: Message-ID: On Fri, 26 May 2023 20:46:29 GMT, Kelvin Nilsen wrote: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Issues already reported to GenShen engineers: gc/shenandoah/TestElasticTLAB.java#generational #? Internal Error (src\hotspot\share\gc\shenandoah\shenandoahFreeSet.cpp:695), pid=23288, tid=23784 #? assert(size % CardTable::card_size_in_words() == 0) failed: size must be multiple of card table size, was 258 gc/stress/gcold/TestGCOldWithShenandoah.java#generational #? Internal Error (src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp:82), pid=20828, tid=5836 #? assert(_old_generation->available() > old_evacuation_budget) failed: Cannot budget more than is available gc/shenandoah/oom/TestAllocOutOfMemory.java#large Execution failed: `main' threw exception: java.lang.RuntimeException: 'java.lang.OutOfMemoryError: Java heap space' missing from stdout/stderr (Issue with 64k Pages) gc/shenandoah/TestRetainObjects.java#no-tlab gc/shenandoah/TestSieveObjects.java#no-tlab Timeouts. gc/shenandoah/TestAllocObjects.java#generational gc/shenandoah/TestDynamicSoftMaxHeapSize.java#generational #? Internal Error src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp:664), pid=18434, tid=29955 #? assert(is_global() || ShenandoahHeap::heap()->is_full_gc_in_progress() || (_used + _humongous_waste <= _affiliated_region_count * ShenandoahHeapRegion::region_size_bytes())) failed: used cannot exceed regions ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1571947174 From alanb at openjdk.org Thu Jun 1 12:37:20 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 1 Jun 2023 12:37:20 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v8] In-Reply-To: References: Message-ID: > This is the implementation for JEP 451. There are two parts to this: > > 1. A multi-line warning is printed when a JVM TI or Java agent is loaded into a running VM. For JVM TI, the message is printed to stderr from JvmtiAgent::load. For Java agents, it is printed to System.err (as that may be redirected) in the JPLIS (j.l.instrumentation) implementation. This part includes an update to the JVM TI spec and API docs to require the warning. > > 2. If running with -Djdk.instrument.traceUsage or -Djdk.instrument.traceUsage=true, the calls to the Instrumentation API print a trace message and stack trace. > > Testing: tier1-6 Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 19 additional commits since the last revision: - Split is_loaded, typos in comments - Merge - Add impl note to document the XX option - Cleanup - Merge - Allow for warning to be skipped when same agent loaded a second/subsequent time - Merge - Tweak javadoc, update test to use more test infra - Merge - Merge - ... and 9 more: https://git.openjdk.org/jdk/compare/eb72f87a...cbd9bb1d ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13899/files - new: https://git.openjdk.org/jdk/pull/13899/files/2d9d5922..cbd9bb1d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13899&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13899&range=06-07 Stats: 2137 lines in 162 files changed: 788 ins; 840 del; 509 mod Patch: https://git.openjdk.org/jdk/pull/13899.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13899/head:pull/13899 PR: https://git.openjdk.org/jdk/pull/13899 From alanb at openjdk.org Thu Jun 1 12:37:21 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 1 Jun 2023 12:37:21 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v6] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 05:55:43 GMT, Alan Bateman wrote: >> src/hotspot/share/prims/jvmtiAgentList.cpp line 231: >> >>> 229: if (agent->is_static_lib() && agent->is_loaded()) { >>> 230: return true; >>> 231: } >> >> This doesn't make sense to me. If you pass in `null` for `os_lib`, then we return true if any loaded static lib is found. Is this an attempt to limit the warning to just the first static lib that is loaded? Also, why would `null` ever be passed in if there wasn't at least one static lib. Some clarify comments would be useful. > > load_agent_from_executable has a comment to explain how statically linked agents are started, that's why it needs to use agent->is_static_lib().&& agent->is_loaded() here. There isn't currently a way to test this but there is other work going to support static builds so it might be possible to write some automated tests at that point. I've updated this so that is_loaded is split into one function to test if a statically linked agent is already loaded and another for agent libraries. That might be clearer to understand. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1213080073 From stuefe at openjdk.org Thu Jun 1 13:01:21 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 1 Jun 2023 13:01:21 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) In-Reply-To: References: Message-ID: On Fri, 26 May 2023 20:46:29 GMT, Kelvin Nilsen wrote: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Hi Kevin, First off, kudos. This is impressive work by you and your Amazon colleagues! I have one particular worry, though, how to verify that this experimental feature does not cause regressions with traditional Shenandoah? The PR is massive (+18kloc) and targeted for JDK 21. Rampdown P1 is in a week. By all accounts, JDK 21 will be a massive release, so we will all have our hands full, fixing stuff and plugging holes. Oracle did put the sources for the Generational ZGC beside the old sources, thereby somewhat guaranteeing traditional ZGC does not regress. Could we follow the same cautionary process here? I am not a Shenandoah expert, but to me the new feature seems intertwined with normal code paths. It's difficult to ensure, via review, that traditional Shenandoah will not suffer regressions. So close to rampdown this is a bit scary. The JEP mentions several "Risks and Assumptions", but it is unclear whether these risks also affect traditional Shenandoah. Cheers, Thomas ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1572008074 From duke at openjdk.org Thu Jun 1 13:11:26 2023 From: duke at openjdk.org (JoKern65) Date: Thu, 1 Jun 2023 13:11:26 GMT Subject: Withdrawn: JDK-8308288: Fix xlc17 clang warnings in shared code In-Reply-To: References: Message-ID: On Thu, 25 May 2023 09:14:14 GMT, JoKern65 wrote: > When using the new xlc17 compiler (based on a recent clang) to build OpenJDk on AIX , we run into various "warnings as errors". > Some of those are in shared codebase and could be addressed by small adjustments. > A lot of those changes are in hotspot, some might be somewhere else in the OpenJDK C/C++ code. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/14146 From duke at openjdk.org Thu Jun 1 13:11:24 2023 From: duke at openjdk.org (JoKern65) Date: Thu, 1 Jun 2023 13:11:24 GMT Subject: RFR: JDK-8308288: Fix xlc17 clang warnings in shared code [v2] In-Reply-To: References: Message-ID: On Fri, 26 May 2023 08:31:46 GMT, JoKern65 wrote: >> When using the new xlc17 compiler (based on a recent clang) to build OpenJDk on AIX , we run into various "warnings as errors". >> Some of those are in shared codebase and could be addressed by small adjustments. >> A lot of those changes are in hotspot, some might be somewhere else in the OpenJDK C/C++ code. > > JoKern65 has updated the pull request incrementally with one additional commit since the last revision: > > forgotton _ Hi, As this PR is big and spans several components I split off the java.base, java.desktop and the sercivability/security issues into extra JBS issues. https://bugs.openjdk.org/browse/JDK-8309219 Fix xlc17 clang 1.5 warnings in java.base https://bugs.openjdk.org/browse/JDK-8309224 Fix xlc17 clang 1.5 warnings in java.desktop https://bugs.openjdk.org/browse/JDK-8309225 Fix xlc17 clang 1.5 warnings in security and servicability I?ll move the changes from this pull request into new pull requests. I will incorporate the requested changes right in the new PRs. I will reuse this issue 8308388 for the hotspot changes but come up with a new, smaller PR. @colleenp, I will move alloca.h to the globalDefinitions_xlc.hpp. @prrace, I will come up with an identical PR for the client files (java.desktop), but improve the comment as @kimbarrett proposed @mbaesken, I will use AIX and take up some of the other fixes you proposed. I guess we need to find a way to fix the issue with the malloc in globalDefinitions_xlc.hpp in the upcoming PR for hotspot. Thanks for your help so far! Hi, As this PR is big and spans several components I split off the java.base, java.desktop and the sercivability/security issues into extra JBS issues. https://bugs.openjdk.org/browse/JDK-8309219 Fix xlc17 clang 1.5 warnings in java.base https://bugs.openjdk.org/browse/JDK-8309224 Fix xlc17 clang 1.5 warnings in java.desktop https://bugs.openjdk.org/browse/JDK-8309225 Fix xlc17 clang 1.5 warnings in security and servicability I?ll move the changes from this pull request into new pull requests. I will incorporate the requested changes right in the new PRs. I will reuse this issue 8308388 for the hotspot changes but come up with a new, smaller PR. @colleenp, I will move alloca.h to the globalDefinitions_xlc.hpp. @prrace, I will come up with an identical PR for the client files (java.desktop), but improve the comment as @kimbarrett proposed @MBaesken, I will use AIX and take up some of the other fixes you proposed. I guess we need to find a way to fix the issue with the malloc in globalDefinitions_xlc.hpp in the upcoming PR for hotspot. Thanks for your help so far! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14146#issuecomment-1572023812 PR Comment: https://git.openjdk.org/jdk/pull/14146#issuecomment-1572024628 From stuefe at openjdk.org Thu Jun 1 14:33:27 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 1 Jun 2023 14:33:27 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) In-Reply-To: References: Message-ID: On Fri, 26 May 2023 20:46:29 GMT, Kelvin Nilsen wrote: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. I did a first read through the tests to check if any test changes affect traditional Shenandoah. To see if regression tests for non-generational are unchanged. All good, I did not find anything noteworthy. test/hotspot/jtreg/gc/shenandoah/TestEvilSyncBug.java line 33: > 31: * @modules java.base/jdk.internal.misc > 32: * java.management > 33: * @run driver/timeout=480 TestEvilSyncBug -XX:ShenandoahGCHeuristics=aggressive Probably fine, but why this change to non-generational testing? Will aggressive heuristic sharpen the test? test/hotspot/jtreg/gc/shenandoah/mxbeans/TestChurnNotifications.java line 169: > 167: > 168: MemoryUsage before = getUsage(mapBefore); > 169: MemoryUsage after = getUsage(mapAfter); This also changes test logic for traditional Shenandoah, but its harmless. Nit: more precise would be to require "Young Gen" pool to only exist for -XX:ShenandoahGCMode=generational. test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 23: > 21: * questions. > 22: * > 23: */ Three tests folded into one, but it does not look like functionality changed for testing traditional Shenandoah. Okay. test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 92: > 90: expectFailure("-Xmx16m", > 91: "-XX:+UnlockExperimentalVMOptions", > 92: "-XX:+UseShenandoahGC", Nit: should not need UnlockExperimentalVMOptions anymore. test/hotspot/jtreg/gc/shenandoah/oom/TestClassLoaderLeak.java line 132: > 130: {{"iu"}, {"adaptive", "aggressive"}}, > 131: {{"passive"}, {"passive"}}, > 132: {{"generational"}, {"adaptive"}} Curious, here and in similar places, why only test adaptive heuristic for generational, if we test satb with all variants? ------------- PR Review: https://git.openjdk.org/jdk/pull/14185#pullrequestreview-1455490699 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213164085 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213232252 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213243280 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213242053 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213244749 From sspitsyn at openjdk.org Thu Jun 1 14:43:14 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 1 Jun 2023 14:43:14 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v8] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 12:37:20 GMT, Alan Bateman wrote: >> This is the implementation for JEP 451. There are two parts to this: >> >> 1. A multi-line warning is printed when a JVM TI or Java agent is loaded into a running VM. For JVM TI, the message is printed to stderr from JvmtiAgent::load. For Java agents, it is printed to System.err (as that may be redirected) in the JPLIS (j.l.instrumentation) implementation. This part includes an update to the JVM TI spec and API docs to require the warning. >> >> 2. If running with -Djdk.instrument.traceUsage or -Djdk.instrument.traceUsage=true, the calls to the Instrumentation API print a trace message and stack trace. >> >> Testing: tier1-6 > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 19 additional commits since the last revision: > > - Split is_loaded, typos in comments > - Merge > - Add impl note to document the XX option > - Cleanup > - Merge > - Allow for warning to be skipped when same agent loaded a second/subsequent time > - Merge > - Tweak javadoc, update test to use more test infra > - Merge > - Merge > - ... and 9 more: https://git.openjdk.org/jdk/compare/6cea493a...cbd9bb1d The update looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/13899#pullrequestreview-1455647274 From asemenov at openjdk.org Thu Jun 1 15:01:37 2023 From: asemenov at openjdk.org (Artem Semenov) Date: Thu, 1 Jun 2023 15:01:37 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v4] In-Reply-To: References: Message-ID: > When using the clang compiler to build OpenJDk on Linux, we encounter various "warnings as errors". > They can be fixed with small changes. Artem Semenov has updated the pull request incrementally with one additional commit since the last revision: update ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14033/files - new: https://git.openjdk.org/jdk/pull/14033/files/d5b70cb2..b423bd4c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14033&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14033&range=02-03 Stats: 7 lines in 2 files changed: 1 ins; 4 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14033.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14033/head:pull/14033 PR: https://git.openjdk.org/jdk/pull/14033 From asemenov at openjdk.org Thu Jun 1 15:07:11 2023 From: asemenov at openjdk.org (Artem Semenov) Date: Thu, 1 Jun 2023 15:07:11 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v3] In-Reply-To: References: Message-ID: On Wed, 31 May 2023 13:52:39 GMT, Weijun Wang wrote: >> Artem Semenov has updated the pull request incrementally with one additional commit since the last revision: >> >> update > > src/java.security.jgss/share/native/libj2gss/gssapi.h line 47: > >> 45: >> 46: // Condition was copied from >> 47: // Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/gssapi/gssapi.h > > In the current version of the file above, it's written as > > #if defined(__APPLE__) && (defined(__ppc__) ||... > > Is it better? done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14033#discussion_r1213297788 From weijun at openjdk.org Thu Jun 1 15:07:12 2023 From: weijun at openjdk.org (Weijun Wang) Date: Thu, 1 Jun 2023 15:07:12 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v3] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 15:02:16 GMT, Artem Semenov wrote: >> src/java.security.jgss/share/native/libj2gss/gssapi.h line 47: >> >>> 45: >>> 46: // Condition was copied from >>> 47: // Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/gssapi/gssapi.h >> >> In the current version of the file above, it's written as >> >> #if defined(__APPLE__) && (defined(__ppc__) ||... >> >> Is it better? > > done I didn't ask to revert the change. It's `s/TARGET_OS_MAC/defined(__APPLE__)/`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14033#discussion_r1213300305 From cjplummer at openjdk.org Thu Jun 1 15:18:18 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 15:18:18 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v7] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 06:46:41 GMT, Serguei Spitsyn wrote: >>> The use of `!JvmtiAgentList::is_loaded(library)` here is a bit confusing. Isn't the library always already loaded by the time we get here (the assert below seems to imply that)? >> >> JvmtiAgentList::load_agent creates the JvmtiAgent, attempts to load it, and adds it to the agent list if is succeeds. The test that you are looking is checking the list of already loaded agents. Maybe the function name "is_loaded" is the confusion here, maybe a better name is needed. > > I see the source of my confusion. > The function `invoke_Agent_OnAttach` is called from the `JvmtiAgent::load`. > But at the time of `invoke_Agent_OnAttach` the agent has not been added to the list yet. > It is added after the `JvmtiAgent::load` with the function `JvmtiAgentList::add`: > > jint JvmtiAgentList::load_agent(const char* agent_name, const char* absParam, > const char* options, outputStream* st) { > // The abs parameter should be "true" or "false" > const bool is_absolute_path = (absParam != nullptr) && (strcmp(absParam, "true") == 0); > JvmtiAgent* const agent = new JvmtiAgent(agent_name, options, is_absolute_path, /* dynamic agent */ true); > if (agent->load(st)) { > add(agent); > . . . > > Now I see the code is correct. > I'd suggest to add a small comment before the line 512 saying > that the agent has not been included into the agents list yet. > JvmtiAgentList::load_agent creates the JvmtiAgent, attempts to load it, and adds it to the agent list if is succeeds. The test that you are looking is checking the list of already loaded agents. Maybe the function name "is_loaded" is the confusion here, maybe a better name is needed. Yes, I did notice that there were two different `is_loaded` checks going on. What's not clear is the relation between agent->is_loaded() and JvmtiAgentList::is_loaded(library). I assume that at this point in time the library is not on the list but the agent is loaded, so that's why it works, but it's not clear to me why this is the case. I think a comment here explaining why would be helpful so the reader doesn't need to track down when each of these "loaded" conditions becomes true. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13899#discussion_r1213314470 From cjplummer at openjdk.org Thu Jun 1 15:27:12 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 15:27:12 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v2] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Wed, 31 May 2023 10:30:22 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > minor renaming in new test TestJcmdNoAgentLoad.java test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 68: > 66: System.out.println("Found output line with the expected error message:\n" + line); > 67: } > 68: } Why not just use `OutputAnalyzer.shouldContain()`? test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 70: > 68: } > 69: if (!seenPattern) { > 70: throw new RuntimeException("Not found expected error message in output"); "Not found" -> "Did not find" Also, you should include the expected error message in the output. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1212367038 PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1212367594 From cjplummer at openjdk.org Thu Jun 1 15:27:09 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 15:27:09 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v3] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Wed, 31 May 2023 23:39:19 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > move check for EnableDynamicAgentLoading to invoke_Agent_OnAttach Except for the possible addition of some comments regarding what is meant by "loaded", the changes look good. ------------- Marked as reviewed by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14244#pullrequestreview-1455750113 From alanb at openjdk.org Thu Jun 1 15:31:04 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 1 Jun 2023 15:31:04 GMT Subject: RFR: 8309146: extend JDI StackFrame.setValue() and JDWP StackFrame.setValues minimal support for virtual threads In-Reply-To: References: Message-ID: On Tue, 30 May 2023 23:44:28 GMT, Chris Plummer wrote: > The JDWP spec for StackFrame.SetValue currently states: > > "If the thread is a virtual thread then this command can be used to set " > "the value of local variables in the top-most frame when the thread is " > "suspended at a breakpoint or single step event. The target VM may support " > "setting local variables in other cases." > > The JDI spec for StackFrame.setValue() has similar wording. In [JDK-8308814](https://bugs.openjdk.org/browse/JDK-8308814) the JVMTI spec clarified support to be for a thread suspended at any event, not just a breakpoint or single step. That same clarification is needed in the JDWP and JDI specs. No implementation changes are needed. Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14235#pullrequestreview-1455759449 From cjplummer at openjdk.org Thu Jun 1 15:32:16 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 15:32:16 GMT Subject: RFR: 8308232: nsk/jdb tests don't pass -verbose flag to the debuggee In-Reply-To: References: Message-ID: On Wed, 31 May 2023 00:27:16 GMT, Chris Plummer wrote: > The nsk/jdb tests are not passing the test args on to the debuggee. I found a way to pass all of them (see the CR for details), but Windows had trouble with the parsing. It turns out the only args that the debuggee really cares about are -verbose and -waittime, so I settled on just passing these two, and Windows is happy. > > Note only a handful of tests run with -verbose as the default. However, with this change at least now if you add -verbose when debugging an issue, you will see verbose log output from the debuggee. Previously you would not. Thanks for the reviews Serguei and Leonid! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14236#issuecomment-1572270309 From cjplummer at openjdk.org Thu Jun 1 15:32:16 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 15:32:16 GMT Subject: Integrated: 8308232: nsk/jdb tests don't pass -verbose flag to the debuggee In-Reply-To: References: Message-ID: On Wed, 31 May 2023 00:27:16 GMT, Chris Plummer wrote: > The nsk/jdb tests are not passing the test args on to the debuggee. I found a way to pass all of them (see the CR for details), but Windows had trouble with the parsing. It turns out the only args that the debuggee really cares about are -verbose and -waittime, so I settled on just passing these two, and Windows is happy. > > Note only a handful of tests run with -verbose as the default. However, with this change at least now if you add -verbose when debugging an issue, you will see verbose log output from the debuggee. Previously you would not. This pull request has now been integrated. Changeset: c6f20db9 Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/c6f20db945c6217aea84cebd6c97dbf8b93c48a4 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod 8308232: nsk/jdb tests don't pass -verbose flag to the debuggee Reviewed-by: sspitsyn, lmesnik ------------- PR: https://git.openjdk.org/jdk/pull/14236 From cjplummer at openjdk.org Thu Jun 1 15:36:18 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 15:36:18 GMT Subject: RFR: 8309146: extend JDI StackFrame.setValue() and JDWP StackFrame.setValues minimal support for virtual threads In-Reply-To: References: Message-ID: On Tue, 30 May 2023 23:44:28 GMT, Chris Plummer wrote: > The JDWP spec for StackFrame.SetValue currently states: > > "If the thread is a virtual thread then this command can be used to set " > "the value of local variables in the top-most frame when the thread is " > "suspended at a breakpoint or single step event. The target VM may support " > "setting local variables in other cases." > > The JDI spec for StackFrame.setValue() has similar wording. In [JDK-8308814](https://bugs.openjdk.org/browse/JDK-8308814) the JVMTI spec clarified support to be for a thread suspended at any event, not just a breakpoint or single step. That same clarification is needed in the JDWP and JDI specs. No implementation changes are needed. Thanks for the reviews Serguei and Alan! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14235#issuecomment-1572279020 From cjplummer at openjdk.org Thu Jun 1 15:36:18 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 15:36:18 GMT Subject: Integrated: 8309146: extend JDI StackFrame.setValue() and JDWP StackFrame.setValues minimal support for virtual threads In-Reply-To: References: Message-ID: On Tue, 30 May 2023 23:44:28 GMT, Chris Plummer wrote: > The JDWP spec for StackFrame.SetValue currently states: > > "If the thread is a virtual thread then this command can be used to set " > "the value of local variables in the top-most frame when the thread is " > "suspended at a breakpoint or single step event. The target VM may support " > "setting local variables in other cases." > > The JDI spec for StackFrame.setValue() has similar wording. In [JDK-8308814](https://bugs.openjdk.org/browse/JDK-8308814) the JVMTI spec clarified support to be for a thread suspended at any event, not just a breakpoint or single step. That same clarification is needed in the JDWP and JDI specs. No implementation changes are needed. This pull request has now been integrated. Changeset: e8271649 Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/e8271649e00771a8bbee240aa1bbbc27a672b22a Stats: 5 lines in 2 files changed: 0 ins; 0 del; 5 mod 8309146: extend JDI StackFrame.setValue() and JDWP StackFrame.setValues minimal support for virtual threads Reviewed-by: sspitsyn, alanb ------------- PR: https://git.openjdk.org/jdk/pull/14235 From daniel.daugherty at oracle.com Thu Jun 1 15:49:37 2023 From: daniel.daugherty at oracle.com (daniel.daugherty at oracle.com) Date: Thu, 1 Jun 2023 11:49:37 -0400 Subject: Clarifying jmethodID Usage and Potential JVM Crashes In-Reply-To: References: <31e606ae-f54e-c8e6-c00b-720ef192d279@oracle.com> Message-ID: <154b6144-c209-af23-cc63-91b40f51076f@oracle.com> Hmmm... seems like concurrent class unloading has revealed a situation in some code where class refs should have been held, but were not held because it's very difficult to do, and the code (mostly) got away with it for a long time... David's quote from the JNI spec makes things very, very clear: > "A field or method ID does not prevent the VM from unloading the class from > which the ID has been derived. After the class is unloaded, the method or > field ID becomes invalid and may not be passed to any function taking such > an ID. The native code, therefore, must make sure to: >?? - keep a live reference to the underlying class, or >?? - recompute the method or field ID > > if it intends to use a method or field ID for an extended period of time." and the JVM/TI spec does depend on the JNI spec for certain pieces like the semantics of jmethodID. I agree with David that fixing this is going to be difficult and a major effort which is why we've all only chipped at this boulder before... Dan On 6/1/23 4:28 AM, David Holmes wrote: > On 1/06/2023 5:16 pm, Jaroslav Bachor?k wrote: >> Hi David, >> >> On Thu, Jun 1, 2023 at 3:56?AM David Holmes > > wrote: >> >> ??? Hi Jaroslav, >> >> ??? On 31/05/2023 9:12 pm, Jaroslav Bachor?k wrote: >> ???? > Dear Team, >> ???? > >> ???? > I've been investigating the unusual JVM crashes occurring in >> ??? JVMTI calls >> ???? > on a J9 JVM. During my investigation, I scrutinized the >> `jmethodID` >> ???? > definition closely, available here: [jmethodID >> ???? > >> definition](https://docs.oracle.com/en/java/javase/17/docs/specs/jvmti.html#jmethodID >> >> > >). >> >> ???? > >> ???? > To paraphrase, the definition suggests that `jmethodID` >> identifies a >> ???? > Java method, initializer, or constructor. These identifiers, once >> ???? > returned by JVM TI functions and events, can be safely stored. >> ??? However, >> ???? > when the class is unloaded, they become invalid, rendering them >> ???? > unsuitable for use. >> ???? > >> ???? > My interpretation is that the JVMTI user should verify the >> ??? validity of a >> ???? > `jmethodID` value before using it to prevent potential crashes. >> ??? Would >> ???? > you agree with this interpretation? >> >> ??? Not quite - as you note you can't verify the jmethodID validity. >> What >> ??? the user needs to do, in line with what Dan was saying, is ensure >> that >> ??? they keep track of the classes to which the methods belong and keep >> ??? them >> ??? alive if necessary. Now that may be easier said than done, but >> that is >> ??? the gist of it. This comes from the JNI spec: >> >> ??? "A field or method ID does not prevent the VM from unloading the >> class >> ??? from which the ID has been derived. After the class is unloaded, the >> ??? method or field ID becomes invalid and may not be passed to any >> ??? function >> ??? taking such an ID. The native code, therefore, must make sure to: >> >> ???? ? ? ?keep a live reference to the underlying class, or >> ???? ? ? ?recompute the method or field ID >> >> ??? if it intends to use a method or field ID for an extended period of >> ??? time." >> >> ???? > This sounds like a sensible requirement, but its practical >> ??? application >> ???? > remains unclear. As far as I know, methods can be unloaded >> ??? concurrently >> ???? > to the native code executing JVMTI functions. This introduces a >> ???? > potential race condition where the JVM unloads the methods during >> ??? the >> ???? > check->use flow, making it only a partial solution. To complicate >> ???? > matters further, no method exists to confirm whether a >> ??? `jmethodID` is valid. >> ???? > >> ???? > Theoretically, we could monitor the `CompiledMethodUnload` >> event to >> ???? > track the validity state, creating a constantly expanding set of >> ???? > unloaded `jmethodID` values or a bloom filter, if one does not >> care >> ???? > about few potential false positives. This strategy, however, >> doesn't >> ???? > address the potential race condition, and it could even >> ??? exacerbate it >> ???? > due to possible event delays. This delay might mistakenly >> validate a >> ???? > `jmethodID` value that has already been unloaded, but for >> which the >> ???? > event hasn't been delivered yet. >> ???? > >> ???? > Honestly, I don't see a way to use `jmethodID` safely unless the >> ??? code >> ???? > using it suspends the entire JVM and doesn't resume until it's >> ??? finished >> ???? > with that `jmethodID`. Any other approach might lead to JVM >> ??? crashes, as >> ???? > we've observed with J9. >> ???? > >> ???? > Lastly, it's noteworthy that Hotspot takes meticulous measures to >> ??? ensure >> ???? > that using jmethodIDs for unloaded methods doesn't crash the >> JVM and >> ???? > even provides useful information. This observation has led me to >> ???? > question whether the documentation aligns with the Hotspot >> ???? > implementation, especially given that following closely the >> ???? > documentation appears to increase the risk associated with the >> ??? use of >> ???? > `jmethodID` values. >> >> ??? There have been folk who wanted to make this area more user-friendly >> ??? but >> ??? that shouldn't be mistaken for moving towards a world where >> jmethodIDs >> ??? are always safe to use. >> >> >> Yes, I see your point. Unfortunately, this confirms my worries that >> using AsyncGetCallTrace (ASGCT) on a system strictly adhering to the >> JVMTI spec of jmethoID is not really possible without risking random >> and quite frequent crashes on systems with concurrent class unloading >> enabled. >> FTR, ASGCT will record the stack trace as a list of frames, each one >> containing?the corresponding jmethodID value. Considering that the >> most common usage of ASGCT is in a signal handler it makes it >> impossible to use JVMTI calls to resolve the holder class and create >> a strong reference to prevent it from being unloaded. >> And even if this would be possible we would need to figure out when >> to release the class reference when it is no more needed - and it is >> not really clear how we could do that reliably, leaving us with the >> option of holding the class references indefinitely or risking >> crashing JVM. > > I would have to agree you cannot use jmethodID's for that purpose, not > without also recording (and keeping-alive) the associated classes. As > to when you would clear such references I can't say as I don't know > how the ASGCT stack record would be used. > >> I want to emphasize that not being able to resolve additional details >> for a jmethodID pointing to a method of an unloaded class is not an >> issue, as long as the JVMTI call does not crash. I think that >> https://bugs.openjdk.org/browse/JDK-8268364 >> did address exactly the >> problem of concurrent class unloading causing races in the code that >> is checking for validity of jmethodID and then using it. > > Yes but internal to the JVMTI implementation. The basic null check > that was used was found to be insufficient when used with ZGC and so > we would crash more often than was considered reasonable. There is a > quoted comment in the code in > https://bugs.openjdk.org/browse/JDK-8268088: > > ? // These aren't deallocated and are going to look like a leak, but > that's > ? // needed because we can't really get rid of jmethodIDs because we > don't > ? // know when native code is going to stop using them. The spec says > that > ? // they're "invalid" but existing programs likely rely on their being > ? // NULL after class unloading. > > that kind of sums up the position of trying to accommodate "bad code" > in a reasonable way. > >> Can this be summarize in a way that the user is not guaranteed to get >> any additional information for an invalid jmethodID but it would be >> really nice for JVM not to crash when jmethodID becomes invalid as >> there is no way for the user to check for its validity in an atomic >> manner >> - and yes, even calling GetMethodDeclaringClass in order to obtain >> the class one could create a strong reference is a subject to racy >> behaviour so it really can not be used as a workaround. > > The only real solution IMO would be to encode jmethodIDs in a way that > validity can be tracked and then have all JVMTI methods be able to > check that validity and guarantee atomicity of the method such that > the id can't become invalid whilst in use. Whether that is at all > feasible/practical I do not know - but it sounds like a major effort > to me. > > Cheers, > David > >> Cheers, >> >> -JB- >> >> >> ??? Cheers, >> ??? David >> >> ???? > I welcome your thoughts and perspectives on this matter. >> ???? > >> ???? > Best regards, >> ???? > >> ???? > Jaroslav >> From cjplummer at openjdk.org Thu Jun 1 15:59:10 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 15:59:10 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v3] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Wed, 31 May 2023 23:39:19 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > move check for EnableDynamicAgentLoading to invoke_Agent_OnAttach Sorry, previous comment was for another review. ------------- Changes requested by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14244#pullrequestreview-1455813557 From cjplummer at openjdk.org Thu Jun 1 16:00:12 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 16:00:12 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v8] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 12:37:20 GMT, Alan Bateman wrote: >> This is the implementation for JEP 451. There are two parts to this: >> >> 1. A multi-line warning is printed when a JVM TI or Java agent is loaded into a running VM. For JVM TI, the message is printed to stderr from JvmtiAgent::load. For Java agents, it is printed to System.err (as that may be redirected) in the JPLIS (j.l.instrumentation) implementation. This part includes an update to the JVM TI spec and API docs to require the warning. >> >> 2. If running with -Djdk.instrument.traceUsage or -Djdk.instrument.traceUsage=true, the calls to the Instrumentation API print a trace message and stack trace. >> >> Testing: tier1-6 > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 19 additional commits since the last revision: > > - Split is_loaded, typos in comments > - Merge > - Add impl note to document the XX option > - Cleanup > - Merge > - Allow for warning to be skipped when same agent loaded a second/subsequent time > - Merge > - Tweak javadoc, update test to use more test infra > - Merge > - Merge > - ... and 9 more: https://git.openjdk.org/jdk/compare/25f0fc82...cbd9bb1d Except for the possible addition of some comments regarding what is meant by "loaded", the changes look good. ------------- Marked as reviewed by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/13899#pullrequestreview-1455815464 From alanb at openjdk.org Thu Jun 1 16:56:17 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 1 Jun 2023 16:56:17 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v9] In-Reply-To: References: Message-ID: <9REwwZtBlDauy0udqsxSLqEXpXkt5Jp3Wv-BOnYTGu0=.c10bfd0a-5b8a-45a6-9570-1b4d939a2c33@github.com> > This is the implementation for JEP 451. There are two parts to this: > > 1. A multi-line warning is printed when a JVM TI or Java agent is loaded into a running VM. For JVM TI, the message is printed to stderr from JvmtiAgent::load. For Java agents, it is printed to System.err (as that may be redirected) in the JPLIS (j.l.instrumentation) implementation. This part includes an update to the JVM TI spec and API docs to require the warning. > > 2. If running with -Djdk.instrument.traceUsage or -Djdk.instrument.traceUsage=true, the calls to the Instrumentation API print a trace message and stack trace. > > Testing: tier1-6 Alan Bateman has updated the pull request incrementally with one additional commit since the last revision: Add comment to invoke_Agent_OnAttach ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13899/files - new: https://git.openjdk.org/jdk/pull/13899/files/cbd9bb1d..71f9886a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13899&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13899&range=07-08 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/13899.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13899/head:pull/13899 PR: https://git.openjdk.org/jdk/pull/13899 From ysr at openjdk.org Thu Jun 1 17:55:20 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 1 Jun 2023 17:55:20 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 12:15:37 GMT, Martin Doerr wrote: > Issues already reported to GenShen engineers: > > gc/shenandoah/TestElasticTLAB.java#generational #? Internal Error (src\hotspot\share\gc\shenandoah\shenandoahFreeSet.cpp:695), pid=23288, tid=23784 #? assert(size % CardTable::card_size_in_words() == 0) failed: size must be multiple of card table size, was 258 > > gc/stress/gcold/TestGCOldWithShenandoah.java#generational #? Internal Error (src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp:82), pid=20828, tid=5836 #? assert(_old_generation->available() > old_evacuation_budget) failed: Cannot budget more than is available > > gc/shenandoah/oom/TestAllocOutOfMemory.java#large Execution failed: `main' threw exception: java.lang.RuntimeException: 'java.lang.OutOfMemoryError: Java heap space' missing from stdout/stderr (Issue with 64k Pages) > > gc/shenandoah/TestRetainObjects.java#no-tlab gc/shenandoah/TestSieveObjects.java#no-tlab Timeouts. > > gc/shenandoah/TestAllocObjects.java#generational gc/shenandoah/TestDynamicSoftMaxHeapSize.java#generational #? Internal Error src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp:664), pid=18434, tid=29955 #? assert(is_global() || ShenandoahHeap::heap()->is_full_gc_in_progress() || (_used + _humongous_waste <= _affiliated_region_count * ShenandoahHeapRegion::region_size_bytes())) failed: used cannot exceed regions Thanks @TheRealMDoerr ; could you specify the platforms on which you see these failures, so we have a better chance at reproducing them? Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1572528277 From kdnilsen at openjdk.org Thu Jun 1 18:07:53 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 1 Jun 2023 18:07:53 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v2] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Make the order of young/old collector checks consistent (#1) ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/aa85a907..eb656ec2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=00-01 Stats: 16 lines in 1 file changed: 7 ins; 8 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From wkemper at openjdk.org Thu Jun 1 18:42:25 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 1 Jun 2023 18:42:25 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v2] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 10:12:02 GMT, Stefan Karlsson wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Make the order of young/old collector checks consistent (#1) > > src/hotspot/share/gc/shared/gcConfiguration.cpp line 88: > >> 86: } >> 87: #endif >> 88: return NA; > > You moved the order between Shenandoah and ZGC in `young_collector()`, so you should probably do the same here. Fixed. Thank you for the review. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213538282 From cjplummer at openjdk.org Thu Jun 1 19:41:13 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 19:41:13 GMT Subject: RFR: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents [v9] In-Reply-To: <9REwwZtBlDauy0udqsxSLqEXpXkt5Jp3Wv-BOnYTGu0=.c10bfd0a-5b8a-45a6-9570-1b4d939a2c33@github.com> References: <9REwwZtBlDauy0udqsxSLqEXpXkt5Jp3Wv-BOnYTGu0=.c10bfd0a-5b8a-45a6-9570-1b4d939a2c33@github.com> Message-ID: <3ieyvgzfEF6sPJy4o-mxJx4NdL9T1i3115ktZMZ0oKA=.f1324163-b27c-4938-b5e6-8423e27ccad7@github.com> On Thu, 1 Jun 2023 16:56:17 GMT, Alan Bateman wrote: >> This is the implementation for JEP 451. There are two parts to this: >> >> 1. A multi-line warning is printed when a JVM TI or Java agent is loaded into a running VM. For JVM TI, the message is printed to stderr from JvmtiAgent::load. For Java agents, it is printed to System.err (as that may be redirected) in the JPLIS (j.l.instrumentation) implementation. This part includes an update to the JVM TI spec and API docs to require the warning. >> >> 2. If running with -Djdk.instrument.traceUsage or -Djdk.instrument.traceUsage=true, the calls to the Instrumentation API print a trace message and stack trace. >> >> Testing: tier1-6 > > Alan Bateman has updated the pull request incrementally with one additional commit since the last revision: > > Add comment to invoke_Agent_OnAttach Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/13899#pullrequestreview-1456185001 From mseledtsov at openjdk.org Thu Jun 1 22:01:08 2023 From: mseledtsov at openjdk.org (Mikhailo Seledtsov) Date: Thu, 1 Jun 2023 22:01:08 GMT Subject: RFR: 8308090: Add container tests for on-the-fly resource quota updates [v2] In-Reply-To: References: Message-ID: On Tue, 23 May 2023 09:04:11 GMT, Severin Gehwolf wrote: >> Please review these test changes which implement automatic testing of container resource updates without JVM restart. Note that this merely tests container detection code handling this case. It doesn't do anything special for the JVM itself, though it might make sense to add some sanity checks should we detect certain limits changing. In another PR, though. >> >> As to the test design, it works similar to the shared temp tests: Interact between the two containers by virtue of a shared filesystem `/tmp` and creating marker files there in order to make them cooperate. Note that the new test needs `podman` version `4.3.0` and better (`4.5` is current). >> >> Testing: >> - [x] GHA >> - [x] Linux x86_64 container tests on cg v1 and cg v2 system >> - [x] Newly added tests on Linux x86_64 cg v1 and cg v2 (`podman` and `docker`) > > Severin Gehwolf has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - Merge branch 'master' into jdk-8308090-tests-container-on-fly-updates > - Fix whitespace > - 8308090: Add container tests for on-the-fly resource quota updates Changes look good to me. Thank you for adding these tests. ------------- Marked as reviewed by mseledtsov (Committer). PR Review: https://git.openjdk.org/jdk/pull/14090#pullrequestreview-1456374316 From sspitsyn at openjdk.org Thu Jun 1 22:51:23 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 1 Jun 2023 22:51:23 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v4] In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: <9DAQ0XpB6JlntsaEu1TZNirVwkT7kewRhKdfeIgoN4s=.d8741a54-876e-4691-9471-a77d392f9ff9@github.com> > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: use output.shouldContain() ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14244/files - new: https://git.openjdk.org/jdk/pull/14244/files/be0ec0c9..062b3f85 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=02-03 Stats: 15 lines in 1 file changed: 0 ins; 14 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14244.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14244/head:pull/14244 PR: https://git.openjdk.org/jdk/pull/14244 From sspitsyn at openjdk.org Thu Jun 1 22:51:25 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 1 Jun 2023 22:51:25 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v2] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Wed, 31 May 2023 22:08:43 GMT, Chris Plummer wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> minor renaming in new test TestJcmdNoAgentLoad.java > > test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 68: > >> 66: System.out.println("Found output line with the expected error message:\n" + line); >> 67: } >> 68: } > > Why not just use `OutputAnalyzer.shouldContain()`? Good suggestion, thanks. I've not found this method. :) > test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 70: > >> 68: } >> 69: if (!seenPattern) { >> 70: throw new RuntimeException("Not found expected error message in output"); > > "Not found" -> "Did not find" > > Also, you should include the expected error message in the output. Okay. But I've replaced this code with `OutputAnalyzer.shouldContain()` as you suggested. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1213750622 PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1213751619 From cjplummer at openjdk.org Thu Jun 1 23:11:23 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 23:11:23 GMT Subject: RFR: 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit Message-ID: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> Virtual threads are always daemon threads, so tests that previously did not explicitly wait for test threads to exit sometimes fail with virtual threads due to the test exiting before the test threads have exited. A join() for each test thread is needed to fix this issue. com/sun/jdi/DeferredStepTest.java is one such tests. I looked at the other com/sun/jdi failures listed in [JDK-8285422](https://bugs.openjdk.org/browse/JDK-8285422) and didn't see any others that might be failing for this same reason. I tested locally with `JTREG_TEST_THREAD_FACTORY=Virtual`. I'll also run the appropriate mach5 tier that tests com/sun/jdi with virtual threads. ------------- Commit messages: - Use join() to make sure test threads have exited Changes: https://git.openjdk.org/jdk/pull/14275/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14275&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309329 Stats: 12 lines in 2 files changed: 9 ins; 1 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14275.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14275/head:pull/14275 PR: https://git.openjdk.org/jdk/pull/14275 From cjplummer at openjdk.org Thu Jun 1 23:17:07 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 1 Jun 2023 23:17:07 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v4] In-Reply-To: <9DAQ0XpB6JlntsaEu1TZNirVwkT7kewRhKdfeIgoN4s=.d8741a54-876e-4691-9471-a77d392f9ff9@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> <9DAQ0XpB6JlntsaEu1TZNirVwkT7kewRhKdfeIgoN4s=.d8741a54-876e-4691-9471-a77d392f9ff9@github.com> Message-ID: On Thu, 1 Jun 2023 22:51:23 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: use output.shouldContain() test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 57: > 55: OutputAnalyzer output = JcmdBase.jcmd(jcmdArgs); > 56: > 57: assertEquals(output.getExitValue(), 0); I think tests normally just use `output.shouldHaveExitValue(0);` ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1213766546 From amenkov at openjdk.org Fri Jun 2 00:01:07 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 2 Jun 2023 00:01:07 GMT Subject: RFR: 8308978: regression with a deadlock involving FollowReferences [v3] In-Reply-To: <6xcKqU3mLr9TocEUpoXXzcNWSnKijSlhcyxIfXXrFD0=.2987b8de-827e-43d0-907b-7ef2016ddab4@github.com> References: <2J1qItzUgmfjRPS0xUbHgXZQ-b12JBxe8XPRftU2GyA=.025e7855-5df4-413a-bea7-585a53832025@github.com> <6xcKqU3mLr9TocEUpoXXzcNWSnKijSlhcyxIfXXrFD0=.2987b8de-827e-43d0-907b-7ef2016ddab4@github.com> Message-ID: On Wed, 31 May 2023 21:33:13 GMT, Alex Menkov wrote: >> The change fixes regression from JDK-8299414. >> There is a deadlock between JvmtiVTMSTransitionDisabler and EscapeBarrier when virtual threads are in mount/unmount transition: >> EscapeBarrier requests deoptimization which requires thread suspension. >> JvmtiVTMSTransitionDisabler ctor waits until all in progress VTMS transitions complete, but they cannot be completed as thread is suspended. >> To avoid the deadlock mount/unmount transitions should be completed before EscapeBarrier stuff. > > Alex Menkov has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: > > - Merge branch 'follow_ref_deadlock' of github.com:alexmenkov/jdk into follow_ref_deadlock > - Merge branch 'openjdk:master' into follow_ref_deadlock > - Merge branch 'follow_ref_deadlock' of github.com:alexmenkov/jdk into follow_ref_deadlock > - fix > - unproblem-list tests > - fix > - unproblem-list tests > - fix tier 1-5 passed (there are 3 failed tests, but they are not related to FollowReferences) ------------- PR Comment: https://git.openjdk.org/jdk/pull/14233#issuecomment-1572934722 From amenkov at openjdk.org Fri Jun 2 00:09:09 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 2 Jun 2023 00:09:09 GMT Subject: RFR: 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit In-Reply-To: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> References: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> Message-ID: On Thu, 1 Jun 2023 23:03:47 GMT, Chris Plummer wrote: > Virtual threads are always daemon threads, so tests that previously did not explicitly wait for test threads to exit sometimes fail with virtual threads due to the test exiting before the test threads have exited. A join() for each test thread is needed to fix this issue. > > com/sun/jdi/DeferredStepTest.java is one such tests. I looked at the other com/sun/jdi failures listed in [JDK-8285422](https://bugs.openjdk.org/browse/JDK-8285422) and didn't see any others that might be failing for this same reason. > > I tested locally with `JTREG_TEST_THREAD_FACTORY=Virtual`. I'll also run the appropriate mach5 tier that tests com/sun/jdi with virtual threads. test/jdk/com/sun/jdi/DeferredStepTest.java line 80: > 78: Thread jj2 = TestScaffold.newThread(obj2, "jj2"); > 79: jj1.start(); > 80: jj2.start(); It looks strange that there are no errors about conflict between variable names and class names. Anyway I think it would be better to rename the variables ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14275#discussion_r1213789142 From kdnilsen at openjdk.org Fri Jun 2 02:30:21 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 2 Jun 2023 02:30:21 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v3] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Assert bounds only when allocations succeed, increase test timeouts (#2) ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/eb656ec2..5bf6e7e0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=01-02 Stats: 3 lines in 3 files changed: 0 ins; 1 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From kdnilsen at openjdk.org Fri Jun 2 02:49:25 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 2 Jun 2023 02:49:25 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: References: Message-ID: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Force PLAB sizes to align on card-table size ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/5bf6e7e0..d4d2f1cf Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=02-03 Stats: 8 lines in 1 file changed: 8 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From kdnilsen at openjdk.org Fri Jun 2 02:56:23 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 2 Jun 2023 02:56:23 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 18:39:05 GMT, William Kemper wrote: >> src/hotspot/share/gc/shared/gcConfiguration.cpp line 88: >> >>> 86: } >>> 87: #endif >>> 88: return NA; >> >> You moved the order between Shenandoah and ZGC in `young_collector()`, so you should probably do the same here. > > Fixed. Thank you for the review. Thanks. We've made your suggested change. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213862862 From kdnilsen at openjdk.org Fri Jun 2 02:56:25 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 2 Jun 2023 02:56:25 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: References: Message-ID: <3lkkOfu3WxdRPQ4Y0uiOl6znrpRNDrcMn4ecDhQAcuU=.d742a62e-6ec7-444f-a925-f70dfeaf7df9@github.com> On Thu, 1 Jun 2023 14:27:12 GMT, Thomas Stuefe wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Force PLAB sizes to align on card-table size > > test/hotspot/jtreg/gc/shenandoah/oom/TestClassLoaderLeak.java line 132: > >> 130: {{"iu"}, {"adaptive", "aggressive"}}, >> 131: {{"passive"}, {"passive"}}, >> 132: {{"generational"}, {"adaptive"}} > > Curious, here and in similar places, why only test adaptive heuristic for generational, if we test satb with all variants? Generational mode only works with adaptive heuristic. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213863860 From sspitsyn at openjdk.org Fri Jun 2 03:14:08 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 2 Jun 2023 03:14:08 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v4] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> <9DAQ0XpB6JlntsaEu1TZNirVwkT7kewRhKdfeIgoN4s=.d8741a54-876e-4691-9471-a77d392f9ff9@github.com> Message-ID: On Thu, 1 Jun 2023 23:14:23 GMT, Chris Plummer wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: use output.shouldContain() > > test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 57: > >> 55: OutputAnalyzer output = JcmdBase.jcmd(jcmdArgs); >> 56: >> 57: assertEquals(output.getExitValue(), 0); > > I think tests normally just use `output.shouldHaveExitValue(0);` Thank you for the suggestion. I've derived this code from one of the existing jcmd tests. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1213871217 From sspitsyn at openjdk.org Fri Jun 2 03:20:37 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 2 Jun 2023 03:20:37 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v5] In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: use output.shouldHaveExitValue(0) in the test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14244/files - new: https://git.openjdk.org/jdk/pull/14244/files/062b3f85..6712f24a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=03-04 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14244.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14244/head:pull/14244 PR: https://git.openjdk.org/jdk/pull/14244 From sspitsyn at openjdk.org Fri Jun 2 03:29:04 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 2 Jun 2023 03:29:04 GMT Subject: RFR: 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit In-Reply-To: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> References: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> Message-ID: On Thu, 1 Jun 2023 23:03:47 GMT, Chris Plummer wrote: > Virtual threads are always daemon threads, so tests that previously did not explicitly wait for test threads to exit sometimes fail with virtual threads due to the test exiting before the test threads have exited. A join() for each test thread is needed to fix this issue. > > com/sun/jdi/DeferredStepTest.java is one such tests. I looked at the other com/sun/jdi failures listed in [JDK-8285422](https://bugs.openjdk.org/browse/JDK-8285422) and didn't see any others that might be failing for this same reason. > > I tested locally with `JTREG_TEST_THREAD_FACTORY=Virtual`. I'll also run the appropriate mach5 tier that tests com/sun/jdi with virtual threads. This looks good modulo the comment from Alex about renaming. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14275#pullrequestreview-1456594772 From sspitsyn at openjdk.org Fri Jun 2 03:44:48 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 2 Jun 2023 03:44:48 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v6] In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: - Merge - review: use output.shouldHaveExitValue(0) in the test - review: use output.shouldContain() - move check for EnableDynamicAgentLoading to invoke_Agent_OnAttach - minor renaming in new test TestJcmdNoAgentLoad.java - 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14244/files - new: https://git.openjdk.org/jdk/pull/14244/files/6712f24a..18cad064 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=04-05 Stats: 8499 lines in 119 files changed: 7217 ins; 645 del; 637 mod Patch: https://git.openjdk.org/jdk/pull/14244.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14244/head:pull/14244 PR: https://git.openjdk.org/jdk/pull/14244 From cjplummer at openjdk.org Fri Jun 2 04:17:07 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 2 Jun 2023 04:17:07 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v6] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: <6fBgCBJY5gKKOpNCLfJDaI78LB5xma3LwnXz4t7YDXk=.eebf1635-3c17-4101-9f33-9f234197f478@github.com> On Fri, 2 Jun 2023 03:44:48 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: > > - Merge > - review: use output.shouldHaveExitValue(0) in the test > - review: use output.shouldContain() > - move check for EnableDynamicAgentLoading to invoke_Agent_OnAttach > - minor renaming in new test TestJcmdNoAgentLoad.java > - 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading Looks good. ------------- Marked as reviewed by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14244#pullrequestreview-1456621661 From cjplummer at openjdk.org Fri Jun 2 04:29:16 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 2 Jun 2023 04:29:16 GMT Subject: RFR: 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit [v2] In-Reply-To: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> References: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> Message-ID: > Virtual threads are always daemon threads, so tests that previously did not explicitly wait for test threads to exit sometimes fail with virtual threads due to the test exiting before the test threads have exited. A join() for each test thread is needed to fix this issue. > > com/sun/jdi/DeferredStepTest.java is one such tests. I looked at the other com/sun/jdi failures listed in [JDK-8285422](https://bugs.openjdk.org/browse/JDK-8285422) and didn't see any others that might be failing for this same reason. > > I tested locally with `JTREG_TEST_THREAD_FACTORY=Virtual`. I'll also run the appropriate mach5 tier that tests com/sun/jdi with virtual threads. Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: some variable renaming ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14275/files - new: https://git.openjdk.org/jdk/pull/14275/files/24dfa505..f83cd766 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14275&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14275&range=00-01 Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/14275.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14275/head:pull/14275 PR: https://git.openjdk.org/jdk/pull/14275 From thartmann at openjdk.org Fri Jun 2 05:23:23 2023 From: thartmann at openjdk.org (Tobias Hartmann) Date: Fri, 2 Jun 2023 05:23:23 GMT Subject: RFR: 8309044: Replace NULL with nullptr, final sweep of hotspot code [v2] In-Reply-To: References: <3FoMnGeBp8DqkpVb6YGXKxdPsgGz6ej-jrf2U2stVfU=.56a11e19-38dd-420a-a07d-3b025120f194@github.com> Message-ID: On Tue, 30 May 2023 19:15:38 GMT, Johan Sj?len wrote: >> A final sweep of Hotspot to remove all re-added NULLs. With only 110 changes I'd appreciate if this was considered trivial. > > Johan Sj?len has updated the pull request incrementally with two additional commits since the last revision: > > - Align > - Suggestions I think if we just rely on reviews, NULLs will slip through again and we would need to have regular cleanup PRs. Doug's idea seems simple enough to implement in Skara/jcheck. An alternative to whitelisting would be a warning in the offending PR or a requirement for "special approvement" of such changes (for example, via a Skara command). ------------- PR Comment: https://git.openjdk.org/jdk/pull/14198#issuecomment-1573169963 From alanb at openjdk.org Fri Jun 2 06:00:27 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 2 Jun 2023 06:00:27 GMT Subject: Integrated: 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents In-Reply-To: References: Message-ID: On Wed, 10 May 2023 11:12:49 GMT, Alan Bateman wrote: > This is the implementation for JEP 451. There are two parts to this: > > 1. A multi-line warning is printed when a JVM TI or Java agent is loaded into a running VM. For JVM TI, the message is printed to stderr from JvmtiAgent::load. For Java agents, it is printed to System.err (as that may be redirected) in the JPLIS (j.l.instrumentation) implementation. This part includes an update to the JVM TI spec and API docs to require the warning. > > 2. If running with -Djdk.instrument.traceUsage or -Djdk.instrument.traceUsage=true, the calls to the Instrumentation API print a trace message and stack trace. > > Testing: tier1-6 This pull request has now been integrated. Changeset: 5bd2af26 Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/5bd2af26e66a863edc670229444b3282ba639563 Stats: 1199 lines in 21 files changed: 941 ins; 99 del; 159 mod 8307478: Implementation of Prepare to Restrict The Dynamic Loading of Agents Reviewed-by: sspitsyn, cjplummer ------------- PR: https://git.openjdk.org/jdk/pull/13899 From sspitsyn at openjdk.org Fri Jun 2 07:14:23 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 2 Jun 2023 07:14:23 GMT Subject: RFR: 8309044: Replace NULL with nullptr, final sweep of hotspot code [v2] In-Reply-To: References: <3FoMnGeBp8DqkpVb6YGXKxdPsgGz6ej-jrf2U2stVfU=.56a11e19-38dd-420a-a07d-3b025120f194@github.com> Message-ID: On Tue, 30 May 2023 19:15:38 GMT, Johan Sj?len wrote: >> A final sweep of Hotspot to remove all re-added NULLs. With only 110 changes I'd appreciate if this was considered trivial. > > Johan Sj?len has updated the pull request incrementally with two additional commits since the last revision: > > - Align > - Suggestions Looks good. Thanks, Serguei ------------- PR Review: https://git.openjdk.org/jdk/pull/14198#pullrequestreview-1456781527 From stefank at openjdk.org Fri Jun 2 07:50:21 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Fri, 2 Jun 2023 07:50:21 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: On Fri, 2 Jun 2023 02:49:25 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Force PLAB sizes to align on card-table size I've reviewed the shared code and think that looks good. ------------- Marked as reviewed by stefank (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14185#pullrequestreview-1456831456 From sgehwolf at openjdk.org Fri Jun 2 09:01:16 2023 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Fri, 2 Jun 2023 09:01:16 GMT Subject: RFR: 8308090: Add container tests for on-the-fly resource quota updates [v2] In-Reply-To: References: Message-ID: On Tue, 23 May 2023 09:04:11 GMT, Severin Gehwolf wrote: >> Please review these test changes which implement automatic testing of container resource updates without JVM restart. Note that this merely tests container detection code handling this case. It doesn't do anything special for the JVM itself, though it might make sense to add some sanity checks should we detect certain limits changing. In another PR, though. >> >> As to the test design, it works similar to the shared temp tests: Interact between the two containers by virtue of a shared filesystem `/tmp` and creating marker files there in order to make them cooperate. Note that the new test needs `podman` version `4.3.0` and better (`4.5` is current). >> >> Testing: >> - [x] GHA >> - [x] Linux x86_64 container tests on cg v1 and cg v2 system >> - [x] Newly added tests on Linux x86_64 cg v1 and cg v2 (`podman` and `docker`) > > Severin Gehwolf has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - Merge branch 'master' into jdk-8308090-tests-container-on-fly-updates > - Fix whitespace > - 8308090: Add container tests for on-the-fly resource quota updates Thanks for the review! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14090#issuecomment-1573393329 From sgehwolf at openjdk.org Fri Jun 2 09:01:19 2023 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Fri, 2 Jun 2023 09:01:19 GMT Subject: Integrated: 8308090: Add container tests for on-the-fly resource quota updates In-Reply-To: References: Message-ID: On Mon, 22 May 2023 16:40:40 GMT, Severin Gehwolf wrote: > Please review these test changes which implement automatic testing of container resource updates without JVM restart. Note that this merely tests container detection code handling this case. It doesn't do anything special for the JVM itself, though it might make sense to add some sanity checks should we detect certain limits changing. In another PR, though. > > As to the test design, it works similar to the shared temp tests: Interact between the two containers by virtue of a shared filesystem `/tmp` and creating marker files there in order to make them cooperate. Note that the new test needs `podman` version `4.3.0` and better (`4.5` is current). > > Testing: > - [x] GHA > - [x] Linux x86_64 container tests on cg v1 and cg v2 system > - [x] Newly added tests on Linux x86_64 cg v1 and cg v2 (`podman` and `docker`) This pull request has now been integrated. Changeset: dc8bc6c9 Author: Severin Gehwolf URL: https://git.openjdk.org/jdk/commit/dc8bc6c98ca1f9b441cf71c641675fe29dda9162 Stats: 434 lines in 5 files changed: 431 ins; 2 del; 1 mod 8308090: Add container tests for on-the-fly resource quota updates Reviewed-by: dholmes, mseledtsov ------------- PR: https://git.openjdk.org/jdk/pull/14090 From fjiang at openjdk.org Fri Jun 2 09:35:22 2023 From: fjiang at openjdk.org (Feilong Jiang) Date: Fri, 2 Jun 2023 09:35:22 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: <9zx2L2htHAAmC0dInZaMaMESDiueZJAn7Tv9XzznJJc=.c3aadf60-97f6-46fe-8bee-a55baeb9bc67@github.com> On Fri, 2 Jun 2023 02:49:25 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Force PLAB sizes to align on card-table size Hi, I have built this pr based on aa85a90, Tier1 tests failed on `gc/TestAllocHumongousFragment.java#generational` on Linux/RISC-V with the following output: # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (shenandoahVerifier.cpp:1244), pid=2951116, tid=2951124 # Error: Verify init-mark remembered set violation; clean card should be dirty # # JRE version: OpenJDK Runtime Environment (21.0) (build 21-internal-adhoc.ubuntu.jdk) # Java VM: OpenJDK 64-Bit Server VM (21-internal-adhoc.ubuntu.jdk, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-riscv64) Looks like Generational Shenandoah does not fully support RISC-V port, should we disable this test on RISC-V port for now? ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1573436275 From duke at openjdk.org Fri Jun 2 10:26:30 2023 From: duke at openjdk.org (JoKern65) Date: Fri, 2 Jun 2023 10:26:30 GMT Subject: RFR: JDK-8309225: Fix xlc17 clang 15 warnings in security and servicability Message-ID: This pr is a split off from JDK-8308288: Fix xlc17 clang warnings in shared code https://github.com/openjdk/jdk/pull/14146 It handles the part in security and servicability. Compiling on AIX with xlc17 which contains the new clang 15 frontend shows the following warnings: src/java.security.jgss/share/native/libj2gss/NativeUtil.h:30: src/java.security.jgss/share/native/libj2gss/gssapi.h:48:5: error: 'TARGET_OS_MAC' is not defined, evaluates to 0 [-Werror,-Wundef] #if TARGET_OS_MAC && (defined(ppc) || defined(ppc64) || defined(i386) || defined(x86_64)) ^ src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:718:33: error: suggest braces around initialization of subobject [-Werror,-Wmissing-braces] struct in6_addr mappedAny = IN6ADDR_ANY_INIT; ^~~~~~~~~~~~~~~~ /usr/include/netinet/in.h:454:32: note: expanded from macro 'IN6ADDR_ANY_INIT' #define IN6ADDR_ANY_INIT {0, 0, 0, 0} ------------- Commit messages: - JDK-8309225 Changes: https://git.openjdk.org/jdk/pull/14282/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14282&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309225 Stats: 4 lines in 2 files changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14282.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14282/head:pull/14282 PR: https://git.openjdk.org/jdk/pull/14282 From lmesnik at openjdk.org Fri Jun 2 16:33:10 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Fri, 2 Jun 2023 16:33:10 GMT Subject: RFR: 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit [v2] In-Reply-To: References: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> Message-ID: On Fri, 2 Jun 2023 04:29:16 GMT, Chris Plummer wrote: >> Virtual threads are always daemon threads, so tests that previously did not explicitly wait for test threads to exit sometimes fail with virtual threads due to the test exiting before the test threads have exited. A join() for each test thread is needed to fix this issue. >> >> com/sun/jdi/DeferredStepTest.java is one such tests. I looked at the other com/sun/jdi failures listed in [JDK-8285422](https://bugs.openjdk.org/browse/JDK-8285422) and didn't see any others that might be failing for this same reason. >> >> I tested locally with `JTREG_TEST_THREAD_FACTORY=Virtual`. I'll also run the appropriate mach5 tier that tests com/sun/jdi with virtual threads. > > Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: > > some variable renaming Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14275#pullrequestreview-1457971775 From ysr at openjdk.org Fri Jun 2 17:59:26 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 2 Jun 2023 17:59:26 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: On Fri, 2 Jun 2023 02:49:25 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Force PLAB sizes to align on card-table size src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1285: > 1283: if (unalignment != 0) { > 1284: word_size = word_size - unalignment + CardTable::card_size_in_words(); > 1285: } Probably not a big deal since this is only used when refilling a PLAB, which is an infrequent operation, but `mod` is an expensive operation, in general, and best to avoid in our code except in assertion checks (or even there given recent experiences with debug tests timing out). Since card size is a power of 2, may be we could use addition and masking instead. Something like defining the following inline in the CardTable class and using it everywhere where card alignment granularity is sought. There may even be a macro or method defined for this already perhaps: (FOO + CardSize - 1) & ~((1 << LogCardSize) - 1) One could even store the mask to avoid the arithmetic to produce the mask although it's pretty cheap. This may turn out to be less expensive than mod, test, and branch, but as I said probably not a big deal here. We should make sure we don't overuse mods in our allocation paths much. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1214658808 From mdoerr at openjdk.org Fri Jun 2 18:13:24 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Fri, 2 Jun 2023 18:13:24 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: On Fri, 2 Jun 2023 02:49:25 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Force PLAB sizes to align on card-table size > # Internal Error (shenandoahVerifier.cpp:1244), pid=2951116, tid=2951124 > # Error: Verify init-mark remembered set violation; clean card should be dirty I've seen the same issue on PPC64: https://bugs.openjdk.org/browse/JDK-8309371 ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1574125489 From ysr at openjdk.org Fri Jun 2 18:27:21 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 2 Jun 2023 18:27:21 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: <7uARcGDHOuSUugc2zRg7JQgC2dSPBDOjeWGPjBPO2qs=.a09479b8-ba9b-4596-bc5a-7ace0968fe31@github.com> On Fri, 2 Jun 2023 02:49:25 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Force PLAB sizes to align on card-table size src/hotspot/cpu/riscv/gc/shenandoah/c1/shenandoahBarrierSetC1_riscv.cpp line 4: > 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. > 3: * Copyright (c) 2020, 2021, Huawei Technologies Co., Ltd. All rights reserved. > 4: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. This should be backed out, since it seems that there is no (other) change to this fie. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1214682175 From mdoerr at openjdk.org Fri Jun 2 18:41:25 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Fri, 2 Jun 2023 18:41:25 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <7uARcGDHOuSUugc2zRg7JQgC2dSPBDOjeWGPjBPO2qs=.a09479b8-ba9b-4596-bc5a-7ace0968fe31@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> <7uARcGDHOuSUugc2zRg7JQgC2dSPBDOjeWGPjBPO2qs=.a09479b8-ba9b-4596-bc5a-7ace0968fe31@github.com> Message-ID: <95yaqYTYoGnlkrDMbvZ-NTyVbGmHrL4DUYYIlV3wkwQ=.b6d787d2-3858-4486-b3dc-428a87969109@github.com> On Fri, 2 Jun 2023 18:24:16 GMT, Y. Srinivas Ramakrishna wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Force PLAB sizes to align on card-table size > > src/hotspot/cpu/riscv/gc/shenandoah/c1/shenandoahBarrierSetC1_riscv.cpp line 4: > >> 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. >> 3: * Copyright (c) 2020, 2021, Huawei Technologies Co., Ltd. All rights reserved. >> 4: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > This should be backed out, since it seems that there is no (other) change to this fie. Yes. And also from files which were changed by non-Amazon employees only, please. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1214693296 From amenkov at openjdk.org Fri Jun 2 19:13:13 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 2 Jun 2023 19:13:13 GMT Subject: RFR: 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit [v2] In-Reply-To: References: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> Message-ID: On Fri, 2 Jun 2023 04:29:16 GMT, Chris Plummer wrote: >> Virtual threads are always daemon threads, so tests that previously did not explicitly wait for test threads to exit sometimes fail with virtual threads due to the test exiting before the test threads have exited. A join() for each test thread is needed to fix this issue. >> >> com/sun/jdi/DeferredStepTest.java is one such tests. I looked at the other com/sun/jdi failures listed in [JDK-8285422](https://bugs.openjdk.org/browse/JDK-8285422) and didn't see any others that might be failing for this same reason. >> >> I tested locally with `JTREG_TEST_THREAD_FACTORY=Virtual`. I'll also run the appropriate mach5 tier that tests com/sun/jdi with virtual threads. > > Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: > > some variable renaming Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14275#pullrequestreview-1458344825 From cjplummer at openjdk.org Fri Jun 2 19:13:16 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 2 Jun 2023 19:13:16 GMT Subject: Integrated: 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit In-Reply-To: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> References: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> Message-ID: On Thu, 1 Jun 2023 23:03:47 GMT, Chris Plummer wrote: > Virtual threads are always daemon threads, so tests that previously did not explicitly wait for test threads to exit sometimes fail with virtual threads due to the test exiting before the test threads have exited. A join() for each test thread is needed to fix this issue. > > com/sun/jdi/DeferredStepTest.java is one such tests. I looked at the other com/sun/jdi failures listed in [JDK-8285422](https://bugs.openjdk.org/browse/JDK-8285422) and didn't see any others that might be failing for this same reason. > > I tested locally with `JTREG_TEST_THREAD_FACTORY=Virtual`. I'll also run the appropriate mach5 tier that tests com/sun/jdi with virtual threads. This pull request has now been integrated. Changeset: 1bb037bd Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/1bb037bdc6cd76a15f85c05d8c1972ac53b9b3cd Stats: 12 lines in 2 files changed: 9 ins; 1 del; 2 mod 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit Reviewed-by: sspitsyn, lmesnik, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/14275 From cjplummer at openjdk.org Fri Jun 2 19:13:14 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 2 Jun 2023 19:13:14 GMT Subject: RFR: 8309329: com/sun/jdi/DeferredStepTest.java fails with virtual threads due to not waiting for threads to exit [v2] In-Reply-To: References: <9WLOY1a8lk3w-ZTrN-SUCOQXdwHideyuexQNjKXPJ9M=.2f555619-bcb4-4f30-8d17-97b0147c58e1@github.com> Message-ID: On Fri, 2 Jun 2023 04:29:16 GMT, Chris Plummer wrote: >> Virtual threads are always daemon threads, so tests that previously did not explicitly wait for test threads to exit sometimes fail with virtual threads due to the test exiting before the test threads have exited. A join() for each test thread is needed to fix this issue. >> >> com/sun/jdi/DeferredStepTest.java is one such tests. I looked at the other com/sun/jdi failures listed in [JDK-8285422](https://bugs.openjdk.org/browse/JDK-8285422) and didn't see any others that might be failing for this same reason. >> >> I tested locally with `JTREG_TEST_THREAD_FACTORY=Virtual`. I'll also run the appropriate mach5 tier that tests com/sun/jdi with virtual threads. > > Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: > > some variable renaming Thanks for the reviews Alex, Serguei, and Leonid. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14275#issuecomment-1574190569 From amenkov at openjdk.org Fri Jun 2 19:30:11 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 2 Jun 2023 19:30:11 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v6] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Fri, 2 Jun 2023 03:44:48 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: > > - Merge > - review: use output.shouldHaveExitValue(0) in the test > - review: use output.shouldContain() > - move check for EnableDynamicAgentLoading to invoke_Agent_OnAttach > - minor renaming in new test TestJcmdNoAgentLoad.java > - 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 42: > 40: * @library /test/lib > 41: * > 42: * @run main/othervm -XX:-EnableDynamicAgentLoading TestJcmdNoAgentLoad The test verifies only one scenario. I think we need to test 3 scenarios: - -XX:-EnableDynamicAgentLoading - +XX:-EnableDynamicAgentLoading - default (no EnableDynamicAgentLoading arg) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1214741247 From amenkov at openjdk.org Fri Jun 2 19:30:11 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 2 Jun 2023 19:30:11 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v6] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Fri, 2 Jun 2023 19:23:23 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: >> >> - Merge >> - review: use output.shouldHaveExitValue(0) in the test >> - review: use output.shouldContain() >> - move check for EnableDynamicAgentLoading to invoke_Agent_OnAttach >> - minor renaming in new test TestJcmdNoAgentLoad.java >> - 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 42: > >> 40: * @library /test/lib >> 41: * >> 42: * @run main/othervm -XX:-EnableDynamicAgentLoading TestJcmdNoAgentLoad > > The test verifies only one scenario. I think we need to test 3 scenarios: > - -XX:-EnableDynamicAgentLoading > - +XX:-EnableDynamicAgentLoading > - default (no EnableDynamicAgentLoading arg) I didn't find tests for EnableDynamicAgentLoading for attach API too (to ensure this fix doesn't break it) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1214745149 From ysr at openjdk.org Fri Jun 2 20:15:29 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 2 Jun 2023 20:15:29 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: <9O5Q9i_7sOIZLqYlRGDMhGOkhCS2aZ2_gFObV9LbjdY=.310b660d-a48e-4ba8-b61a-126ef483f5bc@github.com> On Fri, 2 Jun 2023 02:49:25 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Force PLAB sizes to align on card-table size Thanks for issue reports, for which we are filing JBS tickets as we receive reports, and will be resolving them as we go. The issues reported to us have been filed as issues linked to the JBS ticket for this PR. Reviewers and testers should please feel free to file tickets as they find other issues, given testing notes in the PR, and platforms on which we have inadequate or no testing or coverage. We appreciate the help! We will be incrementally resolving as many of the reported issues as we are able to, including in follow up PRs as bug fixes as appropriate. There were some issues reported with copyright headers, and those will be fixed soon in this PR. Please follow the tickets for further updates. Many thanks to all the reviewers so far. I have been a reviewer at the project level for several of the commits that comprise this PR, so I am happy to review and approve this pull request as a reviewer, as well as a project participant and partial implementor for some fixes. Onwards to the brave new generation of Shenandoah! ;-) ? ? ------------- Marked as reviewed by ysr (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14185#pullrequestreview-1458425985 From cjplummer at openjdk.org Fri Jun 2 21:48:13 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 2 Jun 2023 21:48:13 GMT Subject: RFR: 8309397: com/sun/jdi/JdbXXX tests fail due to not being run with -Djdk.trackAllThreads Message-ID: The com/sun/jdi/JdbXXX tests rely on the jdb "threads" command output to find the main thread. If it is a virtual thread, it will not be included in the "threads" output unless the debuggee is run with -Djdk.trackAllThreads, so we need to make sure to include this option when launching the debuggee. The following tests are impacted. com/sun/jdi/JdbMethodExitTest.java com/sun/jdi/JdbStepTest.java com/sun/jdi/JdbStopThreadTest.java com/sun/jdi/JdbStopThreadidTest.java Note that all these tests also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which needs to be fixed first. Also JdbMethodExitTest.java will fail due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which should be fixed after this CR. I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the 3 tests being removed from the problem list by this PR also pass. ------------- Commit messages: - Launch debuggee with -Djdk.trackAllThreads Changes: https://git.openjdk.org/jdk/pull/14293/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14293&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309397 Stats: 4 lines in 2 files changed: 1 ins; 3 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14293.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14293/head:pull/14293 PR: https://git.openjdk.org/jdk/pull/14293 From cjplummer at openjdk.org Fri Jun 2 21:49:58 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 2 Jun 2023 21:49:58 GMT Subject: RFR: 8309397: com/sun/jdi/JdbXXX tests fail due to not being run with -Djdk.trackAllThreads Message-ID: The com/sun/jdi/JdbXXX tests rely on the jdb "threads" command output to find the main thread. If it is a virtual thread, it will not be included in the "threads" output unless the debuggee is run with -Djdk.trackAllThreads, so we need to make sure to include this option when launching the debuggee. The following tests are impacted. com/sun/jdi/JdbMethodExitTest.java com/sun/jdi/JdbStepTest.java com/sun/jdi/JdbStopThreadTest.java com/sun/jdi/JdbStopThreadidTest.java Note that all these tests also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which needs to be fixed first. Also JdbMethodExitTest.java will fail due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which should be fixed after this CR. I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the 3 tests being removed from the problem list by this PR also pass. ------------- Commit messages: - Launch debuggee with -Djdk.trackAllThreads Changes: https://git.openjdk.org/jdk/pull/14293/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14293&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309397 Stats: 4 lines in 2 files changed: 1 ins; 3 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14293.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14293/head:pull/14293 PR: https://git.openjdk.org/jdk/pull/14293 From cjplummer at openjdk.org Fri Jun 2 21:55:27 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 2 Jun 2023 21:55:27 GMT Subject: RFR: 8309396: com/sun/jdi/JdbMethodExitTest.java fails with virtual threads due to a bug in determining the main thread id Message-ID: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> JdbMethodExitTest.java tries to determine the jdb threadID for the "main" thread, and then later use it in jdb commands that require a threadID. It does this by first having the debuggee execute the following: System.out.println("threadid="+Thread.currentThread().getId()); And then later on the test parses the threadID from this output. The problem is that the id returned by getId() has no relation to threadIDs used by jdb, which are actually JDWP ObjectIDs. In the past this has worked due to some dumb luck. getID() always returns 1 for the main thread, which is always the thread we are executing in. Coincidentally the JDWP ObjectID for the main Thread object is also always 1 because this is the first ObjectID that the debug agent ever needs to create. However, when the debuggee main thread is a virtual thread, neither getId() nor JDWP assign 1 to the threadID, and in fact both will end up with very different values for the threadID. The end result is errors from jdb for using an invalid threadID. The correct threadID can be obtained by executing the jdb "threads" command and parsing it from a line that looks like the following: (java.lang.VirtualThread)694 main running (at breakpoint) Note this test will also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334) and [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which should be fixed first. I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 and JDK-8309397 are fixed, before integrating this PR I'll first merge and verify that the test being removed from the problem list by this PR also passes. ------------- Commit messages: - remove test from problem list - properly determine the main threadId Changes: https://git.openjdk.org/jdk/pull/14294/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14294&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309396 Stats: 11 lines in 2 files changed: 5 ins; 2 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14294.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14294/head:pull/14294 PR: https://git.openjdk.org/jdk/pull/14294 From amenkov at openjdk.org Fri Jun 2 22:05:16 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 2 Jun 2023 22:05:16 GMT Subject: Integrated: 8308978: regression with a deadlock involving FollowReferences In-Reply-To: <2J1qItzUgmfjRPS0xUbHgXZQ-b12JBxe8XPRftU2GyA=.025e7855-5df4-413a-bea7-585a53832025@github.com> References: <2J1qItzUgmfjRPS0xUbHgXZQ-b12JBxe8XPRftU2GyA=.025e7855-5df4-413a-bea7-585a53832025@github.com> Message-ID: On Tue, 30 May 2023 22:58:58 GMT, Alex Menkov wrote: > The change fixes regression from JDK-8299414. > There is a deadlock between JvmtiVTMSTransitionDisabler and EscapeBarrier when virtual threads are in mount/unmount transition: > EscapeBarrier requests deoptimization which requires thread suspension. > JvmtiVTMSTransitionDisabler ctor waits until all in progress VTMS transitions complete, but they cannot be completed as thread is suspended. > To avoid the deadlock mount/unmount transitions should be completed before EscapeBarrier stuff. This pull request has now been integrated. Changeset: 62c935d4 Author: Alex Menkov URL: https://git.openjdk.org/jdk/commit/62c935d4fa09ed557d301bc28d9bf1480b344989 Stats: 22 lines in 2 files changed: 6 ins; 16 del; 0 mod 8308978: regression with a deadlock involving FollowReferences Reviewed-by: sspitsyn, lmesnik ------------- PR: https://git.openjdk.org/jdk/pull/14233 From cjplummer at openjdk.org Fri Jun 2 22:32:14 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 2 Jun 2023 22:32:14 GMT Subject: RFR: 8309391: Remove non-failing tests from test/jdk/ProblemList-Virtual.txt Message-ID: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> The following two tests no longer seem to be failing with the virtual thread test factory: com/sun/jdi/RedefineG.java 8285422 generic-all com/sun/jdi/RedefineTTYLineNumber.java 8285422 generic-all They can be removed from the problem list. I also updated a couple of entries to refer to existing CRs that give more specific reasons for why the tests are failing. I tested locally using JTREG_TEST_THREAD_FACTORY=Virtual. I also have a mach5 tier5 job in progress that will do all the virtual thread testing. I'd like to push this as a trivial change. ------------- Commit messages: - remove some non-failings tests from problem list Changes: https://git.openjdk.org/jdk/pull/14290/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14290&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309391 Stats: 4 lines in 1 file changed: 0 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14290.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14290/head:pull/14290 PR: https://git.openjdk.org/jdk/pull/14290 From amenkov at openjdk.org Fri Jun 2 23:00:08 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 2 Jun 2023 23:00:08 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 21:30:46 GMT, Chris Plummer wrote: > Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: > > com/sun/jdi/JdbMethodExitTest.java > com/sun/jdi/JdbStepTest.java > com/sun/jdi/JdbStopThreadTest.java > com/sun/jdi/JdbStopThreadidTest.java > > These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. > > Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. > > Testing was done by running all of tier1 and tier5. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14292#pullrequestreview-1458569741 From amenkov at openjdk.org Fri Jun 2 23:08:05 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 2 Jun 2023 23:08:05 GMT Subject: RFR: 8309396: com/sun/jdi/JdbMethodExitTest.java fails with virtual threads due to a bug in determining the main thread id In-Reply-To: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> References: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> Message-ID: On Fri, 2 Jun 2023 21:47:47 GMT, Chris Plummer wrote: > JdbMethodExitTest.java tries to determine the jdb threadID for the "main" thread, and then later use it in jdb commands that require a threadID. It does this by first having the debuggee execute the following: > > System.out.println("threadid="+Thread.currentThread().getId()); > > And then later on the test parses the threadID from this output. The problem is that the id returned by getId() has no relation to threadIDs used by jdb, which are actually JDWP ObjectIDs. In the past this has worked due to some dumb luck. getID() always returns 1 for the main thread, which is always the thread we are executing in. Coincidentally the JDWP ObjectID for the main Thread object is also always 1 because this is the first ObjectID that the debug agent ever needs to create. However, when the debuggee main thread is a virtual thread, neither getId() nor JDWP assign 1 to the threadID, and in fact both will end up with very different values for the threadID. The end result is errors from jdb for using an invalid threadID. > > The correct threadID can be obtained by executing the jdb "threads" command and parsing it from a line that looks like the following: > > (java.lang.VirtualThread)694 main running (at breakpoint) > > Note this test will also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334) and [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which should be fixed first. > > I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 and JDK-8309397 are fixed, before integrating this PR I'll first merge and verify that the test being removed from the problem list by this PR also passes. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14294#pullrequestreview-1458578358 From dcubed at openjdk.org Fri Jun 2 23:36:04 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Fri, 2 Jun 2023 23:36:04 GMT Subject: RFR: 8309391: Remove non-failing tests from test/jdk/ProblemList-Virtual.txt In-Reply-To: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> References: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> Message-ID: On Fri, 2 Jun 2023 20:08:15 GMT, Chris Plummer wrote: > The following two tests no longer seem to be failing with the virtual thread test factory: > > com/sun/jdi/RedefineG.java 8285422 generic-all > com/sun/jdi/RedefineTTYLineNumber.java 8285422 generic-all > > They can be removed from the problem list. I also updated a couple of entries to refer to existing CRs that give more specific reasons for why the tests are failing. > > I tested locally using JTREG_TEST_THREAD_FACTORY=Virtual. I also have a mach5 tier5 job in progress that will do all the virtual thread testing. > > I'd like to push this as a trivial change. Thumbs up. This is a trivial fix. ------------- Marked as reviewed by dcubed (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14290#pullrequestreview-1458595858 From lmesnik at openjdk.org Sat Jun 3 01:11:03 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Sat, 3 Jun 2023 01:11:03 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 21:30:46 GMT, Chris Plummer wrote: > Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: > > com/sun/jdi/JdbMethodExitTest.java > com/sun/jdi/JdbStepTest.java > com/sun/jdi/JdbStopThreadTest.java > com/sun/jdi/JdbStopThreadidTest.java > > These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. > > Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. > > Testing was done by running all of tier1 and tier5. Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14292#pullrequestreview-1458712928 From lmesnik at openjdk.org Sat Jun 3 01:12:04 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Sat, 3 Jun 2023 01:12:04 GMT Subject: RFR: 8309391: Remove non-failing tests from test/jdk/ProblemList-Virtual.txt In-Reply-To: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> References: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> Message-ID: On Fri, 2 Jun 2023 20:08:15 GMT, Chris Plummer wrote: > The following two tests no longer seem to be failing with the virtual thread test factory: > > com/sun/jdi/RedefineG.java 8285422 generic-all > com/sun/jdi/RedefineTTYLineNumber.java 8285422 generic-all > > They can be removed from the problem list. I also updated a couple of entries to refer to existing CRs that give more specific reasons for why the tests are failing. > > I tested locally using JTREG_TEST_THREAD_FACTORY=Virtual. I also have a mach5 tier5 job in progress that will do all the virtual thread testing. > > I'd like to push this as a trivial change. Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14290#pullrequestreview-1458715315 From alanb at openjdk.org Sat Jun 3 07:13:03 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sat, 3 Jun 2023 07:13:03 GMT Subject: RFR: 8309397: com/sun/jdi/JdbXXX tests fail due to not being run with -Djdk.trackAllThreads In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 21:41:32 GMT, Chris Plummer wrote: > The com/sun/jdi/JdbXXX tests rely on the jdb "threads" command output to find the main thread. If it is a virtual thread, it will not be included in the "threads" output unless the debuggee is run with -Djdk.trackAllThreads, so we need to make sure to include this option when launching the debuggee. The following tests are impacted. > > com/sun/jdi/JdbMethodExitTest.java > com/sun/jdi/JdbStepTest.java > com/sun/jdi/JdbStopThreadTest.java > com/sun/jdi/JdbStopThreadidTest.java > > Note that all these tests also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which needs to be fixed first. Also JdbMethodExitTest.java will fail due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which should be fixed after this CR. > > I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the 3 tests being removed from the problem list by this PR also pass. Would it be possible to expand a bit on why jdk.trackAllThreads is needed? I would have expected jdb "threads" command to be sensitive to whether the debuggee is launched with the JDWP includevirtualthreads option but jdk.trackAllThreads isn't used by the JDWP agent, it is instead used to determine if virtual threads created directly with the Thread API show up in thread dumps or not. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14293#issuecomment-1574736478 From sspitsyn at openjdk.org Sat Jun 3 11:25:23 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sat, 3 Jun 2023 11:25:23 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING Message-ID: When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. The fix adds: - a special case for passive carrier threads - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. Testing: - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` - submitted mach5 tiers 1-5 - TBD: to submit mach5 tier 6 ------------- Commit messages: - minor tweaks in libThreadStateTest.cpp - 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING Changes: https://git.openjdk.org/jdk/pull/14298/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8307153 Stats: 71 lines in 4 files changed: 61 ins; 0 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/14298.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14298/head:pull/14298 PR: https://git.openjdk.org/jdk/pull/14298 From alanb at openjdk.org Sat Jun 3 14:11:05 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sat, 3 Jun 2023 14:11:05 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 21:30:46 GMT, Chris Plummer wrote: > Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: > > com/sun/jdi/JdbMethodExitTest.java > com/sun/jdi/JdbStepTest.java > com/sun/jdi/JdbStopThreadTest.java > com/sun/jdi/JdbStopThreadidTest.java > > These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. > > Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. > > Testing was done by running all of tier1 and tier5. test/lib/jdk/test/lib/process/ProcessTools.java line 899: > 897: // when main is executed in virtual thread > 898: MainThreadGroup tg = new MainThreadGroup(); > 899: Thread vthread = createVirtualThread(() -> { You can replace this with `Thread.ofVirtual().unstarted(() -> {` and remove the helper method. I think that helper method dates from the jtreg wrapper solution. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14292#discussion_r1215549528 From ysr at openjdk.org Sat Jun 3 15:21:22 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Sat, 3 Jun 2023 15:21:22 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <95yaqYTYoGnlkrDMbvZ-NTyVbGmHrL4DUYYIlV3wkwQ=.b6d787d2-3858-4486-b3dc-428a87969109@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> <7uARcGDHOuSUugc2zRg7JQgC2dSPBDOjeWGPjBPO2qs=.a09479b8-ba9b-4596-bc5a-7ace0968fe31@github.com> <95yaqYTYoGnlkrDMbvZ-NTyVbGmHrL4DUYYIlV3wkwQ=.b6d787d2-3858-4486-b3dc-428a87969109@github.com> Message-ID: <-jrieUm3r32vA5At0baw1nTndtNGoxG6EBrcEDjwyZw=.0a95dc08-1259-418d-a9bb-b2ba86b18c51@github.com> On Fri, 2 Jun 2023 18:38:41 GMT, Martin Doerr wrote: >> src/hotspot/cpu/riscv/gc/shenandoah/c1/shenandoahBarrierSetC1_riscv.cpp line 4: >> >>> 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. >>> 3: * Copyright (c) 2020, 2021, Huawei Technologies Co., Ltd. All rights reserved. >>> 4: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. >> >> This should be backed out, since it seems that there is no (other) change to this fie. > > Yes. And also from files which were changed by non-Amazon employees only, please. Thanks, Martin. Yes, we have noted that there were a few other files that were inadvertently caught in a copyright header dragnet. These will be reviewed and fixed in https://bugs.openjdk.org/browse/JDK-8309392 . ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1215600873 From cjplummer at openjdk.org Sat Jun 3 18:24:04 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 3 Jun 2023 18:24:04 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper In-Reply-To: References: Message-ID: <1BSXXnlH2oe9TenR2t-oiwJAqWlrEjyhSZk_fht2SDE=.d27a4cd5-1b74-4974-8f51-4f73816fcb98@github.com> On Sat, 3 Jun 2023 14:07:52 GMT, Alan Bateman wrote: >> Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: >> >> com/sun/jdi/JdbMethodExitTest.java >> com/sun/jdi/JdbStepTest.java >> com/sun/jdi/JdbStopThreadTest.java >> com/sun/jdi/JdbStopThreadidTest.java >> >> These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. >> >> Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. >> >> Testing was done by running all of tier1 and tier5. > > test/lib/jdk/test/lib/process/ProcessTools.java line 899: > >> 897: // when main is executed in virtual thread >> 898: MainThreadGroup tg = new MainThreadGroup(); >> 899: Thread vthread = createVirtualThread(() -> { > > You can replace this with `Thread.ofVirtual().unstarted(() -> {` and remove the helper method. I think that helper method dates from the jtreg wrapper solution. Better still, maybe you could use `Thread.ofVirtual().name("main").start(() -> {`. Yes, I already brought this up with Leonid. I was going to file a separate bug for it, but I can fix it with this CR if you'd like. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14292#discussion_r1215705793 From cjplummer at openjdk.org Sat Jun 3 19:02:04 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 3 Jun 2023 19:02:04 GMT Subject: RFR: 8309397: com/sun/jdi/JdbXXX tests fail due to not being run with -Djdk.trackAllThreads In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 07:09:54 GMT, Alan Bateman wrote: > Would it be possible to expand a bit on why jdk.trackAllThreads is needed? I would have expected jdb "threads" command to be sensitive to whether the debuggee is launched with the JDWP includevirtualthreads option but jdk.trackAllThreads isn't used by the JDWP agent, it is instead used to determine if virtual threads created directly with the Thread API show up in thread dumps or not. I think you are right that this change should not be needed. I was working on addressing a number of different issues with launching debuggees properly and the tests being able to discover virtual threads, and I think maybe I had done this "fix" at some point, and then did [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), (and [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396)), and thought that this fix was still needed. I thought for sure it was helping, but I've removed it and the tests still seem to pass with just JDK-8309334 and JDK-8309396. jdb by default does not track all virtual threads. It does have a a `-trackallthreads` command line option, which triggers passing `includevirtualthreads=y` to the debug agent (assuming jdb is actually doing the launching and not just attaching). However, given that these tests do not pass `includevirtualthreads=y`, I didn't find myself wondering how they were passing after just fixing [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334). The answer is that jdb does track any virtual thread that it gets an event for, so as it turns out when these 4 jdb tests look for a specific thread in the `threads` output, it's always a thread that jdb already learned about via an event. I might do a separate PR just to force either `-trackallthreads` or `includevirtualthreads=y`, but currently that doesn't seem to be needed by any of the jdb tests. So I think I should close this PR and update the [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334) PR to remove the 3 tests that seem to pass with it and don't also need this PR to pass. The 4th test `com/sun/jdi/JdbMethodExitTest.java` needs an additional fix that I need to push separately. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14293#issuecomment-1575129126 From cjplummer at openjdk.org Sat Jun 3 20:29:13 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 3 Jun 2023 20:29:13 GMT Subject: RFR: 8309391: Remove non-failing tests from test/jdk/ProblemList-Virtual.txt In-Reply-To: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> References: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> Message-ID: On Fri, 2 Jun 2023 20:08:15 GMT, Chris Plummer wrote: > The following two tests no longer seem to be failing with the virtual thread test factory: > > com/sun/jdi/RedefineG.java 8285422 generic-all > com/sun/jdi/RedefineTTYLineNumber.java 8285422 generic-all > > They can be removed from the problem list. I also updated a couple of entries to refer to existing CRs that give more specific reasons for why the tests are failing. > > I tested locally using JTREG_TEST_THREAD_FACTORY=Virtual. I also have a mach5 tier5 job in progress that will do all the virtual thread testing. > > I'd like to push this as a trivial change. Thanks for the reviews Dan and Leonid! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14290#issuecomment-1575176851 From cjplummer at openjdk.org Sat Jun 3 20:29:13 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 3 Jun 2023 20:29:13 GMT Subject: Integrated: 8309391: Remove non-failing tests from test/jdk/ProblemList-Virtual.txt In-Reply-To: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> References: <-SNTXZ_30agcgbvnhSKMSP_3mBhTrt_rC0IHwIBQ7I0=.24fa8e1d-3c7b-4020-bcb0-a80c45aa03d5@github.com> Message-ID: On Fri, 2 Jun 2023 20:08:15 GMT, Chris Plummer wrote: > The following two tests no longer seem to be failing with the virtual thread test factory: > > com/sun/jdi/RedefineG.java 8285422 generic-all > com/sun/jdi/RedefineTTYLineNumber.java 8285422 generic-all > > They can be removed from the problem list. I also updated a couple of entries to refer to existing CRs that give more specific reasons for why the tests are failing. > > I tested locally using JTREG_TEST_THREAD_FACTORY=Virtual. I also have a mach5 tier5 job in progress that will do all the virtual thread testing. > > I'd like to push this as a trivial change. This pull request has now been integrated. Changeset: fdb5893b Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/fdb5893bf0ad277dbaa2aaffb76478d808d2431d Stats: 4 lines in 1 file changed: 0 ins; 2 del; 2 mod 8309391: Remove non-failing tests from test/jdk/ProblemList-Virtual.txt Reviewed-by: dcubed, lmesnik ------------- PR: https://git.openjdk.org/jdk/pull/14290 From cjplummer at openjdk.org Sat Jun 3 21:18:23 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 3 Jun 2023 21:18:23 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper [v2] In-Reply-To: References: Message-ID: > Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: > > com/sun/jdi/JdbMethodExitTest.java > com/sun/jdi/JdbStepTest.java > com/sun/jdi/JdbStopThreadTest.java > com/sun/jdi/JdbStopThreadidTest.java > > These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. > > Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. > > Testing was done by running all of tier1 and tier5. Chris Plummer has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Merge branch 'master' into 8309334_processtools merge - when using virtual threads, properly name the main threads ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14292/files - new: https://git.openjdk.org/jdk/pull/14292/files/e53abaf4..de5ca147 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14292&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14292&range=00-01 Stats: 4052 lines in 65 files changed: 3213 ins; 625 del; 214 mod Patch: https://git.openjdk.org/jdk/pull/14292.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14292/head:pull/14292 PR: https://git.openjdk.org/jdk/pull/14292 From cjplummer at openjdk.org Sat Jun 3 21:34:16 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 3 Jun 2023 21:34:16 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper [v3] In-Reply-To: References: Message-ID: > Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: > > com/sun/jdi/JdbMethodExitTest.java > com/sun/jdi/JdbStepTest.java > com/sun/jdi/JdbStopThreadTest.java > com/sun/jdi/JdbStopThreadidTest.java > > These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. > > Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. > > Testing was done by running all of tier1 and tier5. Chris Plummer has updated the pull request incrementally with two additional commits since the last revision: - No longer need reflection to call Thread.ofVirtual().unstarted() - Remove from problem list tests that are fixed by this PR. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14292/files - new: https://git.openjdk.org/jdk/pull/14292/files/de5ca147..9bcc200b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14292&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14292&range=01-02 Stats: 17 lines in 2 files changed: 0 ins; 16 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14292.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14292/head:pull/14292 PR: https://git.openjdk.org/jdk/pull/14292 From sspitsyn at openjdk.org Sat Jun 3 23:11:07 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sat, 3 Jun 2023 23:11:07 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper [v3] In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 21:34:16 GMT, Chris Plummer wrote: >> Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: >> >> com/sun/jdi/JdbMethodExitTest.java >> com/sun/jdi/JdbStepTest.java >> com/sun/jdi/JdbStopThreadTest.java >> com/sun/jdi/JdbStopThreadidTest.java >> >> These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. >> >> Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. >> >> Testing was done by running all of tier1 and tier5. > > Chris Plummer has updated the pull request incrementally with two additional commits since the last revision: > > - No longer need reflection to call Thread.ofVirtual().unstarted() > - Remove from problem list tests that are fixed by this PR. Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14292#pullrequestreview-1460389726 From alanb at openjdk.org Sun Jun 4 07:18:08 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 4 Jun 2023 07:18:08 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper [v3] In-Reply-To: References: Message-ID: <5DZfskNZrZY5sENy22fgCrs6CAsoNo9GgnsM3gYVBdU=.dbbf65ec-dd7e-4afe-b80d-49151b23511e@github.com> On Sat, 3 Jun 2023 21:34:16 GMT, Chris Plummer wrote: >> Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: >> >> com/sun/jdi/JdbMethodExitTest.java >> com/sun/jdi/JdbStepTest.java >> com/sun/jdi/JdbStopThreadTest.java >> com/sun/jdi/JdbStopThreadidTest.java >> >> These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. >> >> Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. >> >> Testing was done by running all of tier1 and tier5. > > Chris Plummer has updated the pull request incrementally with two additional commits since the last revision: > > - No longer need reflection to call Thread.ofVirtual().unstarted() > - Remove from problem list tests that are fixed by this PR. Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14292#pullrequestreview-1460999128 From alanb at openjdk.org Sun Jun 4 08:08:04 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 4 Jun 2023 08:08:04 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 10:53:04 GMT, Serguei Spitsyn wrote: > When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. > The fix adds: > - a special case for passive carrier threads > - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. > > Testing: > - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` > - submitted mach5 tiers 1-5 > - TBD: to submit mach5 tier 6 src/hotspot/share/prims/jvmtiEnvBase.cpp line 764: > 762: > 763: if (is_passive_carrier_thread(jt, thread_oop)) { > 764: state |= (JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY); This is testing if the jt is carrying thread_oop and it's okay for the JVMTI state to reported as WAITING when waiting for something other than Object.wait. One thing that is a bit confusing is the function name "is_passive_carrier_thread". A platform thread is either a carrier or not. Maybe for a different PR but I think is_passive_carrier_thread should be renamed to avoid the use of the word "passive". ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1216368303 From sspitsyn at openjdk.org Sun Jun 4 08:30:04 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sun, 4 Jun 2023 08:30:04 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 08:05:34 GMT, Alan Bateman wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 764: > >> 762: >> 763: if (is_passive_carrier_thread(jt, thread_oop)) { >> 764: state |= (JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY); > > This is testing if the jt is carrying thread_oop and it's okay for the JVMTI state to reported as WAITING when waiting for something other than Object.wait. > > One thing that is a bit confusing is the function name "is_passive_carrier_thread". A platform thread is either a carrier or not. Maybe for a different PR but I think is_passive_carrier_thread should be renamed to avoid the use of the word "passive". The lines 763-764 are to correct the state exactly for passive carrier thread, a carrier thread which can't progress until the execution control has not been returned from a virtual thread executed on the top. It is never for a platform thread which is not a carrier thread. "Passive" is the best word I was able to find for this meaning. Do you have any other word/suggestion in mind? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1216390108 From sspitsyn at openjdk.org Sun Jun 4 09:38:51 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sun, 4 Jun 2023 09:38:51 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v7] In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: add to TestJcmdNoAgentLoad default and enabled dynamic loading modes ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14244/files - new: https://git.openjdk.org/jdk/pull/14244/files/18cad064..f82ce73e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=05-06 Stats: 36 lines in 1 file changed: 31 ins; 1 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14244.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14244/head:pull/14244 PR: https://git.openjdk.org/jdk/pull/14244 From aph at openjdk.org Sun Jun 4 10:20:23 2023 From: aph at openjdk.org (Andrew Haley) Date: Sun, 4 Jun 2023 10:20:23 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: On Fri, 2 Jun 2023 02:49:25 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Force PLAB sizes to align on card-table size This patch was submitted 2023-05-26, two week before RDP1. It is large and complex. My biggest concern it that it doesn't seem to be well-isolated from the existing Shenandoah code. The biggest risk is that it breaks Trad (i.e. non-generational) Shenandoah. A patch like this takes a few weeks to review properly. I don't believe we should hurry the review process. While this looks interesting, and it has promise for the future, it's not worth breaking anything for. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1575504522 From alanb at openjdk.org Sun Jun 4 11:17:04 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 4 Jun 2023 11:17:04 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 08:26:06 GMT, Serguei Spitsyn wrote: > The lines 763-764 are to correct the state exactly for passive carrier thread, a carrier thread which can't progress until the execution control has not been returned from a virtual thread executed on the top. It is never for a platform thread which is not a carrier thread. "Passive" is the best word I was able to find for this meaning. Do you have any other word/suggestion in mind? It's just a carrier. A platform thread becomes a carrier when a virtual thread is mounted, it ceases to be a carrier once the virtual thread is unmounted. The mental model is that the carrier is blocked so reporting its state as waiting indefinitely is correct. Maybe you don't want to rename it in this PR but renaming this function to something like is_carrying would convey that it's asking the question if a given JavaThread is carrying the given virtual thread oop. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1216568827 From alanb at openjdk.org Sun Jun 4 11:40:13 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 4 Jun 2023 11:40:13 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v7] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Sun, 4 Jun 2023 09:38:51 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: add to TestJcmdNoAgentLoad default and enabled dynamic loading modes Implementation change looks fine. Once your branch is sync up to main line then it should mean EnableDynamicAgentLoading is only used in one function, so easy to understand. test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 68: > 66: private static final String[] CMD = new String[] { "JVMTI.agent_load", "Agent.jar" }; > 67: private static final String PTRN = "Dynamic agent loading is not enabled"; > 68: private static boolean enableDynLoad = true; It might be clear to change this to be a static final field name "dynamicLoadingEnabled", just suggesting "enabled" rather than "enable" as the usage in this test is to see if the option is enabled. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14244#pullrequestreview-1461132592 PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1216591017 From alanb at openjdk.org Sun Jun 4 11:43:13 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 4 Jun 2023 11:43:13 GMT Subject: RFR: 8309397: com/sun/jdi/JdbXXX tests fail due to not being run with -Djdk.trackAllThreads In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 18:57:55 GMT, Chris Plummer wrote: > So I think I should close this PR and update the [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334) PR to remove the 3 tests that seem to pass with it and don't also need this PR to pass. The 4th test `com/sun/jdi/JdbMethodExitTest.java` needs an additional fix that I need to push separately. Okay with me, thanks for confirming that jdk.trackAllThreads has no effect here. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14293#issuecomment-1575534183 From cjplummer at openjdk.org Sun Jun 4 18:21:12 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sun, 4 Jun 2023 18:21:12 GMT Subject: RFR: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper [v3] In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 21:34:16 GMT, Chris Plummer wrote: >> Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: >> >> com/sun/jdi/JdbMethodExitTest.java >> com/sun/jdi/JdbStepTest.java >> com/sun/jdi/JdbStopThreadTest.java >> com/sun/jdi/JdbStopThreadidTest.java >> >> These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. >> >> Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. >> >> Testing was done by running all of tier1 and tier5. > > Chris Plummer has updated the pull request incrementally with two additional commits since the last revision: > > - No longer need reflection to call Thread.ofVirtual().unstarted() > - Remove from problem list tests that are fixed by this PR. Thanks for the reviews Leonid, Alan, Serguei and Alex! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14292#issuecomment-1575663199 From cjplummer at openjdk.org Sun Jun 4 18:21:14 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sun, 4 Jun 2023 18:21:14 GMT Subject: Integrated: 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 21:30:46 GMT, Chris Plummer wrote: > Normally when a virtual thread wrapper is used to run a test, the main thread is renamed to "old-m-a-i-n" and the new virtual thread that will act as the main thread is named "main". Neither is being done by `ProcessTools.main()`. This can cause problems for tests that expect the main thread that the test is running in to be called "main". It is instead left unnamed. This is causing the following 4 tests to fail: > > com/sun/jdi/JdbMethodExitTest.java > com/sun/jdi/JdbStepTest.java > com/sun/jdi/JdbStopThreadTest.java > com/sun/jdi/JdbStopThreadidTest.java > > These tests also fail due to [JDK-8309397](https://bugs.openjdk.org/browse/JDK-8309397), which will be fixed after this CR, and also com/sun/jdi/JdbMethodExitTest.java fails due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which will also subsequently be fixed. > > Note this fix messed up one runtime test. It was expecting an exception message to mention the "main" thread rather than "old-m-a-i-n". Loosening the exception message matching pattern a bit solved the problem. > > Testing was done by running all of tier1 and tier5. This pull request has now been integrated. Changeset: ecb17532 Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/ecb17532dc8f3e271ad2d6550127a2253569cf9b Stats: 24 lines in 3 files changed: 5 ins; 16 del; 3 mod 8309334: ProcessTools.main() does not properly set thread names when using the virtual thread wrapper Reviewed-by: amenkov, lmesnik, sspitsyn, alanb ------------- PR: https://git.openjdk.org/jdk/pull/14292 From cjplummer at openjdk.org Sun Jun 4 18:47:13 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sun, 4 Jun 2023 18:47:13 GMT Subject: RFR: 8309396: com/sun/jdi/JdbMethodExitTest.java fails with virtual threads due to a bug in determining the main thread id [v2] In-Reply-To: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> References: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> Message-ID: > JdbMethodExitTest.java tries to determine the jdb threadID for the "main" thread, and then later use it in jdb commands that require a threadID. It does this by first having the debuggee execute the following: > > ` System.out.println("threadid="+Thread.currentThread().getId());` > > And then later on the test parses the threadID from this output. The problem is that the id returned by getId() has no relation to threadIDs used by jdb, which are actually JDWP ObjectIDs. In the past this has worked due to some dumb luck. getID() always returns 1 for the main thread, which is always the thread we are executing in. Coincidentally the JDWP ObjectID for the main Thread object is also always 1 because this is the first ObjectID that the debug agent ever needs to create. However, when the debuggee main thread is a virtual thread, neither getId() nor JDWP assign 1 to the threadID, and in fact both will end up with very different values for the threadID. The end result is errors from jdb for using an invalid threadID. > > The correct threadID can be obtained by executing the jdb "threads" command and parsing it from a line that looks like the following: > > ` (java.lang.VirtualThread)694 main running (at breakpoint)` > > Note this test will also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which should be fixed first. > > I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the test being removed from the problem list by this PR also passes. Chris Plummer has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: - fix minor merge issue - merge - remove test from problem list - properly determine the main threadId ------------- Changes: https://git.openjdk.org/jdk/pull/14294/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14294&range=01 Stats: 11 lines in 2 files changed: 5 ins; 2 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14294.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14294/head:pull/14294 PR: https://git.openjdk.org/jdk/pull/14294 From kdnilsen at openjdk.org Sun Jun 4 21:39:58 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Sun, 4 Jun 2023 21:39:58 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Remove three asserts making comparisons between atomic volatile variables Though changes to the volatile variables are individually protected by Atomic load and store operations, these asserts were not assuring atomic access to multiple volatile variables, each of which could be modified independently of the others. The asserts were therefore not trustworthy, as has been confirmed by more extensive testing. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/d4d2f1cf..8d80780a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=03-04 Stats: 18 lines in 2 files changed: 0 ins; 18 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From cjplummer at openjdk.org Mon Jun 5 04:11:28 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 5 Jun 2023 04:11:28 GMT Subject: RFR: 8309420: con/sun/jdi/StepTest.java fails with virtual thread wrapper Message-ID: The test has two issues. The first is that it assume that once the VMStart event has arrived and one "step into" is done, it will be in the main method of the debuggee. Once there, it determines the debuggee class name by looking at the classtype of topmost frame. The problems is when using virtual threads, it is actually in TestScaffold.main() at this point, so the wrong class name is gleaned from the frame. To fix this the test just saves away the debuggee class name, which is passed to the test as the 4th argument. The other issue is that the test assumes once it gets to the debuggee go() method, there are only two frames on the stack. It's more like 16 when using virtual threads. The test needs to account for this by counting the number of frames when go() is entered rather than assuming it will be 2. Tested locally with and without the wrapper and by running tier5 svc tests. ------------- Commit messages: - Fix StepTest for virtual threads wrapper Changes: https://git.openjdk.org/jdk/pull/14307/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14307&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309420 Stats: 21 lines in 2 files changed: 10 ins; 4 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/14307.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14307/head:pull/14307 PR: https://git.openjdk.org/jdk/pull/14307 From alanb at openjdk.org Mon Jun 5 06:19:27 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 5 Jun 2023 06:19:27 GMT Subject: RFR: 8309408: Thread.sleep cleanup Message-ID: Thread.sleep has had quite a bit of churn recently to support virtual threads, add sleep(Duration), a JFR event, and the change the underlying implementation to support sub-millis precision. I think the changes have settled down now so we can do some small cleanups that came up in PR discussions. The cleanups were kicked down the road as it requires tracking down faraway tests that depend on the stack depth and the names of internal methods. The two cleanups proposed here are: 1. Add a private sleepNanos method that creates/commits the JFR event around the sleep, this avoids duplicate code in the 3 sleep methods. 2. Rename JVM_Sleep to JVM_SleepNanos to make it clear that it takes the sleep time in nanoseconds, esp. when Thread.sleep's parameter is milliseconds. ------------- Commit messages: - Initial commit Changes: https://git.openjdk.org/jdk/pull/14303/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14303&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309408 Stats: 75 lines in 10 files changed: 27 ins; 32 del; 16 mod Patch: https://git.openjdk.org/jdk/pull/14303.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14303/head:pull/14303 PR: https://git.openjdk.org/jdk/pull/14303 From dholmes at openjdk.org Mon Jun 5 07:31:12 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 5 Jun 2023 07:31:12 GMT Subject: RFR: 8309408: Thread.sleep cleanup In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 11:28:33 GMT, Alan Bateman wrote: > Thread.sleep has had quite a bit of churn recently to support virtual threads, add sleep(Duration), a JFR event, and the change the underlying implementation to support sub-millis precision. I think the changes have settled down now so we can do some small cleanups that came up in PR discussions. The cleanups were kicked down the road as it requires tracking down faraway tests that depend on the stack depth and the names of internal methods. The two cleanups proposed here are: > > 1. Add a private sleepNanos method that creates/commits the JFR event around the sleep, this avoids duplicate code in the 3 sleep methods. > 2. Rename JVM_Sleep to JVM_SleepNanos to make it clear that it takes the sleep time in nanoseconds, esp. when Thread.sleep's parameter is milliseconds. Looks good! Keeping these tests up-to-date is painful, but as you note this is hopefully stabilized now. There is one potential, pre-existing, test omission noted below. Thanks. test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/ThreadController.java line 660: > 658: expectedMethods.add(Thread.class.getName() + ".sleep"); > 659: expectedMethods.add(Thread.class.getName() + ".sleepNanos"); > 660: expectedMethods.add(Thread.class.getName() + ".sleepNanos0"); I'm surprised this test doesn't list `beforeSleep` and `afterSleep`. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14303#pullrequestreview-1461876843 PR Review Comment: https://git.openjdk.org/jdk/pull/14303#discussion_r1217641284 From stefank at openjdk.org Mon Jun 5 08:23:05 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Mon, 5 Jun 2023 08:23:05 GMT Subject: RFR: 8309408: Thread.sleep cleanup In-Reply-To: References: Message-ID: <5QMR3WIYHEm2QUPqwMEb5fX_eHEYZv9qrv9BeJeeooI=.c53f6306-26cf-4d32-966a-c6f6795b760c@github.com> On Sun, 4 Jun 2023 11:28:33 GMT, Alan Bateman wrote: > Thread.sleep has had quite a bit of churn recently to support virtual threads, add sleep(Duration), a JFR event, and the change the underlying implementation to support sub-millis precision. I think the changes have settled down now so we can do some small cleanups that came up in PR discussions. The cleanups were kicked down the road as it requires tracking down faraway tests that depend on the stack depth and the names of internal methods. The two cleanups proposed here are: > > 1. Add a private sleepNanos method that creates/commits the JFR event around the sleep, this avoids duplicate code in the 3 sleep methods. > 2. Rename JVM_Sleep to JVM_SleepNanos to make it clear that it takes the sleep time in nanoseconds, esp. when Thread.sleep's parameter is milliseconds. I welcome this change. We were looking at this code on Friday and was thinking that it would have been good to rename sleep0 and JVM_Sleep. :) ------------- Marked as reviewed by stefank (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14303#pullrequestreview-1461982066 From kdnilsen at openjdk.org Mon Jun 5 12:50:26 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 5 Jun 2023 12:50:26 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 660: > 658: void ShenandoahGeneration::increase_used(size_t bytes) { > 659: Atomic::add(&_used, bytes); > 660: } Note that C++ subexpression evaluation order is undefined. Here is an example of what can go wrong with the removed assertion: ThisThread: fetches _affiliated_region_count * begins to multiply with region_size OtherThread: increases _used and increases _affiliated_region_count appropriately (for a large allocation) ThisThread: fetches _used and observes assert violation ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1218025633 From kdnilsen at openjdk.org Mon Jun 5 13:06:25 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 5 Jun 2023 13:06:25 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 12:46:58 GMT, Kelvin Nilsen wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove three asserts making comparisons between atomic volatile variables >> >> Though changes to the volatile variables are individually protected by >> Atomic load and store operations, these asserts were not assuring >> atomic access to multiple volatile variables, each of which could be >> modified independently of the others. The asserts were therefore not >> trustworthy, as has been confirmed by more extensive testing. > > src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 660: > >> 658: void ShenandoahGeneration::increase_used(size_t bytes) { >> 659: Atomic::add(&_used, bytes); >> 660: } > > Note that C++ subexpression evaluation order is undefined. Here is an example of what can go wrong with the removed assertion: > > ThisThread: fetches _affiliated_region_count * begins to multiply with region_size > OtherThread: increases _used and increases _affiliated_region_count appropriately (for a large allocation) > ThisThread: fetches _used and observes assert violation We have seen this race manifest in actual testing, which is what has motivated us to remove the assertions. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1218045035 From kdnilsen at openjdk.org Mon Jun 5 13:12:26 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 5 Jun 2023 13:12:26 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.inline.hpp line 85: > 83: inline void ShenandoahHeapRegion::internal_increase_live_data(size_t s) { > 84: size_t new_live_data = Atomic::add(&_live_data, s, memory_order_relaxed); > 85: #ifdef ASSERT We have not observed violation of this assert during testing. However, it appears unreliable in that _live_data increases monotonically under Atomic volatile math, whereas used() increases monotonically under heap lock, which we do not hold at the point of this assertion. It is possible that another allocating thread increases both used() and _live_data. We will see the increase in _live_data because _live_data changes are volatile Atomic. However, we may not see the increase in used() because we did not acquire the heap lock and the value of _top() that contributes to calculation of used() is not volatile. This situation can lead to an assertion failure. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1218051609 From goetz at openjdk.org Mon Jun 5 15:04:07 2023 From: goetz at openjdk.org (Goetz Lindenmaier) Date: Mon, 5 Jun 2023 15:04:07 GMT Subject: RFR: JDK-8309225: Fix xlc17 clang 15 warnings in security and servicability In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 10:19:53 GMT, JoKern65 wrote: > This pr is a split off from JDK-8308288: Fix xlc17 clang warnings in shared code https://github.com/openjdk/jdk/pull/14146 > It handles the part in security and servicability. > > Compiling on AIX with xlc17 which contains the new clang 15 frontend shows the following warnings: > > src/java.security.jgss/share/native/libj2gss/NativeUtil.h:30: > src/java.security.jgss/share/native/libj2gss/gssapi.h:48:5: error: 'TARGET_OS_MAC' is not defined, evaluates to 0 [-Werror,-Wundef] > #if TARGET_OS_MAC && (defined(ppc) || defined(ppc64) || defined(i386) || defined(x86_64)) > ^ > TARGET_OS_MAC is not defined. Instead of disabling the warning, I could > ` #ifndef TARGET_OS_MAC` > `#define TARGET_OS_MAC=0` > `#endif` > But this is already handled by disabling the warning for gcc. > > src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:718:33: error: suggest braces around initialization of subobject [-Werror,-Wmissing-braces] > struct in6_addr mappedAny = IN6ADDR_ANY_INIT; > ^~~~~~~~~~~~~~~~ > /usr/include/netinet/in.h:454:32: note: expanded from macro 'IN6ADDR_ANY_INIT' > #define IN6ADDR_ANY_INIT {0, 0, 0, 0} Pre-submit failures: C:/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/11.0.19-7/x64/bin/javac: Bad address on both windows, unrelated to this change. This was seen lately repeatedly. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14282#issuecomment-1576966850 From goetz at openjdk.org Mon Jun 5 15:08:08 2023 From: goetz at openjdk.org (Goetz Lindenmaier) Date: Mon, 5 Jun 2023 15:08:08 GMT Subject: RFR: JDK-8309225: Fix xlc17 clang 15 warnings in security and servicability In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 10:19:53 GMT, JoKern65 wrote: > This pr is a split off from JDK-8308288: Fix xlc17 clang warnings in shared code https://github.com/openjdk/jdk/pull/14146 > It handles the part in security and servicability. > > Compiling on AIX with xlc17 which contains the new clang 15 frontend shows the following warnings: > > src/java.security.jgss/share/native/libj2gss/NativeUtil.h:30: > src/java.security.jgss/share/native/libj2gss/gssapi.h:48:5: error: 'TARGET_OS_MAC' is not defined, evaluates to 0 [-Werror,-Wundef] > #if TARGET_OS_MAC && (defined(ppc) || defined(ppc64) || defined(i386) || defined(x86_64)) > ^ > TARGET_OS_MAC is not defined. Instead of disabling the warning, I could > ` #ifndef TARGET_OS_MAC` > `#define TARGET_OS_MAC=0` > `#endif` > But this is already handled by disabling the warning for gcc. > > src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:718:33: error: suggest braces around initialization of subobject [-Werror,-Wmissing-braces] > struct in6_addr mappedAny = IN6ADDR_ANY_INIT; > ^~~~~~~~~~~~~~~~ > /usr/include/netinet/in.h:454:32: note: expanded from macro 'IN6ADDR_ANY_INIT' > #define IN6ADDR_ANY_INIT {0, 0, 0, 0} TARGET_OS_MAC: I agree we should not fix this one occurance as long as the warning is disabled in gcc. IN6ADDR_ANY_INIT Cannot change the macro, it comes from a system header. ------------- Marked as reviewed by goetz (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14282#pullrequestreview-1462766607 From alanb at openjdk.org Mon Jun 5 15:13:06 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 5 Jun 2023 15:13:06 GMT Subject: RFR: 8309408: Thread.sleep cleanup In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 07:26:57 GMT, David Holmes wrote: >> Thread.sleep has had quite a bit of churn recently to support virtual threads, add sleep(Duration), a JFR event, and the change the underlying implementation to support sub-millis precision. I think the changes have settled down now so we can do some small cleanups that came up in PR discussions. The cleanups were kicked down the road as it requires tracking down faraway tests that depend on the stack depth and the names of internal methods. The two cleanups proposed here are: >> >> 1. Add a private sleepNanos method that creates/commits the JFR event around the sleep, this avoids duplicate code in the 3 sleep methods. >> 2. Rename JVM_Sleep to JVM_SleepNanos to make it clear that it takes the sleep time in nanoseconds, esp. when Thread.sleep's parameter is milliseconds. > > test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/ThreadController.java line 660: > >> 658: expectedMethods.add(Thread.class.getName() + ".sleep"); >> 659: expectedMethods.add(Thread.class.getName() + ".sleepNanos"); >> 660: expectedMethods.add(Thread.class.getName() + ".sleepNanos0"); > > I'm surprised this test doesn't list `beforeSleep` and `afterSleep`. > There is one potential, pre-existing, test omission noted below. > I'm surprised this test doesn't list `beforeSleep` and `afterSleep`. The monitoring/stress/thread tests will fail if they observe an unexpected method name in the stack trace. I don't think it can happen because the tests poll the thread state and for SleepingThread, it will sample the stack trace when the thread state is timed-wait. The beforeSleep/afterSleep methods won't in the stack trace when sleeping. It would be harmless to add them in that they aren't going to cause these tests to fail but might help with any further changes. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14303#discussion_r1218215964 From alanb at openjdk.org Mon Jun 5 15:54:33 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 5 Jun 2023 15:54:33 GMT Subject: RFR: 8309406: Change jdk.trackAllThreads to default to true Message-ID: Virtual threads created directly with the Thread API are not included in the new thread dump by default. This was a source of confusion in JDK 19/20 when virtual threads were in preview. We have decided to switch the default so that all virtual threads are observable. Future work will reduce/remove the overhead of tracking threads in the root container. The change proposed here has no impact to virtual threads created with the virtual ThreadPerTaskExecutor or StructuredTaskScope. With jdk.trackAllThreads defaulting to true then the need for SharedThreadContainer to count threads can go away, as does the need for RootContainer::threads to return virtual threads blocked on I/O. Most of the changes are to the test HotSpotDiagnosticMXBean/DumpThreads.java as this is expanded to cover more cases, and specifically more cases with thread groupings that correspond to the ThreadPerTaskExecutor and thread pools. Testing: Test1-5 ------------- Commit messages: - requires vm.continuations - Initial commit Changes: https://git.openjdk.org/jdk/pull/14300/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14300&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309406 Stats: 319 lines in 6 files changed: 183 ins; 81 del; 55 mod Patch: https://git.openjdk.org/jdk/pull/14300.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14300/head:pull/14300 PR: https://git.openjdk.org/jdk/pull/14300 From duke at openjdk.org Mon Jun 5 16:22:28 2023 From: duke at openjdk.org (Christine Flood) Date: Mon, 5 Jun 2023 16:22:28 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: On Fri, 2 Jun 2023 02:49:25 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Force PLAB sizes to align on card-table size There are some applications where generational collectors are effective, but there are some applications where they aren't, like LRU caches. I would like to see the changes made to Shenandoah to make it generational cleanly isolated from traditional Shenandoah so that both options remain available to our customers moving forward. It seems unlikely that such changes can be completed before the deadline for RDP1. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1575572311 From shade at openjdk.org Mon Jun 5 17:28:08 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 5 Jun 2023 17:28:08 GMT Subject: RFR: 8309408: Thread.sleep cleanup In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 11:28:33 GMT, Alan Bateman wrote: > Thread.sleep has had quite a bit of churn recently to support virtual threads, add sleep(Duration), a JFR event, and the change the underlying implementation to support sub-millis precision. I think the changes have settled down now so we can do some small cleanups that came up in PR discussions. The cleanups were kicked down the road as it requires tracking down faraway tests that depend on the stack depth and the names of internal methods. The two cleanups proposed here are: > > 1. Add a private sleepNanos method that creates/commits the JFR event around the sleep, this avoids duplicate code in the 3 sleep methods. > 2. Rename JVM_Sleep to JVM_SleepNanos to make it clear that it takes the sleep time in nanoseconds, esp. when Thread.sleep's parameter is milliseconds. I think we need to delay this until [JDK-8309361](https://bugs.openjdk.org/browse/JDK-8309361) is resolved, in case we would like to revert [JDK-8305092](https://bugs.openjdk.org/browse/JDK-8305092). ------------- PR Review: https://git.openjdk.org/jdk/pull/14303#pullrequestreview-1463041304 From rpressler at openjdk.org Mon Jun 5 17:53:06 2023 From: rpressler at openjdk.org (Ron Pressler) Date: Mon, 5 Jun 2023 17:53:06 GMT Subject: RFR: 8309406: Change jdk.trackAllThreads to default to true In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 13:41:53 GMT, Alan Bateman wrote: > Virtual threads created directly with the Thread API are not included in the new thread dump by default. This was a source of confusion in JDK 19/20 when virtual threads were in preview. We have decided to switch the default so that all virtual threads are observable. Future work will reduce/remove the overhead of tracking threads in the root container. The change proposed here has no impact to virtual threads created with the virtual ThreadPerTaskExecutor or StructuredTaskScope. > > With jdk.trackAllThreads defaulting to true then the need for SharedThreadContainer to count threads can go away, as does the need for RootContainer::threads to return virtual threads blocked on I/O. > > Most of the changes are to the test HotSpotDiagnosticMXBean/DumpThreads.java as this is expanded to cover more cases, and specifically more cases with thread groupings that correspond to the ThreadPerTaskExecutor and thread pools. > > Testing: Test1-5 Marked as reviewed by rpressler (Committer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14300#pullrequestreview-1463085546 From mchung at openjdk.org Mon Jun 5 18:17:47 2023 From: mchung at openjdk.org (Mandy Chung) Date: Mon, 5 Jun 2023 18:17:47 GMT Subject: RFR: 8309406: Change jdk.trackAllThreads to default to true In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 13:41:53 GMT, Alan Bateman wrote: > Virtual threads created directly with the Thread API are not included in the new thread dump by default. This was a source of confusion in JDK 19/20 when virtual threads were in preview. We have decided to switch the default so that all virtual threads are observable. Future work will reduce/remove the overhead of tracking threads in the root container. The change proposed here has no impact to virtual threads created with the virtual ThreadPerTaskExecutor or StructuredTaskScope. > > With jdk.trackAllThreads defaulting to true then the need for SharedThreadContainer to count threads can go away, as does the need for RootContainer::threads to return virtual threads blocked on I/O. > > Most of the changes are to the test HotSpotDiagnosticMXBean/DumpThreads.java as this is expanded to cover more cases, and specifically more cases with thread groupings that correspond to the ThreadPerTaskExecutor and thread pools. > > Testing: Test1-5 Looks okay to me. ------------- Marked as reviewed by mchung (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14300#pullrequestreview-1463111013 From sspitsyn at openjdk.org Mon Jun 5 18:43:04 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 18:43:04 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v8] In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: <7ZA346z0mMulmI2vNQOcPn43KGYEvxIWT5Va704TnJw=.1c168e3f-bac6-49da-b49c-a3753bd892e6@github.com> > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: add real java agent to TestJcmdNoAgentLoad ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14244/files - new: https://git.openjdk.org/jdk/pull/14244/files/f82ce73e..cf12a0ba Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=06-07 Stats: 35 lines in 2 files changed: 27 ins; 0 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/14244.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14244/head:pull/14244 PR: https://git.openjdk.org/jdk/pull/14244 From sspitsyn at openjdk.org Mon Jun 5 18:43:07 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 18:43:07 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v7] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Sun, 4 Jun 2023 11:36:05 GMT, Alan Bateman wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: add to TestJcmdNoAgentLoad default and enabled dynamic loading modes > > test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 68: > >> 66: private static final String[] CMD = new String[] { "JVMTI.agent_load", "Agent.jar" }; >> 67: private static final String PTRN = "Dynamic agent loading is not enabled"; >> 68: private static boolean enableDynLoad = true; > > It might be clearer to change this to be a static final field name "dynamicLoadingEnabled", just suggesting "enabled" rather than "enable" as the usage in this test is to see if the option is enabled. Good suggestion, thanks. Fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1218449527 From sspitsyn at openjdk.org Mon Jun 5 18:44:37 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 18:44:37 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v6] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Fri, 2 Jun 2023 19:27:28 GMT, Alex Menkov wrote: >> test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java line 42: >> >>> 40: * @library /test/lib >>> 41: * >>> 42: * @run main/othervm -XX:-EnableDynamicAgentLoading TestJcmdNoAgentLoad >> >> The test verifies only one scenario. I think we need to test 3 scenarios: >> - -XX:-EnableDynamicAgentLoading >> - -XX:+EnableDynamicAgentLoading >> - default (no EnableDynamicAgentLoading arg) > > I didn't find tests for EnableDynamicAgentLoading for attach API too (to ensure this fix doesn't break it) Good suggestions, thanks. Added three modes: default, disabled and enabled. Also, added a real java agent to check if it has been loaded. The jcmd is using the dynamic attach to load the agents, so it has to be tested as well. Alan recently added new tests for this as well. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14244#discussion_r1218453869 From sspitsyn at openjdk.org Mon Jun 5 19:00:49 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 19:00:49 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v2] In-Reply-To: References: Message-ID: <5-_kURDcYKd5WYKi9B331c8h6okVmGfvjdy_xqi1UqU=.012c360b-1b57-4773-b64b-d727dbf4daeb@github.com> > When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. > The fix adds: > - a special case for passive carrier threads > - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. > > Testing: > - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` > - submitted mach5 tiers 1-5 > - TBD: to submit mach5 tier 6 Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - Merge - minor tweaks in libThreadStateTest.cpp - 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14298/files - new: https://git.openjdk.org/jdk/pull/14298/files/84d8825f..e60da02e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=00-01 Stats: 10489 lines in 228 files changed: 8499 ins; 1035 del; 955 mod Patch: https://git.openjdk.org/jdk/pull/14298.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14298/head:pull/14298 PR: https://git.openjdk.org/jdk/pull/14298 From sspitsyn at openjdk.org Mon Jun 5 19:01:11 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 19:01:11 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v9] In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: <6Jkd1TVSYX3__pNjLCxkHFVmgXm2NtY8JENyaDFPLZA=.903329bb-b573-4af8-ba0f-ed2833684047@github.com> > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision: - Merge - review: add real java agent to TestJcmdNoAgentLoad - review: add to TestJcmdNoAgentLoad default and enabled dynamic loading modes - Merge - review: use output.shouldHaveExitValue(0) in the test - review: use output.shouldContain() - move check for EnableDynamicAgentLoading to invoke_Agent_OnAttach - minor renaming in new test TestJcmdNoAgentLoad.java - 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14244/files - new: https://git.openjdk.org/jdk/pull/14244/files/cf12a0ba..ada88c6d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=07-08 Stats: 10489 lines in 228 files changed: 8499 ins; 1035 del; 955 mod Patch: https://git.openjdk.org/jdk/pull/14244.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14244/head:pull/14244 PR: https://git.openjdk.org/jdk/pull/14244 From sspitsyn at openjdk.org Mon Jun 5 19:10:40 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 19:10:40 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v10] In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: fix a typo and minor cleanup ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14244/files - new: https://git.openjdk.org/jdk/pull/14244/files/ada88c6d..16bbf3ca Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=08-09 Stats: 5 lines in 1 file changed: 0 ins; 1 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14244.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14244/head:pull/14244 PR: https://git.openjdk.org/jdk/pull/14244 From sspitsyn at openjdk.org Mon Jun 5 19:13:30 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 19:13:30 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v11] In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: removed an obsolete comment ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14244/files - new: https://git.openjdk.org/jdk/pull/14244/files/16bbf3ca..4e8fe970 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14244&range=09-10 Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14244.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14244/head:pull/14244 PR: https://git.openjdk.org/jdk/pull/14244 From alanb at openjdk.org Mon Jun 5 19:46:53 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 5 Jun 2023 19:46:53 GMT Subject: RFR: 8309408: Thread.sleep cleanup In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 17:25:43 GMT, Aleksey Shipilev wrote: > I think we need to delay this until [JDK-8309361](https://bugs.openjdk.org/browse/JDK-8309361) is resolved, in case we would like to revert [JDK-8305092](https://bugs.openjdk.org/browse/JDK-8305092). Okay, I won't integrate this until we see what the issue is. That said, if the sub-mills support needs to be reverted then I think we should keep the interface as nanos on the bound between the VM and the libraries. The reason is that it's mostly nanos at the Java level now, meaning 2 out of the 3 sleep methods support sub-mills, and the virtual thread sleep and the JFR event are in nanos too. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14303#issuecomment-1577374410 From amenkov at openjdk.org Mon Jun 5 19:59:56 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Mon, 5 Jun 2023 19:59:56 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v11] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Mon, 5 Jun 2023 19:13:30 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > removed an obsolete comment Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14244#pullrequestreview-1463299507 From cjplummer at openjdk.org Mon Jun 5 20:10:54 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 5 Jun 2023 20:10:54 GMT Subject: RFR: 8309408: Thread.sleep cleanup In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 15:10:18 GMT, Alan Bateman wrote: >> test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/ThreadController.java line 660: >> >>> 658: expectedMethods.add(Thread.class.getName() + ".sleep"); >>> 659: expectedMethods.add(Thread.class.getName() + ".sleepNanos"); >>> 660: expectedMethods.add(Thread.class.getName() + ".sleepNanos0"); >> >> I'm surprised this test doesn't list `beforeSleep` and `afterSleep`. > >> There is one potential, pre-existing, test omission noted below. >> I'm surprised this test doesn't list `beforeSleep` and `afterSleep`. > > The monitoring/stress/thread tests will fail if they observe an unexpected method name in the stack trace. I don't think it can happen because the tests poll the thread state and for SleepingThread, it will sample the stack trace when the thread state is timed-wait. The beforeSleep/afterSleep methods won't in the stack trace when sleeping. It would be harmless to add them in that they aren't going to cause these tests to fail but might help with any further changes. The following commit in loom heavily modified this file with a lot of added expected methods. There are other related tests with similar changes. I'm not so sure I understand the need for so many additions, and also why expectedLength is so out of sync with the number of added method. I don't believe this commit was reviewed individually, but was just part of the overall loom review when merge into jdk. Perhaps it should be revisited. https://github.com/openjdk/loom/commit/26e66bc1a6a0dd735c8138a696809caba3e82b26 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14303#discussion_r1218539693 From kdnilsen at openjdk.org Mon Jun 5 20:13:11 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 5 Jun 2023 20:13:11 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 733: > 731: } else { > 732: iu_barrier(masm, val, tmp3); > 733: // TODO: store_check missing in upstream Remove this comment with integration. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1218543687 From cjplummer at openjdk.org Mon Jun 5 20:33:52 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 5 Jun 2023 20:33:52 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v2] In-Reply-To: <5-_kURDcYKd5WYKi9B331c8h6okVmGfvjdy_xqi1UqU=.012c360b-1b57-4773-b64b-d727dbf4daeb@github.com> References: <5-_kURDcYKd5WYKi9B331c8h6okVmGfvjdy_xqi1UqU=.012c360b-1b57-4773-b64b-d727dbf4daeb@github.com> Message-ID: On Mon, 5 Jun 2023 19:00:49 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - Merge > - minor tweaks in libThreadStateTest.cpp > - 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING Without the fix in place, do your new tests reproduce the issue> I'm trying to recall the origins of the filing of this CR. I thought I had noticed this issue while working with a JDI test and discussed it with you and Alan. Just wondering if there is something that can be done to a JDI test to also test for this. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14298#issuecomment-1577432927 From cjplummer at openjdk.org Mon Jun 5 20:36:55 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 5 Jun 2023 20:36:55 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v11] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Mon, 5 Jun 2023 19:13:30 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > removed an obsolete comment Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14244#pullrequestreview-1463368797 From sspitsyn at openjdk.org Mon Jun 5 21:08:57 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 21:08:57 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v2] In-Reply-To: <5-_kURDcYKd5WYKi9B331c8h6okVmGfvjdy_xqi1UqU=.012c360b-1b57-4773-b64b-d727dbf4daeb@github.com> References: <5-_kURDcYKd5WYKi9B331c8h6okVmGfvjdy_xqi1UqU=.012c360b-1b57-4773-b64b-d727dbf4daeb@github.com> Message-ID: On Mon, 5 Jun 2023 19:00:49 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - Merge > - minor tweaks in libThreadStateTest.cpp > - 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING > ------------- PR Comment: https://git.openjdk.org/jdk/pull/14298#issuecomment-1577475470 From sspitsyn at openjdk.org Mon Jun 5 21:29:54 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 5 Jun 2023 21:29:54 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v2] In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 11:14:06 GMT, Alan Bateman wrote: >> The lines 763-764 are to correct the state exactly for passive carrier thread, a carrier thread which can't progress until the execution control has not been returned from a virtual thread executed on the top. It is never for a platform thread which is not a carrier thread. "Passive" is the best word I was able to find for this meaning. Do you have any other word/suggestion in mind? > >> The lines 763-764 are to correct the state exactly for passive carrier thread, a carrier thread which can't progress until the execution control has not been returned from a virtual thread executed on the top. It is never for a platform thread which is not a carrier thread. "Passive" is the best word I was able to find for this meaning. Do you have any other word/suggestion in mind? > > It's just a carrier. A platform thread becomes a carrier when a virtual thread is mounted, it ceases to be a carrier once the virtual thread is unmounted. The mental model is that the carrier is blocked so reporting its state as waiting indefinitely is correct. Maybe you don't want to rename it in this PR but renaming this function to something like is_carrying would convey that it's asking the question if a given JavaThread is carrying the given virtual thread oop. Okay, I see you point. Unfortunately, I've always referred the platform thread with an executed FJP schedular as a carrier thread. The term 'carrier' with this meaning is everywhere in the JVMTI code. It looks very confusing to call a thread to be a carrier thread only during some phases of its execution. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1218613536 From cjplummer at openjdk.org Mon Jun 5 21:43:52 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 5 Jun 2023 21:43:52 GMT Subject: RFR: 8309406: Change jdk.trackAllThreads to default to true In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 13:41:53 GMT, Alan Bateman wrote: > Virtual threads created directly with the Thread API are not included in the new thread dump by default. This was a source of confusion in JDK 19/20 when virtual threads were in preview. We have decided to switch the default so that all virtual threads are observable. Future work will reduce/remove the overhead of tracking threads in the root container. The change proposed here has no impact to virtual threads created with the virtual ThreadPerTaskExecutor or StructuredTaskScope. > > With jdk.trackAllThreads defaulting to true then the need for SharedThreadContainer to count threads can go away, as does the need for RootContainer::threads to return virtual threads blocked on I/O. > > Most of the changes are to the test HotSpotDiagnosticMXBean/DumpThreads.java as this is expanded to cover more cases, and specifically more cases with thread groupings that correspond to the ThreadPerTaskExecutor and thread pools. > > Testing: Test1-5 Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14300#pullrequestreview-1463531717 From cjplummer at openjdk.org Mon Jun 5 22:02:01 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 5 Jun 2023 22:02:01 GMT Subject: RFR: 8309505: com/sun/jdi/MethodEntryExitEvents.java due to finding wrong main thread Message-ID: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> The test fails because it tries to determine the main debuggee thread by allowing it run until the debuggee class is loaded (it waits for the ClassPrepareEvent). Normally this would be done on the main debuggee thread. However, when using virtual threads, the main thread has yet to spawn the virtual thread to run the test on. The test is fixed by instead just waiting until the debuggee main method is entered. Tested by running the test locally with and without the virtual test thread wrapper, and also all of tier1 plus tier5 svc tests. ------------- Commit messages: - Fix virtual thread testing issue by waiting until debuggee main method is entered before gleaning the main thread. Changes: https://git.openjdk.org/jdk/pull/14322/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14322&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309505 Stats: 7 lines in 2 files changed: 1 ins; 1 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/14322.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14322/head:pull/14322 PR: https://git.openjdk.org/jdk/pull/14322 From cjplummer at openjdk.org Mon Jun 5 22:20:56 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 5 Jun 2023 22:20:56 GMT Subject: RFR: 8309506: com/sun/jdi/MultiBreakpointsTest.java fails with virtual test thread factory Message-ID: The test fails when the main debuggee thread is a virtual thread, because virtual threads are always daemon threads. Because of this all the test threads that the debuggee creates are also daemon threads. The main deuggee thread immediately exits after creating the test threads, and at that point there are no non-daemon threads left running to keep the debuggee process alive, so the test threads never get a chance to do all the work that is expected of them. The fix is to use Thread.join() to prevent the main test thread from exiting until all the created tests threads have exited. I also updated the test so when run using the virtual test thread factory, the debuggee test threads will now be virtual threads. Previously just the main debuggee thread would be a virtual thread. This is not a bug fix, but does provide better virtual thread testing. ------------- Commit messages: - Fix virtual thread issues by using join(). Also make test threads be virtual theads when the main debuggee thread is a virtual thread. Changes: https://git.openjdk.org/jdk/pull/14323/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14323&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309506 Stats: 16 lines in 2 files changed: 10 ins; 2 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14323.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14323/head:pull/14323 PR: https://git.openjdk.org/jdk/pull/14323 From amenkov at openjdk.org Mon Jun 5 23:46:53 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Mon, 5 Jun 2023 23:46:53 GMT Subject: RFR: JDK-8306441: Enhance parallel heap dump [v6] In-Reply-To: <4O3fdBsNuvwrymizPAQ1aXDMBVkHo9n841Ez5NqgY98=.28c881c8-2ae0-4736-bb25-7422ff9f60a3@github.com> References: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> <4O3fdBsNuvwrymizPAQ1aXDMBVkHo9n841Ez5NqgY98=.28c881c8-2ae0-4736-bb25-7422ff9f60a3@github.com> Message-ID: On Wed, 17 May 2023 09:23:17 GMT, Yi Yang wrote: >> ### Motivation and proposal >> Hi, heap dump brings about pauses for application's execution(STW), this is a well-known pain. JDK-8252842 have added parallel support to heapdump in an attempt to alleviate this issue. However, all concurrent threads competitively write heap data to the same file, and more memory is required to maintain the concurrent buffer queue. In experiments, we did not feel a significant performance improvement from that. >> >> The minor-pause solution, which is presented in this PR, is a two-stage segmented heap dump: >> >> 1. Stage One(STW): Concurrent threads directly write data to multiple heap files. >> 2. Stage Two(Non-STW): Merge multiple heap files into one complete heap dump file. >> >> Now concurrent worker threads are not required to maintain a buffer queue, which would result in more memory overhead, nor do they need to compete for locks. >> >> ### Performance evaluation >> | memory | numOfThread | STW | Total | Compression | >> | --- | --------- | -------------- | ------------ | -------- | >> | 8g | 1 thread | 15.612 secs | 15.612 secs | N | >> | 8g | 32 thread | 2.5617250 secs | 14.498 secs | N | >> | 8g | 32 thread | 2.3084878 secs | 3.198 secs | Compress1 | >> | 8g | 32 thread | 10.9355128 secs | 21.882 secs | Compress2 | >> | 8g | 96 thread | 2.6790452 secs | 14.012 secs | N | >> | 8g | 96 thread | 2.3044796 secs | 3.589 secs | Compress1 | >> | 8g | 96 thread | 9.7585151 secs | 20.219 secs| Compress2 | >> | 16g | 1 thread | 26.278 secs | 26.278 secs | N | >> | 16g | 32 thread | 5.2313740 secs | 26.417 secs | N | >> | 16g | 32 thread | 5.6946983 secs | 6.538 secs | Compress1 | >> | 16g | 32 thread | 21.8211105 secs | 41.133 secs | Compress2 | >> | 16g | 96 thread | 6.2445556 secs | 27.141 secs | N | >> | 16g | 96 thread | 4.6007096 secs | 6.259 secs | Compress1 | >> | 16g | 96 thread | 19.2965783 secs | 39.007 secs | Compress2 | >> | 32g | 1 thread | 48.149 secs | 48.149 secs | N | >> | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | >> | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | >> | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | >> | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | >> | 32g | 96 thread | 9.0954641 secs | 9.885 secs | Compress1 | >> | 32g | 96 thread | 38.9900931 secs | 80.574 secs | Compress2 | >> | 64g | 1 thread | 100.583 secs | 100.583 secs | N | >> | 64g | 32 thread | 20.9233744 secs | 134.701 secs | N | >> | 64g | 32 thread | 18.5023784 secs | 19.358 secs | Compre... > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > rename back to heapdumpCompression A lot of code movement is caused by moving AbstractDumpWriter class from heapDumper.cpp to heapDumpCompression.hpp/cpp I'm not happy with huge heapDumper.cpp file, but this refactoring does not look good to me. Currently all logic about hprof file structure is in heapDumper.cpp, it's not visible outside. With your fix the logic is spread in heapDumper.hpp/.cpp and heapDumpCompression.hpp/cpp ------------- PR Comment: https://git.openjdk.org/jdk/pull/13667#issuecomment-1577693210 From kdnilsen at openjdk.org Mon Jun 5 23:57:09 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 5 Jun 2023 23:57:09 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: <58eIIR00lhW278uv5z9Klo0SBJJWfn6D4JQFqgslqdE=.ca795de6-2480-4bb4-bbdc-12388f8fe388@github.com> On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. src/hotspot/share/gc/shenandoah/heuristics/shenandoahPassiveHeuristics.cpp line 3: > 1: /* > 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Copyright overreach. Revert. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1218719643 From sspitsyn at openjdk.org Tue Jun 6 00:03:03 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 00:03:03 GMT Subject: Integrated: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading In-Reply-To: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Wed, 31 May 2023 10:08:46 GMT, Serguei Spitsyn wrote: > The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. > > The CSR is: > [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading > > Testing: > - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` > - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced This pull request has now been integrated. Changeset: 4b153498 Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/4b1534989b39139aaa2952564e2b4098181a9eee Stats: 124 lines in 4 files changed: 117 ins; 4 del; 3 mod 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading Reviewed-by: cjplummer, alanb, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/14244 From cjplummer at openjdk.org Tue Jun 6 00:09:53 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 00:09:53 GMT Subject: RFR: 8309509: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java fails with virtual test thread factory Message-ID: The test fails with the virtual test thread factory because it tries to find the "main" thread in the list of threads returned by JDI, but "main" is a virtual thread and will only be returned by JDI if the debug agent is launched with includevirtualthreads=y. As a result the thread is not found and the test asserts: java.lang.RuntimeException: assertTrue: expected true, was false at jdk.test.lib.Asserts.fail(Asserts.java:594) at jdk.test.lib.Asserts.assertTrue(Asserts.java:486) at jdk.test.lib.Asserts.assertTrue(Asserts.java:472) at TestNestmateAttr.checkGoodTransforms(TestNestmateAttr.java:511) at TestNestmateAttr.methodEntered(TestNestmateAttr.java:320) at TestScaffold$EventHandler.notifyEvent(TestScaffold.java:205) at TestScaffold$EventHandler.run(TestScaffold.java:279) at java.base/java.lang.Thread.run(Thread.java:1583) The fix is to always run the debug agent with includevirtualthreads=y. Tested by running all com/sun/jdi tests locally with and without the virtual test thread factory. Also ran tier1 and tier5 svc test tasks. ------------- Commit messages: - Make sure we always run debug agent with includevirtualthreads=y Changes: https://git.openjdk.org/jdk/pull/14324/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14324&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309509 Stats: 7 lines in 2 files changed: 3 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14324.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14324/head:pull/14324 PR: https://git.openjdk.org/jdk/pull/14324 From amenkov at openjdk.org Tue Jun 6 00:12:52 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 00:12:52 GMT Subject: RFR: 8309505: com/sun/jdi/MethodEntryExitEvents.java due to finding wrong main thread In-Reply-To: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> References: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> Message-ID: On Mon, 5 Jun 2023 21:55:02 GMT, Chris Plummer wrote: > The test fails because it tries to determine the main debuggee thread by allowing it run until the debuggee class is loaded (it waits for the ClassPrepareEvent). Normally this would be done on the main debuggee thread. However, when using virtual threads, the main thread has yet to spawn the virtual thread to run the test on. The test is fixed by instead just waiting until the debuggee main method is entered. > > Tested by running the test locally with and without the virtual test thread wrapper, and also all of tier1 plus tier5 svc tests. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14322#pullrequestreview-1463861905 From sspitsyn at openjdk.org Tue Jun 6 00:15:04 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 00:15:04 GMT Subject: RFR: 8304438: jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading [v11] In-Reply-To: References: <9H36lIpFiwG4v0oGPE2zCIawMJcNIhuGguz--qxjRnQ=.0fc08ed3-2d12-42d0-8b6c-3fa801d55427@github.com> Message-ID: On Mon, 5 Jun 2023 19:13:30 GMT, Serguei Spitsyn wrote: >> The VM option EnableDynamicAgentLoading was added in JDK 9, default true, to allow deployment to choose whether to allow agents to be loaded/started in the VM. The VM option does the right thing for tools using the Attach API but jcmd JVMTI.agent_load was missed. This should be fixed to disallow loading JVMTI agents when the EnableDynamicAgentLoading is false. >> >> The CSR is: >> [JDK-8309250](https://bugs.openjdk.org/browse/JDK-8309250): jcmd JVMTI.agent_load should obey EnableDynamicAgentLoading >> >> Testing: >> - run new test `test/jdk/sun/tools/jcmd/TestJcmdNoAgentLoad.java` >> - TBD: submit mach5 tiers 1-5 to make sure no new regressions are introduced > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > removed an obsolete comment Alan, Chris and Alex, thank you for review! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14244#issuecomment-1577715724 From amenkov at openjdk.org Tue Jun 6 00:17:52 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 00:17:52 GMT Subject: RFR: 8309506: com/sun/jdi/MultiBreakpointsTest.java fails with virtual test thread factory In-Reply-To: References: Message-ID: <1FarD5pLXkE9_NmVBgprY572HBMvpRTDdvevHaQHRls=.40ed74a6-032f-4b26-a1b5-b5e82813564e@github.com> On Mon, 5 Jun 2023 22:13:29 GMT, Chris Plummer wrote: > The test fails when the main debuggee thread is a virtual thread, because virtual threads are always daemon threads. Because of this all the test threads that the debuggee creates are also daemon threads. The main deuggee thread immediately exits after creating the test threads, and at that point there are no non-daemon threads left running to keep the debuggee process alive, so the test threads never get a chance to do all the work that is expected of them. > > The fix is to use Thread.join() to prevent the main test thread from exiting until all the created tests threads have exited. I also updated the test so when run using the virtual test thread factory, the debuggee test threads will now be virtual threads. Previously just the main debuggee thread would be a virtual thread. This is not a bug fix, but does provide better virtual thread testing. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14323#pullrequestreview-1463871332 From cjplummer at openjdk.org Tue Jun 6 00:39:59 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 00:39:59 GMT Subject: Withdrawn: 8309397: com/sun/jdi/JdbXXX tests fail due to not being run with -Djdk.trackAllThreads In-Reply-To: References: Message-ID: <8IilWRrlJ6j7KLI-7NAIU5MoqAHszkXn04Rtv_yax2w=.3e732bd9-1521-4021-95b6-b5dcab38cbc3@github.com> On Fri, 2 Jun 2023 21:41:32 GMT, Chris Plummer wrote: > The com/sun/jdi/JdbXXX tests rely on the jdb "threads" command output to find the main thread. If it is a virtual thread, it will not be included in the "threads" output unless the debuggee is run with -Djdk.trackAllThreads, so we need to make sure to include this option when launching the debuggee. The following tests are impacted. > > com/sun/jdi/JdbMethodExitTest.java > com/sun/jdi/JdbStepTest.java > com/sun/jdi/JdbStopThreadTest.java > com/sun/jdi/JdbStopThreadidTest.java > > Note that all these tests also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which needs to be fixed first. Also JdbMethodExitTest.java will fail due to [JDK-8309396](https://bugs.openjdk.org/browse/JDK-8309396), which should be fixed after this CR. > > I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the 3 tests being removed from the problem list by this PR also pass. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/14293 From cjplummer at openjdk.org Tue Jun 6 00:44:08 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 00:44:08 GMT Subject: RFR: 8309510: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java no longer needs to override startup() method Message-ID: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java currently overrides the TestScaffold.startup() method: // override this to correct a bug so arguments can be passed to // the Target class protected void startUp(String targetName) { List argList = new ArrayList<>(Arrays.asList(args)); argList.add(0, targetName); // pre-pend so it becomes the first "app" arg println("run args: " + argList); connect((String[]) argList.toArray(args)); waitForVMStart(); } This issue of passing app args was fixed recently by [JDK-8308481](https://bugs.openjdk.org/browse/JDK-8308481), so the override is no longer needed. ------------- Commit messages: - No longer need to override starupt() after JDK-8308481 fix. Changes: https://git.openjdk.org/jdk/pull/14325/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14325&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309510 Stats: 10 lines in 1 file changed: 0 ins; 10 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14325.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14325/head:pull/14325 PR: https://git.openjdk.org/jdk/pull/14325 From sspitsyn at openjdk.org Tue Jun 6 01:34:26 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 01:34:26 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread Message-ID: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `GetSingleStackTraceClosure` and `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. A minor refactoring of the `GetSingleStackTraceClosure` was made to fix the issue. Also, a new test was added to provide coverage: - `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` Testing: - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` - TBD: tiers 1-6 ------------- Commit messages: - remove a trailing space in new ThreadListStackTracesTest.java - 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread Changes: https://git.openjdk.org/jdk/pull/14326/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8295976 Stats: 241 lines in 5 files changed: 224 ins; 10 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/14326.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14326/head:pull/14326 PR: https://git.openjdk.org/jdk/pull/14326 From yyang at openjdk.org Tue Jun 6 01:49:54 2023 From: yyang at openjdk.org (Yi Yang) Date: Tue, 6 Jun 2023 01:49:54 GMT Subject: RFR: JDK-8306441: Enhance parallel heap dump [v6] In-Reply-To: References: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> <4O3fdBsNuvwrymizPAQ1aXDMBVkHo9n841Ez5NqgY98=.28c881c8-2ae0-4736-bb25-7422ff9f60a3@github.com> Message-ID: On Mon, 5 Jun 2023 23:44:04 GMT, Alex Menkov wrote: > A lot of code movement is caused by moving AbstractDumpWriter class from heapDumper.cpp to heapDumpCompression.hpp/cpp > I'm not happy with huge heapDumper.cpp file, but this refactoring does not look good to me. > Currently all logic about hprof file structure is in heapDumper.cpp, it's not visible outside. > With your fix the logic is spread in heapDumper.hpp/.cpp and heapDumpCompression.hpp/cpp I will undo all refactor code. Let us firstly focuse on proposal itself. The refactor can be discussed later. ------------- PR Comment: https://git.openjdk.org/jdk/pull/13667#issuecomment-1577770928 From yyang at openjdk.org Tue Jun 6 02:22:33 2023 From: yyang at openjdk.org (Yi Yang) Date: Tue, 6 Jun 2023 02:22:33 GMT Subject: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v7] In-Reply-To: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> References: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> Message-ID: > ### Motivation and proposal > Hi, heap dump brings about pauses for application's execution(STW), this is a well-known pain. JDK-8252842 have added parallel support to heapdump in an attempt to alleviate this issue. However, all concurrent threads competitively write heap data to the same file, and more memory is required to maintain the concurrent buffer queue. In experiments, we did not feel a significant performance improvement from that. > > The minor-pause solution, which is presented in this PR, is a two-stage segmented heap dump: > > 1. Stage One(STW): Concurrent threads directly write data to multiple heap files. > 2. Stage Two(Non-STW): Merge multiple heap files into one complete heap dump file. > > Now concurrent worker threads are not required to maintain a buffer queue, which would result in more memory overhead, nor do they need to compete for locks. > > ### Performance evaluation > | memory | numOfThread | STW | Total | Compression | > | --- | --------- | -------------- | ------------ | -------- | > | 8g | 1 thread | 15.612 secs | 15.612 secs | N | > | 8g | 32 thread | 2.5617250 secs | 14.498 secs | N | > | 8g | 32 thread | 2.3084878 secs | 3.198 secs | Compress1 | > | 8g | 32 thread | 10.9355128 secs | 21.882 secs | Compress2 | > | 8g | 96 thread | 2.6790452 secs | 14.012 secs | N | > | 8g | 96 thread | 2.3044796 secs | 3.589 secs | Compress1 | > | 8g | 96 thread | 9.7585151 secs | 20.219 secs| Compress2 | > | 16g | 1 thread | 26.278 secs | 26.278 secs | N | > | 16g | 32 thread | 5.2313740 secs | 26.417 secs | N | > | 16g | 32 thread | 5.6946983 secs | 6.538 secs | Compress1 | > | 16g | 32 thread | 21.8211105 secs | 41.133 secs | Compress2 | > | 16g | 96 thread | 6.2445556 secs | 27.141 secs | N | > | 16g | 96 thread | 4.6007096 secs | 6.259 secs | Compress1 | > | 16g | 96 thread | 19.2965783 secs | 39.007 secs | Compress2 | > | 32g | 1 thread | 48.149 secs | 48.149 secs | N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.0954641 secs | 9.885 secs | Compress1 | > | 32g | 96 thread | 38.9900931 secs | 80.574 secs | Compress2 | > | 64g | 1 thread | 100.583 secs | 100.583 secs | N | > | 64g | 32 thread | 20.9233744 secs | 134.701 secs | N | > | 64g | 32 thread | 18.5023784 secs | 19.358 secs | Compress1 | > | 64g | 32 thread | 86.4748377 ... Yi Yang has updated the pull request incrementally with one additional commit since the last revision: undo refactor ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13667/files - new: https://git.openjdk.org/jdk/pull/13667/files/7485e6a7..1432cdaa Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13667&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13667&range=05-06 Stats: 1332 lines in 4 files changed: 651 ins; 654 del; 27 mod Patch: https://git.openjdk.org/jdk/pull/13667.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13667/head:pull/13667 PR: https://git.openjdk.org/jdk/pull/13667 From yyang at openjdk.org Tue Jun 6 02:35:30 2023 From: yyang at openjdk.org (Yi Yang) Date: Tue, 6 Jun 2023 02:35:30 GMT Subject: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v8] In-Reply-To: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> References: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> Message-ID: > ### Motivation and proposal > Hi, heap dump brings about pauses for application's execution(STW), this is a well-known pain. JDK-8252842 have added parallel support to heapdump in an attempt to alleviate this issue. However, all concurrent threads competitively write heap data to the same file, and more memory is required to maintain the concurrent buffer queue. In experiments, we did not feel a significant performance improvement from that. > > The minor-pause solution, which is presented in this PR, is a two-stage segmented heap dump: > > 1. Stage One(STW): Concurrent threads directly write data to multiple heap files. > 2. Stage Two(Non-STW): Merge multiple heap files into one complete heap dump file. > > Now concurrent worker threads are not required to maintain a buffer queue, which would result in more memory overhead, nor do they need to compete for locks. > > ### Performance evaluation > | memory | numOfThread | STW | Total | Compression | > | --- | --------- | -------------- | ------------ | -------- | > | 8g | 1 thread | 15.612 secs | 15.612 secs | N | > | 8g | 32 thread | 2.5617250 secs | 14.498 secs | N | > | 8g | 32 thread | 2.3084878 secs | 3.198 secs | Compress1 | > | 8g | 32 thread | 10.9355128 secs | 21.882 secs | Compress2 | > | 8g | 96 thread | 2.6790452 secs | 14.012 secs | N | > | 8g | 96 thread | 2.3044796 secs | 3.589 secs | Compress1 | > | 8g | 96 thread | 9.7585151 secs | 20.219 secs| Compress2 | > | 16g | 1 thread | 26.278 secs | 26.278 secs | N | > | 16g | 32 thread | 5.2313740 secs | 26.417 secs | N | > | 16g | 32 thread | 5.6946983 secs | 6.538 secs | Compress1 | > | 16g | 32 thread | 21.8211105 secs | 41.133 secs | Compress2 | > | 16g | 96 thread | 6.2445556 secs | 27.141 secs | N | > | 16g | 96 thread | 4.6007096 secs | 6.259 secs | Compress1 | > | 16g | 96 thread | 19.2965783 secs | 39.007 secs | Compress2 | > | 32g | 1 thread | 48.149 secs | 48.149 secs | N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.0954641 secs | 9.885 secs | Compress1 | > | 32g | 96 thread | 38.9900931 secs | 80.574 secs | Compress2 | > | 64g | 1 thread | 100.583 secs | 100.583 secs | N | > | 64g | 32 thread | 20.9233744 secs | 134.701 secs | N | > | 64g | 32 thread | 18.5023784 secs | 19.358 secs | Compress1 | > | 64g | 32 thread | 86.4748377 ... Yi Yang has updated the pull request incrementally with one additional commit since the last revision: undo refactor -- done ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13667/files - new: https://git.openjdk.org/jdk/pull/13667/files/1432cdaa..c8e80e9e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13667&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13667&range=06-07 Stats: 53 lines in 4 files changed: 19 ins; 23 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/13667.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13667/head:pull/13667 PR: https://git.openjdk.org/jdk/pull/13667 From sspitsyn at openjdk.org Tue Jun 6 03:09:52 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 03:09:52 GMT Subject: RFR: 8309509: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java fails with virtual test thread factory In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 00:02:47 GMT, Chris Plummer wrote: > The test fails with the virtual test thread factory because it tries to find the "main" thread in the list of threads returned by JDI, but "main" is a virtual thread and will only be returned by JDI if the debug agent is launched with includevirtualthreads=y. As a result the thread is not found and the test asserts: > > java.lang.RuntimeException: assertTrue: expected true, was false > at jdk.test.lib.Asserts.fail(Asserts.java:594) > at jdk.test.lib.Asserts.assertTrue(Asserts.java:486) > at jdk.test.lib.Asserts.assertTrue(Asserts.java:472) > at TestNestmateAttr.checkGoodTransforms(TestNestmateAttr.java:511) > at TestNestmateAttr.methodEntered(TestNestmateAttr.java:320) > at TestScaffold$EventHandler.notifyEvent(TestScaffold.java:205) > at TestScaffold$EventHandler.run(TestScaffold.java:279) > at java.base/java.lang.Thread.run(Thread.java:1583) > > The fix is to always run the debug agent with includevirtualthreads=y. > > Tested by running all com/sun/jdi tests locally with and without the virtual test thread factory. Also ran tier1 and tier5 svc test tasks. Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14324#pullrequestreview-1464169042 From sspitsyn at openjdk.org Tue Jun 6 03:21:53 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 03:21:53 GMT Subject: RFR: 8309505: com/sun/jdi/MethodEntryExitEvents.java due to finding wrong main thread In-Reply-To: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> References: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> Message-ID: On Mon, 5 Jun 2023 21:55:02 GMT, Chris Plummer wrote: > The test fails because it tries to determine the main debuggee thread by allowing it run until the debuggee class is loaded (it waits for the ClassPrepareEvent). Normally this would be done on the main debuggee thread. However, when using virtual threads, the main thread has yet to spawn the virtual thread to run the test on. The test is fixed by instead just waiting until the debuggee main method is entered. > > Tested by running the test locally with and without the virtual test thread wrapper, and also all of tier1 plus tier5 svc tests. Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14322#pullrequestreview-1464181798 From sspitsyn at openjdk.org Tue Jun 6 04:06:52 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 04:06:52 GMT Subject: RFR: 8309510: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java no longer needs to override startup() method In-Reply-To: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> References: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> Message-ID: On Tue, 6 Jun 2023 00:36:12 GMT, Chris Plummer wrote: > com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java currently overrides the TestScaffold.startup() method: > > // override this to correct a bug so arguments can be passed to > // the Target class > protected void startUp(String targetName) { > List argList = new ArrayList<>(Arrays.asList(args)); > argList.add(0, targetName); // pre-pend so it becomes the first "app" arg > println("run args: " + argList); > connect((String[]) argList.toArray(args)); > waitForVMStart(); > } > > This issue of passing app args was fixed recently by [JDK-8308481](https://bugs.openjdk.org/browse/JDK-8308481), so the override is no longer needed. Looks good in general. The test is in the `test/jdk/ProblemList-Virtual.txt` marked with the bug 8285422 which very general. Is it a good time to remove it from the problem list? Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14325#pullrequestreview-1464220984 From sspitsyn at openjdk.org Tue Jun 6 04:10:54 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 04:10:54 GMT Subject: RFR: 8309510: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java no longer needs to override startup() method In-Reply-To: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> References: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> Message-ID: On Tue, 6 Jun 2023 00:36:12 GMT, Chris Plummer wrote: > com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java currently overrides the TestScaffold.startup() method: > > // override this to correct a bug so arguments can be passed to > // the Target class > protected void startUp(String targetName) { > List argList = new ArrayList<>(Arrays.asList(args)); > argList.add(0, targetName); // pre-pend so it becomes the first "app" arg > println("run args: " + argList); > connect((String[]) argList.toArray(args)); > waitForVMStart(); > } > > This issue of passing app args was fixed recently by [JDK-8308481](https://bugs.openjdk.org/browse/JDK-8308481), so the override is no longer needed. Marked as reviewed by sspitsyn (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14325#pullrequestreview-1464224287 From alanb at openjdk.org Tue Jun 6 06:05:00 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 6 Jun 2023 06:05:00 GMT Subject: RFR: 8309406: Change jdk.trackAllThreads to default to true In-Reply-To: References: Message-ID: <0Y9RTCgPVUDGR8NHe925jykMIGN26CoJZcJIw9Qn4Vc=.60a9624d-bec0-4f97-9c76-9f61178557af@github.com> On Sat, 3 Jun 2023 13:41:53 GMT, Alan Bateman wrote: > Virtual threads created directly with the Thread API are not included in the new thread dump by default. This was a source of confusion in JDK 19/20 when virtual threads were in preview. We have decided to switch the default so that all virtual threads are observable. Future work will reduce/remove the overhead of tracking threads in the root container. The change proposed here has no impact to virtual threads created with the virtual ThreadPerTaskExecutor or StructuredTaskScope. > > With jdk.trackAllThreads defaulting to true then the need for SharedThreadContainer to count threads can go away, as does the need for RootContainer::threads to return virtual threads blocked on I/O. > > Most of the changes are to the test HotSpotDiagnosticMXBean/DumpThreads.java as this is expanded to cover more cases, and specifically more cases with thread groupings that correspond to the ThreadPerTaskExecutor and thread pools. > > Testing: Test1-5 Thanks for reviews. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14300#issuecomment-1577964177 From alanb at openjdk.org Tue Jun 6 06:05:02 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 6 Jun 2023 06:05:02 GMT Subject: Integrated: 8309406: Change jdk.trackAllThreads to default to true In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 13:41:53 GMT, Alan Bateman wrote: > Virtual threads created directly with the Thread API are not included in the new thread dump by default. This was a source of confusion in JDK 19/20 when virtual threads were in preview. We have decided to switch the default so that all virtual threads are observable. Future work will reduce/remove the overhead of tracking threads in the root container. The change proposed here has no impact to virtual threads created with the virtual ThreadPerTaskExecutor or StructuredTaskScope. > > With jdk.trackAllThreads defaulting to true then the need for SharedThreadContainer to count threads can go away, as does the need for RootContainer::threads to return virtual threads blocked on I/O. > > Most of the changes are to the test HotSpotDiagnosticMXBean/DumpThreads.java as this is expanded to cover more cases, and specifically more cases with thread groupings that correspond to the ThreadPerTaskExecutor and thread pools. > > Testing: Test1-5 This pull request has now been integrated. Changeset: 2e9eff56 Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/2e9eff56418273e85accc43dcef533995c6be8bf Stats: 319 lines in 6 files changed: 183 ins; 81 del; 55 mod 8309406: Change jdk.trackAllThreads to default to true Reviewed-by: rpressler, mchung, cjplummer ------------- PR: https://git.openjdk.org/jdk/pull/14300 From sspitsyn at openjdk.org Tue Jun 6 06:11:53 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 06:11:53 GMT Subject: RFR: 8309506: com/sun/jdi/MultiBreakpointsTest.java fails with virtual test thread factory In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 22:13:29 GMT, Chris Plummer wrote: > The test fails when the main debuggee thread is a virtual thread, because virtual threads are always daemon threads. Because of this all the test threads that the debuggee creates are also daemon threads. The main deuggee thread immediately exits after creating the test threads, and at that point there are no non-daemon threads left running to keep the debuggee process alive, so the test threads never get a chance to do all the work that is expected of them. > > The fix is to use Thread.join() to prevent the main test thread from exiting until all the created tests threads have exited. I also updated the test so when run using the virtual test thread factory, the debuggee test threads will now be virtual threads. Previously just the main debuggee thread would be a virtual thread. This is not a bug fix, but does provide better virtual thread testing. Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14323#pullrequestreview-1464335395 From dholmes at openjdk.org Tue Jun 6 07:32:53 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 6 Jun 2023 07:32:53 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread In-Reply-To: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: On Tue, 6 Jun 2023 00:50:34 GMT, Serguei Spitsyn wrote: > The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. > The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `GetSingleStackTraceClosure` and `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. > > Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. > A minor refactoring of the `GetSingleStackTraceClosure` was made to fix the issue. > > Also, a new test was added to provide coverage: > - `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > > Testing: > - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > - TBD: tiers 1-6 Just a passing comment but I happened to notice today that when a virtual thread blocks on a legacy synchronization mechanism, it delegates to its carrier thread to report its state. It is not at all clear to me how this is handled at the JVMTI level. ------------- PR Review: https://git.openjdk.org/jdk/pull/14326#pullrequestreview-1464472318 From alanb at openjdk.org Tue Jun 6 07:42:52 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 6 Jun 2023 07:42:52 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread In-Reply-To: References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: <4CPJ19jOAYq8cQqJWqGRgwO-kbLv0glRGTna0FY4jtE=.c1ca455b-80f6-47e9-ba37-1db3b33bb95c@github.com> On Tue, 6 Jun 2023 07:30:26 GMT, David Holmes wrote: > Just a passing comment but I happened to notice today that when a virtual thread blocks on a legacy synchronization mechanism, it delegates to its carrier thread to report its state. It is not at all clear to me how this is handled at the JVMTI level. When mounted, the virtual thread state comes from its carrier. JVM TI GetThreadState and other functions that return state do the same. Somehow the bulk function GetThreadListStackTraces was missed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14326#issuecomment-1578097719 From duke at openjdk.org Tue Jun 6 09:58:54 2023 From: duke at openjdk.org (JoKern65) Date: Tue, 6 Jun 2023 09:58:54 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop Message-ID: The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. So we have (at least) 3 ways to circumvent this behavior. 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): `#include ` `static void` `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` `{` ` while (1) {` ` sleep(1);` ` }` `}` 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: `static void` `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` `{` ` volatile int i = 1;` ` while (i) {` ` i += 2;` ` }` `}` 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: `static void` `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` `{` ` while (1) {` ` }` `}` To make the third approach platform independent, we can implement it in the following way: In globalDefinitions.hpp `#ifndef OPTNONE` `#define OPTNONE` `#endif` In globalDefinitions_xlc.hpp `// optnone support` `//` `// To use if a function should not be optimized` `// Usage:` `// void* func(size_t size) OPTNONE {...}` `#define OPTNONE __attribute__(( optnone))`` With this we can change libagentthr001.cpp in a platform independent way to `static void` `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` `{` ` while (1) {` ` }` `}` ------------- Commit messages: - JDK-8309462 Changes: https://git.openjdk.org/jdk/pull/14330/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14330&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309462 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14330.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14330/head:pull/14330 PR: https://git.openjdk.org/jdk/pull/14330 From alanb at openjdk.org Tue Jun 6 10:12:54 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 6 Jun 2023 10:12:54 GMT Subject: RFR: 8309408: Thread.sleep cleanup In-Reply-To: References: Message-ID: <8l9FG4c1c4Ydlv3gQBiramw0YpSthxC-4wXOxVWiIOE=.b14f4319-2ecf-47ea-8118-5a3fe60d92c2@github.com> On Mon, 5 Jun 2023 20:05:24 GMT, Chris Plummer wrote: > The following commit in loom heavily modified this file with a lot of added expected methods. There are other related tests with similar changes. I'm not so sure I understand the need for so many additions, and also why expectedLength is so out of sync with the number of added method. I don't believe this commit was reviewed individually, but was just part of the overall loom review when merge into jdk. Perhaps it should be revisited. These tests aren't easy to read or maintain, it would be good to re-visit them. In some cases, the tests capture the stack trace asynchronously so the test needs to know about all code paths. As regards ThreadController, used by the nsk/monitoring/stress/thread/straceXXX tests, the main thread waits at a barrier (a CountDownLatch) until all sleeping threads are ready to sleep. Once the main thread is released, it checks all the sleepers are in TIMED_WAITING state and samples their stack traces with the ThreadMXBean and related APIs. The test fails if there are frames corresponding to methods that the test doesn't know about. If a thread is sleeping then we shouldn't see frames for beforeSleep/afterSlee. My reading of these tests is that the main thread could poll a SleepingThread after it counts down and before it parks in sleep. It's doing an expensive ThreadMXBean::getAllThreadIds once released and that may explain why it hasn't been seen. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14303#discussion_r1219348675 From mbaesken at openjdk.org Tue Jun 6 10:24:52 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Tue, 6 Jun 2023 10:24:52 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 09:51:09 GMT, JoKern65 wrote: > The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 > The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. > So we have (at least) 3 ways to circumvent this behavior. > > 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): > `#include ` > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` while (1) {` > ` sleep(1);` > ` }` > `}` > > 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` volatile int i = 1;` > ` while (i) {` > ` i += 2;` > ` }` > `}` > > 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` > `{` > ` while (1) {` > ` }` > `}` > > To make the third approach platform independent, we can implement it in the following way: > In globalDefinitions.hpp > `#ifndef OPTNONE` > `#define OPTNONE` > `#endif` > > In globalDefinitions_xlc.hpp > `// optnone support` > `//` > `// To use if a function should not be optimized` > `// Usage:` > `// void* func(size_t size) OPTNONE {...}` > `#define OPTNONE __attribute__(( optnone))`` > > With this we can change libagentthr001.cpp in a platform independent way to > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` > `{` > ` while (1) {` > ` }` > `}` looks okay to me ------------- Marked as reviewed by mbaesken (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14330#pullrequestreview-1464841555 From liach at openjdk.org Tue Jun 6 11:16:57 2023 From: liach at openjdk.org (Chen Liang) Date: Tue, 6 Jun 2023 11:16:57 GMT Subject: RFR: 8294977: Convert test/jdk/java tests from ASM library to Classfile API [v8] In-Reply-To: References: Message-ID: On Tue, 9 May 2023 04:17:28 GMT, Chen Liang wrote: >> Summaries: >> 1. A few recommendations about updating the constant API is made at https://mail.openjdk.org/pipermail/classfile-api-dev/2023-March/000233.html and I may update this patch shall the API changes be integrated before >> 2. One ASM library-specific test, `LambdaAsm` is removed. Others have their code generation infrastructure upgraded from ASM to Classfile API. >> 3. Most tests are included in tier1, but some are not: >> In `:jdk_io`: (tier2, part 2) >> >> test/jdk/java/io/Serializable/records/SerialPersistentFieldsTest.java >> test/jdk/java/io/Serializable/records/ProhibitedMethods.java >> test/jdk/java/io/Serializable/records/BadCanonicalCtrTest.java >> >> In `:jdk_instrument`: (tier 3) >> >> test/jdk/java/lang/instrument/RetransformAgent.java >> test/jdk/java/lang/instrument/NativeMethodPrefixAgent.java >> test/jdk/java/lang/instrument/asmlib/Instrumentor.java >> >> >> @asotona Would you mind reviewing? > > Chen Liang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits: > > - Merge branch 'master' into invoke-test-classfile > - Switch to ConstantDescs for and void constants > - Merge AnnotationsTest, remove ModuleTargetAttribute call > - Merge branch 'invoke-test-classfile' of https://github.com/liachmodded/jdk into invoke-test-classfile > - Update test/jdk/java/lang/invoke/8022701/MHIllegalAccess.java > > Co-authored-by: Andrey Turbanov > - Merge branch 'master' into invoke-test-classfile > - Fix LambdaStackTrace after running > - formatting > - Fix failed LambdaStackTrace test, use more convenient APIs > - Merge branch 'master' of https://git.openjdk.java.net/jdk into invoke-test-classfile > - ... and 2 more: https://git.openjdk.org/jdk/compare/d9052b94...5db1e957 keep-alive; this will be updated for the classfile object update. ------------- PR Comment: https://git.openjdk.org/jdk/pull/13009#issuecomment-1578507931 From mdoerr at openjdk.org Tue Jun 6 12:10:52 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 6 Jun 2023 12:10:52 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 09:51:09 GMT, JoKern65 wrote: > The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 > The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. > So we have (at least) 3 ways to circumvent this behavior. > > 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): > `#include ` > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` while (1) {` > ` sleep(1);` > ` }` > `}` > > 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` volatile int i = 1;` > ` while (i) {` > ` i += 2;` > ` }` > `}` > > 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` > `{` > ` while (1) {` > ` }` > `}` > > To make the third approach platform independent, we can implement it in the following way: > In globalDefinitions.hpp > `#ifndef OPTNONE` > `#define OPTNONE` > `#endif` > > In globalDefinitions_xlc.hpp > `// optnone support` > `//` > `// To use if a function should not be optimized` > `// Usage:` > `// void* func(size_t size) OPTNONE {...}` > `#define OPTNONE __attribute__(( optnone))`` > > With this we can change libagentthr001.cpp in a platform independent way to > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` > `{` > ` while (1) {` > ` }` > `}` LGTM. `sleep(1)` is my preferred solution. Thanks for fixing it! ------------- Marked as reviewed by mdoerr (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14330#pullrequestreview-1465026778 From stuefe at openjdk.org Tue Jun 6 12:44:13 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 6 Jun 2023 12:44:13 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. Hi @kdnilsen, I see that following settings changed default values for all of Shenandoah: - ShenandoahLearningSteps (was 10, now 5) - ShenandoahImmediateThreshold (was 90, now 70) - ShenandoahAdaptiveDecayFactor (was 0.5, now 0.1) - ShenandoahFullGCThreshold (was 3, now 64) Assuming that the behavior of legacy Shenandoah remains unchanged, I assume the switches are now handled differently to arrive at the same behavior. I see that we now have ShenandoahOOMGCRetries. Does the changed default for ShenandoahFullGCThreshold and this new ShenandoahOOMGCRetries switch mean the degeneration behavior of legacy Shenandoah did change? I think the general thrust of my questions is, you assured us that legacy Shenandoah will show the same behavior post-patch, but since the settings changed, I assume that the meaning of these settings did change. We will need to document these effects for users of legacy Shenandoah, in case they need to translate existing settings in their environment. A release note would be really helpful. Cheers, Thomas ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1578693858 From mdoerr at openjdk.org Tue Jun 6 12:58:56 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 6 Jun 2023 12:58:56 GMT Subject: RFR: JDK-8309225: Fix xlc17 clang 15 warnings in security and servicability In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 10:19:53 GMT, JoKern65 wrote: > This pr is a split off from JDK-8308288: Fix xlc17 clang warnings in shared code https://github.com/openjdk/jdk/pull/14146 > It handles the part in security and servicability. > > Compiling on AIX with xlc17 which contains the new clang 15 frontend shows the following warnings: > > src/java.security.jgss/share/native/libj2gss/NativeUtil.h:30: > src/java.security.jgss/share/native/libj2gss/gssapi.h:48:5: error: 'TARGET_OS_MAC' is not defined, evaluates to 0 [-Werror,-Wundef] > #if TARGET_OS_MAC && (defined(ppc) || defined(ppc64) || defined(i386) || defined(x86_64)) > ^ > TARGET_OS_MAC is not defined. Instead of disabling the warning, I could > ` #ifndef TARGET_OS_MAC` > `#define TARGET_OS_MAC=0` > `#endif` > But this is already handled by disabling the warning for gcc. > > src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:718:33: error: suggest braces around initialization of subobject [-Werror,-Wmissing-braces] > struct in6_addr mappedAny = IN6ADDR_ANY_INIT; > ^~~~~~~~~~~~~~~~ > /usr/include/netinet/in.h:454:32: note: expanded from macro 'IN6ADDR_ANY_INIT' > #define IN6ADDR_ANY_INIT {0, 0, 0, 0} LGTM. ------------- Marked as reviewed by mdoerr (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14282#pullrequestreview-1465140605 From sspitsyn at openjdk.org Tue Jun 6 13:31:11 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 13:31:11 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v2] In-Reply-To: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: > The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. > The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `GetSingleStackTraceClosure` and `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. > > Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. > A minor refactoring of the `GetSingleStackTraceClosure` was made to fix the issue. > > Also, a new test was added to provide coverage: > - `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > > Testing: > - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > - TBD: tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: simplify GetSingleStackTraceClosure, fix issue in VM_GetThreadListStackTraces::doit, improve test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14326/files - new: https://git.openjdk.org/jdk/pull/14326/files/77718470..4e794bd5 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=00-01 Stats: 27 lines in 4 files changed: 11 ins; 5 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/14326.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14326/head:pull/14326 PR: https://git.openjdk.org/jdk/pull/14326 From sspitsyn at openjdk.org Tue Jun 6 13:37:03 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 13:37:03 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v3] In-Reply-To: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: > The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. > The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `GetSingleStackTraceClosure` and `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. > > Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. > A minor refactoring of the `GetSingleStackTraceClosure` was made to fix the issue. > > Also, a new test was added to provide coverage: > - `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > > Testing: > - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > - TBD: tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with two additional commits since the last revision: - fixed typo in a comment in jvmtiEnvBase.cpp - nit: restored one comment as was before ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14326/files - new: https://git.openjdk.org/jdk/pull/14326/files/4e794bd5..d20e1221 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=01-02 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14326.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14326/head:pull/14326 PR: https://git.openjdk.org/jdk/pull/14326 From kdnilsen at openjdk.org Tue Jun 6 13:48:18 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 13:48:18 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: <28NBsNni8p68iy2K_TjXIC1oScdrAtyK-NJd5OH0gVc=.c4adc4a5-82c9-4afa-b1a9-9e5b99b2ecdd@github.com> On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. Thanks Thomas for the feedback: These proposed changes represent improvements to both Generational and Non-generational modes of operation. We can revert if that is desired, or we can specialize Generational versions of these parameters so that they can have different values in different modes, but here is a bit of background. We've done considerable testing on a variety of synthetic workloads and some limited testing on production workloads. As we move towards upstream integration, we expect this will help us gain exposure to more production workloads. The following changes were based on results of this testing: * Decrease ShenandoahLearningSteps to 5 (from 10): For some workloads, we observed that there were "way too many" learning cycles being triggered. We also observed that the learning achieved during learning cycles was not as trustworthy as the learning achieved during actual operation, because these learning cycles typically trigger during initialization phases which are not representative of real-world operation and because they usually trigger so prematurely that there has not been enough time for allocated objects to die before we garbage collect. * Change ShenandoahImmediateThreshold to 70 from 90: We discovered during experiments with settings on certain real production workloads that reducing the threshold for abbreviated cycles significantly improved throughput, reduced degenerated cycles, and reduced high percentile end-to-end latency on the relevant services. These experiments were based on single-generation Shenandoah. We saw no negative impact of making this change on our various workloads. * I'll let @earthling-amzn comment on the change to ShenandoahAdaptiveDecayFactor. My recollection is that this change was also motivated by experience with single-generation Shenandoah on a real production workload. * The change of ShenandoahFullGCThreshold from 3 to 64 was motivated by some observations with specjbb performance as it ratchets up the workload to determine MaxJOPS. We observed that for both single-generation Shenandoah and generational Shenandoah, the typical behavior was that a single Full GC trigger causes an "infinite" sequence of Full GC, even though we may have only lost the concurrent GC race by a small amount. This is because (1) Full GC discards all the incremental work of the concurrent GC that was just interrupted, (2) STW Full GC creates a situation in which pent up demand for execution and allocation accumulates during the STW pause so there's a huge demand for allocation immediately following the end of Full GC, (3) The concurrent GC that triggers immediately after Full GC completes is "destined" to fail because no garbage has been introduced since Full GC finished and since SATB does not collect floating garbage that accumulates after the start of concurrent GC a nd since the allocation spike is so high immediately following the Full GC (e.g. 11GB/s instead of 3GB/s normally). This change allows a sequence of degenerated GCs to manage slow evolution and sudden bursts of allocation rate much more effectively than the original code. This is accompanied by a change in how we detect and throw OOM. We wait for at least one Full GC but we don't force ShenandoahFullGCThreshold allocation failures before thowing OOM. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1578800487 From jwaters at openjdk.org Tue Jun 6 15:37:00 2023 From: jwaters at openjdk.org (Julian Waters) Date: Tue, 6 Jun 2023 15:37:00 GMT Subject: RFR: 8305341: Alignment should be enforced by alignas instead of compiler specific attributes [v4] In-Reply-To: References: <2d60fxZxeWZEngMaSE1N4JZz07XkvbXj8jrN_hMbo-0=.51ffb82f-2beb-43f7-9195-062555599d0b@github.com> Message-ID: On Wed, 12 Apr 2023 07:12:10 GMT, Julian Waters wrote: >> C11 has been stable for a long time on all platforms, so native code can use the standard alignas operator for alignment requirements > > Julian Waters has updated the pull request incrementally with four additional commits since the last revision: > > - Restore visCPP > - Restore gcc attribute > - Revert gcc > - Revert Bumping ------------- PR Comment: https://git.openjdk.org/jdk/pull/13258#issuecomment-1578994674 From wkemper at openjdk.org Tue Jun 6 15:44:15 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 6 Jun 2023 15:44:15 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. Lowering `ShenandoahAdaptiveDecayFactor` allows the heuristic to give more weight to older samples of the allocation rate and cycle times. We found that with the original value (0.5), the heuristics would "forget" history too soon. With the original value, the heuristics were more likely to mistime their trigger because of a few recent, short cycles. This was particularly true after we lowered `ShenandoahImmediateThreshold`, which resulted in more cycles which could skip evacuation and updating refs. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1579008730 From alanb at openjdk.org Tue Jun 6 15:45:57 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 6 Jun 2023 15:45:57 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v2] In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 21:26:39 GMT, Serguei Spitsyn wrote: > Okay, I see you point. Unfortunately, I've always referred the platform thread with an executed FJP schedular as a carrier thread. The term 'carrier' with this meaning is everywhere in the JVMTI code. It looks very confusing to call a thread to be a carrier thread only during some phases of its execution. Okay, I'm just pointing out that is_passive_carrier_thread is confusing looks a bit strange here as the is testing if a JavaThread is carrying a virtual thread oop - it's not testing if the thread is owned by the virtual thread scheduler. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1219888219 From asemenov at openjdk.org Tue Jun 6 17:35:34 2023 From: asemenov at openjdk.org (Artem Semenov) Date: Tue, 6 Jun 2023 17:35:34 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v3] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 15:04:09 GMT, Weijun Wang wrote: >> done > > I didn't ask to revert the change. It's `s/TARGET_OS_MAC/defined(__APPLE__)/`. This is rarely used in the code and is not the essence of the current changes. If you introduce such changes, then throughout the code. Moreover, this can lead to problems, such as, for example, here: https://bugs.openjdk.org/browse/JDK-8309225 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14033#discussion_r1220056527 From weijun at openjdk.org Tue Jun 6 17:50:00 2023 From: weijun at openjdk.org (Weijun Wang) Date: Tue, 6 Jun 2023 17:50:00 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v3] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 17:32:35 GMT, Artem Semenov wrote: >> I didn't ask to revert the change. It's `s/TARGET_OS_MAC/defined(__APPLE__)/`. > > This is rarely used in the code and is not the essence of the current changes. > If you introduce such changes, then throughout the code. > Moreover, this can lead to problems, such as, for example, here: https://bugs.openjdk.org/browse/JDK-8309225 I'm not a clang expect, I was just asking if modifying the current `#if TARGET_OS_MAC` check into `#if defined(__APPLE__)` is also a solution. The comment on lines 46-47 says the condition was copied from a macOS SDK file and that's what the file is using now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14033#discussion_r1220071779 From cjplummer at openjdk.org Tue Jun 6 18:20:08 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 18:20:08 GMT Subject: RFR: 8309506: com/sun/jdi/MultiBreakpointsTest.java fails with virtual test thread factory In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 22:13:29 GMT, Chris Plummer wrote: > The test fails when the main debuggee thread is a virtual thread, because virtual threads are always daemon threads. Because of this all the test threads that the debuggee creates are also daemon threads. The main deuggee thread immediately exits after creating the test threads, and at that point there are no non-daemon threads left running to keep the debuggee process alive, so the test threads never get a chance to do all the work that is expected of them. > > The fix is to use Thread.join() to prevent the main test thread from exiting until all the created tests threads have exited. I also updated the test so when run using the virtual test thread factory, the debuggee test threads will now be virtual threads. Previously just the main debuggee thread would be a virtual thread. This is not a bug fix, but does provide better virtual thread testing. Thanks for the reviews Alex and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14323#issuecomment-1579243690 From cjplummer at openjdk.org Tue Jun 6 18:20:11 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 18:20:11 GMT Subject: Integrated: 8309506: com/sun/jdi/MultiBreakpointsTest.java fails with virtual test thread factory In-Reply-To: References: Message-ID: <1UoNIjR2ZKcfI8wUHZuermi6dRtmQOih1tLTvMrJt90=.cc498b09-8c37-4aa5-a41f-e31737c4884f@github.com> On Mon, 5 Jun 2023 22:13:29 GMT, Chris Plummer wrote: > The test fails when the main debuggee thread is a virtual thread, because virtual threads are always daemon threads. Because of this all the test threads that the debuggee creates are also daemon threads. The main deuggee thread immediately exits after creating the test threads, and at that point there are no non-daemon threads left running to keep the debuggee process alive, so the test threads never get a chance to do all the work that is expected of them. > > The fix is to use Thread.join() to prevent the main test thread from exiting until all the created tests threads have exited. I also updated the test so when run using the virtual test thread factory, the debuggee test threads will now be virtual threads. Previously just the main debuggee thread would be a virtual thread. This is not a bug fix, but does provide better virtual thread testing. This pull request has now been integrated. Changeset: 571fbdc3 Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/571fbdc3110440ec3a36bb6005dc5a0358696df5 Stats: 16 lines in 2 files changed: 10 ins; 2 del; 4 mod 8309506: com/sun/jdi/MultiBreakpointsTest.java fails with virtual test thread factory Reviewed-by: amenkov, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/14323 From cjplummer at openjdk.org Tue Jun 6 18:31:54 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 18:31:54 GMT Subject: RFR: 8309505: com/sun/jdi/MethodEntryExitEvents.java due to finding wrong main thread In-Reply-To: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> References: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> Message-ID: On Mon, 5 Jun 2023 21:55:02 GMT, Chris Plummer wrote: > The test fails because it tries to determine the main debuggee thread by allowing it run until the debuggee class is loaded (it waits for the ClassPrepareEvent). Normally this would be done on the main debuggee thread. However, when using virtual threads, the main thread has yet to spawn the virtual thread to run the test on. The test is fixed by instead just waiting until the debuggee main method is entered. > > Tested by running the test locally with and without the virtual test thread wrapper, and also all of tier1 plus tier5 svc tests. Thanks for the reviews Alex and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14322#issuecomment-1579258734 From cjplummer at openjdk.org Tue Jun 6 18:38:17 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 18:38:17 GMT Subject: Integrated: 8309505: com/sun/jdi/MethodEntryExitEvents.java due to finding wrong main thread In-Reply-To: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> References: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> Message-ID: On Mon, 5 Jun 2023 21:55:02 GMT, Chris Plummer wrote: > The test fails because it tries to determine the main debuggee thread by allowing it run until the debuggee class is loaded (it waits for the ClassPrepareEvent). Normally this would be done on the main debuggee thread. However, when using virtual threads, the main thread has yet to spawn the virtual thread to run the test on. The test is fixed by instead just waiting until the debuggee main method is entered. > > Tested by running the test locally with and without the virtual test thread wrapper, and also all of tier1 plus tier5 svc tests. This pull request has now been integrated. Changeset: 16ab7bfe Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/16ab7bfe22b96ec8c4b7b7197d55fa41d36e3875 Stats: 7 lines in 2 files changed: 1 ins; 1 del; 5 mod 8309505: com/sun/jdi/MethodEntryExitEvents.java due to finding wrong main thread Reviewed-by: amenkov, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/14322 From cjplummer at openjdk.org Tue Jun 6 18:38:16 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 18:38:16 GMT Subject: RFR: 8309505: com/sun/jdi/MethodEntryExitEvents.java due to finding wrong main thread [v2] In-Reply-To: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> References: <5zpDBAYAdkyhhkEzww1cgfUqHLhb7BFbd1fosEydwoQ=.4d263e58-ef38-4430-8ff0-722dec02c722@github.com> Message-ID: > The test fails because it tries to determine the main debuggee thread by allowing it run until the debuggee class is loaded (it waits for the ClassPrepareEvent). Normally this would be done on the main debuggee thread. However, when using virtual threads, the main thread has yet to spawn the virtual thread to run the test on. The test is fixed by instead just waiting until the debuggee main method is entered. > > Tested by running the test locally with and without the virtual test thread wrapper, and also all of tier1 plus tier5 svc tests. Chris Plummer has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - Merge - Fix virtual thread testing issue by waiting until debuggee main method is entered before gleaning the main thread. ------------- Changes: https://git.openjdk.org/jdk/pull/14322/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14322&range=01 Stats: 7 lines in 2 files changed: 1 ins; 1 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/14322.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14322/head:pull/14322 PR: https://git.openjdk.org/jdk/pull/14322 From sspitsyn at openjdk.org Tue Jun 6 18:42:59 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 18:42:59 GMT Subject: RFR: 8309396: com/sun/jdi/JdbMethodExitTest.java fails with virtual threads due to a bug in determining the main thread id [v2] In-Reply-To: References: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> Message-ID: On Sun, 4 Jun 2023 18:47:13 GMT, Chris Plummer wrote: >> JdbMethodExitTest.java tries to determine the jdb threadID for the "main" thread, and then later use it in jdb commands that require a threadID. It does this by first having the debuggee execute the following: >> >> ` System.out.println("threadid="+Thread.currentThread().getId());` >> >> And then later on the test parses the threadID from this output. The problem is that the id returned by getId() has no relation to threadIDs used by jdb, which are actually JDWP ObjectIDs. In the past this has worked due to some dumb luck. getID() always returns 1 for the main thread, which is always the thread we are executing in. Coincidentally the JDWP ObjectID for the main Thread object is also always 1 because this is the first ObjectID that the debug agent ever needs to create. However, when the debuggee main thread is a virtual thread, neither getId() nor JDWP assign 1 to the threadID, and in fact both will end up with very different values for the threadID. The end result is errors from jdb for using an invalid threadID. >> >> The correct threadID can be obtained by executing the jdb "threads" command and parsing it from a line that looks like the following: >> >> ` (java.lang.VirtualThread)694 main running (at breakpoint)` >> >> Note this test will also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which should be fixed first. >> >> I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the test being removed from the problem list by this PR also passes. > > Chris Plummer has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: > > - fix minor merge issue > - merge > - remove test from problem list > - properly determine the main threadId Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14294#pullrequestreview-1465846971 From duke at openjdk.org Tue Jun 6 18:48:14 2023 From: duke at openjdk.org (Christine Flood) Date: Tue, 6 Jun 2023 18:48:14 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: <5kABzdo1H-Hs5ncz0KFnNH_9Ae0Z7hwPM8Qigg_0bjU=.b42d954e-5b69-4e77-82d8-217b6acb3e43@github.com> On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. I wrote an LRU program back in 2017 which allocates trees and stores them in an array in a round robin fashion, freeing the last allocated. At the time this was written it's purpose was to show how generational GCs can hit the wall and start performing very badly. I ran this on a clean openjdk build, a genshen build in generational mode and a genshen build in non-generational mode. These results are repeatable for me. I would like to understand where the degradation is coming from before moving forward with this patch since it appears to penalize those who wish to just run traditional Shenandoah. Clean cflood at fedora java_programs]$ ~/genshen/cleanjdk/build/linux-x86_64-server-release/images/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC LRU 1000 1000 Took 341892ms to allocate 1000000 trees in a cache of 1000 Genshen generational (we expect this to be bad) [cflood at fedora java_programs]$ ~/genshen/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ShenandoahGCMode=generational LRU 1000 1000 Took 442012ms to allocate 1000000 trees in a cache of 1000 Genshen non-generational (shows what I feel is a significant degradation from the clean build) [cflood at fedora java_programs]$ ~/genshen/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC LRU 1000 1000 Took 395679ms to allocate 1000000 trees in a cache of 1000 I think that generational Shenandoah can be a big win for some applications, but I want to fully understand the cost for all applications. I can't attach a .java file so here it is inline in the post. class TreeNode { public TreeNode left, right; public int val; } public class LRU { static int cache_size; static int reps; static int tree_height=16; private static TreeNode[] trees; private static int getIndex(int i) {return i % cache_size;} private static TreeNode makeTree(int h) { if (h == 0) { return null;} else { TreeNode res = new TreeNode(); res.left = makeTree(h - 1); res.right = makeTree(h - 1); res.val = h; return res; } } public static void main(String[] args) { if (args.length != 2) { System.err.println("LRU requires args: cache_size reps"); return; } cache_size = Integer.parseInt(args[0]); reps = Integer.parseInt(args[1]) * cache_size; trees = new TreeNode[cache_size]; long start = System.currentTimeMillis(); for (int i = 0; i < reps; i++) trees[getIndex(i)] = makeTree(tree_height); long end = System.currentTimeMillis(); long ms = end - start; System.out.println("Took " + ms + "ms to allocate " + reps + " trees in a cache of " + cache_size); } } ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1579278537 From sspitsyn at openjdk.org Tue Jun 6 18:52:56 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 18:52:56 GMT Subject: RFR: 8309420: con/sun/jdi/StepTest.java fails with virtual thread wrapper In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 04:03:57 GMT, Chris Plummer wrote: > The test has two issues. The first is that it assume that once the VMStart event has arrived and one "step into" is done, it will be in the main method of the debuggee. Once there, it determines the debuggee class name by looking at the classtype of topmost frame. The problems is when using virtual threads, it is actually in TestScaffold.main() at this point, so the wrong class name is gleaned from the frame. To fix this the test just saves away the debuggee class name, which is passed to the test as the 4th argument. > > The other issue is that the test assumes once it gets to the debuggee go() method, there are only two frames on the stack. It's more like 16 when using virtual threads. The test needs to account for this by counting the number of frames when go() is entered rather than assuming it will be 2. > > Tested locally with and without the wrapper and by running tier5 svc tests. Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14307#pullrequestreview-1465860157 From phh at openjdk.org Tue Jun 6 19:51:15 2023 From: phh at openjdk.org (Paul Hohensee) Date: Tue, 6 Jun 2023 19:51:15 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. Thanks for finding the single-gen regression, we're very happy you took the time to run it and write up your results. We're very concerned about single-gen regressions too because we have single-gen Shen in production for several critical services. We'd like to propose to push now, and tackle/fix the single-gen issue you identified during RDP1, as well as any other significant single-gen regressions that may come up. We have four Shen experts on board, Roman, Aleksey, Kelvin, and William, so believe it's doable before RDP2 in July. In the worst case that we fail, we'd emulate ZGC and move GenShen to it's own directory as an entirely separate collector before RDP2. Make sense? ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1579351993 From aph at openjdk.org Tue Jun 6 19:59:13 2023 From: aph at openjdk.org (Andrew Haley) Date: Tue, 6 Jun 2023 19:59:13 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. Marked as reviewed by aph (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14185#pullrequestreview-1465971078 From aph at openjdk.org Tue Jun 6 19:59:14 2023 From: aph at openjdk.org (Andrew Haley) Date: Tue, 6 Jun 2023 19:59:14 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 19:48:05 GMT, Paul Hohensee wrote: > We'd like to propose to push now, and tackle/fix the single-gen issue you identified during RDP1, as well as any other significant single-gen regressions that may come up. We have four Shen experts on board, Roman, Aleksey, Kelvin, and William, so believe it's doable before RDP2 in July. In the worst case that we fail, we'd emulate ZGC and move GenShen to it's own directory as an entirely separate collector before RDP2. Make sense? That sounds great to me. I'll approve this PR now, but please wait for Christine's ack. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1579360356 From duke at openjdk.org Tue Jun 6 20:04:22 2023 From: duke at openjdk.org (Christine Flood) Date: Tue, 6 Jun 2023 20:04:22 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: <7VPTUAp0zrlWrCGVgmWqkw5dXda3-1UPA8fcTEW6KoA=.9188218c-7216-4b74-b467-8dab75131578@github.com> On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. This sounds good to me. On Tue, Jun 6, 2023 at 3:55?PM Andrew Haley ***@***.***> wrote: > We'd like to propose to push now, and tackle/fix the single-gen issue you > identified during RDP1, as well as any other significant single-gen > regressions that may come up. We have four Shen experts on board, Roman, > Aleksey, Kelvin, and William, so believe it's doable before RDP2 in July. > In the worst case that we fail, we'd emulate ZGC and move GenShen to it's > own directory as an entirely separate collector before RDP2. Make sense? > > That sounds great to me. I'll approve this PR now, but please wait for > Christine's ack. > > ? > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1579367147 From cjplummer at openjdk.org Tue Jun 6 20:07:33 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 20:07:33 GMT Subject: RFR: 8309396: com/sun/jdi/JdbMethodExitTest.java fails with virtual threads due to a bug in determining the main thread id [v3] In-Reply-To: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> References: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> Message-ID: <-uk6ffen0XX1qSXW5KjuJqbAu0wBKdSHkej3npQiZ44=.ebff7c7e-c92e-4830-88e0-84f8cebbcec1@github.com> > JdbMethodExitTest.java tries to determine the jdb threadID for the "main" thread, and then later use it in jdb commands that require a threadID. It does this by first having the debuggee execute the following: > > ` System.out.println("threadid="+Thread.currentThread().getId());` > > And then later on the test parses the threadID from this output. The problem is that the id returned by getId() has no relation to threadIDs used by jdb, which are actually JDWP ObjectIDs. In the past this has worked due to some dumb luck. getID() always returns 1 for the main thread, which is always the thread we are executing in. Coincidentally the JDWP ObjectID for the main Thread object is also always 1 because this is the first ObjectID that the debug agent ever needs to create. However, when the debuggee main thread is a virtual thread, neither getId() nor JDWP assign 1 to the threadID, and in fact both will end up with very different values for the threadID. The end result is errors from jdb for using an invalid threadID. > > The correct threadID can be obtained by executing the jdb "threads" command and parsing it from a line that looks like the following: > > ` (java.lang.VirtualThread)694 main running (at breakpoint)` > > Note this test will also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which should be fixed first. > > I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the test being removed from the problem list by this PR also passes. Chris Plummer has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - Merge - fix minor merge issue - merge - remove test from problem list - properly determine the main threadId ------------- Changes: https://git.openjdk.org/jdk/pull/14294/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14294&range=02 Stats: 11 lines in 2 files changed: 5 ins; 2 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14294.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14294/head:pull/14294 PR: https://git.openjdk.org/jdk/pull/14294 From mr at openjdk.org Tue Jun 6 20:19:12 2023 From: mr at openjdk.org (Mark Reinhold) Date: Tue, 6 Jun 2023 20:19:12 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: <7VPTUAp0zrlWrCGVgmWqkw5dXda3-1UPA8fcTEW6KoA=.9188218c-7216-4b74-b467-8dab75131578@github.com> References: <7VPTUAp0zrlWrCGVgmWqkw5dXda3-1UPA8fcTEW6KoA=.9188218c-7216-4b74-b467-8dab75131578@github.com> Message-ID: On Tue, 6 Jun 2023 20:01:02 GMT, Christine Flood wrote: > We'd like to propose to push now, and tackle/fix the single-gen issue you identified during RDP1, as well as any other significant single-gen regressions that may come up. We have four Shen experts on board, Roman, Aleksey, Kelvin, and William, so believe it's doable before RDP2 in July. In the worst case that we fail, we'd emulate ZGC and move GenShen to it's own directory as an entirely separate collector before RDP2. Make sense? Unsolicited advice: If you?re planning for this amount of change during RDP 1 then I?d say that you?re not ready for RDP 1. If this patch were less isolated from the rest of HotSpot then I?d be extremely nervous. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1579385819 From mdoerr at openjdk.org Tue Jun 6 20:46:54 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 6 Jun 2023 20:46:54 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 09:51:09 GMT, JoKern65 wrote: > The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 > The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. > So we have (at least) 3 ways to circumvent this behavior. > > 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): > `#include ` > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` while (1) {` > ` sleep(1);` > ` }` > `}` > > 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` volatile int i = 1;` > ` while (i) {` > ` i += 2;` > ` }` > `}` > > 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` > `{` > ` while (1) {` > ` }` > `}` > > To make the third approach platform independent, we can implement it in the following way: > In globalDefinitions.hpp > `#ifndef OPTNONE` > `#define OPTNONE` > `#endif` > > In globalDefinitions_xlc.hpp > `// optnone support` > `//` > `// To use if a function should not be optimized` > `// Usage:` > `// void* func(size_t size) OPTNONE {...}` > `#define OPTNONE __attribute__(( optnone))` > > With this we can change libagentthr001.cpp in a platform independent way to > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` > `{` > ` while (1) {` > ` }` > `}` Changes requested by mdoerr (Reviewer). test/hotspot/jtreg/vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/agentthr001.cpp line 26: > 24: #include > 25: #include > 26: #include Breaks Windows build: agentthr001.cpp(26): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory ------------- PR Review: https://git.openjdk.org/jdk/pull/14330#pullrequestreview-1466074354 PR Review Comment: https://git.openjdk.org/jdk/pull/14330#discussion_r1220327772 From phh at openjdk.org Tue Jun 6 20:59:12 2023 From: phh at openjdk.org (Paul Hohensee) Date: Tue, 6 Jun 2023 20:59:12 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v5] In-Reply-To: References: Message-ID: <3GigY7Hg2Clm2MqFsQnCWHxLB77_d-WM2c1kOi6wavA=.cc31a2e4-424d-48d1-9016-c409be4790c3@github.com> On Sun, 4 Jun 2023 21:39:58 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove three asserts making comparisons between atomic volatile variables > > Though changes to the volatile variables are individually protected by > Atomic load and store operations, these asserts were not assuring > atomic access to multiple volatile variables, each of which could be > modified independently of the others. The asserts were therefore not > trustworthy, as has been confirmed by more extensive testing. We understand, and would not have proposed the last chance split-directory alternative without the level of isolation Hotspot's GC interface enables. We've added single-gen performance enhancements on the way and would like to keep them! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1579439750 From kdnilsen at openjdk.org Tue Jun 6 21:12:41 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 21:12:41 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v6] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Remove an inappropriate copyright notice ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/8d80780a..9811d2aa Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=04-05 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From cjplummer at openjdk.org Tue Jun 6 21:15:55 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 21:15:55 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v3] In-Reply-To: References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: On Tue, 6 Jun 2023 13:37:03 GMT, Serguei Spitsyn wrote: >> The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. >> The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. >> Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. >> >> The list of changes is: >> - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. >> - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. >> - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> >> Testing: >> - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> - TBD: tiers 1-6 (all are good) > > Serguei Spitsyn has updated the pull request incrementally with two additional commits since the last revision: > > - fixed typo in a comment in jvmtiEnvBase.cpp > - nit: restored one comment as was before Changes requested by cjplummer (Reviewer). test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 63: > 61: public void run() { > 62: log("TestTask.run()"); > 63: } I think this should be an abstract method. test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 106: > 104: final Thread.State expState = Thread.State.WAITING; > 105: reentrantLock.lock(); > 106: String name = "ObjectMonitorTestTask"; Should be "ReentrantLockTestTask" test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/libThreadListStackTracesTest.cpp line 35: > 33: extern "C" { > 34: > 35: JNIEXPORT jint JNICALL Java_ThreadListStackTracesTest_getStateSingle(JNIEnv* jni, jclass clazz, jthread vthread) { I'd suggest splitting into 2 lines just like Java_ThreadListStackTracesTest_getStateMultiple() for the sake of consistency and being able to more easily compare the two. ------------- PR Review: https://git.openjdk.org/jdk/pull/14326#pullrequestreview-1466112714 PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220365810 PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220367970 PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220361274 From amenkov at openjdk.org Tue Jun 6 21:25:56 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 21:25:56 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v2] In-Reply-To: <5-_kURDcYKd5WYKi9B331c8h6okVmGfvjdy_xqi1UqU=.012c360b-1b57-4773-b64b-d727dbf4daeb@github.com> References: <5-_kURDcYKd5WYKi9B331c8h6okVmGfvjdy_xqi1UqU=.012c360b-1b57-4773-b64b-d727dbf4daeb@github.com> Message-ID: On Mon, 5 Jun 2023 19:00:49 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - Merge > - minor tweaks in libThreadStateTest.cpp > - 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING src/hotspot/share/prims/jvmtiEnvBase.cpp line 764: > 762: > 763: if (is_passive_carrier_thread(jt, thread_oop)) { > 764: state |= (JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY); Not sure I understand this. I'd expect `JVMTI_THREAD_STATE_ALIVE | JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY` to be returned in the case. How can a thread be JVMTI_THREAD_STATE_RUNNABLE and JVMTI_THREAD_STATE_WAITING at the same time? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220384120 From sspitsyn at openjdk.org Tue Jun 6 21:32:02 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 21:32:02 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v3] In-Reply-To: References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: On Tue, 6 Jun 2023 21:07:46 GMT, Chris Plummer wrote: >> Serguei Spitsyn has updated the pull request incrementally with two additional commits since the last revision: >> >> - fixed typo in a comment in jvmtiEnvBase.cpp >> - nit: restored one comment as was before > > test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 63: > >> 61: public void run() { >> 62: log("TestTask.run()"); >> 63: } > > I think this should be an abstract method. Thanks. Fixed now. > test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 106: > >> 104: final Thread.State expState = Thread.State.WAITING; >> 105: reentrantLock.lock(); >> 106: String name = "ObjectMonitorTestTask"; > > Should be "ReentrantLockTestTask" Thanks. Fixed now. > test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/libThreadListStackTracesTest.cpp line 35: > >> 33: extern "C" { >> 34: >> 35: JNIEXPORT jint JNICALL Java_ThreadListStackTracesTest_getStateSingle(JNIEnv* jni, jclass clazz, jthread vthread) { > > I'd suggest splitting into 2 lines just like > Java_ThreadListStackTracesTest_getStateMultiple() for the sake of consistency and being able to more easily compare the two. Thanks. I've overlooked this. Fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220390676 PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220392423 PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220387806 From kdnilsen at openjdk.org Tue Jun 6 21:36:44 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 21:36:44 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v7] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Exit during initialization on unsupported platforms ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/9811d2aa..cc149904 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=05-06 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From sspitsyn at openjdk.org Tue Jun 6 21:37:25 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 21:37:25 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v4] In-Reply-To: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: > The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. > The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. > Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. > > The list of changes is: > - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. > - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. > - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > > Testing: > - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > - TBD: tiers 1-6 (all are good) Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: addressed new test related review comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14326/files - new: https://git.openjdk.org/jdk/pull/14326/files/d20e1221..e982f97e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=02-03 Stats: 6 lines in 2 files changed: 1 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/14326.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14326/head:pull/14326 PR: https://git.openjdk.org/jdk/pull/14326 From kdnilsen at openjdk.org Tue Jun 6 21:46:12 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 21:46:12 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v7] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 13:32:49 GMT, Thomas Stuefe wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Exit during initialization on unsupported platforms > > test/hotspot/jtreg/gc/shenandoah/TestEvilSyncBug.java line 33: > >> 31: * @modules java.base/jdk.internal.misc >> 32: * java.management >> 33: * @run driver/timeout=480 TestEvilSyncBug -XX:ShenandoahGCHeuristics=aggressive > > Probably fine, but why this change to non-generational testing? Will aggressive heuristic sharpen the test? We moved this argument from the source code (original line 64) to here so that we can invoke the test differently in generational mode. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1220410522 From cjplummer at openjdk.org Tue Jun 6 21:50:55 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 21:50:55 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v4] In-Reply-To: References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: <14FIsTgic9d0rLeA9rAVsfPlBdWnyQkvXKGnYg2j9to=.f5e31a08-e29b-4e3a-ba44-15206f05ff83@github.com> On Tue, 6 Jun 2023 21:37:25 GMT, Serguei Spitsyn wrote: >> The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. >> The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. >> Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. >> >> The list of changes is: >> - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. >> - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. >> - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> >> Testing: >> - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> - TBD: tiers 1-6 (all are good) > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > addressed new test related review comments Looks good. ------------- Marked as reviewed by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14326#pullrequestreview-1466172053 From clanger at openjdk.org Tue Jun 6 21:53:56 2023 From: clanger at openjdk.org (Christoph Langer) Date: Tue, 6 Jun 2023 21:53:56 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: <365iru-S_oNYKt8H6TCLjXeguy6o3-jWEEvCVS7r2GQ=.e80fa994-ba93-469f-a73d-0a5f9489f504@github.com> On Tue, 6 Jun 2023 09:51:09 GMT, JoKern65 wrote: > The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 > The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. > So we have (at least) 3 ways to circumvent this behavior. > > 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): > `#include ` > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` while (1) {` > ` sleep(1);` > ` }` > `}` > > 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` volatile int i = 1;` > ` while (i) {` > ` i += 2;` > ` }` > `}` > > 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` > `{` > ` while (1) {` > ` }` > `}` > > To make the third approach platform independent, we can implement it in the following way: > In globalDefinitions.hpp > `#ifndef OPTNONE` > `#define OPTNONE` > `#endif` > > In globalDefinitions_xlc.hpp > `// optnone support` > `//` > `// To use if a function should not be optimized` > `// Usage:` > `// void* func(size_t size) OPTNONE {...}` > `#define OPTNONE __attribute__(( optnone))` > > With this we can change libagentthr001.cpp in a platform independent way to > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` > `{` > ` while (1) {` > ` }` > `}` The solution to use sleep looks fine. Please address the comments and also update the copyright year. test/hotspot/jtreg/vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/agentthr001.cpp line 141: > 139: sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) { > 140: while (1) { > 141: sleep(1); Maybe you could add a comment here that the call to sleep is necessary to avoid the compiler optimization to elide the loop. ------------- Changes requested by clanger (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14330#pullrequestreview-1466167108 PR Review Comment: https://git.openjdk.org/jdk/pull/14330#discussion_r1220417459 From clanger at openjdk.org Tue Jun 6 21:54:00 2023 From: clanger at openjdk.org (Christoph Langer) Date: Tue, 6 Jun 2023 21:54:00 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 20:43:36 GMT, Martin Doerr wrote: >> The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 >> The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. >> So we have (at least) 3 ways to circumvent this behavior. >> >> 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): >> `#include ` >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` >> `{` >> ` while (1) {` >> ` sleep(1);` >> ` }` >> `}` >> >> 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` >> `{` >> ` volatile int i = 1;` >> ` while (i) {` >> ` i += 2;` >> ` }` >> `}` >> >> 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` >> `{` >> ` while (1) {` >> ` }` >> `}` >> >> To make the third approach platform independent, we can implement it in the following way: >> In globalDefinitions.hpp >> `#ifndef OPTNONE` >> `#define OPTNONE` >> `#endif` >> >> In globalDefinitions_xlc.hpp >> `// optnone support` >> `//` >> `// To use if a function should not be optimized` >> `// Usage:` >> `// void* func(size_t size) OPTNONE {...}` >> `#define OPTNONE __attribute__(( optnone))` >> >> With this we can change libagentthr001.cpp in a platform independent way to >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` >> `{` >> ` while (1) {` >> ` }` >> `}` > > test/hotspot/jtreg/vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/agentthr001.cpp line 26: > >> 24: #include >> 25: #include >> 26: #include > > Breaks Windows build: agentthr001.cpp(26): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory On Windows I think you need windows.h. I guess you can do it like here: https://github.com/openjdk/jdk/blob/4a75fd462c002a209201d8bfc8d6c9eb286a7444/src/java.base/share/native/libjli/wildcard.c#L99 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14330#discussion_r1220410431 From kdnilsen at openjdk.org Tue Jun 6 21:54:14 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 21:54:14 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v7] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 21:43:36 GMT, Kelvin Nilsen wrote: >> test/hotspot/jtreg/gc/shenandoah/TestEvilSyncBug.java line 33: >> >>> 31: * @modules java.base/jdk.internal.misc >>> 32: * java.management >>> 33: * @run driver/timeout=480 TestEvilSyncBug -XX:ShenandoahGCHeuristics=aggressive >> >> Probably fine, but why this change to non-generational testing? Will aggressive heuristic sharpen the test? > > We moved this argument from the source code (original line 64) to here so that we can invoke the test differently in generational mode. See line 64 of the original source code. We did not change the behavior. We only changed how the behavior is realized. This enables generalization of the test to generational mode. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1220418431 From kdnilsen at openjdk.org Tue Jun 6 21:54:16 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 21:54:16 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v7] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 14:25:19 GMT, Thomas Stuefe wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Exit during initialization on unsupported platforms > > test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 92: > >> 90: expectFailure("-Xmx16m", >> 91: "-XX:+UnlockExperimentalVMOptions", >> 92: "-XX:+UseShenandoahGC", > > Nit: should not need UnlockExperimentalVMOptions anymore. We actually do need to UnlockExperimentalVMOptions because generational mode is currently an experimental feature. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1220417069 From clanger at openjdk.org Tue Jun 6 21:57:55 2023 From: clanger at openjdk.org (Christoph Langer) Date: Tue, 6 Jun 2023 21:57:55 GMT Subject: RFR: JDK-8309225: Fix xlc17 clang 15 warnings in security and servicability In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 10:19:53 GMT, JoKern65 wrote: > This pr is a split off from JDK-8308288: Fix xlc17 clang warnings in shared code https://github.com/openjdk/jdk/pull/14146 > It handles the part in security and servicability. > > Compiling on AIX with xlc17 which contains the new clang 15 frontend shows the following warnings: > > src/java.security.jgss/share/native/libj2gss/NativeUtil.h:30: > src/java.security.jgss/share/native/libj2gss/gssapi.h:48:5: error: 'TARGET_OS_MAC' is not defined, evaluates to 0 [-Werror,-Wundef] > #if TARGET_OS_MAC && (defined(ppc) || defined(ppc64) || defined(i386) || defined(x86_64)) > ^ > TARGET_OS_MAC is not defined. Instead of disabling the warning, I could > ` #ifndef TARGET_OS_MAC` > `#define TARGET_OS_MAC=0` > `#endif` > But this is already handled by disabling the warning for gcc. > > src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:718:33: error: suggest braces around initialization of subobject [-Werror,-Wmissing-braces] > struct in6_addr mappedAny = IN6ADDR_ANY_INIT; > ^~~~~~~~~~~~~~~~ > /usr/include/netinet/in.h:454:32: note: expanded from macro 'IN6ADDR_ANY_INIT' > #define IN6ADDR_ANY_INIT {0, 0, 0, 0} Marked as reviewed by clanger (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14282#pullrequestreview-1466180987 From cjplummer at openjdk.org Tue Jun 6 22:00:05 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 22:00:05 GMT Subject: RFR: 8309396: com/sun/jdi/JdbMethodExitTest.java fails with virtual threads due to a bug in determining the main thread id [v3] In-Reply-To: <-uk6ffen0XX1qSXW5KjuJqbAu0wBKdSHkej3npQiZ44=.ebff7c7e-c92e-4830-88e0-84f8cebbcec1@github.com> References: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> <-uk6ffen0XX1qSXW5KjuJqbAu0wBKdSHkej3npQiZ44=.ebff7c7e-c92e-4830-88e0-84f8cebbcec1@github.com> Message-ID: <4okHmqxNLArxtH8q6yeSYG-U4vX9RIkRQV6UCydUBAI=.87e6ab7e-c5d4-4bd5-9ace-34268f6dd4a7@github.com> On Tue, 6 Jun 2023 20:07:33 GMT, Chris Plummer wrote: >> JdbMethodExitTest.java tries to determine the jdb threadID for the "main" thread, and then later use it in jdb commands that require a threadID. It does this by first having the debuggee execute the following: >> >> ` System.out.println("threadid="+Thread.currentThread().getId());` >> >> And then later on the test parses the threadID from this output. The problem is that the id returned by getId() has no relation to threadIDs used by jdb, which are actually JDWP ObjectIDs. In the past this has worked due to some dumb luck. getID() always returns 1 for the main thread, which is always the thread we are executing in. Coincidentally the JDWP ObjectID for the main Thread object is also always 1 because this is the first ObjectID that the debug agent ever needs to create. However, when the debuggee main thread is a virtual thread, neither getId() nor JDWP assign 1 to the threadID, and in fact both will end up with very different values for the threadID. The end result is errors from jdb for using an invalid threadID. >> >> The correct threadID can be obtained by executing the jdb "threads" command and parsing it from a line that looks like the following: >> >> ` (java.lang.VirtualThread)694 main running (at breakpoint)` >> >> Note this test will also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which should be fixed first. >> >> I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the test being removed from the problem list by this PR also passes. > > Chris Plummer has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Merge > - fix minor merge issue > - merge > - remove test from problem list > - properly determine the main threadId Thanks for the reviews Alex and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14294#issuecomment-1579501697 From cjplummer at openjdk.org Tue Jun 6 22:00:07 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 6 Jun 2023 22:00:07 GMT Subject: Integrated: 8309396: com/sun/jdi/JdbMethodExitTest.java fails with virtual threads due to a bug in determining the main thread id In-Reply-To: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> References: <_rWgi41CJ8newOTI2hqbjeNgOCCDTnNq030O6wXi_RI=.436a6404-306a-4001-b193-1206ee40f85d@github.com> Message-ID: On Fri, 2 Jun 2023 21:47:47 GMT, Chris Plummer wrote: > JdbMethodExitTest.java tries to determine the jdb threadID for the "main" thread, and then later use it in jdb commands that require a threadID. It does this by first having the debuggee execute the following: > > ` System.out.println("threadid="+Thread.currentThread().getId());` > > And then later on the test parses the threadID from this output. The problem is that the id returned by getId() has no relation to threadIDs used by jdb, which are actually JDWP ObjectIDs. In the past this has worked due to some dumb luck. getID() always returns 1 for the main thread, which is always the thread we are executing in. Coincidentally the JDWP ObjectID for the main Thread object is also always 1 because this is the first ObjectID that the debug agent ever needs to create. However, when the debuggee main thread is a virtual thread, neither getId() nor JDWP assign 1 to the threadID, and in fact both will end up with very different values for the threadID. The end result is errors from jdb for using an invalid threadID. > > The correct threadID can be obtained by executing the jdb "threads" command and parsing it from a line that looks like the following: > > ` (java.lang.VirtualThread)694 main running (at breakpoint)` > > Note this test will also fail due to [JDK-8309334](https://bugs.openjdk.org/browse/JDK-8309334), which should be fixed first. > > I've tested with mach5 tier5 in a workspace that has integrated the various CRs mentioned. Once JDK-8309334 is fixed, before integrating this PR I'll first merge and verify that the test being removed from the problem list by this PR also passes. This pull request has now been integrated. Changeset: 65bdbc7a Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/65bdbc7a8c06e5f119c9db832734664780933e01 Stats: 11 lines in 2 files changed: 5 ins; 2 del; 4 mod 8309396: com/sun/jdi/JdbMethodExitTest.java fails with virtual threads due to a bug in determining the main thread id Reviewed-by: amenkov, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/14294 From sspitsyn at openjdk.org Tue Jun 6 22:07:14 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 22:07:14 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v3] In-Reply-To: References: Message-ID: > When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. > The fix adds: > - a special case for passive carrier threads > - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. > > Testing: > - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` > - submitted mach5 tiers 1-5 > - TBD: to submit mach5 tier 6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: removed JVMTI_THREAD_STATE_RUNNABLE from a carrier thread state ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14298/files - new: https://git.openjdk.org/jdk/pull/14298/files/e60da02e..b5eb3835 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=01-02 Stats: 4 lines in 2 files changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14298.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14298/head:pull/14298 PR: https://git.openjdk.org/jdk/pull/14298 From sspitsyn at openjdk.org Tue Jun 6 22:07:18 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 22:07:18 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v2] In-Reply-To: References: <5-_kURDcYKd5WYKi9B331c8h6okVmGfvjdy_xqi1UqU=.012c360b-1b57-4773-b64b-d727dbf4daeb@github.com> Message-ID: On Tue, 6 Jun 2023 21:22:40 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - Merge >> - minor tweaks in libThreadStateTest.cpp >> - 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 764: > >> 762: >> 763: if (is_passive_carrier_thread(jt, thread_oop)) { >> 764: state |= (JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY); > > Not sure I understand this. > I'd expect > `JVMTI_THREAD_STATE_ALIVE | JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY` to be returned in the case. > How can a thread be JVMTI_THREAD_STATE_RUNNABLE and JVMTI_THREAD_STATE_WAITING at the same time? Good catch, thanks. Fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220428448 From sspitsyn at openjdk.org Tue Jun 6 22:09:54 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 22:09:54 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v3] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 15:43:02 GMT, Alan Bateman wrote: >> Okay, I see you point. Unfortunately, I've always referred the platform thread with an executed FJP schedular as a carrier thread. The term 'carrier' with this meaning is everywhere in the JVMTI code. It looks very confusing to call a thread to be a carrier thread only during some phases of its execution. > >> Okay, I see you point. Unfortunately, I've always referred the platform thread with an executed FJP schedular as a carrier thread. The term 'carrier' with this meaning is everywhere in the JVMTI code. It looks very confusing to call a thread to be a carrier thread only during some phases of its execution. > > Okay, I'm just pointing out that is_passive_carrier_thread looks a bit strange here as it is testing if a JavaThread is carrying a virtual thread oop - it's not testing if the thread is owned by the virtual thread scheduler. I'm still thinking what identifier to use instead of `is_passive_carrier_thread`. Just `is_carrier_thread` is going to be confusing as well. What about `is_carrier_thread_waiting_for_virtual` or `is_carrier_thread_waiting_for_virtual_to_unmount`? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220435738 From amenkov at openjdk.org Tue Jun 6 22:20:59 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 22:20:59 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v3] In-Reply-To: References: Message-ID: <96WXnBOucA3CuQDIfxYFBKhHC1_Uo651LphuRFeLgcI=.6cd798f1-2849-4871-94e3-ab9df30f393d@github.com> On Tue, 6 Jun 2023 22:07:14 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: removed JVMTI_THREAD_STATE_RUNNABLE from a carrier thread state src/hotspot/share/prims/jvmtiEnvBase.cpp line 768: > 766: } > 767: return state; > 768: } You don't need to call get_thread_state_base in case "passive carrier thread": if (is_passive_carrier_thread(jt, thread_oop)) { return JVMTI_THREAD_STATE_ALIVE | JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; } return get_thread_state_base(thread_oop, jt); ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220447606 From amenkov at openjdk.org Tue Jun 6 22:39:09 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 22:39:09 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v4] In-Reply-To: References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: <7-6AydLkwjCOkTHA2sI-f7axGSMpCq5havZC5K6qz5U=.93d29276-ab0e-444e-b22c-d4b5f1c9b68b@github.com> On Tue, 6 Jun 2023 21:37:25 GMT, Serguei Spitsyn wrote: >> The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. >> The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. >> Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. >> >> The list of changes is: >> - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. >> - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. >> - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> >> Testing: >> - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> - TBD: tiers 1-6 (all are good) > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > addressed new test related review comments Marked as reviewed by amenkov (Reviewer). test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 29: > 27: * @summary GetThreadListStackTraces returns wrong state for blocked VirtualThread > 28: * @requires vm.continuations > 29: * @modules java.base/java.lang:+open I think `@modules` it's not needed test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 34: > 32: > 33: import java.util.concurrent.locks.ReentrantLock; > 34: import java.util.concurrent.*; unused imports ------------- PR Review: https://git.openjdk.org/jdk/pull/14326#pullrequestreview-1466246664 PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220481289 PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220480268 From sspitsyn at openjdk.org Tue Jun 6 22:41:54 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 22:41:54 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: Message-ID: > When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. > The fix adds: > - a special case for passive carrier threads > - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. > > Testing: > - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` > - submitted mach5 tiers 1-5 > - TBD: to submit mach5 tier 6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: call get_thread_state_base only when needed ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14298/files - new: https://git.openjdk.org/jdk/pull/14298/files/b5eb3835..77771816 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=02-03 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14298.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14298/head:pull/14298 PR: https://git.openjdk.org/jdk/pull/14298 From amenkov at openjdk.org Tue Jun 6 22:41:57 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 22:41:57 GMT Subject: RFR: 8309510: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java no longer needs to override startup() method In-Reply-To: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> References: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> Message-ID: On Tue, 6 Jun 2023 00:36:12 GMT, Chris Plummer wrote: > com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java currently overrides the TestScaffold.startup() method: > > // override this to correct a bug so arguments can be passed to > // the Target class > protected void startUp(String targetName) { > List argList = new ArrayList<>(Arrays.asList(args)); > argList.add(0, targetName); // pre-pend so it becomes the first "app" arg > println("run args: " + argList); > connect((String[]) argList.toArray(args)); > waitForVMStart(); > } > > This issue of passing app args was fixed recently by [JDK-8308481](https://bugs.openjdk.org/browse/JDK-8308481), so the override is no longer needed. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14325#pullrequestreview-1466250289 From sspitsyn at openjdk.org Tue Jun 6 22:41:58 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 22:41:58 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v3] In-Reply-To: <96WXnBOucA3CuQDIfxYFBKhHC1_Uo651LphuRFeLgcI=.6cd798f1-2849-4871-94e3-ab9df30f393d@github.com> References: <96WXnBOucA3CuQDIfxYFBKhHC1_Uo651LphuRFeLgcI=.6cd798f1-2849-4871-94e3-ab9df30f393d@github.com> Message-ID: On Tue, 6 Jun 2023 22:17:57 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: removed JVMTI_THREAD_STATE_RUNNABLE from a carrier thread state > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 768: > >> 766: } >> 767: return state; >> 768: } > > You don't need to call get_thread_state_base in case "passive carrier thread": > > if (is_passive_carrier_thread(jt, thread_oop)) { > return JVMTI_THREAD_STATE_ALIVE | JVMTI_THREAD_STATE_WAITING > | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; > } > return get_thread_state_base(thread_oop, jt); Thanks. Yes, noticed it. :) Fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220484149 From sspitsyn at openjdk.org Tue Jun 6 22:54:04 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 22:54:04 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v5] In-Reply-To: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: > The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. > The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. > Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. > > The list of changes is: > - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. > - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. > - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > > Testing: > - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > - TBD: tiers 1-6 (all are good) Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: ThreadListStackTracesTest cleanup ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14326/files - new: https://git.openjdk.org/jdk/pull/14326/files/e982f97e..6b685ca3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=03-04 Stats: 3 lines in 1 file changed: 0 ins; 2 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14326.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14326/head:pull/14326 PR: https://git.openjdk.org/jdk/pull/14326 From sspitsyn at openjdk.org Tue Jun 6 22:54:06 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 22:54:06 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v4] In-Reply-To: References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: <0U4aaMnv803nuwN5lempsD-eUFIEiC6WbQr0lCnwTKo=.6b13a626-79ff-4968-ae05-6da81ed5e997@github.com> On Tue, 6 Jun 2023 21:37:25 GMT, Serguei Spitsyn wrote: >> The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. >> The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. >> Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. >> >> The list of changes is: >> - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. >> - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. >> - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> >> Testing: >> - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> - TBD: tiers 1-6 (all are good) > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > addressed new test related review comments Chris and Alex, thank you for review! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14326#issuecomment-1579560041 From sspitsyn at openjdk.org Tue Jun 6 22:54:07 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 22:54:07 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v4] In-Reply-To: <7-6AydLkwjCOkTHA2sI-f7axGSMpCq5havZC5K6qz5U=.93d29276-ab0e-444e-b22c-d4b5f1c9b68b@github.com> References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> <7-6AydLkwjCOkTHA2sI-f7axGSMpCq5havZC5K6qz5U=.93d29276-ab0e-444e-b22c-d4b5f1c9b68b@github.com> Message-ID: On Tue, 6 Jun 2023 22:34:33 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> addressed new test related review comments > > test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 29: > >> 27: * @summary GetThreadListStackTraces returns wrong state for blocked VirtualThread >> 28: * @requires vm.continuations >> 29: * @modules java.base/java.lang:+open > > I think `@modules` it's not needed Thanks. Removed now. > test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 34: > >> 32: >> 33: import java.util.concurrent.locks.ReentrantLock; >> 34: import java.util.concurrent.*; > > unused imports Thanks. Removed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220495535 PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220495047 From amenkov at openjdk.org Tue Jun 6 23:01:51 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 23:01:51 GMT Subject: RFR: 8309509: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java fails with virtual test thread factory In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 00:02:47 GMT, Chris Plummer wrote: > The test fails with the virtual test thread factory because it tries to find the "main" thread in the list of threads returned by JDI, but "main" is a virtual thread and will only be returned by JDI if the debug agent is launched with includevirtualthreads=y. As a result the thread is not found and the test asserts: > > java.lang.RuntimeException: assertTrue: expected true, was false > at jdk.test.lib.Asserts.fail(Asserts.java:594) > at jdk.test.lib.Asserts.assertTrue(Asserts.java:486) > at jdk.test.lib.Asserts.assertTrue(Asserts.java:472) > at TestNestmateAttr.checkGoodTransforms(TestNestmateAttr.java:511) > at TestNestmateAttr.methodEntered(TestNestmateAttr.java:320) > at TestScaffold$EventHandler.notifyEvent(TestScaffold.java:205) > at TestScaffold$EventHandler.run(TestScaffold.java:279) > at java.base/java.lang.Thread.run(Thread.java:1583) > > The fix is to always run the debug agent with includevirtualthreads=y. > > Tested by running all com/sun/jdi tests locally with and without the virtual test thread factory. Also ran tier1 and tier5 svc test tasks. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14324#pullrequestreview-1466268017 From amenkov at openjdk.org Tue Jun 6 23:04:54 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 23:04:54 GMT Subject: RFR: 8309420: com/sun/jdi/StepTest.java fails with virtual thread wrapper In-Reply-To: References: Message-ID: On Mon, 5 Jun 2023 04:03:57 GMT, Chris Plummer wrote: > The test has two issues. The first is that it assume that once the VMStart event has arrived and one "step into" is done, it will be in the main method of the debuggee. Once there, it determines the debuggee class name by looking at the classtype of topmost frame. The problems is when using virtual threads, it is actually in TestScaffold.main() at this point, so the wrong class name is gleaned from the frame. To fix this the test just saves away the debuggee class name, which is passed to the test as the 4th argument. > > The other issue is that the test assumes once it gets to the debuggee go() method, there are only two frames on the stack. It's more like 16 when using virtual threads. The test needs to account for this by counting the number of frames when go() is entered rather than assuming it will be 2. > > Tested locally with and without the wrapper and by running tier5 svc tests. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14307#pullrequestreview-1466272033 From amenkov at openjdk.org Tue Jun 6 23:23:55 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 23:23:55 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v5] In-Reply-To: References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: On Tue, 6 Jun 2023 22:54:04 GMT, Serguei Spitsyn wrote: >> The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. >> The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. >> Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. >> >> The list of changes is: >> - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. >> - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. >> - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> >> Testing: >> - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` >> - TBD: tiers 1-6 (all are good) > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: ThreadListStackTracesTest cleanup test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 37: > 35: import java.util.List; > 36: import java.lang.reflect.Constructor; > 37: import java.lang.reflect.InvocationTargetException; I tried to comment all this lines, but something went wrong.. AFAIC the only required import is java.util.concurrent.locks.ReentrantLock, the rest are unused ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220526323 From kdnilsen at openjdk.org Tue Jun 6 23:24:33 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 23:24:33 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v8] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Improve efficiency of card-size alignment calculations ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/cc149904..8f9e2a84 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=06-07 Stats: 8 lines in 1 file changed: 0 ins; 4 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From kdnilsen at openjdk.org Tue Jun 6 23:24:34 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 23:24:34 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> Message-ID: On Fri, 2 Jun 2023 17:55:56 GMT, Y. Srinivas Ramakrishna wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Force PLAB sizes to align on card-table size > > src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1285: > >> 1283: if (unalignment != 0) { >> 1284: word_size = word_size - unalignment + CardTable::card_size_in_words(); >> 1285: } > > Probably not a big deal since this is only used when refilling a PLAB, which is an infrequent operation, but `mod` is an expensive operation, in general, and best to avoid in our code except in assertion checks (or even there given recent experiences with debug tests timing out). Since card size is a power of 2, may be we could use addition and masking instead. Something like defining the following inline in the CardTable class and using it everywhere where card alignment granularity is sought. There may even be a macro or method defined for this already perhaps: > > > (FOO + CardSize - 1) & ~((1 << LogCardSize) - 1) > > > One could even store the mask to avoid the arithmetic to produce the mask although it's pretty cheap. > > This may turn out to be less expensive than mod, test, and branch, but as I said probably not a big deal here. We should make sure we don't overuse mods in our allocation paths much. Thanks for this suggestion. I've modified the code. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1220523992 From amenkov at openjdk.org Tue Jun 6 23:39:55 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 23:39:55 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 22:06:14 GMT, Serguei Spitsyn wrote: >>> Okay, I see you point. Unfortunately, I've always referred the platform thread with an executed FJP schedular as a carrier thread. The term 'carrier' with this meaning is everywhere in the JVMTI code. It looks very confusing to call a thread to be a carrier thread only during some phases of its execution. >> >> Okay, I'm just pointing out that is_passive_carrier_thread looks a bit strange here as it is testing if a JavaThread is carrying a virtual thread oop - it's not testing if the thread is owned by the virtual thread scheduler. > > I'm still thinking what identifier to use instead of `is_passive_carrier_thread`. > Just `is_carrier_thread` is going to be confusing as well. > What about `is_carrier_thread_waiting_for_virtual` or `is_carrier_thread_waiting_for_virtual_to_unmount`? `is_carrying_carrier_thread`? a bit artificial, but it's a carrier thread and it's carrying a virtual thread ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220541192 From kdnilsen at openjdk.org Tue Jun 6 23:42:08 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 6 Jun 2023 23:42:08 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v4] In-Reply-To: <-jrieUm3r32vA5At0baw1nTndtNGoxG6EBrcEDjwyZw=.0a95dc08-1259-418d-a9bb-b2ba86b18c51@github.com> References: <2sgbRGVCiStjmAspEqqpyWAM0IzbZfjFC6HHXlhbcyE=.9637c274-1b10-4103-b528-34719037362b@github.com> <7uARcGDHOuSUugc2zRg7JQgC2dSPBDOjeWGPjBPO2qs=.a09479b8-ba9b-4596-bc5a-7ace0968fe31@github.com> <95yaqYTYoGnlkrDMbvZ-NTyVbGmHrL4DUYYIlV3wkwQ=.b6d787d2-3858-4486-b3dc-428a87969109@github.com> <-jrieUm3r32vA5At0baw1nTndtNGoxG6EBrcEDjwyZw=.0a95dc08-1259-418d-a9bb-b2ba86b18c51@github.com> Message-ID: On Sat, 3 Jun 2023 15:17:37 GMT, Y. Srinivas Ramakrishna wrote: >> Yes. And also from files which were changed by non-Amazon employees only, please. > > Thanks, Martin. Yes, we have noted that there were a few other files that were inadvertently caught in a copyright header dragnet. These will be reviewed and fixed in https://bugs.openjdk.org/browse/JDK-8309392 . I'm fixing this copyright notice and others. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1220542561 From amenkov at openjdk.org Tue Jun 6 23:44:55 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 6 Jun 2023 23:44:55 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 22:41:54 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: call get_thread_state_base only when needed Marked as reviewed by amenkov (Reviewer). src/hotspot/share/prims/jvmtiEnvBase.hpp line 386: > 384: > 385: // get platform thread state > 386: static jint get_thread_state_base(oop thread_oop, JavaThread* jt); maybe rename it to `get_platform_thread_state`? ------------- PR Review: https://git.openjdk.org/jdk/pull/14298#pullrequestreview-1466301122 PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220543652 From sspitsyn at openjdk.org Tue Jun 6 23:44:57 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 23:44:57 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 23:37:03 GMT, Alex Menkov wrote: > is_carrying_carrier_thread? a bit artificial, but it's a carrier thread and it's carrying a virtual thread I guess, your suggestion is `is_carrying_virtual_thread`. Is it right? If so, I like this suggestion. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220546031 From sspitsyn at openjdk.org Tue Jun 6 23:45:22 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 23:45:22 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v6] In-Reply-To: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: > The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. > The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. > Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. > > The list of changes is: > - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. > - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. > - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > > Testing: > - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > - TBD: tiers 1-6 (all are good) Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: remove unused imports from ThreadListStackTracesTest.java ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14326/files - new: https://git.openjdk.org/jdk/pull/14326/files/6b685ca3..7506b539 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14326&range=04-05 Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14326.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14326/head:pull/14326 PR: https://git.openjdk.org/jdk/pull/14326 From sspitsyn at openjdk.org Tue Jun 6 23:45:22 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 23:45:22 GMT Subject: RFR: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread [v5] In-Reply-To: References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: On Tue, 6 Jun 2023 23:20:53 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: ThreadListStackTracesTest cleanup > > test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest/ThreadListStackTracesTest.java line 37: > >> 35: import java.util.List; >> 36: import java.lang.reflect.Constructor; >> 37: import java.lang.reflect.InvocationTargetException; > > I tried to comment all this lines, but something went wrong.. > AFAIC the only required import is java.util.concurrent.locks.ReentrantLock, the rest are unused Thanks. Fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14326#discussion_r1220538628 From sspitsyn at openjdk.org Tue Jun 6 23:56:55 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 6 Jun 2023 23:56:55 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 23:39:54 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: call get_thread_state_base only when needed > > src/hotspot/share/prims/jvmtiEnvBase.hpp line 386: > >> 384: >> 385: // get platform thread state >> 386: static jint get_thread_state_base(oop thread_oop, JavaThread* jt); > > maybe rename it to `get_platform_thread_state`? I was thinking about it. It will be inconsistent with`get_vthread_state`. Ideally, then the `get_vthread_state` needs to be replaced with the `get_virtual_thread_state`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220559583 From kdnilsen at openjdk.org Wed Jun 7 00:39:52 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 00:39:52 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: References: Message-ID: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Update copyright notices ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/8f9e2a84..f6c073a5 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=07-08 Stats: 7 lines in 6 files changed: 0 ins; 5 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From amenkov at openjdk.org Wed Jun 7 01:08:05 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 7 Jun 2023 01:08:05 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 23:42:24 GMT, Serguei Spitsyn wrote: > > is_carrying_carrier_thread? a bit artificial, but it's a carrier thread and it's carrying a virtual thread > > I guess, your suggestion is `is_carrying_virtual_thread`. Is it right? If so, I like this suggestion. Up to you. I think any of this names is better than is_passive_carrier_thread. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220627250 From ysr at openjdk.org Wed Jun 7 03:41:13 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 03:41:13 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices > Hi, I have built this pr based on [aa85a90](https://github.com/openjdk/jdk/commit/aa85a9073e2a71d6bf920409e739d555f9dcf302), Tier1 tests failed on `gc/TestAllocHumongousFragment.java#generational` on Linux/RISC-V with the following output: > > ``` > # > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error (shenandoahVerifier.cpp:1244), pid=2951116, tid=2951124 > # Error: Verify init-mark remembered set violation; clean card should be dirty > # > # JRE version: OpenJDK Runtime Environment (21.0) (build 21-internal-adhoc.ubuntu.jdk) > # Java VM: OpenJDK 64-Bit Server VM (21-internal-adhoc.ubuntu.jdk, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-riscv64) > ``` > > Looks like Generational Shenandoah does not fully support RISC-V port, should we disable this test on RISC-V port for now? Fixed (platform disabled) by @kdnilsen in https://github.com/openjdk/jdk/pull/14185/commits/cc149904d76c78355fc994da171f0f21411e903f ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1579829038 From cjplummer at openjdk.org Wed Jun 7 04:48:59 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 04:48:59 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: Message-ID: <0g78gMSXw683BQh9UYpDXbpDgBOS5DsIP9df5Qr3XUM=.5c557de4-8b29-4a3f-9e83-ed23a853739f@github.com> On Tue, 6 Jun 2023 22:41:54 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: call get_thread_state_base only when needed Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14298#pullrequestreview-1466591721 From alanb at openjdk.org Wed Jun 7 05:53:56 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 7 Jun 2023 05:53:56 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 01:05:07 GMT, Alex Menkov wrote: > I guess, your suggestion is `is_carrying_virtual_thread`. Is it right? If so, I like this suggestion. Good, I think will be easy to understand at the use sites. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1220891958 From avu at openjdk.org Wed Jun 7 07:00:59 2023 From: avu at openjdk.org (Alexey Ushakov) Date: Wed, 7 Jun 2023 07:00:59 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v4] In-Reply-To: References: Message-ID: On Thu, 1 Jun 2023 15:01:37 GMT, Artem Semenov wrote: >> When using the clang compiler to build OpenJDk on Linux, we encounter various "warnings as errors". >> They can be fixed with small changes. > > Artem Semenov has updated the pull request incrementally with one additional commit since the last revision: > > update Marked as reviewed by avu (Committer). The latest changes look good to me. ------------- PR Review: https://git.openjdk.org/jdk/pull/14033#pullrequestreview-1466776433 PR Comment: https://git.openjdk.org/jdk/pull/14033#issuecomment-1580043192 From ysr at openjdk.org Wed Jun 7 07:25:16 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 07:25:16 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp line 4: > 2: * Copyright (c) 2018, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright (c) 2012, 2022 SAP SE. All rights reserved. > 4: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. I believe line 4 should deleted; the copyright header change here is unnecessary. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221032491 From ysr at openjdk.org Wed Jun 7 07:43:20 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 07:43:20 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices src/hotspot/cpu/riscv/gc/shenandoah/shenandoahBarrierSetAssembler_riscv.cpp line 4: > 2: * Copyright (c) 2018, 2020, Red Hat, Inc. All rights reserved. > 3: * Copyright (c) 2020, 2021, Huawei Technologies Co., Ltd. All rights reserved. > 4: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Remove this line; extent of changes doesn't warrant copyright header change. src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.hpp line 3: > 1: /* > 2: * Copyright (c) 2018, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Should probably be removed. src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.hpp line 3: > 1: /* > 2: * Copyright (c) 2018, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Check if this is necessary. src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp line 4: > 2: * Copyright (c) 2015, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright (C) 2022 THL A29 Limited, a Tencent company. All rights reserved. > 4: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Should be removed. src/hotspot/share/gc/shenandoah/heuristics/shenandoahCompactHeuristics.hpp line 3: > 1: /* > 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Can be removed? src/hotspot/share/gc/shenandoah/heuristics/shenandoahPassiveHeuristics.hpp line 3: > 1: /* > 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Should be removed? src/hotspot/share/gc/shenandoah/heuristics/shenandoahStaticHeuristics.hpp line 3: > 1: /* > 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Should be removed? src/hotspot/share/gc/shenandoah/mode/shenandoahPassiveMode.hpp line 3: > 1: /* > 2: * Copyright (c) 2019, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Should be removed? src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp line 3: > 1: /* > 2: * Copyright (c) 2019, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Can be removed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221051315 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221054767 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221056157 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221056909 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221058681 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221060221 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221060613 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221061900 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221063047 From ysr at openjdk.org Wed Jun 7 07:46:20 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 07:46:20 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices src/hotspot/share/gc/shenandoah/shenandoahBarrierSetClone.inline.hpp line 3: > 1: /* > 2: * Copyright (c) 2013, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary. Delete. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221070505 From stuefe at openjdk.org Wed Jun 7 07:50:20 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 7 Jun 2023 07:50:20 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices > Thanks Thomas for the feedback: > > These proposed changes represent improvements to both Generational and Non-generational modes of operation. We can revert if that is desired, or we can specialize Generational versions of these parameters so that they can have different values in different modes, but here is a bit of background. We've done considerable testing on a variety of synthetic workloads and some limited testing on production workloads. As we move towards upstream integration, we expect this will help us gain exposure to more production workloads. The following changes were based on results of this testing: > Hi Kelvin, thanks for the thorough explanations! It is a pity that these valuable insights are buried in a GH discussion and these changes inside such a large patch. I also looked at the originating patch in openjdk/shenandoah, which I assume is your development repo for Shenandoah (?). Could I convince you to adapt the JBS issue process in the shenandoah repo (so, opening an issue on JBS, with some clear explanation, then fixing the bug)? Roman convinced me of this for the Lilliput repository, and now I think the added work is well worth it. JBS is a treasure trove of insights, if filled with care, and can help us for many years. Some more questions about `ShenandoahFullGCThreshold`: I am looking at the nice ASCII art in `ShenandoahControlThread::service_concurrent_normal_cycle`. IIUC, the cycle goes: Concurrent GC -> Alloc failure -> n x Degenerated GC -> Alloc Failure -> Full GC right? So the change is now in how often we try a degenerated GC before falling back to a full GC? With GenShen, does a degenerated GC still collect only the young regions? And only FullGC does collect all regions? Are comment and ASCII-art still correct for GenShen? E.g. the comment says: // If second allocation failure happens during Degenerated GC cycle (for example, when GC // tries to evac something and no memory is available), cycle degrades to Full GC. Is "second allocation failure" correct? Since even before this patch, we tried three times before falling back to a Full GC. Thank you, Thomas ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1580127311 From ysr at openjdk.org Wed Jun 7 07:50:20 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 07:50:20 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: <11_EM-LgCnPY9x_t3s6wz3-oh8eN_e-GMlo2mWtiHbc=.4ade74ee-6f36-4621-b911-dce9e127cd98@github.com> On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.hpp line 3: > 1: /* > 2: * Copyright (c) 2013, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Probably unnecessary. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221076190 From sspitsyn at openjdk.org Wed Jun 7 07:55:13 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 07:55:13 GMT Subject: Integrated: 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread In-Reply-To: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> References: <_bbL6afkGfa1lw1UFa26F4lGRiLQaiIkjo0tkDMUHm4=.0079cdec-34c0-45f4-836f-61118c111f43@github.com> Message-ID: On Tue, 6 Jun 2023 00:50:34 GMT, Serguei Spitsyn wrote: > The `GetThreadListStackTraces` returns `JVMTI_THREAD_STATE_RUNNABLE` for a VirtualThread blocked on a monitor when called for more than one thread. When called for a single VirtualThread it correctly returns a state that includes the `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` flag. > The `VM_GetThreadListStackTraces::doit` should call the `get_threadOop_and_JavaThread` instead of `cv_external_thread_to_JavaThread`. But the `get_threadOop_and_JavaThread` has a check for the current thread by comparing with the JavaThread::current() which does not work for a `VM_op`. Some refactoring of the `get_threadOop_and_JavaThread` was made to make it working for a `VM_op`. > Also, a minor bug in the `GetSingleStackTraceClosure::do_thread()` was discovered during testing. > > The list of changes is: > - minor refactoring of the function`get_threadOop_and_JavaThread`: added an overloaded version of this function with the extra parameter `JavaThread* cur_thread`. It is called instead of `JvmtiExport::cv_external_thread_to_JavaThread` from the `VM_GetThreadListStackTraces::doit`. > - `GetSingleStackTraceClosure::do_thread()`: The use of `jt->threadObj()` is replaced with the `JNIHandles::resolve_external_guard(_jthread)`. > - added new test to provide needed coverage: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > > Testing: > - ran new test: `test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadListStackTracesTest` > - TBD: tiers 1-6 (all are good) This pull request has now been integrated. Changeset: a25b7b8b Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/a25b7b8b55f2dcd3c2945193d78f754580421733 Stats: 222 lines in 5 files changed: 215 ins; 2 del; 5 mod 8295976: GetThreadListStackTraces returns wrong state for blocked VirtualThread Reviewed-by: cjplummer, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/14326 From ysr at openjdk.org Wed Jun 7 07:58:22 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 07:58:22 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.hpp line 3: > 1: /* > 2: * Copyright (c) 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary. src/hotspot/share/gc/shenandoah/shenandoahFullGC.hpp line 3: > 1: /* > 2: * Copyright (c) 2014, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary src/hotspot/share/gc/shenandoah/shenandoahGC.cpp line 3: > 1: /* > 2: * Copyright (c) 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary src/hotspot/share/gc/shenandoah/shenandoahGC.hpp line 3: > 1: /* > 2: * Copyright (c) 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221087455 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221091340 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221092320 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221093246 From ysr at openjdk.org Wed Jun 7 08:12:28 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 08:12:28 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices src/hotspot/share/gc/shenandoah/shenandoahNMethod.cpp line 3: > 1: /* > 2: * Copyright (c) 2019, 2022, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary? src/hotspot/share/gc/shenandoah/shenandoahNumberSeq.hpp line 3: > 1: /* > 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary? src/hotspot/share/gc/shenandoah/shenandoahSTWMark.hpp line 3: > 1: /* > 2: * Copyright (c) 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. unnecessary. src/hotspot/share/gc/shenandoah/shenandoahUnload.cpp line 3: > 1: /* > 2: * Copyright (c) 2019, 2021, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Delete src/hotspot/share/gc/shenandoah/shenandoahWorkerPolicy.cpp line 3: > 1: /* > 2: * Copyright (c) 2017, 2019, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary src/hotspot/share/gc/shenandoah/shenandoahWorkerPolicy.hpp line 3: > 1: /* > 2: * Copyright (c) 2017, 2022, Red Hat, Inc. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. Unnecessary ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221104857 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221106767 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221110530 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221113191 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221115925 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221116941 From ysr at openjdk.org Wed Jun 7 08:17:22 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 08:17:22 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices test/hotspot/gtest/gc/shenandoah/test_shenandoahNumberSeq.cpp line 2: > 1: /* > 2: * Copyright (c) 2022, 2023, Oracle and/or its affiliates. All rights reserved. This may be deleted as far as I can tell, or we can just leave it in there. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221124209 From ysr at openjdk.org Wed Jun 7 08:26:18 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 7 Jun 2023 08:26:18 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: <47uWbayDcFRkkQ5crcpFyIFGvBwM0yCAXvxeIonOdcI=.5387f07c-d89f-4839-acdd-bb49d686a2d7@github.com> On Wed, 7 Jun 2023 00:39:52 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright notices test/hotspot/jtreg/gc/stress/gcold/TestGCOldWithShenandoah.java line 3: > 1: /* > 2: * Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights reserved. > 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. General comment: Looking at the history, I might have expected RedHat copyright headers also for many of these tests, but that isn't a change that's happened with generational shenandoah. So, nothing for us to do in this PR. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221136441 From sspitsyn at openjdk.org Wed Jun 7 11:31:02 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 11:31:02 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v5] In-Reply-To: References: Message-ID: > When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. > The fix adds: > - a special case for passive carrier threads > - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. > > Testing: > - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` > - submitted mach5 tiers 1-5 > - TBD: to submit mach5 tier 6 Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: - Merge - review: call get_thread_state_base only when needed - review: removed JVMTI_THREAD_STATE_RUNNABLE from a carrier thread state - Merge - minor tweaks in libThreadStateTest.cpp - 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14298/files - new: https://git.openjdk.org/jdk/pull/14298/files/77771816..3e7618c4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=03-04 Stats: 14603 lines in 141 files changed: 9240 ins; 4758 del; 605 mod Patch: https://git.openjdk.org/jdk/pull/14298.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14298/head:pull/14298 PR: https://git.openjdk.org/jdk/pull/14298 From sspitsyn at openjdk.org Wed Jun 7 11:31:06 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 11:31:06 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v5] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 05:50:46 GMT, Alan Bateman wrote: >>> > is_carrying_carrier_thread? a bit artificial, but it's a carrier thread and it's carrying a virtual thread >>> >>> I guess, your suggestion is `is_carrying_virtual_thread`. Is it right? If so, I like this suggestion. >> >> Up to you. I think any of this names is better than is_passive_carrier_thread. > >> I guess, your suggestion is `is_carrying_virtual_thread`. Is it right? If so, I like this suggestion. > > Good, I think will be easy to understand at the use sites. Thank you, Alan and Alex. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1221417248 From sspitsyn at openjdk.org Wed Jun 7 11:48:19 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 11:48:19 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v6] In-Reply-To: References: Message-ID: > When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. > The fix adds: > - a special case for passive carrier threads > - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. > > Testing: > - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` > - submitted mach5 tiers 1-5 > - TBD: to submit mach5 tier 6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: one function renaming ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14298/files - new: https://git.openjdk.org/jdk/pull/14298/files/3e7618c4..a6e6c981 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14298&range=04-05 Stats: 8 lines in 2 files changed: 0 ins; 0 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/14298.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14298/head:pull/14298 PR: https://git.openjdk.org/jdk/pull/14298 From sspitsyn at openjdk.org Wed Jun 7 11:53:57 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 11:53:57 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v6] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 11:48:19 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: one function renaming Thank you for review, Alex and Chris. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14298#issuecomment-1580639648 From kdnilsen at openjdk.org Wed Jun 7 12:31:18 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 12:31:18 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 07:22:13 GMT, Y. Srinivas Ramakrishna wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Update copyright notices > > src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp line 4: > >> 2: * Copyright (c) 2018, 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright (c) 2012, 2022 SAP SE. All rights reserved. >> 4: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > I believe line 4 should deleted; the copyright header change here is unnecessary. will remove. I noticed that amazon had also contributed to this file, but changes were very minor. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221506981 From kdnilsen at openjdk.org Wed Jun 7 12:37:42 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 12:37:42 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v10] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Remove one more extraneous Amazon copyright ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/f6c073a5..221c88ff Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=08-09 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From sspitsyn at openjdk.org Wed Jun 7 12:40:08 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 12:40:08 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 Message-ID: This is a minor update of the `jvmti.xml` file. The JVM TI history table needs to be updated to list new capability, functions and events added to support virtual threads as a permanent feature in JDK 21. Also, it should list a minor update with the `Implementation Note` that dynamic loading of agents into a running VM is now specified to print a warning (JEP 451). The JVM TI history table is maintained for convenience only and does not require a CSR. ------------- Commit messages: - 8309602: update JVMTI history table for jdk 21 Changes: https://git.openjdk.org/jdk/pull/14352/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14352&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309602 Stats: 18 lines in 1 file changed: 17 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14352.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14352/head:pull/14352 PR: https://git.openjdk.org/jdk/pull/14352 From kdnilsen at openjdk.org Wed Jun 7 12:44:53 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 12:44:53 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v11] In-Reply-To: References: Message-ID: <1g_0xe2WOXuGOGYZ02225hi4yyz6OKWsRxVNrpZjhhE=.bacf7c2e-24d4-4060-8cff-7e27cd6f3721@github.com> > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: JDK-8309322: [GenShen] TestAllocOutOfMemory#large failed When generational Shenandoah is used, there may be an additional alignment related heap size adjustment that the test should be cognizant of. Such alignment might also happen in the non-generational case, but in this case the specific size used in the test was affected on machines with larger than usual os page size settings. The alignment related adjustment would have affected all generational collectors (except perhaps Gen Z). In the future, we might try and relax this alignment constraint.alignment. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/221c88ff..88958669 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=09-10 Stats: 27 lines in 1 file changed: 16 ins; 0 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From sspitsyn at openjdk.org Wed Jun 7 12:57:34 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 12:57:34 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 [v2] In-Reply-To: References: Message-ID: > This is a minor update of the `jvmti.xml` file. > The JVM TI history table needs to be updated to list new capability, functions and events added to support virtual threads as a permanent feature in JDK 21. Also, it should list a minor update with the `Implementation Note` that dynamic loading of agents into a running VM is now specified to print a warning (JEP 451). > The JVM TI history table is maintained for convenience only and does not require a CSR. Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: simplified the latest history entries ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14352/files - new: https://git.openjdk.org/jdk/pull/14352/files/cb95bf11..a9e4f8eb Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14352&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14352&range=00-01 Stats: 15 lines in 1 file changed: 0 ins; 13 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14352.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14352/head:pull/14352 PR: https://git.openjdk.org/jdk/pull/14352 From alanb at openjdk.org Wed Jun 7 12:58:56 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 7 Jun 2023 12:58:56 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 [v2] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 12:57:34 GMT, Serguei Spitsyn wrote: >> This is a minor update of the `jvmti.xml` file. >> The JVM TI history table needs to be updated to list: >> - Virtual threads finalized to be a permanent feature. >> - Agent start-up in the live phase now specified to print a warning. >> >> The JVM TI history table is maintained for convenience only and does not require a CSR. > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: simplified the latest history entries A small suggestion for the 19.0.0 note, otherwise good. src/hotspot/share/prims/jvmti.xml line 15459: > 15457: > 15458: > 15459: Preview feature - Support for virtual threads: Maybe "Support for virtual threads as preview feature" or "Support for Virtual Threads (Preview)". src/hotspot/share/prims/jvmti.xml line 15474: > 15472: Virtual threads finalized to be a permanent feature. > 15473: Agent start-up in the live phase now specified to print a warning. > 15474: This is just the history text, no normative changes, shouldn't need a CSR. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14352#pullrequestreview-1467588938 PR Review Comment: https://git.openjdk.org/jdk/pull/14352#discussion_r1221548406 PR Review Comment: https://git.openjdk.org/jdk/pull/14352#discussion_r1221549609 From sspitsyn at openjdk.org Wed Jun 7 13:08:00 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 13:08:00 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 [v3] In-Reply-To: References: Message-ID: <2cVTkVoKxRkH6B_a7B6Toc6hK5IWhnwJkBX0hnP_g-0=.e169eba0-bf85-4a9c-b227-211c5315d74a@github.com> > This is a minor update of the `jvmti.xml` file. > The JVM TI history table needs to be updated to list: > - Virtual threads finalized to be a permanent feature. > - Agent start-up in the live phase now specified to print a warning. > > The JVM TI history table is maintained for convenience only and does not require a CSR. Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: minor history table tweak ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14352/files - new: https://git.openjdk.org/jdk/pull/14352/files/a9e4f8eb..11db4f4f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14352&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14352&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14352.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14352/head:pull/14352 PR: https://git.openjdk.org/jdk/pull/14352 From sspitsyn at openjdk.org Wed Jun 7 13:08:03 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 13:08:03 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 [v2] In-Reply-To: References: Message-ID: <3iju4FBlvcDEeRLIAUCPoV-nZEIuJnrka8cSN-qFmUA=.ab40cb6b-31be-4ecc-aa15-040f1dc9ffac@github.com> On Wed, 7 Jun 2023 12:55:27 GMT, Alan Bateman wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: simplified the latest history entries > > src/hotspot/share/prims/jvmti.xml line 15459: > >> 15457: >> 15458: >> 15459: Preview feature - Support for virtual threads: > > Maybe "Support for virtual threads as preview feature" or "Support for Virtual Threads (Preview)". Thanks. Fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14352#discussion_r1221558022 From sspitsyn at openjdk.org Wed Jun 7 13:09:57 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 13:09:57 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 [v2] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 12:56:18 GMT, Alan Bateman wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: simplified the latest history entries > > src/hotspot/share/prims/jvmti.xml line 15474: > >> 15472: Virtual threads finalized to be a permanent feature. >> 15473: Agent start-up in the live phase now specified to print a warning. >> 15474: > > This is just the history text, no normative changes, shouldn't need a CSR. Okay, thanks. Updated the description. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14352#discussion_r1221564482 From sspitsyn at openjdk.org Wed Jun 7 13:20:02 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 13:20:02 GMT Subject: Integrated: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: On Sat, 3 Jun 2023 10:53:04 GMT, Serguei Spitsyn wrote: > When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. > The fix adds: > - a special case for passive carrier threads > - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. > > Testing: > - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` > - submitted mach5 tiers 1-5 > - TBD: to submit mach5 tier 6 This pull request has now been integrated. Changeset: 177e8327 Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/177e8327d685444d63235567f2a9bde0ec3d51cf Stats: 82 lines in 4 files changed: 65 ins; 0 del; 17 mod 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING Reviewed-by: amenkov, cjplummer ------------- PR: https://git.openjdk.org/jdk/pull/14298 From alanb at openjdk.org Wed Jun 7 13:27:06 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 7 Jun 2023 13:27:06 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v6] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 11:48:19 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: one function renaming src/hotspot/share/prims/jvmtiEnvBase.cpp line 1741: > 1739: "sanity check"); > 1740: > 1741: // An attempt to handshake-suspend a passive carrier thread will result in The rename from is_passive_carrier_thread to is_thread_carrying_vthread looks fine. There are a few stray comments that still say "passive carrier thread" that probably should be cleaned up. I see you've just integrated this change but maybe the next change in the area that do this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1221598356 From sspitsyn at openjdk.org Wed Jun 7 13:36:06 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 13:36:06 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v6] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 13:24:32 GMT, Alan Bateman wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: one function renaming > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 1741: > >> 1739: "sanity check"); >> 1740: >> 1741: // An attempt to handshake-suspend a passive carrier thread will result in > > The rename from is_passive_carrier_thread to is_thread_carrying_vthread looks fine. There are a few stray comments that still say "passive carrier thread" that probably should be cleaned up. I see you've just integrated this change but maybe the next change in the area that do this. Thanks. Alan. Will do this cleanup when there is a chance. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14298#discussion_r1221612240 From kdnilsen at openjdk.org Wed Jun 7 13:37:44 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 13:37:44 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v12] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Remove more extraneous copyright notices ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/88958669..8e5c3b73 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=11 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=10-11 Stats: 10 lines in 10 files changed: 0 ins; 10 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From kdnilsen at openjdk.org Wed Jun 7 13:37:57 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 13:37:57 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 07:54:17 GMT, Y. Srinivas Ramakrishna wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Update copyright notices > > src/hotspot/share/gc/shenandoah/shenandoahFullGC.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2014, 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary fixed. > src/hotspot/share/gc/shenandoah/shenandoahGC.cpp line 3: > >> 1: /* >> 2: * Copyright (c) 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary fixed. > src/hotspot/share/gc/shenandoah/shenandoahGC.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary fixed. > src/hotspot/share/gc/shenandoah/shenandoahNMethod.cpp line 3: > >> 1: /* >> 2: * Copyright (c) 2019, 2022, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary? fixed. > src/hotspot/share/gc/shenandoah/shenandoahNumberSeq.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary? fixed. > src/hotspot/share/gc/shenandoah/shenandoahSTWMark.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > unnecessary. fixed. > src/hotspot/share/gc/shenandoah/shenandoahUnload.cpp line 3: > >> 1: /* >> 2: * Copyright (c) 2019, 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Delete fixed. > src/hotspot/share/gc/shenandoah/shenandoahWorkerPolicy.cpp line 3: > >> 1: /* >> 2: * Copyright (c) 2017, 2019, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary fixed. > src/hotspot/share/gc/shenandoah/shenandoahWorkerPolicy.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2017, 2022, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary fixed. > test/hotspot/gtest/gc/shenandoah/test_shenandoahNumberSeq.cpp line 2: > >> 1: /* >> 2: * Copyright (c) 2022, 2023, Oracle and/or its affiliates. All rights reserved. > > This may be deleted as far as I can tell, or we can just leave it in there. will leave as is. > test/hotspot/jtreg/gc/stress/gcold/TestGCOldWithShenandoah.java line 3: > >> 1: /* >> 2: * Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > General comment: Looking at the history, I might have expected RedHat copyright headers also for many of these tests, but that isn't a change that's happened with generational shenandoah. So, nothing for us to do in this PR. ok. will leave as is. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221595485 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221596525 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221597200 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221597834 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221598462 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221599944 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221599533 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221600615 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221601596 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221602442 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221604163 From duke at openjdk.org Wed Jun 7 13:46:05 2023 From: duke at openjdk.org (JoKern65) Date: Wed, 7 Jun 2023 13:46:05 GMT Subject: Integrated: JDK-8309225: Fix xlc17 clang 15 warnings in security and servicability In-Reply-To: References: Message-ID: On Fri, 2 Jun 2023 10:19:53 GMT, JoKern65 wrote: > This pr is a split off from JDK-8308288: Fix xlc17 clang warnings in shared code https://github.com/openjdk/jdk/pull/14146 > It handles the part in security and servicability. > > Compiling on AIX with xlc17 which contains the new clang 15 frontend shows the following warnings: > > src/java.security.jgss/share/native/libj2gss/NativeUtil.h:30: > src/java.security.jgss/share/native/libj2gss/gssapi.h:48:5: error: 'TARGET_OS_MAC' is not defined, evaluates to 0 [-Werror,-Wundef] > #if TARGET_OS_MAC && (defined(ppc) || defined(ppc64) || defined(i386) || defined(x86_64)) > ^ > TARGET_OS_MAC is not defined. Instead of disabling the warning, I could > ` #ifndef TARGET_OS_MAC` > `#define TARGET_OS_MAC=0` > `#endif` > But this is already handled by disabling the warning for gcc. > > src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:718:33: error: suggest braces around initialization of subobject [-Werror,-Wmissing-braces] > struct in6_addr mappedAny = IN6ADDR_ANY_INIT; > ^~~~~~~~~~~~~~~~ > /usr/include/netinet/in.h:454:32: note: expanded from macro 'IN6ADDR_ANY_INIT' > #define IN6ADDR_ANY_INIT {0, 0, 0, 0} This pull request has now been integrated. Changeset: 89f5baca Author: JoKern65 Committer: Martin Doerr URL: https://git.openjdk.org/jdk/commit/89f5bacaf6ac6d5b3634db2fcde5b9abdc492b64 Stats: 4 lines in 2 files changed: 2 ins; 0 del; 2 mod 8309225: Fix xlc17 clang 15 warnings in security and servicability Reviewed-by: goetz, mdoerr, clanger ------------- PR: https://git.openjdk.org/jdk/pull/14282 From duke at openjdk.org Wed Jun 7 13:52:54 2023 From: duke at openjdk.org (JoKern65) Date: Wed, 7 Jun 2023 13:52:54 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 09:51:09 GMT, JoKern65 wrote: > The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 > The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. > So we have (at least) 3 ways to circumvent this behavior. > > 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): > `#include ` > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` while (1) {` > ` sleep(1);` > ` }` > `}` > > 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` volatile int i = 1;` > ` while (i) {` > ` i += 2;` > ` }` > `}` > > 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` > `{` > ` while (1) {` > ` }` > `}` > > To make the third approach platform independent, we can implement it in the following way: > In globalDefinitions.hpp > `#ifndef OPTNONE` > `#define OPTNONE` > `#endif` > > In globalDefinitions_xlc.hpp > `// optnone support` > `//` > `// To use if a function should not be optimized` > `// Usage:` > `// void* func(size_t size) OPTNONE {...}` > `#define OPTNONE __attribute__(( optnone))` > > With this we can change libagentthr001.cpp in a platform independent way to > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` > `{` > ` while (1) {` > ` }` > `}` Pre-submit failures: C:/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/11.0.19-7/x64/bin/javac: Bad address on both windows, unrelated to this change. This was seen lately repeatedly. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14330#issuecomment-1580865446 From duke at openjdk.org Wed Jun 7 14:01:33 2023 From: duke at openjdk.org (JoKern65) Date: Wed, 7 Jun 2023 14:01:33 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop [v2] In-Reply-To: References: Message-ID: > The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 > The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. > So we have (at least) 3 ways to circumvent this behavior. > > 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): > `#include ` > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` while (1) {` > ` sleep(1);` > ` }` > `}` > > 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` volatile int i = 1;` > ` while (i) {` > ` i += 2;` > ` }` > `}` > > 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` > `{` > ` while (1) {` > ` }` > `}` > > To make the third approach platform independent, we can implement it in the following way: > In globalDefinitions.hpp > `#ifndef OPTNONE` > `#define OPTNONE` > `#endif` > > In globalDefinitions_xlc.hpp > `// optnone support` > `//` > `// To use if a function should not be optimized` > `// Usage:` > `// void* func(size_t size) OPTNONE {...}` > `#define OPTNONE __attribute__(( optnone))` > > With this we can change libagentthr001.cpp in a platform independent way to > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` > `{` > ` while (1) {` > ` }` > `}` JoKern65 has updated the pull request incrementally with one additional commit since the last revision: switched to solution two ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14330/files - new: https://git.openjdk.org/jdk/pull/14330/files/ae0d67ae..e44edba6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14330&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14330&range=00-01 Stats: 32 lines in 2 files changed: 29 ins; 1 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14330.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14330/head:pull/14330 PR: https://git.openjdk.org/jdk/pull/14330 From duke at openjdk.org Wed Jun 7 14:04:14 2023 From: duke at openjdk.org (JoKern65) Date: Wed, 7 Jun 2023 14:04:14 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop [v3] In-Reply-To: References: Message-ID: <-A8xf1yGnRBGrKQ069bz9f60DUOncpaoGSnEbNsGA34=.a754f901-62e0-41cc-9a20-be3ca708a27f@github.com> > The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 > The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. > So we have (at least) 3 ways to circumvent this behavior. > > 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): > `#include ` > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` while (1) {` > ` sleep(1);` > ` }` > `}` > > 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` volatile int i = 1;` > ` while (i) {` > ` i += 2;` > ` }` > `}` > > 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` > `{` > ` while (1) {` > ` }` > `}` > > To make the third approach platform independent, we can implement it in the following way: > In globalDefinitions.hpp > `#ifndef OPTNONE` > `#define OPTNONE` > `#endif` > > In globalDefinitions_xlc.hpp > `// optnone support` > `//` > `// To use if a function should not be optimized` > `// Usage:` > `// void* func(size_t size) OPTNONE {...}` > `#define OPTNONE __attribute__(( optnone))` > > With this we can change libagentthr001.cpp in a platform independent way to > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` > `{` > ` while (1) {` > ` }` > `}` JoKern65 has updated the pull request incrementally with one additional commit since the last revision: remove d042520_libagentthr.diff ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14330/files - new: https://git.openjdk.org/jdk/pull/14330/files/e44edba6..e49f2759 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14330&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14330&range=01-02 Stats: 26 lines in 1 file changed: 0 ins; 26 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14330.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14330/head:pull/14330 PR: https://git.openjdk.org/jdk/pull/14330 From kdnilsen at openjdk.org Wed Jun 7 14:09:23 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 14:09:23 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v12] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 13:37:44 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove more extraneous copyright notices Hi Thomas, Thank you for your followup comments. I am in total agreement that it is a shame the challenges we have faced and the progress we have made is not better documented in the history of JBS tickets. I have been the worst offender. I apologize. One aspect of this problem is that our work has included a large degree of uncertainty and "research", and it is not always clear to us what needs to be addressed until after we finish and test certain fixes as integrated with a variety of other fixes. We will commit to being more engaged with JBS from this point forward, both for any further work done on the Shenandoah branch, and definitely for work done on tip. You are correct that the change is to N, the number of times in a row that we perform degenerated GC before we automatically upgrade to Full GC. It is still possible that we will upgrade to Full GC before N is reached, because there are other situations, such as lack of progress by degenerated GC, that will cause us to upgrade to Full even before N is reached. The comment is still valid as written. During degenerated GC, the mutator threads are all blocked, so the ONLY kind of allocation failure that can occur during degenerated GC is a GC-worker-thread allocation for the purpose of evacuating memory. If we experience an "evacuation failure" during degenerated GC. we will upgrade to Full GC. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1580896496 From kdnilsen at openjdk.org Wed Jun 7 14:09:25 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 14:09:25 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: <4oGdxjol_Xzl_3bE1LgSwsvWzfNgt7yli7dqVBc5yb8=.bab01852-5037-4b19-bc81-f67b6dd13611@github.com> On Wed, 7 Jun 2023 07:52:35 GMT, Y. Srinivas Ramakrishna wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Update copyright notices > > src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary. Thanks Ramki for sifting through these again. Sorry I missed so many. I'm making your suggested fixes. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221612109 From mdoerr at openjdk.org Wed Jun 7 14:10:58 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 7 Jun 2023 14:10:58 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop [v3] In-Reply-To: <-A8xf1yGnRBGrKQ069bz9f60DUOncpaoGSnEbNsGA34=.a754f901-62e0-41cc-9a20-be3ca708a27f@github.com> References: <-A8xf1yGnRBGrKQ069bz9f60DUOncpaoGSnEbNsGA34=.a754f901-62e0-41cc-9a20-be3ca708a27f@github.com> Message-ID: On Wed, 7 Jun 2023 14:04:14 GMT, JoKern65 wrote: >> The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 >> The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. >> So we have (at least) 3 ways to circumvent this behavior. >> >> 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): >> `#include ` >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` >> `{` >> ` while (1) {` >> ` sleep(1);` >> ` }` >> `}` >> >> 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` >> `{` >> ` volatile int i = 1;` >> ` while (i) {` >> ` i += 2;` >> ` }` >> `}` >> >> 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` >> `{` >> ` while (1) {` >> ` }` >> `}` >> >> To make the third approach platform independent, we can implement it in the following way: >> In globalDefinitions.hpp >> `#ifndef OPTNONE` >> `#define OPTNONE` >> `#endif` >> >> In globalDefinitions_xlc.hpp >> `// optnone support` >> `//` >> `// To use if a function should not be optimized` >> `// Usage:` >> `// void* func(size_t size) OPTNONE {...}` >> `#define OPTNONE __attribute__(( optnone))` >> >> With this we can change libagentthr001.cpp in a platform independent way to >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` >> `{` >> ` while (1) {` >> ` }` >> `}` > > JoKern65 has updated the pull request incrementally with one additional commit since the last revision: > > remove d042520_libagentthr.diff I don't like empty endless loops in general, but I think your solution is ok for a test. ------------- Marked as reviewed by mdoerr (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14330#pullrequestreview-1467779695 From kdnilsen at openjdk.org Wed Jun 7 14:38:29 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 14:38:29 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v9] In-Reply-To: References: <5tvdtUZnFnmoUyyVFYoOZm_KtVSzjfzBI7aPXpCpgVw=.2c09ff27-4004-44f3-b86d-a88d2f43a2a8@github.com> Message-ID: On Wed, 7 Jun 2023 07:34:44 GMT, Y. Srinivas Ramakrishna wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Update copyright notices > > src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2018, 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Should probably be removed. removed. > src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2018, 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Check if this is necessary. ok. i'll remove. > src/hotspot/share/gc/shenandoah/heuristics/shenandoahCompactHeuristics.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Can be removed? removed. > src/hotspot/share/gc/shenandoah/heuristics/shenandoahPassiveHeuristics.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Should be removed? removed. > src/hotspot/share/gc/shenandoah/heuristics/shenandoahStaticHeuristics.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2018, 2019, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Should be removed? removed. > src/hotspot/share/gc/shenandoah/mode/shenandoahPassiveMode.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2019, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Should be removed? removed. > src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp line 3: > >> 1: /* >> 2: * Copyright (c) 2019, 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Can be removed. removed. > src/hotspot/share/gc/shenandoah/shenandoahBarrierSetClone.inline.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2013, 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Unnecessary. Delete. removed. > src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.hpp line 3: > >> 1: /* >> 2: * Copyright (c) 2013, 2021, Red Hat, Inc. All rights reserved. >> 3: * Copyright Amazon.com Inc. or its affiliates. All Rights Reserved. > > Probably unnecessary. removed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221700458 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221704676 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221706236 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221707142 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221709229 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221710192 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221711143 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221712173 PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1221713033 From dcubed at openjdk.org Wed Jun 7 14:42:04 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 7 Jun 2023 14:42:04 GMT Subject: RFR: 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING Message-ID: This reverts commit 177e8327d685444d63235567f2a9bde0ec3d51cf. ------------- Commit messages: - Revert "8307153: JVMTI GetThreadState on carrier should return STATE_WAITING" Changes: https://git.openjdk.org/jdk/pull/14359/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14359&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309614 Stats: 82 lines in 4 files changed: 0 ins; 65 del; 17 mod Patch: https://git.openjdk.org/jdk/pull/14359.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14359/head:pull/14359 PR: https://git.openjdk.org/jdk/pull/14359 From dcubed at openjdk.org Wed Jun 7 14:44:10 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 7 Jun 2023 14:44:10 GMT Subject: RFR: 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING [v6] In-Reply-To: References: Message-ID: <89TTwIzHO2gB9XwBttdaPsa09Y00QHzAGZ3YG3IG2Z8=.8d977a17-207c-4728-b6d2-f3d7c12aa13c@github.com> On Wed, 7 Jun 2023 11:48:19 GMT, Serguei Spitsyn wrote: >> When a virtual thread is mounted, the carrier thread should be reported as "waiting" until the virtual thread unmounts. Right now, GetThreadState reports a state based the JavaThread status when it should return JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY. >> The fix adds: >> - a special case for passive carrier threads >> - necessary test coverage to the existing JVMTI test: `serviceability/jvmti/vthread/ThreadStateTest`. >> >> Testing: >> - tested with the updated test: `serviceability/jvmti/vthread/ThreadStateTest` >> - submitted mach5 tiers 1-5 >> - TBD: to submit mach5 tier 6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: one function renaming Looks like this PR has caused regression failures in Tier1. We have between 2 and 5 failures per Tier1. See: [JDK-8309612](https://bugs.openjdk.org/browse/JDK-8309612) serviceability/jvmti/vthread/SuspendResume1/SuspendResume1.java#default fails after JDK-8307153 Because this failure is happening in Tier1, combined with the fact that we get much more JVM/TI testing in the upper Tiers, and tomorrow is the code-fork I'm proceeding with a [BACKOUT] and am testing that [BACKOUT] with an urgent Tier1 right now. See: [JDK-8309614](https://bugs.openjdk.org/browse/JDK-8309614) [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING ------------- PR Comment: https://git.openjdk.org/jdk/pull/14298#issuecomment-1580968712 From azvegint at openjdk.org Wed Jun 7 14:46:55 2023 From: azvegint at openjdk.org (Alexander Zvegintsev) Date: Wed, 7 Jun 2023 14:46:55 GMT Subject: RFR: 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: <_Tr0APesEl_SWoQt5z76pgNj9x6BVD2Nh1pQhaIysIo=.66d68136-15e5-4aad-8992-4207c62f000d@github.com> On Wed, 7 Jun 2023 14:33:51 GMT, Daniel D. Daugherty wrote: > This reverts commit 177e8327d685444d63235567f2a9bde0ec3d51cf. Marked as reviewed by azvegint (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14359#pullrequestreview-1467877421 From kdnilsen at openjdk.org Wed Jun 7 14:47:13 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 14:47:13 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v13] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Remove a few more unneeded copyright notices ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/8e5c3b73..01c62516 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=12 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=11-12 Stats: 9 lines in 9 files changed: 0 ins; 9 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From dcubed at openjdk.org Wed Jun 7 14:50:55 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 7 Jun 2023 14:50:55 GMT Subject: RFR: 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: <_Tr0APesEl_SWoQt5z76pgNj9x6BVD2Nh1pQhaIysIo=.66d68136-15e5-4aad-8992-4207c62f000d@github.com> References: <_Tr0APesEl_SWoQt5z76pgNj9x6BVD2Nh1pQhaIysIo=.66d68136-15e5-4aad-8992-4207c62f000d@github.com> Message-ID: On Wed, 7 Jun 2023 14:43:45 GMT, Alexander Zvegintsev wrote: >> This reverts commit 177e8327d685444d63235567f2a9bde0ec3d51cf. > > Marked as reviewed by azvegint (Reviewer). @azvegint - Thanks for the fast review! I'm still waiting for my urgent Tier1 test results... ------------- PR Comment: https://git.openjdk.org/jdk/pull/14359#issuecomment-1580982837 From clanger at openjdk.org Wed Jun 7 14:54:57 2023 From: clanger at openjdk.org (Christoph Langer) Date: Wed, 7 Jun 2023 14:54:57 GMT Subject: RFR: JDK-8309462: vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop [v3] In-Reply-To: <-A8xf1yGnRBGrKQ069bz9f60DUOncpaoGSnEbNsGA34=.a754f901-62e0-41cc-9a20-be3ca708a27f@github.com> References: <-A8xf1yGnRBGrKQ069bz9f60DUOncpaoGSnEbNsGA34=.a754f901-62e0-41cc-9a20-be3ca708a27f@github.com> Message-ID: On Wed, 7 Jun 2023 14:04:14 GMT, JoKern65 wrote: >> The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 >> The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. >> So we have (at least) 3 ways to circumvent this behavior. >> >> 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): >> `#include ` >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` >> `{` >> ` while (1) {` >> ` sleep(1);` >> ` }` >> `}` >> >> 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` >> `{` >> ` volatile int i = 1;` >> ` while (i) {` >> ` i += 2;` >> ` }` >> `}` >> >> 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` >> `{` >> ` while (1) {` >> ` }` >> `}` >> >> To make the third approach platform independent, we can implement it in the following way: >> In globalDefinitions.hpp >> `#ifndef OPTNONE` >> `#define OPTNONE` >> `#endif` >> >> In globalDefinitions_xlc.hpp >> `// optnone support` >> `//` >> `// To use if a function should not be optimized` >> `// Usage:` >> `// void* func(size_t size) OPTNONE {...}` >> `#define OPTNONE __attribute__(( optnone))` >> >> With this we can change libagentthr001.cpp in a platform independent way to >> `static void` >> `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` >> `{` >> ` while (1) {` >> ` }` >> `}` > > JoKern65 has updated the pull request incrementally with one additional commit since the last revision: > > remove d042520_libagentthr.diff Marked as reviewed by clanger (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14330#pullrequestreview-1467898282 From kdnilsen at openjdk.org Wed Jun 7 15:02:34 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 15:02:34 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v14] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Simplify test logic, fail if name of Shenandoah young gen pool changes (#3) ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/01c62516..240d413d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=13 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=12-13 Stats: 13 lines in 1 file changed: 0 ins; 9 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From dcubed at openjdk.org Wed Jun 7 15:07:56 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 7 Jun 2023 15:07:56 GMT Subject: RFR: 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 14:33:51 GMT, Daniel D. Daugherty wrote: > This reverts commit 177e8327d685444d63235567f2a9bde0ec3d51cf. The linux-x64-debug config has passed with the [BACKOUT]. That config failed in each of the three Tier1 job sets that had sightings of this failure mode. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14359#issuecomment-1581015467 From dcubed at openjdk.org Wed Jun 7 15:15:55 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 7 Jun 2023 15:15:55 GMT Subject: RFR: 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: <9G1JkknuhUm2cDiBnCfkEU3pbTN2a4PeHkMdUWpjmzE=.2b6ab4d8-886e-4105-b550-cd9a50f60224@github.com> On Wed, 7 Jun 2023 14:33:51 GMT, Daniel D. Daugherty wrote: > This reverts commit 177e8327d685444d63235567f2a9bde0ec3d51cf. linux-aarch64-debug has now passed with the [BACKOUT]. That config failed in 2 of the 3 Tier1 job sets that had sightings of this failure mode. windows-x64-debug has now passed with the [BACKOUT]. That config failed in 1 of the 3 Tier1 job sets that had sightings of this failure mode. macosx-x64-debug has now passed with the [BACKOUT]. That config failed in 2 of the 3 Tier1 job sets that had sightings of this failure mode. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14359#issuecomment-1581027842 PR Comment: https://git.openjdk.org/jdk/pull/14359#issuecomment-1581030425 PR Comment: https://git.openjdk.org/jdk/pull/14359#issuecomment-1581033097 From dcubed at openjdk.org Wed Jun 7 15:21:04 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 7 Jun 2023 15:21:04 GMT Subject: Integrated: 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 14:33:51 GMT, Daniel D. Daugherty wrote: > This reverts commit 177e8327d685444d63235567f2a9bde0ec3d51cf. This pull request has now been integrated. Changeset: 33bb64f2 Author: Daniel D. Daugherty URL: https://git.openjdk.org/jdk/commit/33bb64f24fdffdb2b1a5f21ff432b1cee9ead498 Stats: 82 lines in 4 files changed: 0 ins; 65 del; 17 mod 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING Reviewed-by: azvegint ------------- PR: https://git.openjdk.org/jdk/pull/14359 From dcubed at openjdk.org Wed Jun 7 15:24:03 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 7 Jun 2023 15:24:03 GMT Subject: RFR: 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 14:33:51 GMT, Daniel D. Daugherty wrote: > This reverts commit 177e8327d685444d63235567f2a9bde0ec3d51cf. macosx-aarch64-debug has now passed with the [BACKOUT]. That config failed in 2 of the 3 Tier1 job sets that had sightings of this failure mode. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14359#issuecomment-1581049185 From iris at openjdk.org Wed Jun 7 15:41:55 2023 From: iris at openjdk.org (Iris Clark) Date: Wed, 7 Jun 2023 15:41:55 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 [v3] In-Reply-To: <2cVTkVoKxRkH6B_a7B6Toc6hK5IWhnwJkBX0hnP_g-0=.e169eba0-bf85-4a9c-b227-211c5315d74a@github.com> References: <2cVTkVoKxRkH6B_a7B6Toc6hK5IWhnwJkBX0hnP_g-0=.e169eba0-bf85-4a9c-b227-211c5315d74a@github.com> Message-ID: On Wed, 7 Jun 2023 13:08:00 GMT, Serguei Spitsyn wrote: >> This is a minor update of the `jvmti.xml` file. >> The JVM TI history table needs to be updated to list: >> - Virtual threads finalized to be a permanent feature. >> - Agent start-up in the live phase now specified to print a warning. >> >> The JVM TI history table has no normative changes. This update does not need a CSR. > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: minor history table tweak Thanks! ------------- Marked as reviewed by iris (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14352#pullrequestreview-1468021106 From kdnilsen at openjdk.org Wed Jun 7 16:00:37 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 16:00:37 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v15] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with 174 additional commits since the last revision: - 8309614: [BACKOUT] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING Reviewed-by: azvegint - 8308288: Fix xlc17 clang warnings and build errors in hotspot Reviewed-by: goetz, mbaesken - 8309225: Fix xlc17 clang 15 warnings in security and servicability Reviewed-by: goetz, mdoerr, clanger - 8309219: Fix xlc17 clang 15 warnings in java.base Reviewed-by: goetz, mdoerr - 8307153: JVMTI GetThreadState on carrier should return STATE_WAITING Reviewed-by: amenkov, cjplummer - 8309543: Micro-optimize x86 assembler UseCondCardMark Reviewed-by: kvn, mdoerr - 8280982: [Wayland] [XWayland] java.awt.Robot taking screenshots Reviewed-by: prr, kizune, psadhukhan - 8309550: jdk.jfr.internal.Utils::formatDataAmount method should gracefully handle amounts equal to Long.MIN_VALUE Reviewed-by: stuefe, mgronlun - 8308445: Linker should check that capture state segment is big enough Reviewed-by: mcimadamore - 8308031: Linkers should reject unpromoted variadic parameters Reviewed-by: mcimadamore - ... and 164 more: https://git.openjdk.org/jdk/compare/240d413d...8b2edd9c ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/240d413d..8b2edd9c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=14 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=13-14 Stats: 55913 lines in 825 files changed: 45254 ins; 7582 del; 3077 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From kdnilsen at openjdk.org Wed Jun 7 16:51:38 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 16:51:38 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v16] In-Reply-To: References: Message-ID: <9vJprjFu5K2ZymOM-lzCm_IAA2pAnGBgCDAtwQ0sOvw=.47abdb57-237b-4c57-a5e1-25f11ee14a80@github.com> > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 305 commits: - Merge branch 'master' of https://git.openjdk.org/jdk into merge-generational-shenandoah - Simplify test logic, fail if name of Shenandoah young gen pool changes (#3) - Remove a few more unneeded copyright notices - Remove more extraneous copyright notices - JDK-8309322: [GenShen] TestAllocOutOfMemory#large failed When generational Shenandoah is used, there may be an additional alignment related heap size adjustment that the test should be cognizant of. Such alignment might also happen in the non-generational case, but in this case the specific size used in the test was affected on machines with larger than usual os page size settings. The alignment related adjustment would have affected all generational collectors (except perhaps Gen Z). In the future, we might try and relax this alignment constraint.alignment. - Remove one more extraneous Amazon copyright - Update copyright notices - Improve efficiency of card-size alignment calculations - Exit during initialization on unsupported platforms - Remove an inappropriate copyright notice - ... and 295 more: https://git.openjdk.org/jdk/compare/33bb64f2...612072a4 ------------- Changes: https://git.openjdk.org/jdk/pull/14185/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=15 Stats: 20143 lines in 202 files changed: 18218 ins; 916 del; 1009 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From cjplummer at openjdk.org Wed Jun 7 17:51:59 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 17:51:59 GMT Subject: RFR: 8309509: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java fails with virtual test thread factory In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 00:02:47 GMT, Chris Plummer wrote: > The test fails with the virtual test thread factory because it tries to find the "main" thread in the list of threads returned by JDI, but "main" is a virtual thread and will only be returned by JDI if the debug agent is launched with includevirtualthreads=y. As a result the thread is not found and the test asserts: > > java.lang.RuntimeException: assertTrue: expected true, was false > at jdk.test.lib.Asserts.fail(Asserts.java:594) > at jdk.test.lib.Asserts.assertTrue(Asserts.java:486) > at jdk.test.lib.Asserts.assertTrue(Asserts.java:472) > at TestNestmateAttr.checkGoodTransforms(TestNestmateAttr.java:511) > at TestNestmateAttr.methodEntered(TestNestmateAttr.java:320) > at TestScaffold$EventHandler.notifyEvent(TestScaffold.java:205) > at TestScaffold$EventHandler.run(TestScaffold.java:279) > at java.base/java.lang.Thread.run(Thread.java:1583) > > The fix is to always run the debug agent with includevirtualthreads=y. > > Tested by running all com/sun/jdi tests locally with and without the virtual test thread factory. Also ran tier1 and tier5 svc test tasks. Thanks for the reviews Alex and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14324#issuecomment-1581258561 From cjplummer at openjdk.org Wed Jun 7 17:51:59 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 17:51:59 GMT Subject: Integrated: 8309509: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java fails with virtual test thread factory In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 00:02:47 GMT, Chris Plummer wrote: > The test fails with the virtual test thread factory because it tries to find the "main" thread in the list of threads returned by JDI, but "main" is a virtual thread and will only be returned by JDI if the debug agent is launched with includevirtualthreads=y. As a result the thread is not found and the test asserts: > > java.lang.RuntimeException: assertTrue: expected true, was false > at jdk.test.lib.Asserts.fail(Asserts.java:594) > at jdk.test.lib.Asserts.assertTrue(Asserts.java:486) > at jdk.test.lib.Asserts.assertTrue(Asserts.java:472) > at TestNestmateAttr.checkGoodTransforms(TestNestmateAttr.java:511) > at TestNestmateAttr.methodEntered(TestNestmateAttr.java:320) > at TestScaffold$EventHandler.notifyEvent(TestScaffold.java:205) > at TestScaffold$EventHandler.run(TestScaffold.java:279) > at java.base/java.lang.Thread.run(Thread.java:1583) > > The fix is to always run the debug agent with includevirtualthreads=y. > > Tested by running all com/sun/jdi tests locally with and without the virtual test thread factory. Also ran tier1 and tier5 svc test tasks. This pull request has now been integrated. Changeset: a54f4d4a Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/a54f4d4ab9537ac7e070dd82e26f73c90d82290a Stats: 7 lines in 2 files changed: 3 ins; 2 del; 2 mod 8309509: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java fails with virtual test thread factory Reviewed-by: sspitsyn, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/14324 From cjplummer at openjdk.org Wed Jun 7 18:00:32 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 18:00:32 GMT Subject: RFR: 8309510: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java no longer needs to override startUp() method In-Reply-To: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> References: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> Message-ID: On Tue, 6 Jun 2023 00:36:12 GMT, Chris Plummer wrote: > com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java currently overrides the TestScaffold.startup() method: > > // override this to correct a bug so arguments can be passed to > // the Target class > protected void startUp(String targetName) { > List argList = new ArrayList<>(Arrays.asList(args)); > argList.add(0, targetName); // pre-pend so it becomes the first "app" arg > println("run args: " + argList); > connect((String[]) argList.toArray(args)); > waitForVMStart(); > } > > This issue of passing app args was fixed recently by [JDK-8308481](https://bugs.openjdk.org/browse/JDK-8308481), so the override is no longer needed. Thanks for the reviews Alex and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14325#issuecomment-1581267545 From cjplummer at openjdk.org Wed Jun 7 18:00:33 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 18:00:33 GMT Subject: Integrated: 8309510: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java no longer needs to override startUp() method In-Reply-To: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> References: <6kWHttu7fg50M_7FJ_zGVAbJl-UrWu6VfPMwyFCEUUk=.d6e1fcea-f821-4adb-92f4-217051c572e7@github.com> Message-ID: On Tue, 6 Jun 2023 00:36:12 GMT, Chris Plummer wrote: > com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java currently overrides the TestScaffold.startup() method: > > // override this to correct a bug so arguments can be passed to > // the Target class > protected void startUp(String targetName) { > List argList = new ArrayList<>(Arrays.asList(args)); > argList.add(0, targetName); // pre-pend so it becomes the first "app" arg > println("run args: " + argList); > connect((String[]) argList.toArray(args)); > waitForVMStart(); > } > > This issue of passing app args was fixed recently by [JDK-8308481](https://bugs.openjdk.org/browse/JDK-8308481), so the override is no longer needed. This pull request has now been integrated. Changeset: c38abbfc Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/c38abbfcaa80b3bb38bd6fc284e6bc3437199d77 Stats: 10 lines in 1 file changed: 0 ins; 10 del; 0 mod 8309510: com/sun/jdi/RedefineNestmateAttr/TestNestmateAttr.java no longer needs to override startUp() method Reviewed-by: sspitsyn, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/14325 From cjplummer at openjdk.org Wed Jun 7 18:06:46 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 18:06:46 GMT Subject: RFR: 8309420: com/sun/jdi/StepTest.java fails with virtual thread wrapper In-Reply-To: References: Message-ID: <6oxqRDdA94hwUsH4cKxHUhgXKOQV4fJYsDRZYxHN3XU=.89098067-ea69-4072-89e9-13e713a75948@github.com> On Mon, 5 Jun 2023 04:03:57 GMT, Chris Plummer wrote: > The test has two issues. The first is that it assume that once the VMStart event has arrived and one "step into" is done, it will be in the main method of the debuggee. Once there, it determines the debuggee class name by looking at the classtype of topmost frame. The problems is when using virtual threads, it is actually in TestScaffold.main() at this point, so the wrong class name is gleaned from the frame. To fix this the test just saves away the debuggee class name, which is passed to the test as the 4th argument. > > The other issue is that the test assumes once it gets to the debuggee go() method, there are only two frames on the stack. It's more like 16 when using virtual threads. The test needs to account for this by counting the number of frames when go() is entered rather than assuming it will be 2. > > Tested locally with and without the wrapper and by running tier5 svc tests. Thanks for the reviews Alex and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14307#issuecomment-1581278317 From cjplummer at openjdk.org Wed Jun 7 18:06:48 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 18:06:48 GMT Subject: Integrated: 8309420: com/sun/jdi/StepTest.java fails with virtual thread wrapper In-Reply-To: References: Message-ID: <1Z9g1Q1xIykL6UragWbioCzEZY88SzoG9W15fkvCW5w=.448e1733-77fd-4746-9d43-5e128bb4630f@github.com> On Mon, 5 Jun 2023 04:03:57 GMT, Chris Plummer wrote: > The test has two issues. The first is that it assume that once the VMStart event has arrived and one "step into" is done, it will be in the main method of the debuggee. Once there, it determines the debuggee class name by looking at the classtype of topmost frame. The problems is when using virtual threads, it is actually in TestScaffold.main() at this point, so the wrong class name is gleaned from the frame. To fix this the test just saves away the debuggee class name, which is passed to the test as the 4th argument. > > The other issue is that the test assumes once it gets to the debuggee go() method, there are only two frames on the stack. It's more like 16 when using virtual threads. The test needs to account for this by counting the number of frames when go() is entered rather than assuming it will be 2. > > Tested locally with and without the wrapper and by running tier5 svc tests. This pull request has now been integrated. Changeset: e3f3ac08 Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/e3f3ac0825e63ef5cec8e5f7e53ee0dbee263ac4 Stats: 21 lines in 2 files changed: 10 ins; 4 del; 7 mod 8309420: com/sun/jdi/StepTest.java fails with virtual thread wrapper Reviewed-by: sspitsyn, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/14307 From kdnilsen at openjdk.org Wed Jun 7 18:21:43 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 18:21:43 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v17] In-Reply-To: References: Message-ID: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Fix budgeting assertion to allow equal or greater than ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/612072a4..19e62fe0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=16 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=15-16 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185 From stuefe at openjdk.org Wed Jun 7 18:21:45 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 7 Jun 2023 18:21:45 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v12] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 14:06:31 GMT, Kelvin Nilsen wrote: > Hi Thomas, > > Thank you for your followup comments. I am in total agreement that it is a shame the challenges we have faced and the progress we have made is not better documented in the history of JBS tickets. I have been the worst offender. I apologize. Please, no need to apologize. I understand that during early development one needs to move quickly. I just thought that your team's experience with tuning Shenandoah is valuable, and it is regrettable when it is lost. > You are correct that the change is to N, the number of times in a row that we perform degenerated GC before we automatically upgrade to Full GC. It is still possible that we will upgrade to Full GC before N is reached, because there are other situations, such as lack of progress by degenerated GC, that will cause us to upgrade to Full even before N is reached. > > The comment is still valid as written. During degenerated GC, the mutator threads are all blocked, so the ONLY kind of allocation failure that can occur during degenerated GC is a GC-worker-thread allocation for the purpose of evacuating memory. If we experience an "evacuation failure" during degenerated GC. we will upgrade to Full GC. Thank you for the thorough explanation. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1581232734 From stuefe at openjdk.org Wed Jun 7 18:21:48 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 7 Jun 2023 18:21:48 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v16] In-Reply-To: <9vJprjFu5K2ZymOM-lzCm_IAA2pAnGBgCDAtwQ0sOvw=.47abdb57-237b-4c57-a5e1-25f11ee14a80@github.com> References: <9vJprjFu5K2ZymOM-lzCm_IAA2pAnGBgCDAtwQ0sOvw=.47abdb57-237b-4c57-a5e1-25f11ee14a80@github.com> Message-ID: On Wed, 7 Jun 2023 16:51:38 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 305 commits: > > - Merge branch 'master' of https://git.openjdk.org/jdk into merge-generational-shenandoah > - Simplify test logic, fail if name of Shenandoah young gen pool changes (#3) > > - Remove a few more unneeded copyright notices > - Remove more extraneous copyright notices > - JDK-8309322: [GenShen] TestAllocOutOfMemory#large failed > > When generational Shenandoah is used, there may be an additional > alignment related heap size adjustment that the test should be cognizant > of. Such alignment might also happen in the non-generational case, but > in this case the specific size used in the test was affected on machines > with larger than usual os page size settings. > > The alignment related adjustment would have affected all generational > collectors (except perhaps Gen Z). In the future, we might try and relax > this alignment constraint.alignment. > - Remove one more extraneous Amazon copyright > - Update copyright notices > - Improve efficiency of card-size alignment calculations > - Exit during initialization on unsupported platforms > - Remove an inappropriate copyright notice > - ... and 295 more: https://git.openjdk.org/jdk/compare/33bb64f2...612072a4 I won't be able to give reasonable input here in the short time left before RDP1. Nor am I the most qualified to do so. Just wanted to re-iterate that I see this rushed review with worry. Nothing I have not said already, and of course, it does not diminish the massive effort behind this JEP. Process-wise, when talking about Lilliput integration we touched on the idea of moving RDP1 down to a sooner date. I still think this makes sense. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1581244415 From sspitsyn at openjdk.org Wed Jun 7 18:51:55 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 18:51:55 GMT Subject: RFR: 8309612: [R[REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING Message-ID: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). The first push/patch is the original fix of JDK-8307153. The fix of the SUSPEND bit issue will be in the incremental update. It is to simplify the review. Testing: - TBD: mach5 tiers 1-5 ------------- Commit messages: - 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING Changes: https://git.openjdk.org/jdk/pull/14366/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309612 Stats: 82 lines in 4 files changed: 65 ins; 0 del; 17 mod Patch: https://git.openjdk.org/jdk/pull/14366.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14366/head:pull/14366 PR: https://git.openjdk.org/jdk/pull/14366 From sspitsyn at openjdk.org Wed Jun 7 19:27:05 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 19:27:05 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v2] In-Reply-To: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: > This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). > The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. > The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). > > The first push/patch is the original fix of JDK-8307153. > The fix of the SUSPEND bit issue will be in the incremental update. > It is to simplify the review. > > Testing: > - TBD: mach5 tiers 1-5 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: fixed the SUSPEND bit issue in JVMTI thread state of carrier threads ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14366/files - new: https://git.openjdk.org/jdk/pull/14366/files/00f51d34..29adb0af Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=00-01 Stats: 5 lines in 1 file changed: 0 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/14366.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14366/head:pull/14366 PR: https://git.openjdk.org/jdk/pull/14366 From sspitsyn at openjdk.org Wed Jun 7 19:32:23 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 19:32:23 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v3] In-Reply-To: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: > This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). > The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. > The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). > > The first push/patch is the original fix of JDK-8307153. > The fix of the SUSPEND bit issue will be in the incremental update. > It is to simplify the review. > > Testing: > - TBD: mach5 tiers 1-5 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: cleanup in comments: replace confusing term: passive carrier thread ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14366/files - new: https://git.openjdk.org/jdk/pull/14366/files/29adb0af..094b5f28 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=01-02 Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/14366.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14366/head:pull/14366 PR: https://git.openjdk.org/jdk/pull/14366 From sspitsyn at openjdk.org Wed Jun 7 20:05:45 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 20:05:45 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: > This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). > The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. > The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). > > The first push/patch is the original fix of JDK-8307153. > The fix of the SUSPEND bit issue will be in the incremental update. > It is to simplify the review. > > Testing: > - TBD: mach5 tiers 1-5 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: fix trailing space in jvmtiEnvBase.cpp ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14366/files - new: https://git.openjdk.org/jdk/pull/14366/files/094b5f28..4defcf2e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=02-03 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14366.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14366/head:pull/14366 PR: https://git.openjdk.org/jdk/pull/14366 From cjplummer at openjdk.org Wed Jun 7 20:15:54 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 20:15:54 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: On Wed, 7 Jun 2023 20:05:45 GMT, Serguei Spitsyn wrote: >> This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). >> The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. >> The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). >> >> The first push/patch is the original fix of JDK-8307153. >> The fix of the SUSPEND bit issue will be in the incremental update. >> It is to simplify the review. >> >> Testing: >> - TBD: mach5 tiers 1-5 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > fix trailing space in jvmtiEnvBase.cpp Changes requested by cjplummer (Reviewer). src/hotspot/share/prims/jvmtiEnvBase.cpp line 765: > 763: if (is_thread_carrying_vthread(jt, thread_oop)) { > 764: state &= ~JVMTI_THREAD_STATE_RUNNABLE; > 765: state |= JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; How about a comment here: "Clear RUNNABLE state and add WAITING state because..." src/hotspot/share/prims/jvmtiEnvBase.cpp line 1739: > 1737: "sanity check"); > 1738: > 1739: // An attempt to handshake-suspend a thread carrying virtual thread will result in Suggestion: // An attempt to handshake-suspend a thread carrying a virtual thread will result in src/hotspot/share/prims/jvmtiEnvBase.hpp line 99: > 97: static bool is_in_thread_list(jint count, const jthread* list, oop jt_oop); > 98: > 99: // check if thread_oop represents a thread carrying virtual thread Suggestion: // check if thread_oop represents a thread carrying a virtual thread src/hotspot/share/prims/jvmtiEnvBase.hpp line 183: > 181: > 182: // Return true if the thread identified with a pair is current. > 183: // A thread carrying virtual thread is not treated as current. Suggestion: // A thread carrying a virtual thread is not treated as current. ------------- PR Review: https://git.openjdk.org/jdk/pull/14366#pullrequestreview-1468479443 PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222104282 PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222104787 PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222105165 PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222105551 From kdnilsen at openjdk.org Wed Jun 7 21:07:39 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 21:07:39 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v17] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 18:21:43 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. >> >> We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Fix budgeting assertion to allow equal or greater than We would like to thank everyone who has taken time to review and provide feedback on our pull request. Given the risks identified during the review process and the lack of time available to perform the thorough review that such a large contribution of code requires, we have decided to close this PR at the current time. We will seek to target JDK 22. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1581509386 From kdnilsen at openjdk.org Wed Jun 7 21:07:41 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 7 Jun 2023 21:07:41 GMT Subject: Withdrawn: JDK-8307314: Implementation: Generational Shenandoah (Experimental) In-Reply-To: References: Message-ID: On Fri, 26 May 2023 20:46:29 GMT, Kelvin Nilsen wrote: > OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`. The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity. Generational mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed collections. Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation. In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort. Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects. These regions tend to have smaller amounts of garbage, and are less likely to be collected. If they survive a sufficient number of young-generation collections, the ?survivor? regions are promoted into the old generation. > > We expect to refine heuristics as we gain experience with more production workloads. In the future, we plan to remove the ?experimental? qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/14185 From sspitsyn at openjdk.org Wed Jun 7 21:22:41 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 21:22:41 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v5] In-Reply-To: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: > This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). > The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. > The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). > > The first push/patch is the original fix of JDK-8307153. > The fix of the SUSPEND bit issue will be in the incremental update. > It is to simplify the review. > > Testing: > - TBD: mach5 tiers 1-5 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: added/adjusted some comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14366/files - new: https://git.openjdk.org/jdk/pull/14366/files/4defcf2e..8f26e277 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=03-04 Stats: 7 lines in 2 files changed: 4 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/14366.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14366/head:pull/14366 PR: https://git.openjdk.org/jdk/pull/14366 From amenkov at openjdk.org Wed Jun 7 21:22:44 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 7 Jun 2023 21:22:44 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> On Wed, 7 Jun 2023 20:05:45 GMT, Serguei Spitsyn wrote: >> This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). >> The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. >> The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). >> >> The first push/patch is the original fix of JDK-8307153. >> The fix of the SUSPEND bit issue will be in the incremental update. >> It is to simplify the review. >> >> Testing: >> - TBD: mach5 tiers 1-5 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > fix trailing space in jvmtiEnvBase.cpp src/hotspot/share/prims/jvmtiEnvBase.cpp line 765: > 763: if (is_thread_carrying_vthread(jt, thread_oop)) { > 764: state &= ~JVMTI_THREAD_STATE_RUNNABLE; > 765: state |= JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; This does not look correct. GetThreadState spec provides hierarchical set of questions to interpret thread state value. JVMTI_THREAD_STATE_ALIVE | JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY is only one branch and I'd expect all other bits are not set for this state. Need to decide what do we want to report as carrier thread state for all possible values returned by get_thread_state_base(). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222182733 From sspitsyn at openjdk.org Wed Jun 7 21:22:49 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 21:22:49 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: On Wed, 7 Jun 2023 20:10:24 GMT, Chris Plummer wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> fix trailing space in jvmtiEnvBase.cpp > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 765: > >> 763: if (is_thread_carrying_vthread(jt, thread_oop)) { >> 764: state &= ~JVMTI_THREAD_STATE_RUNNABLE; >> 765: state |= JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; > > How about a comment here: > > "Clear RUNNABLE state and add WAITING state because..." Thanks. Added comment. > src/hotspot/share/prims/jvmtiEnvBase.cpp line 1739: > >> 1737: "sanity check"); >> 1738: >> 1739: // An attempt to handshake-suspend a thread carrying virtual thread will result in > > Suggestion: > > // An attempt to handshake-suspend a thread carrying a virtual thread will result in Thanks. Updated now. > src/hotspot/share/prims/jvmtiEnvBase.hpp line 99: > >> 97: static bool is_in_thread_list(jint count, const jthread* list, oop jt_oop); >> 98: >> 99: // check if thread_oop represents a thread carrying virtual thread > > Suggestion: > > // check if thread_oop represents a thread carrying a virtual thread Thanks. Updated now. > src/hotspot/share/prims/jvmtiEnvBase.hpp line 183: > >> 181: >> 182: // Return true if the thread identified with a pair is current. >> 183: // A thread carrying virtual thread is not treated as current. > > Suggestion: > > // A thread carrying a virtual thread is not treated as current. Thanks. Updated now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222185513 PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222185817 PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222185985 PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222186135 From sspitsyn at openjdk.org Wed Jun 7 21:57:49 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 21:57:49 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> Message-ID: On Wed, 7 Jun 2023 21:12:24 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> fix trailing space in jvmtiEnvBase.cpp > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 765: > >> 763: if (is_thread_carrying_vthread(jt, thread_oop)) { >> 764: state &= ~JVMTI_THREAD_STATE_RUNNABLE; >> 765: state |= JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; > > This does not look correct. > GetThreadState spec provides hierarchical set of questions to interpret thread state value. > JVMTI_THREAD_STATE_ALIVE | JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY is only one branch and I'd expect all other bits are not set for this state. > Need to decide what do we want to report as carrier thread state for all possible values returned by get_thread_state_base(). Good concern. There are two bits (and the related RUNNABLE bit) that we care in this sub-tree of state bits: `SUSPENDED` and `INTERRUPTED`. This update clones these two bits. The RUNNABLE bit must be cleared. A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. The state returned by the `get_thread_state_base` is based on the call: ` state = (jint)java_lang_Thread::get_thread_status(thread_oop);` and addition of the derived from JavaThread bits: `SUSPENDED`, `INTERRUPTED` and `IN_NATIVE`. The three bit derived from the JavaThread are not relevant. This call has to be made directly: ` state = (jint)java_lang_Thread::get_thread_status(thread_oop);` The SUSPEND bit has to be based on the call: ` jt->is_carrier_thread_suspended();` ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222212064 From cjplummer at openjdk.org Wed Jun 7 22:23:49 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 22:23:49 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v5] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: On Wed, 7 Jun 2023 21:22:41 GMT, Serguei Spitsyn wrote: >> This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). >> The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. >> The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). >> >> The first push/patch is the original fix of JDK-8307153. >> The fix of the SUSPEND bit issue will be in the incremental update. >> It is to simplify the review. >> >> Testing: >> - TBD: mach5 tiers 1-5 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: added/adjusted some comments Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14366#pullrequestreview-1468678777 From amenkov at openjdk.org Wed Jun 7 22:45:49 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 7 Jun 2023 22:45:49 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> Message-ID: On Wed, 7 Jun 2023 21:52:33 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiEnvBase.cpp line 765: >> >>> 763: if (is_thread_carrying_vthread(jt, thread_oop)) { >>> 764: state &= ~JVMTI_THREAD_STATE_RUNNABLE; >>> 765: state |= JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; >> >> This does not look correct. >> GetThreadState spec provides hierarchical set of questions to interpret thread state value. >> JVMTI_THREAD_STATE_ALIVE | JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY is only one branch and I'd expect all other bits are not set for this state. >> Need to decide what do we want to report as carrier thread state for all possible values returned by get_thread_state_base(). > > Good concern. > There are two bits (and the related RUNNABLE bit) that we care in this sub-tree of state bits: `SUSPENDED` and `INTERRUPTED`. This update clones these two bits. The RUNNABLE bit must be cleared. > A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. > The state returned by the `get_thread_state_base` is based on the call: > ` state = (jint)java_lang_Thread::get_thread_status(thread_oop);` > and addition of the derived from JavaThread bits: `SUSPENDED`, `INTERRUPTED` and `IN_NATIVE`. > The three bits derived from the JavaThread are not relevant. > This call has to be made directly: > ` state = (jint)java_lang_Thread::get_thread_status(thread_oop);` > The SUSPEND bit has to be based on the call: > ` jt->is_carrier_thread_suspended();` > > The function `get_thread_state` will look as below: > > if (is_thread_carrying_vthread(jt, thread_oop)) { > jint state = (jint)java_lang_Thread::get_thread_status(thread_oop); > if (jt->is_carrier_thread_suspended()) { > state |= JVMTI_THREAD_STATE_SUSPENDED; > } > // It's okay for the JVMTI state to be reported as WAITING when waiting > // for something other than an Object.wait. So, we treat a thread carrying > // a virtual thread as waiting indefinitely which is not runnable. > // It is why the RUNNABLE bit is cleared and the WAITING bits are added. > state &= ~JVMTI_THREAD_STATE_RUNNABLE; > state |= JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; > return state; > } else { > return get_thread_state_base(thread_oop, jt); > } Do you need to check jt->is_interrupted(false) and set INTERRUPTED bit? It looks like java_lang_Thread::get_thread_status(thread_oop) can only return RUNNABLE in the case and we clear it, so the call is not needed: if (is_thread_carrying_vthread(jt, thread_oop)) { jint state = JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; if (jt->is_carrier_thread_suspended()) { state |= JVMTI_THREAD_STATE_SUSPENDED; } if (jt->is_interrupted(false)) { state |= JVMTI_THREAD_STATE_INTERRUPTED; } return state; } else ... ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222252628 From amenkov at openjdk.org Wed Jun 7 22:51:49 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 7 Jun 2023 22:51:49 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> Message-ID: <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> On Wed, 7 Jun 2023 22:42:49 GMT, Alex Menkov wrote: >> Good concern. >> There are two bits (and the related RUNNABLE bit) that we care in this sub-tree of state bits: `SUSPENDED` and `INTERRUPTED`. This update clones these two bits. The RUNNABLE bit must be cleared. >> A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. >> The state returned by the `get_thread_state_base` is based on the call: >> ` state = (jint)java_lang_Thread::get_thread_status(thread_oop);` >> and addition of the derived from JavaThread bits: `SUSPENDED`, `INTERRUPTED` and `IN_NATIVE`. >> The three bits derived from the JavaThread are not relevant. >> This call has to be made directly: >> ` state = (jint)java_lang_Thread::get_thread_status(thread_oop);` >> The SUSPEND bit has to be based on the call: >> ` jt->is_carrier_thread_suspended();` >> >> The function `get_thread_state` will look as below: >> >> if (is_thread_carrying_vthread(jt, thread_oop)) { >> jint state = (jint)java_lang_Thread::get_thread_status(thread_oop); >> if (jt->is_carrier_thread_suspended()) { >> state |= JVMTI_THREAD_STATE_SUSPENDED; >> } >> // It's okay for the JVMTI state to be reported as WAITING when waiting >> // for something other than an Object.wait. So, we treat a thread carrying >> // a virtual thread as waiting indefinitely which is not runnable. >> // It is why the RUNNABLE bit is cleared and the WAITING bits are added. >> state &= ~JVMTI_THREAD_STATE_RUNNABLE; >> state |= JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; >> return state; >> } else { >> return get_thread_state_base(thread_oop, jt); >> } > > Do you need to check jt->is_interrupted(false) and set INTERRUPTED bit? > It looks like java_lang_Thread::get_thread_status(thread_oop) can only return RUNNABLE in the case and we clear it, so the call is not needed: > > if (is_thread_carrying_vthread(jt, thread_oop)) { > jint state = JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; > if (jt->is_carrier_thread_suspended()) { > state |= JVMTI_THREAD_STATE_SUSPENDED; > } > if (jt->is_interrupted(false)) { > state |= JVMTI_THREAD_STATE_INTERRUPTED; > } > return state; > } else ... > A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. Actually it can be in native. And if I remember correctly synchronized block pins virtual thread, so inside synchronized we can get other states ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222255498 From sspitsyn at openjdk.org Wed Jun 7 23:11:48 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 7 Jun 2023 23:11:48 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> Message-ID: On Wed, 7 Jun 2023 22:48:47 GMT, Alex Menkov wrote: >> Do you need to check jt->is_interrupted(false) and set INTERRUPTED bit? >> It looks like java_lang_Thread::get_thread_status(thread_oop) can only return RUNNABLE in the case and we clear it, so the call is not needed: >> >> if (is_thread_carrying_vthread(jt, thread_oop)) { >> jint state = JVMTI_THREAD_STATE_WAITING | JVMTI_THREAD_STATE_WAITING_INDEFINITELY; >> if (jt->is_carrier_thread_suspended()) { >> state |= JVMTI_THREAD_STATE_SUSPENDED; >> } >> if (jt->is_interrupted(false)) { >> state |= JVMTI_THREAD_STATE_INTERRUPTED; >> } >> return state; >> } else ... > >> A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. > > Actually it can be in native. > And if I remember correctly synchronized block pins virtual thread, so inside synchronized we can get other states The INTERRUPTED bit we need has to be returned by the `java_lang_Thread::get_thread_status`. Not completely sure but the bit jt->is_interrupted(false) can be set for the mounted virtual thread. The JVMTI InterruptThread calls this function to set interrupt bit for non-virtual threads: ` java_lang_Thread::set_interrupted(thread_obj, true);` ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222268031 From cjplummer at openjdk.org Wed Jun 7 23:23:55 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 7 Jun 2023 23:23:55 GMT Subject: RFR: 8232839: JDI AfterThreadDeathTest.java failed due to "FAILED: Did not get expected IllegalThreadStateException on a StepRequest.enable()" Message-ID: The test waits for a ThreadDeathEvent for "main". Once that arrives, it then waits for the next ThreadStartEvent (for any thread). Once it arrives, the test tries to create and enable a StepRequest on the "main" thread. Since "main" is supposedly dead, the expectation is an IllegalThreadStateException. However, it turns out that sometimes the enabling can in fact succeed. Just because a ThreadDeathEvent has been received for a thread does not mean you can no longer do things with the thread like suspend it or enable a StepEvent. There is a short delay in the debug agent after sending the ThreadDeathEvent before it stops tracking the thread. The thread can still be acted upon until then. The JDWP and JDI specs seem to support doing this: > Notification of a completed thread in the target VM. The notification is generated by the dying thread before it terminates. Because of this timing, it is possible for {@link VirtualMachine#allThreads} to return this thread after this event is received. > > Note that this event gives no information about the lifetime of the thread object. It may or may not be collected soon depending on what references exist in the target VM. What this means is that when the test receives some arbitrary ThreadStartEvent after the "main" ThreadDeathEvent has been received, the test may in fact still be able to enable a StepRequest on the "main" thread, causing the test to fail. What seems to trigger the failure is receiving an unexpected spurious ThreadStartEvent such as from the Common-Clean thread or a carrier thread, although even then it only fails some of the time. In fact if I modify the test to enable the StepRequest when it receives the ThreadDeathEvent for "main", it still almost always passes, but will fail more frequently than it normally does. It seems if the test always waits for the ThreadStartEvent for "DestroyJavaVM", then the "main" thread is truly gone by then and the test always passes, so this is how I've chosen to fix the issue. ------------- Commit messages: - Fix 8232839 Changes: https://git.openjdk.org/jdk/pull/14372/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14372&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8232839 Stats: 24 lines in 2 files changed: 11 ins; 3 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/14372.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14372/head:pull/14372 PR: https://git.openjdk.org/jdk/pull/14372 From sspitsyn at openjdk.org Thu Jun 8 01:42:10 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 8 Jun 2023 01:42:10 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v6] In-Reply-To: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: > This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). > The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. > The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). > > The first push/patch is the original fix of JDK-8307153. > The fix of the SUSPEND bit issue will be in the incremental update. > It is to simplify the review. > > Testing: > - TBD: mach5 tiers 1-5 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: corrected the function get_thread_state for safety ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14366/files - new: https://git.openjdk.org/jdk/pull/14366/files/8f26e277..5fd74f39 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14366&range=04-05 Stats: 13 lines in 1 file changed: 10 ins; 1 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14366.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14366/head:pull/14366 PR: https://git.openjdk.org/jdk/pull/14366 From sspitsyn at openjdk.org Thu Jun 8 01:42:25 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 8 Jun 2023 01:42:25 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> Message-ID: On Wed, 7 Jun 2023 23:08:52 GMT, Serguei Spitsyn wrote: >>> A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. >> >> Actually it can be in native. >> And if I remember correctly synchronized block pins virtual thread, so inside synchronized we can get other states > > The INTERRUPTED bit we need has to be returned by the `java_lang_Thread::get_thread_status`. > Not completely sure but the bit jt->is_interrupted(false) can be set for the mounted virtual thread. > The JVMTI InterruptThread calls this function to set interrupt bit for non-virtual threads: > ` java_lang_Thread::set_interrupted(thread_obj, true);` Corrected the function `get_thread_state()` to make it more safe. Only `ALIVE` and `INTERRUPTED` bits are taken from result of `java_lang_Thread::get_thread_status(thread_oop)`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222352148 From dholmes at openjdk.org Thu Jun 8 02:09:48 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 8 Jun 2023 02:09:48 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> Message-ID: On Thu, 8 Jun 2023 01:40:06 GMT, Serguei Spitsyn wrote: >> The INTERRUPTED bit we need has to be returned by the `java_lang_Thread::get_thread_status`. >> Not completely sure but the bit jt->is_interrupted(false) can be set for the mounted virtual thread. >> The JVMTI InterruptThread calls this function to set interrupt bit for non-virtual threads: >> ` java_lang_Thread::set_interrupted(thread_obj, true);` > > Corrected the function `get_thread_state()` to make it more safe. > Only `ALIVE` and `INTERRUPTED` bits are taken from result of `java_lang_Thread::get_thread_status(thread_oop)`. > A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. A virtual thread can call native code, be blocked on an object monitor, or waiting on an object monitor. Only parking and sleeping are specialized for virtual threads in the list you gave. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222364495 From sspitsyn at openjdk.org Thu Jun 8 03:39:47 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 8 Jun 2023 03:39:47 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> Message-ID: On Thu, 8 Jun 2023 02:07:00 GMT, David Holmes wrote: > > A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. >A virtual thread can call native code, be blocked on an object monitor, or waiting on an object monitor. Only parking and sleeping are specialized for virtual threads in the list you gave. This statement was about carrier thread when there is a virtual thread executed at the top. We are getting state bits with the `java_lang_Thread::get_thread_status(thread_oop)` where the `thread_oop` belongs to the carrier thread. But you are talking about a virtual thread which, of course, can be in almost any state. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222413499 From sspitsyn at openjdk.org Thu Jun 8 04:01:56 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 8 Jun 2023 04:01:56 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 [v4] In-Reply-To: References: Message-ID: <7xXMXXSeQLwosyHqOBA9N6cIKIIzHRfGTOVNxYLcMOY=.4a7ce215-bf6b-4659-a502-fb001293a5c7@github.com> > This is a minor update of the `jvmti.xml` file. > The JVM TI history table needs to be updated to list: > - Virtual threads finalized to be a permanent feature. > - Agent start-up in the live phase now specified to print a warning. > > The JVM TI history table has no normative changes. This update does not need a CSR. Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: improved formatting in the jvmti.xml history table ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14352/files - new: https://git.openjdk.org/jdk/pull/14352/files/11db4f4f..8b0997a2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14352&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14352&range=02-03 Stats: 43 lines in 1 file changed: 0 ins; 14 del; 29 mod Patch: https://git.openjdk.org/jdk/pull/14352.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14352/head:pull/14352 PR: https://git.openjdk.org/jdk/pull/14352 From sspitsyn at openjdk.org Thu Jun 8 04:06:55 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 8 Jun 2023 04:06:55 GMT Subject: RFR: 8309602: update JVMTI history table for jdk 21 [v4] In-Reply-To: <7xXMXXSeQLwosyHqOBA9N6cIKIIzHRfGTOVNxYLcMOY=.4a7ce215-bf6b-4659-a502-fb001293a5c7@github.com> References: <7xXMXXSeQLwosyHqOBA9N6cIKIIzHRfGTOVNxYLcMOY=.4a7ce215-bf6b-4659-a502-fb001293a5c7@github.com> Message-ID: On Thu, 8 Jun 2023 04:01:56 GMT, Serguei Spitsyn wrote: >> This is a minor update of the `jvmti.xml` file. >> The JVM TI history table needs to be updated to list: >> - Virtual threads finalized to be a permanent feature. >> - Agent start-up in the live phase now specified to print a warning. >> >> The JVM TI history table has no normative changes. This update does not need a CSR. > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > improved formatting in the jvmti.xml history table Alan and Iris, thank you for review! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14352#issuecomment-1581857901 From sspitsyn at openjdk.org Thu Jun 8 04:06:57 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 8 Jun 2023 04:06:57 GMT Subject: Integrated: 8309602: update JVMTI history table for jdk 21 In-Reply-To: References: Message-ID: <2z5JUAtnKoIy2wAx3tSYZP_SYkRlvmEm334SnqUzmxI=.5886d218-223f-4a3d-902a-5b92721b20d1@github.com> On Wed, 7 Jun 2023 12:32:14 GMT, Serguei Spitsyn wrote: > This is a minor update of the `jvmti.xml` file. > The JVM TI history table needs to be updated to list: > - Virtual threads finalized to be a permanent feature. > - Agent start-up in the live phase now specified to print a warning. > > The JVM TI history table has no normative changes. This update does not need a CSR. This pull request has now been integrated. Changeset: 5af9d2a0 Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/5af9d2a0ac82ad83dc83461e5b8ce793cc995ad3 Stats: 44 lines in 1 file changed: 0 ins; 10 del; 34 mod 8309602: update JVMTI history table for jdk 21 Reviewed-by: alanb, iris ------------- PR: https://git.openjdk.org/jdk/pull/14352 From dholmes at openjdk.org Thu Jun 8 04:32:48 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 8 Jun 2023 04:32:48 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> Message-ID: On Thu, 8 Jun 2023 03:36:52 GMT, Serguei Spitsyn wrote: >>> A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. >> >> A virtual thread can call native code, be blocked on an object monitor, or waiting on an object monitor. Only parking and sleeping are specialized for virtual threads in the list you gave. > >> > A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. > >>A virtual thread can call native code, be blocked on an object monitor, or waiting on an object monitor. Only parking and sleeping are specialized for virtual threads in the list you gave. > > This statement was about a carrier thread (not a `JavaThread` and not a ` java.lang.VirtualThread`) when there is a virtual thread executed at the top. We are getting state bits with the `java_lang_Thread::get_thread_status(thread_oop)` where the `thread_oop` belongs to the carrier thread. But you are talking about a virtual thread which, of course, can be in almost any state. Thanks for clarifying - it gets very confusing as to which "thread" is being talked about. But if a virtual thread is mounted on this JavaThread then I thought the carrier thread's thread-oop is supposed to be in a blocked state? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222437953 From sspitsyn at openjdk.org Thu Jun 8 04:34:47 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 8 Jun 2023 04:34:47 GMT Subject: RFR: 8232839: JDI AfterThreadDeathTest.java failed due to "FAILED: Did not get expected IllegalThreadStateException on a StepRequest.enable()" In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 23:17:18 GMT, Chris Plummer wrote: > The test waits for a ThreadDeathEvent for "main". Once that arrives, it then waits for the next ThreadStartEvent (for any thread). Once it arrives, the test tries to create and enable a StepRequest on the "main" thread. Since "main" is supposedly dead, the expectation is an IllegalThreadStateException. However, it turns out that sometimes the enabling can in fact succeed. > > Just because a ThreadDeathEvent has been received for a thread does not mean you can no longer do things with the thread like suspend it or enable a StepEvent. There is a short delay in the debug agent after sending the ThreadDeathEvent before it stops tracking the thread. The thread can still be acted upon until then. The JDWP and JDI specs seem to support doing this: > >> Notification of a completed thread in the target VM. The notification is generated by the dying thread before it terminates. Because of this timing, it is possible for {@link VirtualMachine#allThreads} to return this thread after this event is received. >> >> Note that this event gives no information about the lifetime of the thread object. It may or may not be collected soon depending on what references exist in the target VM. > > What this means is that when the test receives some arbitrary ThreadStartEvent after the "main" ThreadDeathEvent has been received, the test may in fact still be able to enable a StepRequest on the "main" thread, causing the test to fail. What seems to trigger the failure is receiving an unexpected spurious ThreadStartEvent such as from the Common-Clean thread or a carrier thread, although even then it only fails some of the time. In fact if I modify the test to enable the StepRequest when it receives the ThreadDeathEvent for "main", it still almost always passes, but will fail more frequently than it normally does. > > It seems if the test always waits for the ThreadStartEvent for "DestroyJavaVM", then the "main" thread is truly gone by then and the test always passes, so this is how I've chosen to fix the issue. > > Tested with tier1, tier2 svc tests, and tier5 svc tests. It looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14372#pullrequestreview-1468978228 From sspitsyn at openjdk.org Thu Jun 8 04:46:50 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 8 Jun 2023 04:46:50 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> Message-ID: On Thu, 8 Jun 2023 04:29:47 GMT, David Holmes wrote: >>> > A thread carrying a virtual thread can not be in native, blocked, parked, sleeping or waiting on some object. >> >>>A virtual thread can call native code, be blocked on an object monitor, or waiting on an object monitor. Only parking and sleeping are specialized for virtual threads in the list you gave. >> >> This statement was about a carrier thread (not a `JavaThread` and not a ` java.lang.VirtualThread`) when there is a virtual thread executed at the top. We are getting state bits with the `java_lang_Thread::get_thread_status(thread_oop)` where the `thread_oop` belongs to the carrier thread. But you are talking about a virtual thread which, of course, can be in almost any state. > > Thanks for clarifying - it gets very confusing as to which "thread" is being talked about. But if a virtual thread is mounted on this JavaThread then I thought the carrier thread's thread-oop is supposed to be in a blocked state? It was decided with Alan that it is okay to be in a waiting state. The `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` state requires a monitor to be blocked on, so it can be confusing. Alan's comment in the original PR [https://github.com/openjdk/jdk/pull/14298](https://github.com/openjdk/jdk/pull/14298) was: > if the jt is carrying thread_oop and it's okay for the JVMTI state to reported as WAITING when waiting for something other than Object.wait. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222442892 From alanb at openjdk.org Thu Jun 8 06:27:49 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 8 Jun 2023 06:27:49 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> Message-ID: On Thu, 8 Jun 2023 04:41:10 GMT, Serguei Spitsyn wrote: >> Thanks for clarifying - it gets very confusing as to which "thread" is being talked about. But if a virtual thread is mounted on this JavaThread then I thought the carrier thread's thread-oop is supposed to be in a blocked state? > > It was decided with Alan that it is okay to be in a waiting state. The `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` state requires a monitor to be blocked on, so it can be confusing. Alan's comment in the original PR [https://github.com/openjdk/jdk/pull/14298](https://github.com/openjdk/jdk/pull/14298) was: >> if the jt is carrying thread_oop and it's okay for the JVMTI state to reported as WAITING when waiting for something other than Object.wait. The mental model is that the carrier is blocked so this is what an observer using the APIs should see. My recollection is that JVMTI_THREAD_STATE_WAITING was okay because there is a wriggle room in the JVM TI spec, it only uses Object.wait as an example. There may be a few rough edges to smooth down in this area. It's okay to take time with this PR and expand the tests to cover more cases and get more confident that there aren't more issues. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1222511997 From jwaters at openjdk.org Thu Jun 8 07:45:53 2023 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 8 Jun 2023 07:45:53 GMT Subject: RFR: 8305341: Alignment should be enforced by alignas instead of compiler specific attributes [v4] In-Reply-To: References: <2d60fxZxeWZEngMaSE1N4JZz07XkvbXj8jrN_hMbo-0=.51ffb82f-2beb-43f7-9195-062555599d0b@github.com> Message-ID: <6x5TNtFwNDmFEx3WL4_uB-PLyFGgi9T0s4rMKEC-iVI=.120ae51e-3d33-46cd-97ce-6cb6b7bde452@github.com> On Wed, 12 Apr 2023 07:12:10 GMT, Julian Waters wrote: >> C11 has been stable for a long time on all platforms, so native code can use the standard alignas operator for alignment requirements > > Julian Waters has updated the pull request incrementally with four additional commits since the last revision: > > - Restore visCPP > - Restore gcc attribute > - Revert gcc > - Revert Anyone? ------------- PR Comment: https://git.openjdk.org/jdk/pull/13258#issuecomment-1582065715 From aph at openjdk.org Thu Jun 8 08:25:35 2023 From: aph at openjdk.org (Andrew Haley) Date: Thu, 8 Jun 2023 08:25:35 GMT Subject: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v17] In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 21:03:47 GMT, Kelvin Nilsen wrote: > We would like to thank everyone who has taken time to review and provide feedback on our pull request. Given the risks identified during the review process and the lack of time available to perform the thorough review that such a large contribution of code requires, we have decided to close this PR at the current time. We will seek to target JDK 22. Thank you for this. It's the right decision. In hindsight, there never was a highly-likely prospect of getting such a substantial and interwoven patch successfully reviewed in such a short time, even with the most skilful and experienced team. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1582124942 From yzheng at openjdk.org Thu Jun 8 17:40:03 2023 From: yzheng at openjdk.org (Yudi Zheng) Date: Thu, 8 Jun 2023 17:40:03 GMT Subject: RFR: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled Message-ID: HeapMonitor checks if System.getProperty("jvmci.Compiler") is graal and will not enforce checking line number derived from uncommon trap debug info. However, Graal does not set this property explicitly. ------------- Commit messages: - Avoid using jvmci.Compiler property to determine if Graal is enabled. Changes: https://git.openjdk.org/jdk/pull/14381/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14381&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309671 Stats: 3 lines in 2 files changed: 0 ins; 2 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14381.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14381/head:pull/14381 PR: https://git.openjdk.org/jdk/pull/14381 From dnsimon at openjdk.org Thu Jun 8 17:40:04 2023 From: dnsimon at openjdk.org (Doug Simon) Date: Thu, 8 Jun 2023 17:40:04 GMT Subject: RFR: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled In-Reply-To: References: Message-ID: <3Sa8G0j18gcxBvSRPq_Uoa1AzqBvST1aW4KNdH8V5oI=.79ebc9bf-8ee7-4c95-8d51-540014d26de1@github.com> On Thu, 8 Jun 2023 17:14:39 GMT, Yudi Zheng wrote: > HeapMonitor checks if System.getProperty("jvmci.Compiler") is graal and will not enforce checking line number derived from uncommon trap debug info. However, Graal does not set this property explicitly. test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 257: > 255: > 256: checkLines = !(enableJVMCI.getValue().equals("true") > 257: && useJVMCICompiler.getValue().equals("true")); Is it possible to use `jdk.test.whitebox.code.Compiler.isGraalEnabled()` here instead? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14381#discussion_r1223342262 From amenkov at openjdk.org Thu Jun 8 18:21:44 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 8 Jun 2023 18:21:44 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v6] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: On Thu, 8 Jun 2023 01:42:10 GMT, Serguei Spitsyn wrote: >> This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). >> The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. >> The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). >> >> The first push/patch is the original fix of JDK-8307153. >> The fix of the SUSPEND bit issue will be in the incremental update. >> It is to simplify the review. >> >> Testing: >> - TBD: mach5 tiers 1-5 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: corrected the function get_thread_state for safety Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14366#pullrequestreview-1470533353 From yzheng at openjdk.org Thu Jun 8 18:32:55 2023 From: yzheng at openjdk.org (Yudi Zheng) Date: Thu, 8 Jun 2023 18:32:55 GMT Subject: RFR: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled In-Reply-To: <3Sa8G0j18gcxBvSRPq_Uoa1AzqBvST1aW4KNdH8V5oI=.79ebc9bf-8ee7-4c95-8d51-540014d26de1@github.com> References: <3Sa8G0j18gcxBvSRPq_Uoa1AzqBvST1aW4KNdH8V5oI=.79ebc9bf-8ee7-4c95-8d51-540014d26de1@github.com> Message-ID: On Thu, 8 Jun 2023 17:19:46 GMT, Doug Simon wrote: >> HeapMonitor checks if System.getProperty("jvmci.Compiler") is graal and will not enforce checking line number derived from uncommon trap debug info. However, Graal does not set this property explicitly. > > test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 257: > >> 255: >> 256: checkLines = !(enableJVMCI.getValue().equals("true") >> 257: && useJVMCICompiler.getValue().equals("true")); > > Is it possible to use `jdk.test.whitebox.code.Compiler.isGraalEnabled()` here instead? To use whitebox I will have to update the config of every test here, which I think is too verbose: diff --git a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java index e08454a4857..f086f744965 100644 --- a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java +++ b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java @@ -27,11 +27,15 @@ package MyPackage; /** * @test * @summary Verifies the JVMTI Heap Monitor sampling interval average. - * @build Frame HeapMonitor + * @library /test/lib + * @build Frame HeapMonitor jdk.test.whitebox.WhiteBox * @compile HeapMonitorStatIntervalTest.java * @requires vm.jvmti * @requires vm.compMode != "Xcomp" - * @run main/othervm/native -agentlib:HeapMonitorTest MyPackage.HeapMonitorStatIntervalTest + * @run driver jdk.test.lib.helpers.ClassFileInstaller jdk.test.whitebox.WhiteBox + * @run main/othervm/native -agentlib:HeapMonitorTest -Xbootclasspath/a:. + * -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI + * MyPackage.HeapMonitorStatIntervalTest */ ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14381#discussion_r1223411115 From dnsimon at openjdk.org Thu Jun 8 19:08:53 2023 From: dnsimon at openjdk.org (Doug Simon) Date: Thu, 8 Jun 2023 19:08:53 GMT Subject: RFR: 8306028: separate ThreadStart/ThreadEnd events posting code in JVMTI VTMS transitions [v8] In-Reply-To: References: Message-ID: <9RJb8PvlZxLYs2zsDb-lDqoILDkiFwPx54vq3NpIdQQ=.c158dc31-68b0-49bd-8ac4-1ff1069be454@github.com> On Tue, 2 May 2023 02:01:44 GMT, Serguei Spitsyn wrote: >> This refactoring to separate ThreadStart/ThreadEnd events posting code in the JVMTI VTMS transitions is needed for future work on JVMTI scalability and performance improvements. It is to easier put this code on slow path. >> >> Testing: mach5 tiers 1-6 were successful. > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > update copyright comments src/hotspot/share/runtime/sharedRuntime.cpp line 641: > 639: JRT_ENTRY(void, SharedRuntime::notify_jvmti_vthread_start(oopDesc* vt, jboolean hide, JavaThread* current)) > 640: assert(hide == JNI_FALSE, "must be VTMS transition finish"); > 641: jobject vthread = JNIHandles::make_local(const_cast(vt)); Since the current thread is in the `current` arg, it could be used here when creating the local handle. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13484#discussion_r1223444559 From dnsimon at openjdk.org Thu Jun 8 19:12:40 2023 From: dnsimon at openjdk.org (Doug Simon) Date: Thu, 8 Jun 2023 19:12:40 GMT Subject: RFR: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled In-Reply-To: References: Message-ID: <8FdRus4tjvM7FAZ9HAOTbQpbgJKfWw8lyJWAZmbnRFU=.8a73dd98-f8ac-421c-bdb6-5fe9733577c3@github.com> On Thu, 8 Jun 2023 17:14:39 GMT, Yudi Zheng wrote: > HeapMonitor checks if System.getProperty("jvmci.Compiler") is graal and will not enforce checking line number derived from uncommon trap debug info. However, Graal does not set this property explicitly. Marked as reviewed by dnsimon (Committer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14381#pullrequestreview-1470601144 From dnsimon at openjdk.org Thu Jun 8 19:12:42 2023 From: dnsimon at openjdk.org (Doug Simon) Date: Thu, 8 Jun 2023 19:12:42 GMT Subject: RFR: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled In-Reply-To: References: <3Sa8G0j18gcxBvSRPq_Uoa1AzqBvST1aW4KNdH8V5oI=.79ebc9bf-8ee7-4c95-8d51-540014d26de1@github.com> Message-ID: On Thu, 8 Jun 2023 18:29:50 GMT, Yudi Zheng wrote: >> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java line 257: >> >>> 255: >>> 256: checkLines = !(enableJVMCI.getValue().equals("true") >>> 257: && useJVMCICompiler.getValue().equals("true")); >> >> Is it possible to use `jdk.test.whitebox.code.Compiler.isGraalEnabled()` here instead? > > To use whitebox I will have to update the config of every test here, which I think is too verbose: > > diff --git a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java > index e08454a4857..f086f744965 100644 > --- a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java > +++ b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java > @@ -27,11 +27,15 @@ package MyPackage; > /** > * @test > * @summary Verifies the JVMTI Heap Monitor sampling interval average. > - * @build Frame HeapMonitor > + * @library /test/lib > + * @build Frame HeapMonitor jdk.test.whitebox.WhiteBox > * @compile HeapMonitorStatIntervalTest.java > * @requires vm.jvmti > * @requires vm.compMode != "Xcomp" > - * @run main/othervm/native -agentlib:HeapMonitorTest MyPackage.HeapMonitorStatIntervalTest > + * @run driver jdk.test.lib.helpers.ClassFileInstaller jdk.test.whitebox.WhiteBox > + * @run main/othervm/native -agentlib:HeapMonitorTest -Xbootclasspath/a:. > + * -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI > + * MyPackage.HeapMonitorStatIntervalTest > */ Ok. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14381#discussion_r1223447124 From phh at openjdk.org Thu Jun 8 19:20:40 2023 From: phh at openjdk.org (Paul Hohensee) Date: Thu, 8 Jun 2023 19:20:40 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives In-Reply-To: References: Message-ID: On Wed, 24 May 2023 00:38:27 GMT, Dmitry Chuyko wrote: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives. Methods in general are often inlined, and this information is hard to track down. > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Obviously there is a performance penalty, so it should be applied with care. Hot code will most likely be recompiled soon, as nothing happens to its hotness. > > A new flag '`-d`' has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and marks for deoptimization those methods that have any active non-default matching compiler directives. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be deoptimized. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be deoptimized, but this can be achieved by having rules for them. > > In addition, a new diagnistic command `Compiler.replace_directives`, has been added for convenience. It's like a combinatio... "refresh" (-r) would be better than "deoptimize" (-d). The latter implies a specific implementation, the former is generic. If the method is to be recompiled, perhaps rather than deopt and wait, add it to the compile queue immediately and deopt the old version when the new compilation is complete, similar to what happens when the c1 version of the method is replaced by the c2 version. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14111#issuecomment-1583199824 From kevinw at openjdk.org Thu Jun 8 19:36:43 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Thu, 8 Jun 2023 19:36:43 GMT Subject: RFR: 8232839: JDI AfterThreadDeathTest.java failed due to "FAILED: Did not get expected IllegalThreadStateException on a StepRequest.enable()" In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 23:17:18 GMT, Chris Plummer wrote: > The test waits for a ThreadDeathEvent for "main". Once that arrives, it then waits for the next ThreadStartEvent (for any thread). Once it arrives, the test tries to create and enable a StepRequest on the "main" thread. Since "main" is supposedly dead, the expectation is an IllegalThreadStateException. However, it turns out that sometimes the enabling can in fact succeed. > > Just because a ThreadDeathEvent has been received for a thread does not mean you can no longer do things with the thread like suspend it or enable a StepEvent. There is a short delay in the debug agent after sending the ThreadDeathEvent before it stops tracking the thread. The thread can still be acted upon until then. The JDWP and JDI specs seem to support doing this: > >> Notification of a completed thread in the target VM. The notification is generated by the dying thread before it terminates. Because of this timing, it is possible for {@link VirtualMachine#allThreads} to return this thread after this event is received. >> >> Note that this event gives no information about the lifetime of the thread object. It may or may not be collected soon depending on what references exist in the target VM. > > What this means is that when the test receives some arbitrary ThreadStartEvent after the "main" ThreadDeathEvent has been received, the test may in fact still be able to enable a StepRequest on the "main" thread, causing the test to fail. What seems to trigger the failure is receiving an unexpected spurious ThreadStartEvent such as from the Common-Clean thread or a carrier thread, although even then it only fails some of the time. In fact if I modify the test to enable the StepRequest when it receives the ThreadDeathEvent for "main", it still almost always passes, but will fail more frequently than it normally does. > > It seems if the test always waits for the ThreadStartEvent for "DestroyJavaVM", then the "main" thread is truly gone by then and the test always passes, so this is how I've chosen to fix the issue. > > Tested with tier1, tier2 svc tests, and tier5 svc tests. I think it makes sense! ------------- Marked as reviewed by kevinw (Committer). PR Review: https://git.openjdk.org/jdk/pull/14372#pullrequestreview-1470630717 From matsaave at openjdk.org Thu Jun 8 22:01:11 2023 From: matsaave at openjdk.org (Matias Saavedra Silva) Date: Thu, 8 Jun 2023 22:01:11 GMT Subject: RFR: 8309673: Refactor ref_at methods in Serviceability ConstantPool Message-ID: The accessor methods in constantpool.cpp were previously cleaned up to allow for different types of indices to be used, distinguishing them by the bytecode. This patch adds the same changes to the hotspot serviceability code. Verified with tier 1-5 tests. ------------- Commit messages: - 8309673: Refactor ref_at methods in Serviceability ConstantPool Changes: https://git.openjdk.org/jdk/pull/14385/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14385&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309673 Stats: 95 lines in 4 files changed: 32 ins; 25 del; 38 mod Patch: https://git.openjdk.org/jdk/pull/14385.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14385/head:pull/14385 PR: https://git.openjdk.org/jdk/pull/14385 From sspitsyn at openjdk.org Fri Jun 9 05:15:56 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 9 Jun 2023 05:15:56 GMT Subject: RFR: 8306028: separate ThreadStart/ThreadEnd events posting code in JVMTI VTMS transitions [v8] In-Reply-To: <9RJb8PvlZxLYs2zsDb-lDqoILDkiFwPx54vq3NpIdQQ=.c158dc31-68b0-49bd-8ac4-1ff1069be454@github.com> References: <9RJb8PvlZxLYs2zsDb-lDqoILDkiFwPx54vq3NpIdQQ=.c158dc31-68b0-49bd-8ac4-1ff1069be454@github.com> Message-ID: On Thu, 8 Jun 2023 19:05:54 GMT, Doug Simon wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> update copyright comments > > src/hotspot/share/runtime/sharedRuntime.cpp line 641: > >> 639: JRT_ENTRY(void, SharedRuntime::notify_jvmti_vthread_start(oopDesc* vt, jboolean hide, JavaThread* current)) >> 640: assert(hide == JNI_FALSE, "must be VTMS transition finish"); >> 641: jobject vthread = JNIHandles::make_local(const_cast(vt)); > > Since the current thread is in the `current` arg, it could be used here when creating the local handle. That's right. Thanks. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13484#discussion_r1223851214 From sspitsyn at openjdk.org Fri Jun 9 05:29:40 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 9 Jun 2023 05:29:40 GMT Subject: RFR: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled In-Reply-To: References: Message-ID: On Thu, 8 Jun 2023 17:14:39 GMT, Yudi Zheng wrote: > HeapMonitor checks if System.getProperty("jvmci.Compiler") is graal and will not enforce checking line number derived from uncommon trap debug info. However, Graal does not set this property explicitly. Looks good. Is JVMCI used by the Graal compiler only? Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14381#pullrequestreview-1471250105 From sspitsyn at openjdk.org Fri Jun 9 06:17:49 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 9 Jun 2023 06:17:49 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v4] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> <8pd-OWsWDM6RK-_X479uitxD-NNERoHXggUi4F7iemM=.229a79e8-5140-456b-80ba-2916e7dd95d9@github.com> <_52Zqcx-rftOA6EzeCRNEcUPBNI-fK3nvYQJ4TItudM=.00584b6b-f4a8-486a-b0c4-40d77bca5f2b@github.com> Message-ID: On Thu, 8 Jun 2023 06:25:20 GMT, Alan Bateman wrote: >> It was decided with Alan that it is okay to be in a waiting state. The `JVMTI_THREAD_STATE_BLOCKED_ON_MONITOR_ENTER` state requires a monitor to be blocked on, so it can be confusing. Alan's comment in the original PR [https://github.com/openjdk/jdk/pull/14298](https://github.com/openjdk/jdk/pull/14298) was: >>> if the jt is carrying thread_oop and it's okay for the JVMTI state to reported as WAITING when waiting for something other than Object.wait. > > The mental model is that the carrier is blocked so this is what an observer using the APIs should see. My recollection is that JVMTI_THREAD_STATE_WAITING was okay because there is a wriggle room in the JVM TI spec, it only uses Object.wait as an example. There may be a few rough edges to smooth down in this area. It's okay to take time with this PR and expand the tests to cover more cases and get more confident that there aren't more issues. We agreed with Alex to file a test RFE to improve test coverage in this area. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14366#discussion_r1223883294 From sspitsyn at openjdk.org Fri Jun 9 06:17:51 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 9 Jun 2023 06:17:51 GMT Subject: Integrated: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING In-Reply-To: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: On Wed, 7 Jun 2023 18:42:34 GMT, Serguei Spitsyn wrote: > This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). > The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. > The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). > > The first push/patch is the original fix of JDK-8307153. > The fix of the SUSPEND bit issue will be in the incremental update. > It is to simplify the review. > > Testing: > - TBD: mach5 tiers 1-5 This pull request has now been integrated. Changeset: f91e9ba7 Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/f91e9ba757f04983655c23542e06973805465249 Stats: 96 lines in 4 files changed: 76 ins; 0 del; 20 mod 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING Reviewed-by: cjplummer, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/14366 From sspitsyn at openjdk.org Fri Jun 9 06:26:50 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 9 Jun 2023 06:26:50 GMT Subject: RFR: 8309612: [REDO] JDK-8307153 JVMTI GetThreadState on carrier should return STATE_WAITING [v6] In-Reply-To: References: <2yWxf_TT2Dw2LLUa9fs8GZM-EfYIAOD-mTv1GLmg6o4=.705caff7-40a4-4dcb-862f-cebac0be68db@github.com> Message-ID: On Thu, 8 Jun 2023 01:42:10 GMT, Serguei Spitsyn wrote: >> This is REDO the fix of [JDK-8307153](https://bugs.openjdk.org/browse/JDK-8307153). >> The last update of the fix in the review cycle was incorrect and incorrectly tested, so the issue has not been noticed. It is why the fix was backed out. >> The issue is that the SUSPEND bit was missed in the JVMTI thread state of platform/carrier threads carrying virtual threads (see`JvmtiEnvBase::get_thread_state` function). >> >> The first push/patch is the original fix of JDK-8307153. >> The fix of the SUSPEND bit issue will be in the incremental update. >> It is to simplify the review. >> >> Testing: >> - TBD: mach5 tiers 1-5 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: corrected the function get_thread_state for safety Chris and Alex, thank you for review! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14366#issuecomment-1584039912 From dnsimon at openjdk.org Fri Jun 9 12:17:42 2023 From: dnsimon at openjdk.org (Doug Simon) Date: Fri, 9 Jun 2023 12:17:42 GMT Subject: RFR: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled In-Reply-To: References: Message-ID: <29D590lYaxvw5XS9uU-De3N2NJ3thhNWoWU7eQkhpfc=.80aa7b3c-7ab2-41a9-940f-8db0caefaaee@github.com> On Fri, 9 Jun 2023 05:26:59 GMT, Serguei Spitsyn wrote: > Is JVMCI used by the Graal compiler only? So far this is true and will probably remain true for the foreseeable future. However, the Right Thing to do long term is to add a `jdk.test.whitebox.code.Compiler.uncommonTrapsHavePreciseBCIs()` method. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14381#issuecomment-1584480496 From mbaesken at openjdk.org Fri Jun 9 13:45:58 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 9 Jun 2023 13:45:58 GMT Subject: RFR: JDK-8309549: com/sun/tools/attach/warnings/DynamicLoadWarningTest.java fails on AIX Message-ID: On AIX , new jtreg test com/sun/tools/attach/warnings/DynamicLoadWarningTest.java always failed with the output : ----------System.err:(294/28579)---------- STARTED DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' SUCCESSFUL DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' STARTED DynamicLoadWarningTest::testLoadOneJvmtiAgent '[1] DynamicLoadWarningTest$$Lambda/0x000000040020bd88 at 600d90bb' org.opentest4j.AssertionFailedError: expected: <1> but was: <2> at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197) at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150) at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145) at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:528) at DynamicLoadWarningTest$AppRunner.stderrShouldContain(DynamicLoadWarningTest.java:298) at DynamicLoadWarningTest.testLoadOneJvmtiAgent(DynamicLoadWarningTest.java:125) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) Reason seems to be the different behavior of dlopen on AIX compared to e.g. Linux This is what I find in the manpage of AIX 7.2 or 7.3 : https://www.ibm.com/docs/en/aix/7.2?topic=d-dlopen-subroutine https://www.ibm.com/docs/en/aix/7.3?topic=d-dlopen-subroutine 'If the module is already loaded, it is not loaded again, but a new, unique value will be returned by the dlopen subroutine.' Sounds different to what Linux documents in the manpage: https://man7.org/linux/man-pages/man3/dlopen.3.html 'If the same shared object is opened again with dlopen(), the same object handle is returned.' We skip this special subtest on AIX . ------------- Commit messages: - JDK-8309549 Changes: https://git.openjdk.org/jdk/pull/14393/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14393&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309549 Stats: 7 lines in 1 file changed: 3 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/14393.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14393/head:pull/14393 PR: https://git.openjdk.org/jdk/pull/14393 From alanb at openjdk.org Fri Jun 9 13:58:42 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 9 Jun 2023 13:58:42 GMT Subject: RFR: JDK-8309549: com/sun/tools/attach/warnings/DynamicLoadWarningTest.java fails on AIX In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 13:39:26 GMT, Matthias Baesken wrote: > On AIX , new jtreg test com/sun/tools/attach/warnings/DynamicLoadWarningTest.java always failed with the output : > > ----------System.err:(294/28579)---------- > STARTED DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' > SUCCESSFUL DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' > STARTED DynamicLoadWarningTest::testLoadOneJvmtiAgent '[1] DynamicLoadWarningTest$$Lambda/0x000000040020bd88 at 600d90bb' > org.opentest4j.AssertionFailedError: expected: <1> but was: <2> > at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) > at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) > at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197) > at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150) > at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145) > at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:528) > at DynamicLoadWarningTest$AppRunner.stderrShouldContain(DynamicLoadWarningTest.java:298) > at DynamicLoadWarningTest.testLoadOneJvmtiAgent(DynamicLoadWarningTest.java:125) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > > Reason seems to be the different behavior of dlopen on AIX compared to e.g. Linux > This is what I find in the manpage of AIX 7.2 or 7.3 : > https://www.ibm.com/docs/en/aix/7.2?topic=d-dlopen-subroutine > https://www.ibm.com/docs/en/aix/7.3?topic=d-dlopen-subroutine > 'If the module is already loaded, it is not loaded again, but a new, unique value will be returned by the dlopen subroutine.' > > Sounds different to what Linux documents in the manpage: > https://man7.org/linux/man-pages/man3/dlopen.3.html > 'If the same shared object is opened again with dlopen(), the same object handle is returned.' > > We skip this special subtest on AIX . Thanks for taking this one. The spec doesn't require warnings be de-duplicated when the same agent library is loaded more than once, thus skipping this test is okay where it's too complex to detect that the agent library is already loaded. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14393#pullrequestreview-1472159054 From coleenp at openjdk.org Fri Jun 9 14:58:42 2023 From: coleenp at openjdk.org (Coleen Phillimore) Date: Fri, 9 Jun 2023 14:58:42 GMT Subject: RFR: 8309673: Refactor ref_at methods in Serviceability ConstantPool In-Reply-To: References: Message-ID: On Thu, 8 Jun 2023 21:42:24 GMT, Matias Saavedra Silva wrote: > The accessor methods in constantpool.cpp were previously cleaned up to allow for different types of indices to be used, distinguishing them by the bytecode. This patch adds the same changes to the hotspot serviceability code. Verified with tier 1-5 tests. Looks good. src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/ConstantPool.java line 262: > 260: case Bytecodes._invokedynamic: > 261: int poolIndex = getCache().getIndyEntryAt(index).getConstantPoolIndex(); > 262: return poolIndex = invokeDynamicNameAndTypeRefIndexAt(poolIndex); probably don't need another assignment to poolIndex in the return statement. ------------- Marked as reviewed by coleenp (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14385#pullrequestreview-1472334909 PR Review Comment: https://git.openjdk.org/jdk/pull/14385#discussion_r1224416163 From fparain at openjdk.org Fri Jun 9 16:55:45 2023 From: fparain at openjdk.org (Frederic Parain) Date: Fri, 9 Jun 2023 16:55:45 GMT Subject: RFR: 8309673: Refactor ref_at methods in Serviceability ConstantPool In-Reply-To: References: Message-ID: On Thu, 8 Jun 2023 21:42:24 GMT, Matias Saavedra Silva wrote: > The accessor methods in constantpool.cpp were previously cleaned up to allow for different types of indices to be used, distinguishing them by the bytecode. This patch adds the same changes to the hotspot serviceability code. Verified with tier 1-5 tests. Marked as reviewed by fparain (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14385#pullrequestreview-1472650636 From cjplummer at openjdk.org Fri Jun 9 17:35:44 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 9 Jun 2023 17:35:44 GMT Subject: RFR: JDK-8309549: com/sun/tools/attach/warnings/DynamicLoadWarningTest.java fails on AIX In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 13:39:26 GMT, Matthias Baesken wrote: > On AIX , new jtreg test com/sun/tools/attach/warnings/DynamicLoadWarningTest.java always failed with the output : > > ----------System.err:(294/28579)---------- > STARTED DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' > SUCCESSFUL DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' > STARTED DynamicLoadWarningTest::testLoadOneJvmtiAgent '[1] DynamicLoadWarningTest$$Lambda/0x000000040020bd88 at 600d90bb' > org.opentest4j.AssertionFailedError: expected: <1> but was: <2> > at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) > at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) > at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197) > at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150) > at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145) > at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:528) > at DynamicLoadWarningTest$AppRunner.stderrShouldContain(DynamicLoadWarningTest.java:298) > at DynamicLoadWarningTest.testLoadOneJvmtiAgent(DynamicLoadWarningTest.java:125) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > > Reason seems to be the different behavior of dlopen on AIX compared to e.g. Linux > This is what I find in the manpage of AIX 7.2 or 7.3 : > https://www.ibm.com/docs/en/aix/7.2?topic=d-dlopen-subroutine > https://www.ibm.com/docs/en/aix/7.3?topic=d-dlopen-subroutine > 'If the module is already loaded, it is not loaded again, but a new, unique value will be returned by the dlopen subroutine.' > > Sounds different to what Linux documents in the manpage: > https://man7.org/linux/man-pages/man3/dlopen.3.html > 'If the same shared object is opened again with dlopen(), the same object handle is returned.' > > We skip this special subtest on AIX . Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14393#pullrequestreview-1472750566 From cjplummer at openjdk.org Fri Jun 9 18:11:41 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 9 Jun 2023 18:11:41 GMT Subject: RFR: 8232839: JDI AfterThreadDeathTest.java failed due to "FAILED: Did not get expected IllegalThreadStateException on a StepRequest.enable()" In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 23:17:18 GMT, Chris Plummer wrote: > The test waits for a ThreadDeathEvent for "main". Once that arrives, it then waits for the next ThreadStartEvent (for any thread). Once it arrives, the test tries to create and enable a StepRequest on the "main" thread. Since "main" is supposedly dead, the expectation is an IllegalThreadStateException. However, it turns out that sometimes the enabling can in fact succeed. > > Just because a ThreadDeathEvent has been received for a thread does not mean you can no longer do things with the thread like suspend it or enable a StepEvent. There is a short delay in the debug agent after sending the ThreadDeathEvent before it stops tracking the thread. The thread can still be acted upon until then. The JDWP and JDI specs seem to support doing this: > >> Notification of a completed thread in the target VM. The notification is generated by the dying thread before it terminates. Because of this timing, it is possible for {@link VirtualMachine#allThreads} to return this thread after this event is received. >> >> Note that this event gives no information about the lifetime of the thread object. It may or may not be collected soon depending on what references exist in the target VM. > > What this means is that when the test receives some arbitrary ThreadStartEvent after the "main" ThreadDeathEvent has been received, the test may in fact still be able to enable a StepRequest on the "main" thread, causing the test to fail. What seems to trigger the failure is receiving an unexpected spurious ThreadStartEvent such as from the Common-Clean thread or a carrier thread, although even then it only fails some of the time. In fact if I modify the test to enable the StepRequest when it receives the ThreadDeathEvent for "main", it still almost always passes, but will fail more frequently than it normally does. > > It seems if the test always waits for the ThreadStartEvent for "DestroyJavaVM", then the "main" thread is truly gone by then and the test always passes, so this is how I've chosen to fix the issue. > > Tested with tier1, tier2 svc tests, and tier5 svc tests. thanks for the reviews Kevin and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14372#issuecomment-1584951868 From cjplummer at openjdk.org Fri Jun 9 18:11:42 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 9 Jun 2023 18:11:42 GMT Subject: Integrated: 8232839: JDI AfterThreadDeathTest.java failed due to "FAILED: Did not get expected IllegalThreadStateException on a StepRequest.enable()" In-Reply-To: References: Message-ID: On Wed, 7 Jun 2023 23:17:18 GMT, Chris Plummer wrote: > The test waits for a ThreadDeathEvent for "main". Once that arrives, it then waits for the next ThreadStartEvent (for any thread). Once it arrives, the test tries to create and enable a StepRequest on the "main" thread. Since "main" is supposedly dead, the expectation is an IllegalThreadStateException. However, it turns out that sometimes the enabling can in fact succeed. > > Just because a ThreadDeathEvent has been received for a thread does not mean you can no longer do things with the thread like suspend it or enable a StepEvent. There is a short delay in the debug agent after sending the ThreadDeathEvent before it stops tracking the thread. The thread can still be acted upon until then. The JDWP and JDI specs seem to support doing this: > >> Notification of a completed thread in the target VM. The notification is generated by the dying thread before it terminates. Because of this timing, it is possible for {@link VirtualMachine#allThreads} to return this thread after this event is received. >> >> Note that this event gives no information about the lifetime of the thread object. It may or may not be collected soon depending on what references exist in the target VM. > > What this means is that when the test receives some arbitrary ThreadStartEvent after the "main" ThreadDeathEvent has been received, the test may in fact still be able to enable a StepRequest on the "main" thread, causing the test to fail. What seems to trigger the failure is receiving an unexpected spurious ThreadStartEvent such as from the Common-Clean thread or a carrier thread, although even then it only fails some of the time. In fact if I modify the test to enable the StepRequest when it receives the ThreadDeathEvent for "main", it still almost always passes, but will fail more frequently than it normally does. > > It seems if the test always waits for the ThreadStartEvent for "DestroyJavaVM", then the "main" thread is truly gone by then and the test always passes, so this is how I've chosen to fix the issue. > > Tested with tier1, tier2 svc tests, and tier5 svc tests. This pull request has now been integrated. Changeset: 84184f94 Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/84184f947342fd1adbe4e3f2230ce3de4ae6007e Stats: 24 lines in 2 files changed: 11 ins; 3 del; 10 mod 8232839: JDI AfterThreadDeathTest.java failed due to "FAILED: Did not get expected IllegalThreadStateException on a StepRequest.enable()" Reviewed-by: sspitsyn, kevinw ------------- PR: https://git.openjdk.org/jdk/pull/14372 From matsaave at openjdk.org Fri Jun 9 18:14:18 2023 From: matsaave at openjdk.org (Matias Saavedra Silva) Date: Fri, 9 Jun 2023 18:14:18 GMT Subject: RFR: 8309673: Refactor ref_at methods in Serviceability ConstantPool [v2] In-Reply-To: References: Message-ID: > The accessor methods in constantpool.cpp were previously cleaned up to allow for different types of indices to be used, distinguishing them by the bytecode. This patch adds the same changes to the hotspot serviceability agent code. Verified with tier 1-5 tests. Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision: Removed unnecessary assignment ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14385/files - new: https://git.openjdk.org/jdk/pull/14385/files/2ee60bd1..6630cb2c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14385&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14385&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14385.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14385/head:pull/14385 PR: https://git.openjdk.org/jdk/pull/14385 From iklam at openjdk.org Fri Jun 9 19:06:44 2023 From: iklam at openjdk.org (Ioi Lam) Date: Fri, 9 Jun 2023 19:06:44 GMT Subject: RFR: 8309673: Refactor ref_at methods in SA ConstantPool [v2] In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 18:14:18 GMT, Matias Saavedra Silva wrote: >> The accessor methods in constantpool.cpp were previously cleaned up to allow for different types of indices to be used, distinguishing them by the bytecode. This patch adds the same changes to the hotspot serviceability agent code. Verified with tier 1-5 tests. > > Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision: > > Removed unnecessary assignment LGTM. I would suggest changing "Serviceability" in the title to "SA" ------------- Marked as reviewed by iklam (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14385#pullrequestreview-1472921430 From matsaave at openjdk.org Fri Jun 9 19:10:51 2023 From: matsaave at openjdk.org (Matias Saavedra Silva) Date: Fri, 9 Jun 2023 19:10:51 GMT Subject: Integrated: 8309673: Refactor ref_at methods in SA ConstantPool In-Reply-To: References: Message-ID: On Thu, 8 Jun 2023 21:42:24 GMT, Matias Saavedra Silva wrote: > The accessor methods in constantpool.cpp were previously cleaned up to allow for different types of indices to be used, distinguishing them by the bytecode. This patch adds the same changes to the hotspot serviceability agent code. Verified with tier 1-5 tests. This pull request has now been integrated. Changeset: 7d6f97d0 Author: Matias Saavedra Silva URL: https://git.openjdk.org/jdk/commit/7d6f97d04d8fac44b9c71ec7e36c27ec61e82445 Stats: 95 lines in 4 files changed: 32 ins; 25 del; 38 mod 8309673: Refactor ref_at methods in SA ConstantPool Reviewed-by: coleenp, fparain, iklam ------------- PR: https://git.openjdk.org/jdk/pull/14385 From matsaave at openjdk.org Fri Jun 9 19:10:49 2023 From: matsaave at openjdk.org (Matias Saavedra Silva) Date: Fri, 9 Jun 2023 19:10:49 GMT Subject: RFR: 8309673: Refactor ref_at methods in SA ConstantPool [v2] In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 14:56:21 GMT, Coleen Phillimore wrote: >> Matias Saavedra Silva has updated the pull request incrementally with one additional commit since the last revision: >> >> Removed unnecessary assignment > > Looks good. Thank you for the reviews @coleenp, @fparain, and @iklam ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14385#issuecomment-1585021327 From sspitsyn at openjdk.org Fri Jun 9 19:25:39 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 9 Jun 2023 19:25:39 GMT Subject: RFR: JDK-8309549: com/sun/tools/attach/warnings/DynamicLoadWarningTest.java fails on AIX In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 13:39:26 GMT, Matthias Baesken wrote: > On AIX , new jtreg test com/sun/tools/attach/warnings/DynamicLoadWarningTest.java always failed with the output : > > ----------System.err:(294/28579)---------- > STARTED DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' > SUCCESSFUL DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' > STARTED DynamicLoadWarningTest::testLoadOneJvmtiAgent '[1] DynamicLoadWarningTest$$Lambda/0x000000040020bd88 at 600d90bb' > org.opentest4j.AssertionFailedError: expected: <1> but was: <2> > at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) > at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) > at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197) > at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150) > at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145) > at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:528) > at DynamicLoadWarningTest$AppRunner.stderrShouldContain(DynamicLoadWarningTest.java:298) > at DynamicLoadWarningTest.testLoadOneJvmtiAgent(DynamicLoadWarningTest.java:125) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > > Reason seems to be the different behavior of dlopen on AIX compared to e.g. Linux > This is what I find in the manpage of AIX 7.2 or 7.3 : > https://www.ibm.com/docs/en/aix/7.2?topic=d-dlopen-subroutine > https://www.ibm.com/docs/en/aix/7.3?topic=d-dlopen-subroutine > 'If the module is already loaded, it is not loaded again, but a new, unique value will be returned by the dlopen subroutine.' > > Sounds different to what Linux documents in the manpage: > https://man7.org/linux/man-pages/man3/dlopen.3.html > 'If the same shared object is opened again with dlopen(), the same object handle is returned.' > > We skip this special subtest on AIX . Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14393#pullrequestreview-1472948256 From cjplummer at openjdk.org Fri Jun 9 23:38:22 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 9 Jun 2023 23:38:22 GMT Subject: RFR: 8309752: com/sun/jdi/SetLocalWhileThreadInNative.java fails with virtual test thread factory due to OpaqueFrameException Message-ID: com/sun/jdi/SetLocalWhileThreadInNative.java is failing with OpaqueFrameException when using the virtual test thread factory. The reason is because JDI only supports calling StackFrame.setValue() on the topmost frame of a virtual thread. The test is calling it on the ThreadReference.frames(2), so the OpaqueFrameException is correct behavior and the test needs to adapt. I could have chosen to just not have this test support running on a virtual thread, but it appears to be the only test we have that attempts StackFrame.setValue() on something other than the topmost frame, so it's good to have it expect the OpaqueFrameException. Tested locally with and without the virtual thread wrapper. tier1 and tier5 svc testing tbd. ------------- Commit messages: - Fix jcheck error - Expect OpaqueFrameException when using virtual threads. Changes: https://git.openjdk.org/jdk/pull/14402/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14402&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309752 Stats: 11 lines in 2 files changed: 7 ins; 1 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/14402.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14402/head:pull/14402 PR: https://git.openjdk.org/jdk/pull/14402 From cjplummer at openjdk.org Sat Jun 10 00:05:48 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 10 Jun 2023 00:05:48 GMT Subject: RFR: 8309757: com/sun/jdi/ReferrersTest.java fails with virtual test thread factor Message-ID: This test launches a debuggee, which creates 11 instances of its main class, stores them in a static array of the main class, and then the debugger side iterates over all referrers to the main class instaces. Usually this is a pretty quick process and doesn't produce much in the way of output while walking the reference tree of referrers. However, with virtual threads the tree walking and output get unwieldy, and eventually it fails with: IOException reading output of child java interpreter:Stream closed java.lang.IllegalThreadStateException at jdk.jdi/com.sun.tools.jdi.JDWPException.toJDIException(JDWPException.java:76) at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.name(ThreadReferenceImpl.java:197) at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.toString(ThreadReferenceImpl.java:637) at java.base/java.lang.String.valueOf(String.java:4461) at ReferrersTest.showReferrers(ReferrersTest.java:438) at ReferrersTest.showReferrers(ReferrersTest.java:466) And ReferrersTest.showReferrers() has recursed about 200 levels deep. I'm not sure the order of these errors can be relied on. It looks like while walking the referrers tree, the test eventually stumbled upon a thread that had exited (but its Thread object was still alive), and this resulted in the test aborting. If I catch these exceptions, eventually the test times out while still working on referrers. Judging by some of the output, it appears that introducing the TestScaffold class as a referrer to the main debuggee class is the root cause of all these extra referrers. The test has a provision to cut off the recursion: // We have to stop going up a referrer chain in some cases Type rt = objRef.type(); if (rt instanceof ClassType) { ClassType ct = (ClassType)rt; String name = ct.name(); if (name.equals("sun.awt.SoftCache$ValueCell")) { return; } if (name.equals("java.lang.ref.Finalizer")) { return; } if (name.equals("java.lang.ref.SoftReference")) { return; } // oh oh, should really check for a subclass of ClassLoader :-) if (name.indexOf("ClassLoader") >= 0) { return; } // No doubt there are other reasons to stop ... } Adding TestScaffold to the list makes it so the referrer tree walking output is almost identical to what it is when not using virtual threads. Adding java.lang.reflect.Method instead does a slightly better job. Tested locally with and without the virtual thread wrapper. tier1 and tier5 svc testing tbd. ------------- Commit messages: - Do some additional pruning needed when run with a virtual thread Changes: https://git.openjdk.org/jdk/pull/14405/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14405&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309757 Stats: 5 lines in 2 files changed: 3 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14405.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14405/head:pull/14405 PR: https://git.openjdk.org/jdk/pull/14405 From sspitsyn at openjdk.org Sat Jun 10 02:07:50 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sat, 10 Jun 2023 02:07:50 GMT Subject: RFR: 8309752: com/sun/jdi/SetLocalWhileThreadInNative.java fails with virtual test thread factory due to OpaqueFrameException In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 20:47:13 GMT, Chris Plummer wrote: > com/sun/jdi/SetLocalWhileThreadInNative.java is failing with OpaqueFrameException when using the virtual test thread factory. The reason is because JDI only supports calling StackFrame.setValue() on the topmost frame of a virtual thread. The test is calling it on the ThreadReference.frames(2), so the OpaqueFrameException is correct behavior and the test needs to adapt. > > I could have chosen to just not have this test support running on a virtual thread, but it appears to be the only test we have that attempts StackFrame.setValue() on something other than the topmost frame, so it's good to have it expect the OpaqueFrameException. > > Tested locally with and without the virtual thread wrapper. tier1 and tier5 svc testing tbd. Looks good. Thanks, Serguei test/jdk/com/sun/jdi/SetLocalWhileThreadInNative.java line 166: > 164: List localVars = frame.visibleVariables(); > 165: boolean changedLocal = false; > 166: boolean caughtOPE = false; Nit: Should it be caughtOFE instead of caughtOPE? ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14402#pullrequestreview-1473268372 PR Review Comment: https://git.openjdk.org/jdk/pull/14402#discussion_r1224999636 From cjplummer at openjdk.org Sat Jun 10 02:56:47 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 10 Jun 2023 02:56:47 GMT Subject: RFR: 8309752: com/sun/jdi/SetLocalWhileThreadInNative.java fails with virtual test thread factory due to OpaqueFrameException [v2] In-Reply-To: References: Message-ID: > com/sun/jdi/SetLocalWhileThreadInNative.java is failing with OpaqueFrameException when using the virtual test thread factory. The reason is because JDI only supports calling StackFrame.setValue() on the topmost frame of a virtual thread. The test is calling it on the ThreadReference.frames(2), so the OpaqueFrameException is correct behavior and the test needs to adapt. > > I could have chosen to just not have this test support running on a virtual thread, but it appears to be the only test we have that attempts StackFrame.setValue() on something other than the topmost frame, so it's good to have it expect the OpaqueFrameException. > > Tested locally with and without the virtual thread wrapper. tier1 and tier5 svc testing tbd. Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: caughtOPE -> caughtOFE ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14402/files - new: https://git.openjdk.org/jdk/pull/14402/files/ee21d434..2799ebe6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14402&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14402&range=00-01 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/14402.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14402/head:pull/14402 PR: https://git.openjdk.org/jdk/pull/14402 From asemenov at openjdk.org Sun Jun 11 15:40:28 2023 From: asemenov at openjdk.org (Artem Semenov) Date: Sun, 11 Jun 2023 15:40:28 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v3] In-Reply-To: References: Message-ID: On Tue, 6 Jun 2023 17:47:02 GMT, Weijun Wang wrote: >> This is rarely used in the code and is not the essence of the current changes. >> If you introduce such changes, then throughout the code. >> Moreover, this can lead to problems, such as, for example, here: https://bugs.openjdk.org/browse/JDK-8309225 > > I'm not a clang expect, I was just asking if modifying the current `#if TARGET_OS_MAC` check into `#if defined(__APPLE__)` is also a solution. The comment on lines 46-47 says the condition was copied from a macOS SDK file and that's what the file is using now. I think you can create a separate ticket and pull request to discuss this issue. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14033#discussion_r1225842637 From asemenov at openjdk.org Sun Jun 11 16:38:31 2023 From: asemenov at openjdk.org (Artem Semenov) Date: Sun, 11 Jun 2023 16:38:31 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v5] In-Reply-To: References: Message-ID: > When using the clang compiler to build OpenJDk on Linux, we encounter various "warnings as errors". > They can be fixed with small changes. Artem Semenov has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - update - update - update - update - 8308286 Fix clang warnings in linux code ------------- Changes: https://git.openjdk.org/jdk/pull/14033/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14033&range=04 Stats: 16 lines in 6 files changed: 14 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14033.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14033/head:pull/14033 PR: https://git.openjdk.org/jdk/pull/14033 From lmesnik at openjdk.org Sun Jun 11 18:16:28 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Sun, 11 Jun 2023 18:16:28 GMT Subject: RFR: 8309752: com/sun/jdi/SetLocalWhileThreadInNative.java fails with virtual test thread factory due to OpaqueFrameException [v2] In-Reply-To: References: Message-ID: On Sat, 10 Jun 2023 02:56:47 GMT, Chris Plummer wrote: >> com/sun/jdi/SetLocalWhileThreadInNative.java is failing with OpaqueFrameException when using the virtual test thread factory. The reason is because JDI only supports calling StackFrame.setValue() on the topmost frame of a virtual thread. The test is calling it on the ThreadReference.frames(2), so the OpaqueFrameException is correct behavior and the test needs to adapt. >> >> I could have chosen to just not have this test support running on a virtual thread, but it appears to be the only test we have that attempts StackFrame.setValue() on something other than the topmost frame, so it's good to have it expect the OpaqueFrameException. >> >> Tested locally with and without the virtual thread wrapper. tier1 and tier5 svc testing tbd. > > Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: > > caughtOPE -> caughtOFE Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14402#pullrequestreview-1473772526 From yyang at openjdk.org Mon Jun 12 06:34:06 2023 From: yyang at openjdk.org (Yi Yang) Date: Mon, 12 Jun 2023 06:34:06 GMT Subject: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v5] In-Reply-To: References: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> Message-ID: On Tue, 16 May 2023 18:41:26 GMT, Chris Plummer wrote: >> Hi, can I have a review for this patch? > > @y1yang0 Sorry no one has been able to review this so far. The serviceability team is very busy for the next few weeks finishing up JDK 21 changes before RDP1. It's unlikely we'll find time for the review before them. > > I did take a very quick look at the changes just to understand the scope. One thing I noticed that makes this PR hard to review is the code refactoring and relocating that you did. At first it looks like a lot of old code was deleted and a lot of new code added, but in fact most of the new code is just relocated old code. It makes it very hard to tell if there have been any changes to this code. Is there anything you can do to lessen the amount of apparent new code that is actually just moved code? Hi @plummercj @kevinjwalls @alexmenkov, sorry to disturb you. RDP1 has ended. Do you have any plans to review this patch recently? Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/13667#issuecomment-1586671507 From mbaesken at openjdk.org Mon Jun 12 07:10:57 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 12 Jun 2023 07:10:57 GMT Subject: Integrated: JDK-8309549: com/sun/tools/attach/warnings/DynamicLoadWarningTest.java fails on AIX In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 13:39:26 GMT, Matthias Baesken wrote: > On AIX , new jtreg test com/sun/tools/attach/warnings/DynamicLoadWarningTest.java always failed with the output : > > ----------System.err:(294/28579)---------- > STARTED DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' > SUCCESSFUL DynamicLoadWarningTest::testLoadJavaAgent 'testLoadJavaAgent()' > STARTED DynamicLoadWarningTest::testLoadOneJvmtiAgent '[1] DynamicLoadWarningTest$$Lambda/0x000000040020bd88 at 600d90bb' > org.opentest4j.AssertionFailedError: expected: <1> but was: <2> > at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) > at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) > at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197) > at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150) > at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145) > at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:528) > at DynamicLoadWarningTest$AppRunner.stderrShouldContain(DynamicLoadWarningTest.java:298) > at DynamicLoadWarningTest.testLoadOneJvmtiAgent(DynamicLoadWarningTest.java:125) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > > Reason seems to be the different behavior of dlopen on AIX compared to e.g. Linux > This is what I find in the manpage of AIX 7.2 or 7.3 : > https://www.ibm.com/docs/en/aix/7.2?topic=d-dlopen-subroutine > https://www.ibm.com/docs/en/aix/7.3?topic=d-dlopen-subroutine > 'If the module is already loaded, it is not loaded again, but a new, unique value will be returned by the dlopen subroutine.' > > Sounds different to what Linux documents in the manpage: > https://man7.org/linux/man-pages/man3/dlopen.3.html > 'If the same shared object is opened again with dlopen(), the same object handle is returned.' > > We skip this special subtest on AIX . This pull request has now been integrated. Changeset: 4d66d977 Author: Matthias Baesken URL: https://git.openjdk.org/jdk/commit/4d66d977450e083214da3dba6ad4ed851c6c1cb4 Stats: 7 lines in 1 file changed: 3 ins; 0 del; 4 mod 8309549: com/sun/tools/attach/warnings/DynamicLoadWarningTest.java fails on AIX Reviewed-by: alanb, cjplummer, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/14393 From duke at openjdk.org Mon Jun 12 10:04:05 2023 From: duke at openjdk.org (JoKern65) Date: Mon, 12 Jun 2023 10:04:05 GMT Subject: Integrated: JDK-8309462: [AIX] vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: <6FqRlF1qvten3FMAVitnVXbHIAH2PGT8r__lviVrDUE=.15db8952-297e-4613-9fe1-49a53261febe@github.com> On Tue, 6 Jun 2023 09:51:09 GMT, JoKern65 wrote: > The sys_thread_3() function contains an empty while loop, which by the standard can be optimized away. Please refer to discussion in https://github.com/llvm/llvm-project/issues/60622 > The xlc17 compiler is doing so, and IBM claims that they are following the standard and will not fix this on compiler side. > So we have (at least) 3 ways to circumvent this behavior. > > 1. we can introduce the call of a nop library function, which will hinder the optimizer to throw away the loop (This is our proposed solution, but instead of a heavy looping thread, the result is a more or less idle thread): > `#include ` > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` while (1) {` > ` sleep(1);` > ` }` > `}` > > 2. We can make use of a volatile variable in the loop body which also hinders the optimizer to throw away the loop: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p)` > `{` > ` volatile int i = 1;` > ` while (i) {` > ` i += 2;` > ` }` > `}` > > 3. we can use the __attribute__ ((optnone)) modifier in the function declaration to suppress the optimization at all: > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) __attribute__ ((optnone))` > `{` > ` while (1) {` > ` }` > `}` > > To make the third approach platform independent, we can implement it in the following way: > In globalDefinitions.hpp > `#ifndef OPTNONE` > `#define OPTNONE` > `#endif` > > In globalDefinitions_xlc.hpp > `// optnone support` > `//` > `// To use if a function should not be optimized` > `// Usage:` > `// void* func(size_t size) OPTNONE {...}` > `#define OPTNONE __attribute__(( optnone))` > > With this we can change libagentthr001.cpp in a platform independent way to > `static void` > `sys_thread_3(jvmtiEnv* jvmti, JNIEnv* jni, void *p) OPTNONE` > `{` > ` while (1) {` > ` }` > `}` This pull request has now been integrated. Changeset: cf9e6353 Author: JoKern65 Committer: Martin Doerr URL: https://git.openjdk.org/jdk/commit/cf9e6353cc6fe9e57a7a9883813d09892e7e7621 Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod 8309462: [AIX] vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop Reviewed-by: mbaesken, mdoerr, clanger ------------- PR: https://git.openjdk.org/jdk/pull/14330 From erikj at openjdk.org Mon Jun 12 12:30:55 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 12 Jun 2023 12:30:55 GMT Subject: RFR: 8308286 Fix clang warnings in linux code [v5] In-Reply-To: References: Message-ID: On Sun, 11 Jun 2023 16:38:31 GMT, Artem Semenov wrote: >> When using the clang compiler to build OpenJDk on Linux, we encounter various "warnings as errors". >> They can be fixed with small changes. > > Artem Semenov has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - update > - update > - update > - update > - 8308286 Fix clang warnings in linux code Build changes look ok. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14033#issuecomment-1587242876 From apangin at openjdk.org Mon Jun 12 13:00:56 2023 From: apangin at openjdk.org (Andrei Pangin) Date: Mon, 12 Jun 2023 13:00:56 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives In-Reply-To: References: Message-ID: On Wed, 24 May 2023 00:38:27 GMT, Dmitry Chuyko wrote: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives. Methods in general are often inlined, and this information is hard to track down. > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Obviously there is a performance penalty, so it should be applied with care. Hot code will most likely be recompiled soon, as nothing happens to its hotness. > > A new flag '`-d`' has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and marks for deoptimization those methods that have any active non-default matching compiler directives. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be deoptimized. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be deoptimized, but this can be achieved by having rules for them. > > In addition, a new diagnistic command `Compiler.replace_directives`, has been added for convenience. It's like a combinatio... src/hotspot/share/code/codeCache.cpp line 1379: > 1377: while(iter.next()) { > 1378: CompiledMethod* nm = iter.method(); > 1379: HandleMark hm(thread); Isn't it better to move `HandleMark` outside the loop? src/hotspot/share/runtime/mutexLocker.cpp line 274: > 272: MUTEX_DEFN(MethodCompileQueue_lock , PaddedMonitor, safepoint); > 273: MUTEX_DEFN(CompileStatistics_lock , PaddedMutex , safepoint); > 274: MUTEX_DEFN(DirectivesStack_lock , PaddedMutex , nosafepoint-3); A comment explaining the rank change would be helpful. src/hotspot/share/services/diagnosticCommand.cpp line 890: > 888: DCmdWithParser(output, heap), > 889: _filename("filename", "Name of the directives file", "STRING", true), > 890: _force_deopt("-d", "Force deoptimization of affected methods.", "BOOLEAN", false, "false") { I agree with Paul a generic alternative like `refresh` would be better. src/hotspot/share/services/diagnosticCommand.cpp line 946: > 944: DeoptimizationScope deopt_scope; > 945: CodeCache::mark_for_deoptimization_directives_matches(&deopt_scope); > 946: DirectivesStack::pop(1); Why deoptimizing methods from the whole stack if we change only the topmost list? src/hotspot/share/services/diagnosticCommand.hpp line 734: > 732: }; > 733: > 734: class CompilerDirectivesReplaceDCmd : public DCmdWithParser { Introducing a new command probably isn't worth it, given the same functionality can be achieved with existing commands. Furthermore, it's not obvious whether "replace" should mean remove+add or clear+add. src/hotspot/share/services/diagnosticCommand.hpp line 745: > 743: } > 744: static const char* description() { > 745: return "Clear derectives stack amd load new compiler directives from file."; Spelling: Clear *directives* stack *and* ... ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1226618742 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1226553192 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1226624671 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1226627598 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1226604676 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1226548048 From stsypanov at openjdk.org Mon Jun 12 13:30:41 2023 From: stsypanov at openjdk.org (Sergey Tsypanov) Date: Mon, 12 Jun 2023 13:30:41 GMT Subject: RFR: 8309408: Thread.sleep cleanup In-Reply-To: References: Message-ID: On Sun, 4 Jun 2023 11:28:33 GMT, Alan Bateman wrote: > Thread.sleep has had quite a bit of churn recently to support virtual threads, add sleep(Duration), a JFR event, and the change the underlying implementation to support sub-millis precision. I think the changes have settled down now so we can do some small cleanups that came up in PR discussions. The cleanups were kicked down the road as it requires tracking down faraway tests that depend on the stack depth and the names of internal methods. The two cleanups proposed here are: > > 1. Add a private sleepNanos method that creates/commits the JFR event around the sleep, this avoids duplicate code in the 3 sleep methods. > 2. Rename JVM_Sleep to JVM_SleepNanos to make it clear that it takes the sleep time in nanoseconds, esp. when Thread.sleep's parameter is milliseconds. test/hotspot/jtreg/vmTestbase/nsk/jdwp/ThreadReference/OwnedMonitorsStackDepthInfo/ownedMonitorsStackDepthInfo001/ownedMonitorsStackDepthInfo001a.java line 113: > 111: return -1; > 112: } else { > 113: return depth + 2; Doesn't it change the logic? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14303#discussion_r1226668010 From mdoerr at openjdk.org Mon Jun 12 14:45:24 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Mon, 12 Jun 2023 14:45:24 GMT Subject: RFR: 8309462: [AIX] vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop Message-ID: Test fix for test failing on AIX because of undefined behavior in current implementation. ------------- Commit messages: - Backport cf9e6353cc6fe9e57a7a9883813d09892e7e7621 Changes: https://git.openjdk.org/jdk21/pull/9/files Webrev: https://webrevs.openjdk.org/?repo=jdk21&pr=9&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309462 Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk21/pull/9.diff Fetch: git fetch https://git.openjdk.org/jdk21.git pull/9/head:pull/9 PR: https://git.openjdk.org/jdk21/pull/9 From mbaesken at openjdk.org Mon Jun 12 14:49:49 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 12 Jun 2023 14:49:49 GMT Subject: RFR: 8309462: [AIX] vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 14:36:56 GMT, Martin Doerr wrote: > Test fix for test failing on AIX because of undefined behavior in current implementation. Marked as reviewed by mbaesken (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk21/pull/9#pullrequestreview-1475121715 From mdoerr at openjdk.org Mon Jun 12 14:49:50 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Mon, 12 Jun 2023 14:49:50 GMT Subject: RFR: 8309462: [AIX] vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 14:36:56 GMT, Martin Doerr wrote: > Test fix for test failing on AIX because of undefined behavior in current implementation. Thanks for the review! I'm planning to integrate tomorrow. ------------- PR Comment: https://git.openjdk.org/jdk21/pull/9#issuecomment-1587497638 From duke at openjdk.org Mon Jun 12 17:59:12 2023 From: duke at openjdk.org (duke) Date: Mon, 12 Jun 2023 17:59:12 GMT Subject: Withdrawn: 8295146: Clean up native code with newer C/C++ language features In-Reply-To: References: Message-ID: On Thu, 10 Nov 2022 06:20:41 GMT, Julian Waters wrote: > After [JDK-8292008](https://bugs.openjdk.org/browse/JDK-8292008) and [JDK-8247283](https://bugs.openjdk.org/browse/JDK-8247283), some C and C++ code across the JDK can be replaced and simplified with cleaner language features that were previously not available due to required compatibility with the now unsupported Visual C++ 2017 compiler. These cleanups were highlighted by the very briefly integrated 8296115 > > No changes to the behaviour of the JDK has resulted in any way from this commit This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/11081 From cjplummer at openjdk.org Mon Jun 12 18:10:59 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 12 Jun 2023 18:10:59 GMT Subject: RFR: 8309752: com/sun/jdi/SetLocalWhileThreadInNative.java fails with virtual test thread factory due to OpaqueFrameException [v2] In-Reply-To: References: Message-ID: On Sat, 10 Jun 2023 02:56:47 GMT, Chris Plummer wrote: >> com/sun/jdi/SetLocalWhileThreadInNative.java is failing with OpaqueFrameException when using the virtual test thread factory. The reason is because JDI only supports calling StackFrame.setValue() on the topmost frame of a virtual thread. The test is calling it on the ThreadReference.frames(2), so the OpaqueFrameException is correct behavior and the test needs to adapt. >> >> I could have chosen to just not have this test support running on a virtual thread, but it appears to be the only test we have that attempts StackFrame.setValue() on something other than the topmost frame, so it's good to have it expect the OpaqueFrameException. >> >> Tested locally with and without the virtual thread wrapper. tier1 and tier5 svc testing tbd. > > Chris Plummer has updated the pull request incrementally with one additional commit since the last revision: > > caughtOPE -> caughtOFE Thanks for the reviews Leonid and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14402#issuecomment-1587826234 From cjplummer at openjdk.org Mon Jun 12 18:10:59 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 12 Jun 2023 18:10:59 GMT Subject: Integrated: 8309752: com/sun/jdi/SetLocalWhileThreadInNative.java fails with virtual test thread factory due to OpaqueFrameException In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 20:47:13 GMT, Chris Plummer wrote: > com/sun/jdi/SetLocalWhileThreadInNative.java is failing with OpaqueFrameException when using the virtual test thread factory. The reason is because JDI only supports calling StackFrame.setValue() on the topmost frame of a virtual thread. The test is calling it on the ThreadReference.frames(2), so the OpaqueFrameException is correct behavior and the test needs to adapt. > > I could have chosen to just not have this test support running on a virtual thread, but it appears to be the only test we have that attempts StackFrame.setValue() on something other than the topmost frame, so it's good to have it expect the OpaqueFrameException. > > Tested locally with and without the virtual thread wrapper. tier1 and tier5 svc testing tbd. This pull request has now been integrated. Changeset: 1612b6dc Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/1612b6dc6e7075e1c0c82d0e92910051c8066fe5 Stats: 11 lines in 2 files changed: 7 ins; 1 del; 3 mod 8309752: com/sun/jdi/SetLocalWhileThreadInNative.java fails with virtual test thread factory due to OpaqueFrameException Reviewed-by: sspitsyn, lmesnik ------------- PR: https://git.openjdk.org/jdk/pull/14402 From iklam at openjdk.org Mon Jun 12 20:00:19 2023 From: iklam at openjdk.org (Ioi Lam) Date: Mon, 12 Jun 2023 20:00:19 GMT Subject: RFR: 8309878: Reduce inclusion of resolvedIndyEntry.hpp Message-ID: resolvedIndyEntry.hpp was added in [JDK-8301995](https://bugs.openjdk.org/browse/JDK-8301995) and is included in the popular cpCache.hpp. As a result, resolvedIndyEntry.hpp is included in 807 out of about 1160 hotspot .o files. The contents of resolvedIndyEntry.hpp is infrequently used. Its inclusion should be moved from cpCache.hpp to cpCache.inline.hpp. This improves hotspot build time. After this PR, resolvedIndyEntry.hpp is included in only 30 hotspot .o files. ------------- Commit messages: - fixed copyright - step2 - step1 Changes: https://git.openjdk.org/jdk/pull/14427/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14427&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309878 Stats: 86 lines in 35 files changed: 61 ins; 13 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/14427.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14427/head:pull/14427 PR: https://git.openjdk.org/jdk/pull/14427 From coleenp at openjdk.org Mon Jun 12 20:16:41 2023 From: coleenp at openjdk.org (Coleen Phillimore) Date: Mon, 12 Jun 2023 20:16:41 GMT Subject: RFR: 8309878: Reduce inclusion of resolvedIndyEntry.hpp In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 19:52:36 GMT, Ioi Lam wrote: > resolvedIndyEntry.hpp was added in [JDK-8301995](https://bugs.openjdk.org/browse/JDK-8301995) and is included in the popular cpCache.hpp. As a result, resolvedIndyEntry.hpp is included in 807 out of about 1160 hotspot .o files. > > The contents of resolvedIndyEntry.hpp is infrequently used. Its inclusion should be moved from cpCache.hpp to cpCache.inline.hpp. This improves hotspot build time. > > After this PR, resolvedIndyEntry.hpp is included in only 30 hotspot .o files. Marked as reviewed by coleenp (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14427#pullrequestreview-1475737898 From jlu at openjdk.org Mon Jun 12 21:28:30 2023 From: jlu at openjdk.org (Justin Lu) Date: Mon, 12 Jun 2023 21:28:30 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update Message-ID: Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) For example, image Please note that the HTML files only apply to .properties, and not .java resources. ------------- Commit messages: - Revert tool's un-translation of 'exception-type' for zh-CN - Move localized currencynames.properties to jdk.localedata - Apply trailing WS tool patch - Remove extra quotes in 'main.usage' values - Revert WinResources changing of 'resource.wxl-file-name' value - Revert MsiInstallerStrings culture value changes - Apply JDK21 Open translation patch Changes: https://git.openjdk.org/jdk21/pull/11/files Webrev: https://webrevs.openjdk.org/?repo=jdk21&pr=11&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309632 Stats: 462 lines in 51 files changed: 279 ins; 48 del; 135 mod Patch: https://git.openjdk.org/jdk21/pull/11.diff Fetch: git fetch https://git.openjdk.org/jdk21.git pull/11/head:pull/11 PR: https://git.openjdk.org/jdk21/pull/11 From jlu at openjdk.org Mon Jun 12 21:28:30 2023 From: jlu at openjdk.org (Justin Lu) Date: Mon, 12 Jun 2023 21:28:30 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update In-Reply-To: References: Message-ID: <-g_p5DXIJa2iFx-wYX9Fl024cy_wOxNlIN3g8TB5V1E=.84d95609-9341-4a50-afa2-e9dcb462a0bb@github.com> On Mon, 12 Jun 2023 21:21:05 GMT, Justin Lu wrote: > Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). > > To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) > > For example, > > image > > > Please note that the HTML files only apply to .properties, and not .java resources. @naotoj @wangweij @sormuras could you please review the changes that affect the languages you are familiar with when you have the chance. Please use the HTML files for this, as they will be in UTF-8 native and not unicode escaped sequences. ------------- PR Comment: https://git.openjdk.org/jdk21/pull/11#issuecomment-1588123540 From jlu at openjdk.org Mon Jun 12 21:29:54 2023 From: jlu at openjdk.org (Justin Lu) Date: Mon, 12 Jun 2023 21:29:54 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 21:21:05 GMT, Justin Lu wrote: > Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). > > To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) > > For example, > > image > > > Please note that the HTML files only apply to .properties, and not .java resources. @prrace @jonathan-gibbons @pavelrappo @lahodaj @wangweij @igraves @AlanBateman @eirbjo could you please review the modules / changes you are familiar with, thank you. ------------- PR Comment: https://git.openjdk.org/jdk21/pull/11#issuecomment-1588130775 From kcr at openjdk.org Mon Jun 12 21:45:55 2023 From: kcr at openjdk.org (Kevin Rushforth) Date: Mon, 12 Jun 2023 21:45:55 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 21:27:05 GMT, Justin Lu wrote: >> Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). >> >> To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) >> >> For example, >> >> image >> >> >> Please note that the HTML files only apply to .properties, and not .java resources. > > @prrace @jonathan-gibbons @pavelrappo @lahodaj @wangweij @igraves @AlanBateman @eirbjo could you please review the modules / changes you are familiar with, thank you. @justin-curtis-lu Unless there is a strong reason to do otherwise, Please submit a PR against jdk mainline first. Then, once that is integrated, you can proceed with this PR as a backport to JDK 21. See: https://mail.openjdk.org/pipermail/jdk-dev/2023-June/007894.html ------------- PR Comment: https://git.openjdk.org/jdk21/pull/11#issuecomment-1588146416 From jlu at openjdk.org Mon Jun 12 21:55:48 2023 From: jlu at openjdk.org (Justin Lu) Date: Mon, 12 Jun 2023 21:55:48 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 21:42:24 GMT, Kevin Rushforth wrote: >> @prrace @jonathan-gibbons @pavelrappo @lahodaj @wangweij @igraves @AlanBateman @eirbjo could you please review the modules / changes you are familiar with, thank you. > > @justin-curtis-lu Unless there is a strong reason to do otherwise, Please submit a PR against jdk mainline first. Then, once that is integrated, you can proceed with this PR as a backport to JDK 21. > > See: https://mail.openjdk.org/pipermail/jdk-dev/2023-June/007894.html @kevinrushforth Thanks for the reminder, will make the change in ML first. ------------- PR Comment: https://git.openjdk.org/jdk21/pull/11#issuecomment-1588156115 From jlu at openjdk.org Mon Jun 12 22:07:11 2023 From: jlu at openjdk.org (Justin Lu) Date: Mon, 12 Jun 2023 22:07:11 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update Message-ID: Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) For example, image Please note that the HTML files only apply to .properties, and not .java resources. ------------- Commit messages: - Merge remote-tracking branch 'justinJDK21/JDK21-RDP1-L10nDrop' into JDK21-L10NDrop-RDP1 - Revert tool's un-translation of 'exception-type' for zh-CN - Move localized currencynames.properties to jdk.localedata - Apply trailing WS tool patch - Remove extra quotes in 'main.usage' values - Revert WinResources changing of 'resource.wxl-file-name' value - Revert MsiInstallerStrings culture value changes - Apply JDK21 Open translation patch Changes: https://git.openjdk.org/jdk/pull/14430/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14430&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8309632 Stats: 462 lines in 51 files changed: 279 ins; 48 del; 135 mod Patch: https://git.openjdk.org/jdk/pull/14430.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14430/head:pull/14430 PR: https://git.openjdk.org/jdk/pull/14430 From jlu at openjdk.org Mon Jun 12 22:16:50 2023 From: jlu at openjdk.org (Justin Lu) Date: Mon, 12 Jun 2023 22:16:50 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 22:00:01 GMT, Justin Lu wrote: > Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). > > To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) > > For example, > > image > > > Please note that the HTML files only apply to .properties, and not .java resources. @naotoj @wangweij @sormuras could you please review the changes that affect the languages you are familiar with when you have the chance. Please use the HTML files for this, as they will be in UTF-8 native and not unicode escaped sequences. @prrace @jonathan-gibbons @pavelrappo @lahodaj @wangweij @igraves @AlanBateman @eirbjo could you please review the modules / changes you are familiar with, thank you. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14430#issuecomment-1588180092 From duke at openjdk.org Mon Jun 12 22:20:53 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Mon, 12 Jun 2023 22:20:53 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 22:00:01 GMT, Justin Lu wrote: > Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). > > To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) > > For example, > > image > > > Please note that the HTML files only apply to .properties, and not .java resources. src/jdk.jartool/share/classes/sun/tools/jar/resources/jar_de.properties line 115: > 113: main.help.opt.main.validate=\ --validate Validiert den Inhalt des JAR-Archivs. Diese Option\n validiert, dass die von einem Multi-Release-JAR-Archiv\n exportierte API \u00FCber die verschiedenen Releaseversionen\n hinweg konsistent ist. > 114: main.help.opt.any=\ In jedem Modus g\u00FCltige Vorgangsmodifikatoren:\n\n -C DIR Zum angegebenen Verzeichnis wechseln und die folgende\n Datei aufnehmen > 115: main.help.opt.any.file=\ -f, --file=FILE Der Name der Archivdatei. Wenn Sie dies lauslassen, wird entweder stdin oder\n stdout verwendet, je nach Vorgang\n --release VERSION Speichert alle der folgenden Dateien in einem versionierten Verzeichnis\n der JAR-Datei (d.h. META-INF/versions/VERSION/) This change looks suspicious to me. The English word used is ?omitted?. ?Auslassen? is to ?Leave out?, while ?lauslassen? is to ?let go?. A native German speaker should confirm, I have only high school level German.. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1227296089 From sspitsyn at openjdk.org Mon Jun 12 22:27:42 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 12 Jun 2023 22:27:42 GMT Subject: RFR: 8309878: Reduce inclusion of resolvedIndyEntry.hpp In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 19:52:36 GMT, Ioi Lam wrote: > resolvedIndyEntry.hpp was added in [JDK-8301995](https://bugs.openjdk.org/browse/JDK-8301995) and is included in the popular cpCache.hpp. As a result, resolvedIndyEntry.hpp is included in 807 out of about 1160 hotspot .o files. > > The contents of resolvedIndyEntry.hpp is infrequently used. Its inclusion should be moved from cpCache.hpp to cpCache.inline.hpp. This improves hotspot build time. > > After this PR, resolvedIndyEntry.hpp is included in only 30 hotspot .o files. Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14427#pullrequestreview-1475909850 From jlu at openjdk.org Mon Jun 12 22:32:14 2023 From: jlu at openjdk.org (Justin Lu) Date: Mon, 12 Jun 2023 22:32:14 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: References: Message-ID: > Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). > > To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) > > For example, > > image > > > Please note that the HTML files only apply to .properties, and not .java resources. Justin Lu has updated the pull request incrementally with one additional commit since the last revision: Revert 'main.help.opt.any.file' value for _de ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14430/files - new: https://git.openjdk.org/jdk/pull/14430/files/ee072231..8ee7bf88 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14430&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14430&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14430.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14430/head:pull/14430 PR: https://git.openjdk.org/jdk/pull/14430 From jlu at openjdk.org Mon Jun 12 22:32:34 2023 From: jlu at openjdk.org (Justin Lu) Date: Mon, 12 Jun 2023 22:32:34 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 22:17:25 GMT, Eirik Bjorsnos wrote: >> Justin Lu has updated the pull request incrementally with one additional commit since the last revision: >> >> Revert 'main.help.opt.any.file' value for _de > > src/jdk.jartool/share/classes/sun/tools/jar/resources/jar_de.properties line 115: > >> 113: main.help.opt.main.validate=\ --validate Validiert den Inhalt des JAR-Archivs. Diese Option\n validiert, dass die von einem Multi-Release-JAR-Archiv\n exportierte API \u00FCber die verschiedenen Releaseversionen\n hinweg konsistent ist. >> 114: main.help.opt.any=\ In jedem Modus g\u00FCltige Vorgangsmodifikatoren:\n\n -C DIR Zum angegebenen Verzeichnis wechseln und die folgende\n Datei aufnehmen >> 115: main.help.opt.any.file=\ -f, --file=FILE Der Name der Archivdatei. Wenn Sie dies lauslassen, wird entweder stdin oder\n stdout verwendet, je nach Vorgang\n --release VERSION Speichert alle der folgenden Dateien in einem versionierten Verzeichnis\n der JAR-Datei (d.h. META-INF/versions/VERSION/) > > This change looks suspicious to me. The English word used is ?omitted?. ?Auslassen? is to ?Leave out?, while ?lauslassen? is to ?let go?. A native German speaker should confirm, I have only high school level German.. Thanks for catching that. The translation tool incorrectly translated this, and we had manually fixed it with `Auslassen` in the last translation drop. This was an instance of the tool simply overwriting it this time. Will file a bug against the translation tool regarding this, to prevent it in future translation drops. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1227305807 From sspitsyn at openjdk.org Mon Jun 12 22:34:50 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 12 Jun 2023 22:34:50 GMT Subject: RFR: 8309757: com/sun/jdi/ReferrersTest.java fails with virtual test thread factory In-Reply-To: References: Message-ID: <51cH2Cvc9gqoLYcEu3kOQVuea2NquATWxdbBl1TZl4A=.2888b97f-24c6-49b3-b405-5c3a9871066b@github.com> On Fri, 9 Jun 2023 23:58:39 GMT, Chris Plummer wrote: > This test launches a debuggee, which creates 11 instances of its main class, stores them in a static array of the main class, and then the debugger side iterates over all referrers to the main class instaces. Usually this is a pretty quick process and doesn't produce much in the way of output while walking the reference tree of referrers. However, with virtual threads the tree walking and output get unwieldy, and eventually it fails with: > > > IOException reading output of child java interpreter:Stream closed > java.lang.IllegalThreadStateException > at jdk.jdi/com.sun.tools.jdi.JDWPException.toJDIException(JDWPException.java:76) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.name(ThreadReferenceImpl.java:197) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.toString(ThreadReferenceImpl.java:637) > at java.base/java.lang.String.valueOf(String.java:4461) > at ReferrersTest.showReferrers(ReferrersTest.java:438) > at ReferrersTest.showReferrers(ReferrersTest.java:466) > > > And ReferrersTest.showReferrers() has recursed about 200 levels deep. I'm not sure the order of these errors can be relied on. It looks like while walking the referrers tree, the test eventually stumbled upon a thread that had exited (but its Thread object was still alive), and this resulted in the test aborting. If I catch these exceptions, eventually the test times out while still working on referrers. > > Judging by some of the output, it appears that introducing the TestScaffold class as a referrer to the main debuggee class is the root cause of all these extra referrers. > > The test has a provision to cut off the recursion: > > > // We have to stop going up a referrer chain in some cases > Type rt = objRef.type(); > if (rt instanceof ClassType) { > ClassType ct = (ClassType)rt; > String name = ct.name(); > if (name.equals("sun.awt.SoftCache$ValueCell")) { > return; > } > if (name.equals("java.lang.ref.Finalizer")) { > return; > } > if (name.equals("java.lang.ref.SoftReference")) { > return; > } > // oh oh, should really check for a subclass of ClassLoader :-) > if (name.indexOf("ClassLoader") >= 0) { > return; > } > // No doubt there are other reasons to stop ... > } > > > Adding TestScaffold to the list makes it so the referrer tree walking output is almost identical to what it is when not using virtual threads. ... Looks good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14405#pullrequestreview-1475915772 From dholmes at openjdk.org Tue Jun 13 02:22:40 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 13 Jun 2023 02:22:40 GMT Subject: RFR: 8309468: Remove jvmti Allocate locker test case In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 16:39:12 GMT, Leo Korinth wrote: > There is a bunch of tests that are used to test critical section/gc locker. One of the test is named jvmti locker. In that test, JNI code is doing a loop of ` jvmti->Allocate()` followed `sleep()` followed by a `jvmti->Deallocate()`. There is no JVM lock implementation to be tested on jvmti Allocate/Deallocate, they are implemented using malloc/free. Let us save test time, code complexity and confusion by removing this test. > > This removal is very similar to https://bugs.openjdk.org/browse/JDK-8309048 > > (Oracle) hs-tier5 testing passed on x86-64. Serviceability folk should probably look at this and @lmesnik. I can't see what this is actually trying to test either, so removing it seems fine to me too. Thanks. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14421#pullrequestreview-1476125547 From lmesnik at openjdk.org Tue Jun 13 04:16:40 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Tue, 13 Jun 2023 04:16:40 GMT Subject: RFR: 8309468: Remove jvmti Allocate locker test case In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 16:39:12 GMT, Leo Korinth wrote: > There is a bunch of tests that are used to test critical section/gc locker. One of the test is named jvmti locker. In that test, JNI code is doing a loop of ` jvmti->Allocate()` followed `sleep()` followed by a `jvmti->Deallocate()`. There is no JVM lock implementation to be tested on jvmti Allocate/Deallocate, they are implemented using malloc/free. Let us save test time, code complexity and confusion by removing this test. > > This removal is very similar to https://bugs.openjdk.org/browse/JDK-8309048 > > (Oracle) hs-tier5 testing passed on x86-64. Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14421#pullrequestreview-1476207969 From sspitsyn at openjdk.org Tue Jun 13 05:41:48 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 13 Jun 2023 05:41:48 GMT Subject: RFR: 8309468: Remove jvmti Allocate locker test case In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 16:39:12 GMT, Leo Korinth wrote: > There is a bunch of tests that are used to test critical section/gc locker. One of the test is named jvmti locker. In that test, JNI code is doing a loop of ` jvmti->Allocate()` followed `sleep()` followed by a `jvmti->Deallocate()`. There is no JVM lock implementation to be tested on jvmti Allocate/Deallocate, they are implemented using malloc/free. Let us save test time, code complexity and confusion by removing this test. > > This removal is very similar to https://bugs.openjdk.org/browse/JDK-8309048 > > (Oracle) hs-tier5 testing passed on x86-64. I also do not see any value in this test, so it looks good to get rid of it. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14421#pullrequestreview-1476285877 From mdoerr at openjdk.org Tue Jun 13 08:30:58 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 13 Jun 2023 08:30:58 GMT Subject: Integrated: 8309462: [AIX] vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 14:36:56 GMT, Martin Doerr wrote: > Test fix for test failing on AIX because of undefined behavior in current implementation. This pull request has now been integrated. Changeset: 08eff92b Author: Martin Doerr URL: https://git.openjdk.org/jdk21/commit/08eff92b5e1e42cf299c1da8d5fa88d92b840505 Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod 8309462: [AIX] vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/TestDescription.java crashing due to empty while loop Reviewed-by: mbaesken Backport-of: cf9e6353cc6fe9e57a7a9883813d09892e7e7621 ------------- PR: https://git.openjdk.org/jdk21/pull/9 From yzheng at openjdk.org Tue Jun 13 08:34:49 2023 From: yzheng at openjdk.org (Yudi Zheng) Date: Tue, 13 Jun 2023 08:34:49 GMT Subject: RFR: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled In-Reply-To: References: Message-ID: On Thu, 8 Jun 2023 17:14:39 GMT, Yudi Zheng wrote: > HeapMonitor checks if System.getProperty("jvmci.Compiler") is graal and will not enforce checking line number derived from uncommon trap debug info. However, Graal does not set this property explicitly. Thanks for the review! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14381#issuecomment-1588815689 From yzheng at openjdk.org Tue Jun 13 09:06:00 2023 From: yzheng at openjdk.org (Yudi Zheng) Date: Tue, 13 Jun 2023 09:06:00 GMT Subject: Integrated: 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled In-Reply-To: References: Message-ID: On Thu, 8 Jun 2023 17:14:39 GMT, Yudi Zheng wrote: > HeapMonitor checks if System.getProperty("jvmci.Compiler") is graal and will not enforce checking line number derived from uncommon trap debug info. However, Graal does not set this property explicitly. This pull request has now been integrated. Changeset: 4f23fc1f Author: Yudi Zheng Committer: Doug Simon URL: https://git.openjdk.org/jdk/commit/4f23fc1f273ea30f49c5412a2f25c07f8982d5b5 Stats: 3 lines in 2 files changed: 0 ins; 2 del; 1 mod 8309671: Avoid using jvmci.Compiler property to determine if Graal is enabled Reviewed-by: dnsimon, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/14381 From kevinw at openjdk.org Tue Jun 13 11:19:49 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Tue, 13 Jun 2023 11:19:49 GMT Subject: RFR: 8309757: com/sun/jdi/ReferrersTest.java fails with virtual test thread factory In-Reply-To: References: Message-ID: On Fri, 9 Jun 2023 23:58:39 GMT, Chris Plummer wrote: > This test launches a debuggee, which creates 11 instances of its main class, stores them in a static array of the main class, and then the debugger side iterates over all referrers to the main class instaces. Usually this is a pretty quick process and doesn't produce much in the way of output while walking the reference tree of referrers. However, with virtual threads the tree walking and output get unwieldy, and eventually it fails with: > > > IOException reading output of child java interpreter:Stream closed > java.lang.IllegalThreadStateException > at jdk.jdi/com.sun.tools.jdi.JDWPException.toJDIException(JDWPException.java:76) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.name(ThreadReferenceImpl.java:197) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.toString(ThreadReferenceImpl.java:637) > at java.base/java.lang.String.valueOf(String.java:4461) > at ReferrersTest.showReferrers(ReferrersTest.java:438) > at ReferrersTest.showReferrers(ReferrersTest.java:466) > > > And ReferrersTest.showReferrers() has recursed about 200 levels deep. I'm not sure the order of these errors can be relied on. It looks like while walking the referrers tree, the test eventually stumbled upon a thread that had exited (but its Thread object was still alive), and this resulted in the test aborting. If I catch these exceptions, eventually the test times out while still working on referrers. > > Judging by some of the output, it appears that introducing the TestScaffold class as a referrer to the main debuggee class is the root cause of all these extra referrers. > > The test has a provision to cut off the recursion: > > > // We have to stop going up a referrer chain in some cases > Type rt = objRef.type(); > if (rt instanceof ClassType) { > ClassType ct = (ClassType)rt; > String name = ct.name(); > if (name.equals("sun.awt.SoftCache$ValueCell")) { > return; > } > if (name.equals("java.lang.ref.Finalizer")) { > return; > } > if (name.equals("java.lang.ref.SoftReference")) { > return; > } > // oh oh, should really check for a subclass of ClassLoader :-) > if (name.indexOf("ClassLoader") >= 0) { > return; > } > // No doubt there are other reasons to stop ... > } > > > Adding TestScaffold to the list makes it so the referrer tree walking output is almost identical to what it is when not using virtual threads. ... Looks good. This test test/jdk/com/sun/jdi/ReferrersTest.java, and also AnyDebuggeeTest.java and InstancesTest.java have comments referring to running them with "runregresS". Maybe this is some old test harness or launcher that no longer exists, probably wants to be deleted or reworded, in a separate update. ------------- Marked as reviewed by kevinw (Committer). PR Review: https://git.openjdk.org/jdk/pull/14405#pullrequestreview-1476899872 PR Comment: https://git.openjdk.org/jdk/pull/14405#issuecomment-1589093501 From lkorinth at openjdk.org Tue Jun 13 12:34:59 2023 From: lkorinth at openjdk.org (Leo Korinth) Date: Tue, 13 Jun 2023 12:34:59 GMT Subject: RFR: 8309468: Remove jvmti Allocate locker test case In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 16:39:12 GMT, Leo Korinth wrote: > There is a bunch of tests that are used to test critical section/gc locker. One of the test is named jvmti locker. In that test, JNI code is doing a loop of ` jvmti->Allocate()` followed `sleep()` followed by a `jvmti->Deallocate()`. There is no JVM lock implementation to be tested on jvmti Allocate/Deallocate, they are implemented using malloc/free. Let us save test time, code complexity and confusion by removing this test. > > This removal is very similar to https://bugs.openjdk.org/browse/JDK-8309048 > > (Oracle) hs-tier5 testing passed on x86-64. Thanks David, Leonid and Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14421#issuecomment-1589211147 From lkorinth at openjdk.org Tue Jun 13 12:35:00 2023 From: lkorinth at openjdk.org (Leo Korinth) Date: Tue, 13 Jun 2023 12:35:00 GMT Subject: Integrated: 8309468: Remove jvmti Allocate locker test case In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 16:39:12 GMT, Leo Korinth wrote: > There is a bunch of tests that are used to test critical section/gc locker. One of the test is named jvmti locker. In that test, JNI code is doing a loop of ` jvmti->Allocate()` followed `sleep()` followed by a `jvmti->Deallocate()`. There is no JVM lock implementation to be tested on jvmti Allocate/Deallocate, they are implemented using malloc/free. Let us save test time, code complexity and confusion by removing this test. > > This removal is very similar to https://bugs.openjdk.org/browse/JDK-8309048 > > (Oracle) hs-tier5 testing passed on x86-64. This pull request has now been integrated. Changeset: c884862a Author: Leo Korinth URL: https://git.openjdk.org/jdk/commit/c884862ad2189654596df27a76ab685dcd7399f6 Stats: 260 lines in 9 files changed: 0 ins; 257 del; 3 mod 8309468: Remove jvmti Allocate locker test case Reviewed-by: dholmes, lmesnik, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/14421 From matsaave at openjdk.org Tue Jun 13 15:13:42 2023 From: matsaave at openjdk.org (Matias Saavedra Silva) Date: Tue, 13 Jun 2023 15:13:42 GMT Subject: RFR: 8309878: Reduce inclusion of resolvedIndyEntry.hpp In-Reply-To: References: Message-ID: <30EVi0eDcl2WRGeDGLrlYWEckKuzAnwH2o9by8w3ihg=.1c26c41a-45f7-4065-b365-96fae46b3c51@github.com> On Mon, 12 Jun 2023 19:52:36 GMT, Ioi Lam wrote: > resolvedIndyEntry.hpp was added in [JDK-8301995](https://bugs.openjdk.org/browse/JDK-8301995) and is included in the popular cpCache.hpp. As a result, resolvedIndyEntry.hpp is included in 807 out of about 1160 hotspot .o files. > > The contents of resolvedIndyEntry.hpp is infrequently used. Its inclusion should be moved from cpCache.hpp to cpCache.inline.hpp. This improves hotspot build time. > > After this PR, resolvedIndyEntry.hpp is included in only 30 hotspot .o files. Looks good, thanks for this! ------------- Marked as reviewed by matsaave (Committer). PR Review: https://git.openjdk.org/jdk/pull/14427#pullrequestreview-1477437064 From cjplummer at openjdk.org Tue Jun 13 18:07:04 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 13 Jun 2023 18:07:04 GMT Subject: RFR: 8309757: com/sun/jdi/ReferrersTest.java fails with virtual test thread factory [v2] In-Reply-To: References: Message-ID: <_hpiQEe-SzfV7Kp1W2KGaHC4AJI632gD8hkpMOtDjzw=.2eab8b3f-4288-47d8-8acf-c99a509dc136@github.com> > This test launches a debuggee, which creates 11 instances of its main class, stores them in a static array of the main class, and then the debugger side iterates over all referrers to the main class instaces. Usually this is a pretty quick process and doesn't produce much in the way of output while walking the reference tree of referrers. However, with virtual threads the tree walking and output get unwieldy, and eventually it fails with: > > > IOException reading output of child java interpreter:Stream closed > java.lang.IllegalThreadStateException > at jdk.jdi/com.sun.tools.jdi.JDWPException.toJDIException(JDWPException.java:76) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.name(ThreadReferenceImpl.java:197) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.toString(ThreadReferenceImpl.java:637) > at java.base/java.lang.String.valueOf(String.java:4461) > at ReferrersTest.showReferrers(ReferrersTest.java:438) > at ReferrersTest.showReferrers(ReferrersTest.java:466) > > > And ReferrersTest.showReferrers() has recursed about 200 levels deep. I'm not sure the order of these errors can be relied on. It looks like while walking the referrers tree, the test eventually stumbled upon a thread that had exited (but its Thread object was still alive), and this resulted in the test aborting. If I catch these exceptions, eventually the test times out while still working on referrers. > > Judging by some of the output, it appears that introducing the TestScaffold class as a referrer to the main debuggee class is the root cause of all these extra referrers. > > The test has a provision to cut off the recursion: > > > // We have to stop going up a referrer chain in some cases > Type rt = objRef.type(); > if (rt instanceof ClassType) { > ClassType ct = (ClassType)rt; > String name = ct.name(); > if (name.equals("sun.awt.SoftCache$ValueCell")) { > return; > } > if (name.equals("java.lang.ref.Finalizer")) { > return; > } > if (name.equals("java.lang.ref.SoftReference")) { > return; > } > // oh oh, should really check for a subclass of ClassLoader :-) > if (name.indexOf("ClassLoader") >= 0) { > return; > } > // No doubt there are other reasons to stop ... > } > > > Adding TestScaffold to the list makes it so the referrer tree walking output is almost identical to what it is when not using virtual threads. ... Chris Plummer has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - Merge - Do some additional pruning needed when run with a virtual thread ------------- Changes: https://git.openjdk.org/jdk/pull/14405/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14405&range=01 Stats: 5 lines in 2 files changed: 3 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/14405.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14405/head:pull/14405 PR: https://git.openjdk.org/jdk/pull/14405 From cjplummer at openjdk.org Tue Jun 13 18:10:59 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 13 Jun 2023 18:10:59 GMT Subject: RFR: 8309757: com/sun/jdi/ReferrersTest.java fails with virtual test thread factory In-Reply-To: References: Message-ID: <69LTJQywkD14tGUrwKrBvlgdmA7Q064RIJsl8aQ0AXM=.33ec5f6c-403d-4135-b877-915d24e45974@github.com> On Fri, 9 Jun 2023 23:58:39 GMT, Chris Plummer wrote: > This test launches a debuggee, which creates 11 instances of its main class, stores them in a static array of the main class, and then the debugger side iterates over all referrers to the main class instaces. Usually this is a pretty quick process and doesn't produce much in the way of output while walking the reference tree of referrers. However, with virtual threads the tree walking and output get unwieldy, and eventually it fails with: > > > IOException reading output of child java interpreter:Stream closed > java.lang.IllegalThreadStateException > at jdk.jdi/com.sun.tools.jdi.JDWPException.toJDIException(JDWPException.java:76) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.name(ThreadReferenceImpl.java:197) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.toString(ThreadReferenceImpl.java:637) > at java.base/java.lang.String.valueOf(String.java:4461) > at ReferrersTest.showReferrers(ReferrersTest.java:438) > at ReferrersTest.showReferrers(ReferrersTest.java:466) > > > And ReferrersTest.showReferrers() has recursed about 200 levels deep. I'm not sure the order of these errors can be relied on. It looks like while walking the referrers tree, the test eventually stumbled upon a thread that had exited (but its Thread object was still alive), and this resulted in the test aborting. If I catch these exceptions, eventually the test times out while still working on referrers. > > Judging by some of the output, it appears that introducing the TestScaffold class as a referrer to the main debuggee class is the root cause of all these extra referrers. > > The test has a provision to cut off the recursion: > > > // We have to stop going up a referrer chain in some cases > Type rt = objRef.type(); > if (rt instanceof ClassType) { > ClassType ct = (ClassType)rt; > String name = ct.name(); > if (name.equals("sun.awt.SoftCache$ValueCell")) { > return; > } > if (name.equals("java.lang.ref.Finalizer")) { > return; > } > if (name.equals("java.lang.ref.SoftReference")) { > return; > } > // oh oh, should really check for a subclass of ClassLoader :-) > if (name.indexOf("ClassLoader") >= 0) { > return; > } > // No doubt there are other reasons to stop ... > } > > > Adding TestScaffold to the list makes it so the referrer tree walking output is almost identical to what it is when not using virtual threads. ... Thanks for the reviews Serguei and Kevin ------------- PR Comment: https://git.openjdk.org/jdk/pull/14405#issuecomment-1589795544 From cjplummer at openjdk.org Tue Jun 13 18:11:00 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 13 Jun 2023 18:11:00 GMT Subject: Integrated: 8309757: com/sun/jdi/ReferrersTest.java fails with virtual test thread factory In-Reply-To: References: Message-ID: <7Jsm60VigvH37nrIKYaByOB9C3xuuCoKPeMV1xAsxyU=.590453b6-b9c7-4661-a380-92395ddc7903@github.com> On Fri, 9 Jun 2023 23:58:39 GMT, Chris Plummer wrote: > This test launches a debuggee, which creates 11 instances of its main class, stores them in a static array of the main class, and then the debugger side iterates over all referrers to the main class instaces. Usually this is a pretty quick process and doesn't produce much in the way of output while walking the reference tree of referrers. However, with virtual threads the tree walking and output get unwieldy, and eventually it fails with: > > > IOException reading output of child java interpreter:Stream closed > java.lang.IllegalThreadStateException > at jdk.jdi/com.sun.tools.jdi.JDWPException.toJDIException(JDWPException.java:76) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.name(ThreadReferenceImpl.java:197) > at jdk.jdi/com.sun.tools.jdi.ThreadReferenceImpl.toString(ThreadReferenceImpl.java:637) > at java.base/java.lang.String.valueOf(String.java:4461) > at ReferrersTest.showReferrers(ReferrersTest.java:438) > at ReferrersTest.showReferrers(ReferrersTest.java:466) > > > And ReferrersTest.showReferrers() has recursed about 200 levels deep. I'm not sure the order of these errors can be relied on. It looks like while walking the referrers tree, the test eventually stumbled upon a thread that had exited (but its Thread object was still alive), and this resulted in the test aborting. If I catch these exceptions, eventually the test times out while still working on referrers. > > Judging by some of the output, it appears that introducing the TestScaffold class as a referrer to the main debuggee class is the root cause of all these extra referrers. > > The test has a provision to cut off the recursion: > > > // We have to stop going up a referrer chain in some cases > Type rt = objRef.type(); > if (rt instanceof ClassType) { > ClassType ct = (ClassType)rt; > String name = ct.name(); > if (name.equals("sun.awt.SoftCache$ValueCell")) { > return; > } > if (name.equals("java.lang.ref.Finalizer")) { > return; > } > if (name.equals("java.lang.ref.SoftReference")) { > return; > } > // oh oh, should really check for a subclass of ClassLoader :-) > if (name.indexOf("ClassLoader") >= 0) { > return; > } > // No doubt there are other reasons to stop ... > } > > > Adding TestScaffold to the list makes it so the referrer tree walking output is almost identical to what it is when not using virtual threads. ... This pull request has now been integrated. Changeset: d7251c17 Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/d7251c1755779f8c4fdeac16ccb797ed45b1dfd4 Stats: 5 lines in 2 files changed: 3 ins; 1 del; 1 mod 8309757: com/sun/jdi/ReferrersTest.java fails with virtual test thread factory Reviewed-by: sspitsyn, kevinw ------------- PR: https://git.openjdk.org/jdk/pull/14405 From naoto at openjdk.org Tue Jun 13 18:41:52 2023 From: naoto at openjdk.org (Naoto Sato) Date: Tue, 13 Jun 2023 18:41:52 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 22:32:14 GMT, Justin Lu wrote: >> Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). >> >> To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) >> >> For example, >> >> image >> >> >> Please note that the HTML files only apply to .properties, and not .java resources. > > Justin Lu has updated the pull request incrementally with one additional commit since the last revision: > > Revert 'main.help.opt.any.file' value for _de Left some comments on the translations mainly in Japanese. It is now very easy to look at the l10n changes in the generated HTML. One small comment to the tool is that it would be nice if the order in HTML (alphabetically sorted currently) aligns with the order in the actual resource file. It is a great improvement anyways. src/java.desktop/macosx/classes/com/apple/laf/resources/aqua_ja.properties line 63: > 61: FileChooser.newFolderExistsError.textAndMnemonic=\u305D\u306E\u540D\u524D\u306F\u3059\u3067\u306B\u4F7F\u7528\u3055\u308C\u3066\u3044\u307E\u3059 > 62: FileChooser.chooseButton.textAndMnemonic=\u9078\u629E > 63: FileChooser.chooseButtonToolTip.textAndMnemonic=\u9078\u629E\u3057\u305F\u30D5\u30A1\u30A4\u30EB\u3092\u8FFD\u52A0\u3057\u307E\u3059 The translation seems wrong. It means to *add* the selected file, which is not the same meaning as in English file src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_de.properties line 140: > 138: > 139: # 0: type, 1: name > 140: compiler.err.annotation.unrecognized.attribute.name=Annotation @{0} weist das unbekannte Attribut "{1}" auf Is it intentional to replace single quotes with double quotes? It is done in several locations in this file, but other languages keep using single quotes as in the source English resource file. src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties line 505: > 503: > 504: compiler.warn.possible.this.escape.location=\u4E8B\u524D\u306B\u6307\u5B9A\u53EF\u80FD\u306A''this''\u30A8\u30B9\u30B1\u30FC\u30D7\u306F\u547C\u51FA\u3057\u3092\u4ECB\u3057\u3066\u3053\u3053\u306B\u51FA\u73FE\u3057\u307E\u3059 > 505: These translations are very cryptic. Cannot understand it at first hand. src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties line 959: > 957: > 958: compiler.err.text.block.template.is.not.well.formed=\u30C6\u30AD\u30B9\u30C8\u30FB\u30D6\u30ED\u30C3\u30AF\u30FB\u30C6\u30F3\u30D7\u30EC\u30FC\u30C8\u304C\u6574\u5F62\u5F0F\u3067\u306F\u3042\u308A\u307E\u305B\u3093 > 959: The translation of "well-formed" is incorrect. It is not ??? ------------- PR Review: https://git.openjdk.org/jdk/pull/14430#pullrequestreview-1477795175 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228530280 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228521409 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228538758 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228529769 From jlu at openjdk.org Tue Jun 13 18:56:44 2023 From: jlu at openjdk.org (Justin Lu) Date: Tue, 13 Jun 2023 18:56:44 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: References: Message-ID: <05VweeA-JZy6fvHzscLEDLi4nPZM0xhDiaOqYcceMgI=.2574b0d1-92ea-4319-9c45-dbd9df91ad4f@github.com> On Tue, 13 Jun 2023 18:13:12 GMT, Naoto Sato wrote: >> Justin Lu has updated the pull request incrementally with one additional commit since the last revision: >> >> Revert 'main.help.opt.any.file' value for _de > > src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_de.properties line 140: > >> 138: >> 139: # 0: type, 1: name >> 140: compiler.err.annotation.unrecognized.attribute.name=Annotation @{0} weist das unbekannte Attribut "{1}" auf > > Is it intentional to replace single quotes with double quotes? It is done in several locations in this file, but other languages keep using single quotes as in the source English resource file. I will consult the language specialist on this. I remember in previous discussions that certain languages would format punctuation differently, but let ask about this specific instance. > src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties line 959: > >> 957: >> 958: compiler.err.text.block.template.is.not.well.formed=\u30C6\u30AD\u30B9\u30C8\u30FB\u30D6\u30ED\u30C3\u30AF\u30FB\u30C6\u30F3\u30D7\u30EC\u30FC\u30C8\u304C\u6574\u5F62\u5F0F\u3067\u306F\u3042\u308A\u307E\u305B\u3093 >> 959: > > The translation of "well-formed" is incorrect. It is not ??? For translations that are incorrect, I can file a bug against the tool. Such a fix would probably not happen soon, can you suggest an alternative for now? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228567669 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228570665 From naoto at openjdk.org Tue Jun 13 20:24:46 2023 From: naoto at openjdk.org (Naoto Sato) Date: Tue, 13 Jun 2023 20:24:46 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 22:32:14 GMT, Justin Lu wrote: >> Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). >> >> To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) >> >> For example, >> >> image >> >> >> Please note that the HTML files only apply to .properties, and not .java resources. > > Justin Lu has updated the pull request incrementally with one additional commit since the last revision: > > Revert 'main.help.opt.any.file' value for _de Left some suggestions. Should be looked at by translators later. src/java.desktop/macosx/classes/com/apple/laf/resources/aqua_ja.properties line 63: > 61: FileChooser.newFolderExistsError.textAndMnemonic=\u305D\u306E\u540D\u524D\u306F\u3059\u3067\u306B\u4F7F\u7528\u3055\u308C\u3066\u3044\u307E\u3059 > 62: FileChooser.chooseButton.textAndMnemonic=\u9078\u629E > 63: FileChooser.chooseButtonToolTip.textAndMnemonic=\u9078\u629E\u3057\u305F\u30D5\u30A1\u30A4\u30EB\u3092\u8FFD\u52A0\u3057\u307E\u3059 Suggestion: FileChooser.chooseButtonToolTip.textAndMnemonic=\u9078\u629e\u3055\u308c\u3066\u3044\u308b\u30d5\u30a1\u30a4\u30eb\u3092\u9078\u629e\u3057\u307e\u3059 src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties line 502: > 500: compiler.warn.forward.ref=\u521D\u671F\u5316\u3055\u308C\u308B\u524D\u306E\u5909\u6570''{0}''\u3092\u53C2\u7167\u3057\u3088\u3046\u3068\u3057\u307E\u3057\u305F > 501: > 502: compiler.warn.possible.this.escape=\u30B5\u30D6\u30AF\u30E9\u30B9\u524D\u306B\u6307\u5B9A\u53EF\u80FD\u306A''this''\u30A8\u30B9\u30B1\u30FC\u30D7\u306F\u5B8C\u5168\u306B\u521D\u671F\u5316\u3055\u308C\u307E\u3059 Suggestion: compiler.warn.possible.this.escape=\u30b5\u30d6\u30af\u30e9\u30b9\u304c\u521d\u671f\u5316\u3055\u308c\u308b\u524d\u306e''this''\u30a8\u30b9\u30b1\u30fc\u30d7\u306e\u53ef\u80fd\u6027\u304c\u3042\u308a\u307e\u3059 src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties line 504: > 502: compiler.warn.possible.this.escape=\u30B5\u30D6\u30AF\u30E9\u30B9\u524D\u306B\u6307\u5B9A\u53EF\u80FD\u306A''this''\u30A8\u30B9\u30B1\u30FC\u30D7\u306F\u5B8C\u5168\u306B\u521D\u671F\u5316\u3055\u308C\u307E\u3059 > 503: > 504: compiler.warn.possible.this.escape.location=\u4E8B\u524D\u306B\u6307\u5B9A\u53EF\u80FD\u306A''this''\u30A8\u30B9\u30B1\u30FC\u30D7\u306F\u547C\u51FA\u3057\u3092\u4ECB\u3057\u3066\u3053\u3053\u306B\u51FA\u73FE\u3057\u307E\u3059 Suggestion: compiler.warn.possible.this.escape.location=\u4e8b\u524d\u306e''this''\u30a8\u30b9\u30b1\u30fc\u30d7\u306f\u3053\u306e\u547c\u3073\u51fa\u3057\u3067\u51fa\u73fe\u3057\u307e\u3059 src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties line 956: > 954: compiler.err.unclosed.text.block=\u9589\u3058\u3089\u308C\u3066\u3044\u306A\u3044\u30C6\u30AD\u30B9\u30C8\u30FB\u30D6\u30ED\u30C3\u30AF > 955: > 956: compiler.err.string.template.is.not.well.formed=\u6587\u5B57\u5217\u30C6\u30F3\u30D7\u30EC\u30FC\u30C8\u304C\u6574\u5F62\u5F0F\u3067\u306F\u3042\u308A\u307E\u305B\u3093 Suggestion: compiler.err.string.template.is.not.well.formed=\u6587\u5b57\u5217\u30c6\u30f3\u30d7\u30ec\u30fc\u30c8\u304c\u9069\u683c\u3067\u306f\u3042\u308a\u307e\u305b\u3093 src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties line 958: > 956: compiler.err.string.template.is.not.well.formed=\u6587\u5B57\u5217\u30C6\u30F3\u30D7\u30EC\u30FC\u30C8\u304C\u6574\u5F62\u5F0F\u3067\u306F\u3042\u308A\u307E\u305B\u3093 > 957: > 958: compiler.err.text.block.template.is.not.well.formed=\u30C6\u30AD\u30B9\u30C8\u30FB\u30D6\u30ED\u30C3\u30AF\u30FB\u30C6\u30F3\u30D7\u30EC\u30FC\u30C8\u304C\u6574\u5F62\u5F0F\u3067\u306F\u3042\u308A\u307E\u305B\u3093 Suggestion: compiler.err.text.block.template.is.not.well.formed=\u30c6\u30ad\u30b9\u30c8\u30fb\u30d6\u30ed\u30c3\u30af\u30fb\u30c6\u30f3\u30d7\u30ec\u30fc\u30c8\u304c\u9069\u683c\u3067\u306f\u3042\u308a\u307e\u305b\u3093 ------------- Changes requested by naoto (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14430#pullrequestreview-1477963618 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228635514 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228648714 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228651272 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228653057 PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228654384 From weijun at openjdk.org Tue Jun 13 20:58:51 2023 From: weijun at openjdk.org (Weijun Wang) Date: Tue, 13 Jun 2023 20:58:51 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 22:32:14 GMT, Justin Lu wrote: >> Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). >> >> To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) >> >> For example, >> >> image >> >> >> Please note that the HTML files only apply to .properties, and not .java resources. > > Justin Lu has updated the pull request incrementally with one additional commit since the last revision: > > Revert 'main.help.opt.any.file' value for _de Some comments: - `src/java.base/share/classes/sun/launcher/resources/launcher` `? private`. "private" should be translated, just like the "static" in the 2nd box. - `src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler` `compiler.err.annotation.unrecognized.attribute.name`: Here "annotation" is translated to "??"? How do you translate "comment"? `compiler.err.enum.label.must.be.enum.constant`: "constant" should be translated. `compiler.warn.possible.this.escape`, `compiler.warn.possible.this.escape.location`: "escape" might not mean "??". It's more like its original meaning "run away" or "referenced". - `src/jdk.compiler/share/classes/com/sun/tools/javac/resources/javac`: `javac.opt.Xlint.desc.output-file-clash`: "native header files" is translated into "??????". The "??" is strange, I'd rather just say "???" as there is no non-native header file. - `src/jdk.jdeps/share/classes/com/sun/tools/javap/resources/javap` I don't think the new translation is better than the old one. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14430#issuecomment-1590013825 From jlu at openjdk.org Tue Jun 13 21:06:25 2023 From: jlu at openjdk.org (Justin Lu) Date: Tue, 13 Jun 2023 21:06:25 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v3] In-Reply-To: References: Message-ID: > Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). > > To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) > > For example, > > image > > > Please note that the HTML files only apply to .properties, and not .java resources. Justin Lu has updated the pull request incrementally with one additional commit since the last revision: Review: use Naoto's recommended ja values ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14430/files - new: https://git.openjdk.org/jdk/pull/14430/files/8ee7bf88..7801eefb Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14430&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14430&range=01-02 Stats: 5 lines in 2 files changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/14430.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14430/head:pull/14430 PR: https://git.openjdk.org/jdk/pull/14430 From jlu at openjdk.org Tue Jun 13 21:06:28 2023 From: jlu at openjdk.org (Justin Lu) Date: Tue, 13 Jun 2023 21:06:28 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: References: Message-ID: On Tue, 13 Jun 2023 19:59:28 GMT, Naoto Sato wrote: >> Justin Lu has updated the pull request incrementally with one additional commit since the last revision: >> >> Revert 'main.help.opt.any.file' value for _de > > src/java.desktop/macosx/classes/com/apple/laf/resources/aqua_ja.properties line 63: > >> 61: FileChooser.newFolderExistsError.textAndMnemonic=\u305D\u306E\u540D\u524D\u306F\u3059\u3067\u306B\u4F7F\u7528\u3055\u308C\u3066\u3044\u307E\u3059 >> 62: FileChooser.chooseButton.textAndMnemonic=\u9078\u629E >> 63: FileChooser.chooseButtonToolTip.textAndMnemonic=\u9078\u629E\u3057\u305F\u30D5\u30A1\u30A4\u30EB\u3092\u8FFD\u52A0\u3057\u307E\u3059 > > Suggestion: > > FileChooser.chooseButtonToolTip.textAndMnemonic=\u9078\u629e\u3055\u308c\u3066\u3044\u308b\u30d5\u30a1\u30a4\u30eb\u3092\u9078\u629e\u3057\u307e\u3059 Thank you Naoto, all your recommended translations have been updated. The HTML diffs have also been updated as well. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1228692234 From amenkov at openjdk.org Tue Jun 13 22:05:03 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 13 Jun 2023 22:05:03 GMT Subject: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v8] In-Reply-To: References: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> Message-ID: On Tue, 6 Jun 2023 02:35:30 GMT, Yi Yang wrote: >> ### Motivation and proposal >> Hi, heap dump brings about pauses for application's execution(STW), this is a well-known pain. JDK-8252842 have added parallel support to heapdump in an attempt to alleviate this issue. However, all concurrent threads competitively write heap data to the same file, and more memory is required to maintain the concurrent buffer queue. In experiments, we did not feel a significant performance improvement from that. >> >> The minor-pause solution, which is presented in this PR, is a two-stage segmented heap dump: >> >> 1. Stage One(STW): Concurrent threads directly write data to multiple heap files. >> 2. Stage Two(Non-STW): Merge multiple heap files into one complete heap dump file. >> >> Now concurrent worker threads are not required to maintain a buffer queue, which would result in more memory overhead, nor do they need to compete for locks. >> >> ### Performance evaluation >> | memory | numOfThread | STW | Total | Compression | >> | --- | --------- | -------------- | ------------ | -------- | >> | 8g | 1 thread | 15.612 secs | 15.612 secs | N | >> | 8g | 32 thread | 2.5617250 secs | 14.498 secs | N | >> | 8g | 32 thread | 2.3084878 secs | 3.198 secs | Compress1 | >> | 8g | 32 thread | 10.9355128 secs | 21.882 secs | Compress2 | >> | 8g | 96 thread | 2.6790452 secs | 14.012 secs | N | >> | 8g | 96 thread | 2.3044796 secs | 3.589 secs | Compress1 | >> | 8g | 96 thread | 9.7585151 secs | 20.219 secs| Compress2 | >> | 16g | 1 thread | 26.278 secs | 26.278 secs | N | >> | 16g | 32 thread | 5.2313740 secs | 26.417 secs | N | >> | 16g | 32 thread | 5.6946983 secs | 6.538 secs | Compress1 | >> | 16g | 32 thread | 21.8211105 secs | 41.133 secs | Compress2 | >> | 16g | 96 thread | 6.2445556 secs | 27.141 secs | N | >> | 16g | 96 thread | 4.6007096 secs | 6.259 secs | Compress1 | >> | 16g | 96 thread | 19.2965783 secs | 39.007 secs | Compress2 | >> | 32g | 1 thread | 48.149 secs | 48.149 secs | N | >> | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | >> | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | >> | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | >> | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | >> | 32g | 96 thread | 9.0954641 secs | 9.885 secs | Compress1 | >> | 32g | 96 thread | 38.9900931 secs | 80.574 secs | Compress2 | >> | 64g | 1 thread | 100.583 secs | 100.583 secs | N | >> | 64g | 32 thread | 20.9233744 secs | 134.701 secs | N | >> | 64g | 32 thread | 18.5023784 secs | 19.358 secs | Compre... > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > undo refactor -- done It looks good in general. I added some change requests. src/hotspot/share/services/heapDumper.cpp line 577: > 575: _in_dump_segment = true; > 576: _is_huge_sub_record = len > buffer_size() - dump_segment_header_size; > 577: ResourceMark rm; Looks like redundant leftover from development src/hotspot/share/services/heapDumper.cpp line 1892: > 1890: } > 1891: > 1892: static int volatile dump_seq = 0; Could you please convert this global variable to field of VM_HeapDumper. The value can be passed to VM_HeapDumpMerge as ctor argument src/hotspot/share/services/heapDumper.cpp line 1894: > 1892: static int volatile dump_seq = 0; > 1893: > 1894: void VM_HeapDumpMerge::merge_done() { Implementation of VM_HeapDumpMerge methods is in the middle of VM_HeapDumper methods. Please put them after VM_HeapDumpMerge declaration (as we have for all other classes) ------------- PR Review: https://git.openjdk.org/jdk/pull/13667#pullrequestreview-1478097941 PR Review Comment: https://git.openjdk.org/jdk/pull/13667#discussion_r1228735182 PR Review Comment: https://git.openjdk.org/jdk/pull/13667#discussion_r1228728069 PR Review Comment: https://git.openjdk.org/jdk/pull/13667#discussion_r1228731666 From jlu at openjdk.org Tue Jun 13 22:09:40 2023 From: jlu at openjdk.org (Justin Lu) Date: Tue, 13 Jun 2023 22:09:40 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v4] In-Reply-To: References: Message-ID: > Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). > > To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) > > For example, > > image > > > Please note that the HTML files only apply to .properties, and not .java resources. Justin Lu has updated the pull request incrementally with three additional commits since the last revision: - Review: correct previous commit, remove ? as well - Review: Remove ?? from 'javac.opt.Xlint.desc.output-file-clash' for zh_CN - Review: Use old 'err.crash' zh_CN value ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14430/files - new: https://git.openjdk.org/jdk/pull/14430/files/7801eefb..f7c06512 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14430&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14430&range=02-03 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14430.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14430/head:pull/14430 PR: https://git.openjdk.org/jdk/pull/14430 From jlu at openjdk.org Tue Jun 13 22:09:43 2023 From: jlu at openjdk.org (Justin Lu) Date: Tue, 13 Jun 2023 22:09:43 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: References: Message-ID: On Tue, 13 Jun 2023 20:56:31 GMT, Weijun Wang wrote: >> Justin Lu has updated the pull request incrementally with one additional commit since the last revision: >> >> Revert 'main.help.opt.any.file' value for _de > > Some comments: > > - `src/java.base/share/classes/sun/launcher/resources/launcher` > > `? private`. "private" should be translated, just like the "static" in the 2nd box. > > - `src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler` > > `compiler.err.annotation.unrecognized.attribute.name`: Here "annotation" is translated to "??"? How do you translate "comment"? > > `compiler.err.enum.label.must.be.enum.constant`: "constant" should be translated. > > `compiler.warn.possible.this.escape`, `compiler.warn.possible.this.escape.location`: "escape" might not mean "??". It's more like its original meaning "run away" or "referenced". > > > - `src/jdk.compiler/share/classes/com/sun/tools/javac/resources/javac`: > > `javac.opt.Xlint.desc.output-file-clash`: "native header files" is translated into "??????". The "??" is strange, I'd rather just say "???" as there is no non-native header file. > > - `src/jdk.jdeps/share/classes/com/sun/tools/javap/resources/javap` > > I don't think the new translation is better than the old one. Thank you for the review @wangweij, I have addressed these recommendations > * `src/jdk.compiler/share/classes/com/sun/tools/javac/resources/javac`: > > > `javac.opt.Xlint.desc.output-file-clash`: "native header files" is translated into "??????". The "??" is strange, I'd rather just say "???" as there is no non-native header file. > > * `src/jdk.jdeps/share/classes/com/sun/tools/javap/resources/javap` > > > I don't think the new translation is better than the old one. For the other comments, I will bring them to the language specialist for updated translations. Ideally we can get those in for this translation drop, but if not, they can go in during the second translation drop during RDP2. I am also hesitant to make too many manual changes, as they will just get overwritten by the tool in the future, so it's best to get it fixed in the tool first. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14430#issuecomment-1590087839 From naoto at openjdk.org Wed Jun 14 00:09:03 2023 From: naoto at openjdk.org (Naoto Sato) Date: Wed, 14 Jun 2023 00:09:03 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v4] In-Reply-To: References: Message-ID: On Tue, 13 Jun 2023 22:09:40 GMT, Justin Lu wrote: >> Please review this PR which updates the JDK's localized resources since the previous L10n translation drop (1/26). >> >> To help with reviewing the changes, @jonathan-gibbons created a tool which displays the localized changes next to the original file's changes in UTF-8 native. Those files can be viewed [here](https://cr.openjdk.org/~jlu/output/) >> >> For example, >> >> image >> >> >> Please note that the HTML files only apply to .properties, and not .java resources. > > Justin Lu has updated the pull request incrementally with three additional commits since the last revision: > > - Review: correct previous commit, remove ? as well > - Review: Remove ?? from 'javac.opt.Xlint.desc.output-file-clash' for zh_CN > - Review: Use old 'err.crash' zh_CN value Looks good, Justin ------------- Marked as reviewed by naoto (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14430#pullrequestreview-1478236336 From iklam at openjdk.org Wed Jun 14 00:19:03 2023 From: iklam at openjdk.org (Ioi Lam) Date: Wed, 14 Jun 2023 00:19:03 GMT Subject: RFR: 8309878: Reduce inclusion of resolvedIndyEntry.hpp In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 22:24:26 GMT, Serguei Spitsyn wrote: >> resolvedIndyEntry.hpp was added in [JDK-8301995](https://bugs.openjdk.org/browse/JDK-8301995) and is included in the popular cpCache.hpp. As a result, resolvedIndyEntry.hpp is included in 807 out of about 1160 hotspot .o files. >> >> The contents of resolvedIndyEntry.hpp is infrequently used. Its inclusion should be moved from cpCache.hpp to cpCache.inline.hpp. This improves hotspot build time. >> >> After this PR, resolvedIndyEntry.hpp is included in only 30 hotspot .o files. > > Looks good. > Thanks, > Serguei Thanks @sspitsyn @coleenp @matias9927 for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14427#issuecomment-1590238854 From iklam at openjdk.org Wed Jun 14 00:19:05 2023 From: iklam at openjdk.org (Ioi Lam) Date: Wed, 14 Jun 2023 00:19:05 GMT Subject: Integrated: 8309878: Reduce inclusion of resolvedIndyEntry.hpp In-Reply-To: References: Message-ID: On Mon, 12 Jun 2023 19:52:36 GMT, Ioi Lam wrote: > resolvedIndyEntry.hpp was added in [JDK-8301995](https://bugs.openjdk.org/browse/JDK-8301995) and is included in the popular cpCache.hpp. As a result, resolvedIndyEntry.hpp is included in 807 out of about 1160 hotspot .o files. > > The contents of resolvedIndyEntry.hpp is infrequently used. Its inclusion should be moved from cpCache.hpp to cpCache.inline.hpp. This improves hotspot build time. > > After this PR, resolvedIndyEntry.hpp is included in only 30 hotspot .o files. This pull request has now been integrated. Changeset: 5d193193 Author: Ioi Lam URL: https://git.openjdk.org/jdk/commit/5d193193a3a4c519e7b3d77b27e6b2bf1b11c7f9 Stats: 86 lines in 35 files changed: 61 ins; 13 del; 12 mod 8309878: Reduce inclusion of resolvedIndyEntry.hpp Reviewed-by: coleenp, sspitsyn, matsaave ------------- PR: https://git.openjdk.org/jdk/pull/14427 From amenkov at openjdk.org Wed Jun 14 01:20:56 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 14 Jun 2023 01:20:56 GMT Subject: RFR: 8309663: test fails "assert(check_alignment(result)) failed: address not aligned: 0x00000008baadbabe" In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 01:02:09 GMT, Alex Menkov wrote: > If virtual thread has frames in stackChunks, need to apply load barriers before processing nmethods (javaVFrame::locals() and javaVFrame::expressions() do it internally) > > Testing: tier1-tier5 (still in progress); > 400 runs of VThreadStackRefTest.java test on linux-x64 and linux-x64-debug with "-XX:+UseZGC -Xcomp -XX:-TieredCompilation" added serviceability as it was not added automatically ------------- PR Comment: https://git.openjdk.org/jdk/pull/14460#issuecomment-1590288091 From dholmes at openjdk.org Wed Jun 14 05:22:19 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 14 Jun 2023 05:22:19 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 Message-ID: Updated the version to 22-ea and year to 2024. The following unpublished changes will also be included in this update: - [JDK-8290626](https://bugs.openjdk.org/browse/JDK-8290626): keytool manpage contains a special character - [JDK-8303928](https://bugs.openjdk.org/browse/JDK-8303928): Update jarsigner man page after [JDK-8303410](https://bugs.openjdk.org/browse/JDK-8303410) - [JDK-8301207](https://bugs.openjdk.org/browse/JDK-8301207): (jdeps) Deprecate jdeps -profile option The following changes were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22: - [JDK-8296656](https://bugs.openjdk.org/browse/JDK-8296656): java.lang.NoClassDefFoundError exception on running fully legitimate code - [JDK-8015831](https://bugs.openjdk.org/browse/JDK-8015831): Add lint check for calling overridable methods from a constructor Thanks. ------------- Commit messages: - 8304478: Initial nroff manpage generation for JDK 22 Changes: https://git.openjdk.org/jdk/pull/14462/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14462&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8304478 Stats: 140 lines in 28 files changed: 1 ins; 105 del; 34 mod Patch: https://git.openjdk.org/jdk/pull/14462.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14462/head:pull/14462 PR: https://git.openjdk.org/jdk/pull/14462 From alanb at openjdk.org Wed Jun 14 06:11:55 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 14 Jun 2023 06:11:55 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 04:53:58 GMT, David Holmes wrote: > Updated the version to 22-ea and year to 2024. > > The following unpublished changes will also be included in this update: > - [JDK-8290626](https://bugs.openjdk.org/browse/JDK-8290626): keytool manpage contains a special character > - [JDK-8303928](https://bugs.openjdk.org/browse/JDK-8303928): Update jarsigner man page after [JDK-8303410](https://bugs.openjdk.org/browse/JDK-8303410) > - [JDK-8301207](https://bugs.openjdk.org/browse/JDK-8301207): (jdeps) Deprecate jdeps -profile option > > The following changes, to `javac.1`, were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22: > - [JDK-8296656](https://bugs.openjdk.org/browse/JDK-8296656): java.lang.NoClassDefFoundError exception on running fully legitimate code > - [JDK-8015831](https://bugs.openjdk.org/browse/JDK-8015831): Add lint check for calling overridable methods from a constructor > > Thanks. Looks fine. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14462#pullrequestreview-1478549017 From dholmes at openjdk.org Wed Jun 14 06:11:55 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 14 Jun 2023 06:11:55 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 06:08:59 GMT, Alan Bateman wrote: >> Updated the version to 22-ea and year to 2024. >> >> The following unpublished changes will also be included in this update: >> - [JDK-8290626](https://bugs.openjdk.org/browse/JDK-8290626): keytool manpage contains a special character >> - [JDK-8303928](https://bugs.openjdk.org/browse/JDK-8303928): Update jarsigner man page after [JDK-8303410](https://bugs.openjdk.org/browse/JDK-8303410) >> - [JDK-8301207](https://bugs.openjdk.org/browse/JDK-8301207): (jdeps) Deprecate jdeps -profile option >> >> The following changes, to `javac.1`, were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22: >> - [JDK-8296656](https://bugs.openjdk.org/browse/JDK-8296656): java.lang.NoClassDefFoundError exception on running fully legitimate code >> - [JDK-8015831](https://bugs.openjdk.org/browse/JDK-8015831): Add lint check for calling overridable methods from a constructor >> >> Thanks. > > Looks fine. Thanks @AlanBateman ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14462#issuecomment-1590531861 From yyang at openjdk.org Wed Jun 14 06:24:28 2023 From: yyang at openjdk.org (Yi Yang) Date: Wed, 14 Jun 2023 06:24:28 GMT Subject: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v9] In-Reply-To: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> References: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> Message-ID: <09RatXWCXaKcJ0qGCB1AjpYwhMvCrpAL1WIu7BXZH6c=.5cae9e7c-719f-461b-9c31-b92d46a9cccc@github.com> > ### Motivation and proposal > Hi, heap dump brings about pauses for application's execution(STW), this is a well-known pain. JDK-8252842 have added parallel support to heapdump in an attempt to alleviate this issue. However, all concurrent threads competitively write heap data to the same file, and more memory is required to maintain the concurrent buffer queue. In experiments, we did not feel a significant performance improvement from that. > > The minor-pause solution, which is presented in this PR, is a two-stage segmented heap dump: > > 1. Stage One(STW): Concurrent threads directly write data to multiple heap files. > 2. Stage Two(Non-STW): Merge multiple heap files into one complete heap dump file. > > Now concurrent worker threads are not required to maintain a buffer queue, which would result in more memory overhead, nor do they need to compete for locks. The changes in the overall design are as follows: > > ![image](https://github.com/openjdk/jdk/assets/5010047/86d2717d-c621-446f-98c2-cce761ec8db5) >

Fig1. Before

> > ![image](https://github.com/openjdk/jdk/assets/5010047/c912c516-83b6-4343-a2e8-5d5bdb99cb87) >

Fig1. After this patch

> > ### Performance evaluation > | memory | numOfThread | STW | Total | Compression | > | --- | --------- | -------------- | ------------ | -------- | > | 8g | 1 thread | 15.612 secs | 15.612 secs | N | > | 8g | 32 thread | 2.5617250 secs | 14.498 secs | N | > | 8g | 32 thread | 2.3084878 secs | 3.198 secs | Compress1 | > | 8g | 32 thread | 10.9355128 secs | 21.882 secs | Compress2 | > | 8g | 96 thread | 2.6790452 secs | 14.012 secs | N | > | 8g | 96 thread | 2.3044796 secs | 3.589 secs | Compress1 | > | 8g | 96 thread | 9.7585151 secs | 20.219 secs| Compress2 | > | 16g | 1 thread | 26.278 secs | 26.278 secs | N | > | 16g | 32 thread | 5.2313740 secs | 26.417 secs | N | > | 16g | 32 thread | 5.6946983 secs | 6.538 secs | Compress1 | > | 16g | 32 thread | 21.8211105 secs | 41.133 secs | Compress2 | > | 16g | 96 thread | 6.2445556 secs | 27.141 secs | N | > | 16g | 96 thread | 4.6007096 secs | 6.259 secs | Compress1 | > | 16g | 96 thread | 19.2965783 secs | 39.007 secs | Compress2 | > | 32g | 1 thread | 48.149 secs | 48.149 secs | N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.0954641 secs | 9.885 secs | C... Yi Yang has updated the pull request incrementally with one additional commit since the last revision: review from Alex ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13667/files - new: https://git.openjdk.org/jdk/pull/13667/files/c8e80e9e..04f54d9a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13667&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13667&range=07-08 Stats: 152 lines in 1 file changed: 78 ins; 70 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/13667.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13667/head:pull/13667 PR: https://git.openjdk.org/jdk/pull/13667 From yyang at openjdk.org Wed Jun 14 06:54:10 2023 From: yyang at openjdk.org (Yi Yang) Date: Wed, 14 Jun 2023 06:54:10 GMT Subject: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v10] In-Reply-To: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> References: <8YqPPHSW4K1s0t317Kp6UqvoGuv5v9oCbjtQ9FX8p2o=.0f6c687b-d031-401d-901d-1ec532715cdc@github.com> Message-ID: > ### Motivation and proposal > Hi, heap dump brings about pauses for application's execution(STW), this is a well-known pain. JDK-8252842 have added parallel support to heapdump in an attempt to alleviate this issue. However, all concurrent threads competitively write heap data to the same file, and more memory is required to maintain the concurrent buffer queue. In experiments, we did not feel a significant performance improvement from that. > > The minor-pause solution, which is presented in this PR, is a two-stage segmented heap dump: > > 1. Stage One(STW): Concurrent threads directly write data to multiple heap files. > 2. Stage Two(Non-STW): Merge multiple heap files into one complete heap dump file. > > Now concurrent worker threads are not required to maintain a buffer queue, which would result in more memory overhead, nor do they need to compete for locks. The changes in the overall design are as follows: > > ![image](https://github.com/openjdk/jdk/assets/5010047/86d2717d-c621-446f-98c2-cce761ec8db5) >

Fig1. Before

> > ![image](https://github.com/openjdk/jdk/assets/5010047/c912c516-83b6-4343-a2e8-5d5bdb99cb87) >

Fig1. After this patch

> > ### Performance evaluation > | memory | numOfThread | STW | Total | Compression | > | --- | --------- | -------------- | ------------ | -------- | > | 8g | 1 thread | 15.612 secs | 15.612 secs | N | > | 8g | 32 thread | 2.5617250 secs | 14.498 secs | N | > | 8g | 32 thread | 2.3084878 secs | 3.198 secs | Compress1 | > | 8g | 32 thread | 10.9355128 secs | 21.882 secs | Compress2 | > | 8g | 96 thread | 2.6790452 secs | 14.012 secs | N | > | 8g | 96 thread | 2.3044796 secs | 3.589 secs | Compress1 | > | 8g | 96 thread | 9.7585151 secs | 20.219 secs| Compress2 | > | 16g | 1 thread | 26.278 secs | 26.278 secs | N | > | 16g | 32 thread | 5.2313740 secs | 26.417 secs | N | > | 16g | 32 thread | 5.6946983 secs | 6.538 secs | Compress1 | > | 16g | 32 thread | 21.8211105 secs | 41.133 secs | Compress2 | > | 16g | 96 thread | 6.2445556 secs | 27.141 secs | N | > | 16g | 96 thread | 4.6007096 secs | 6.259 secs | Compress1 | > | 16g | 96 thread | 19.2965783 secs | 39.007 secs | Compress2 | > | 32g | 1 thread | 48.149 secs | 48.149 secs | N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.0954641 secs | 9.885 secs | C... Yi Yang has updated the pull request incrementally with one additional commit since the last revision: whitespace ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13667/files - new: https://git.openjdk.org/jdk/pull/13667/files/04f54d9a..2280159a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13667&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13667&range=08-09 Stats: 2 lines in 2 files changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/13667.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13667/head:pull/13667 PR: https://git.openjdk.org/jdk/pull/13667 From sspitsyn at openjdk.org Wed Jun 14 09:10:59 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 14 Jun 2023 09:10:59 GMT Subject: RFR: 8309663: test fails "assert(check_alignment(result)) failed: address not aligned: 0x00000008baadbabe" In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 01:02:09 GMT, Alex Menkov wrote: > If virtual thread has frames in stackChunks, need to apply load barriers before processing nmethods (javaVFrame::locals() and javaVFrame::expressions() do it internally) > > Testing: tier1-tier5 (still in progress); > 400 runs of VThreadStackRefTest.java test on linux-x64 and linux-x64-debug with "-XX:+UseZGC -Xcomp -XX:-TieredCompilation" src/hotspot/share/prims/jvmtiTagMap.cpp line 2325: > 2323: // Need to apply load barriers for unmounted vthreads. > 2324: nmethod* nm = jvf->cb()->as_nmethod(); > 2325: nm->run_nmethod_entry_barrier(); This looks good in general. However, I'm looking at the `stackChunkOopDesc::do_barriers0()` implementation and wonder if a similar trick is needed for interpreter frames (to support `Class Redefinition`): template void stackChunkOopDesc::do_barriers0(const StackChunkFrameStream& f, const RegisterMapT* map) { // We need to invoke the write barriers so as not to miss oops in old chunks that haven't yet been concurrently scanned assert (!f.is_done(), ""); if (f.is_interpreted()) { Method* m = f.to_frame().interpreter_frame_method(); // Class redefinition support m->record_gc_epoch(); } else if (f.is_compiled()) { nmethod* nm = f.cb()->as_nmethod(); // The entry barrier takes care of having the right synchronization // when keeping the nmethod alive during concurrent execution. nm->run_nmethod_entry_barrier(); // There is no need to mark the Method, as class redefinition will walk the // CodeCache, noting their Methods } . . . ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14460#discussion_r1229273861 From sspitsyn at openjdk.org Wed Jun 14 09:28:11 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 14 Jun 2023 09:28:11 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 04:53:58 GMT, David Holmes wrote: > Updated the version to 22-ea and year to 2024. > > The following unpublished changes will also be included in this update: > - [JDK-8290626](https://bugs.openjdk.org/browse/JDK-8290626): keytool manpage contains a special character > - [JDK-8303928](https://bugs.openjdk.org/browse/JDK-8303928): Update jarsigner man page after [JDK-8303410](https://bugs.openjdk.org/browse/JDK-8303410) > - [JDK-8301207](https://bugs.openjdk.org/browse/JDK-8301207): (jdeps) Deprecate jdeps -profile option > > The following changes, to `javac.1`, were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22: > - [JDK-8296656](https://bugs.openjdk.org/browse/JDK-8296656): java.lang.NoClassDefFoundError exception on running fully legitimate code > - [JDK-8015831](https://bugs.openjdk.org/browse/JDK-8015831): Add lint check for calling overridable methods from a constructor > > Thanks. The Serviceability changes look good. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14462#pullrequestreview-1478940868 From kevinw at openjdk.org Wed Jun 14 11:47:13 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Wed, 14 Jun 2023 11:47:13 GMT Subject: RFR: 8303916: ThreadLists.java inconsistent results Message-ID: This test fails when additional threads are created, as it compares thread lists from different sources and sees a mismatch. It needs to tolerate some threads with certain names appearing, e.g. JVMCI and ForkJoin threads. If there are extra threads in the thread ID list from the ThreadMBean, remove them from the thread count before comparing. When it fails, additionally print the ids and names of threads that it got from the MBean. I found the test quite confusing, the naming of thread counts, arrays of threads, and arrays of thread stacktraces from different sources, so have renamed some things. ------------- Commit messages: - More concise printing of thread id+name on failure. - Merge remote-tracking branch 'upstream/master' into 8303916_ThreadLists_test - 8303916: ThreadLists.java inconsistent results Changes: https://git.openjdk.org/jdk/pull/14465/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14465&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303916 Stats: 77 lines in 1 file changed: 54 ins; 1 del; 22 mod Patch: https://git.openjdk.org/jdk/pull/14465.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14465/head:pull/14465 PR: https://git.openjdk.org/jdk/pull/14465 From jlu at openjdk.org Wed Jun 14 16:25:00 2023 From: jlu at openjdk.org (Justin Lu) Date: Wed, 14 Jun 2023 16:25:00 GMT Subject: RFR: 8309632: JDK 21 RDP1 L10n resource files update [v2] In-Reply-To: <05VweeA-JZy6fvHzscLEDLi4nPZM0xhDiaOqYcceMgI=.2574b0d1-92ea-4319-9c45-dbd9df91ad4f@github.com> References: <05VweeA-JZy6fvHzscLEDLi4nPZM0xhDiaOqYcceMgI=.2574b0d1-92ea-4319-9c45-dbd9df91ad4f@github.com> Message-ID: On Tue, 13 Jun 2023 18:50:42 GMT, Justin Lu wrote: >> src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_de.properties line 140: >> >>> 138: >>> 139: # 0: type, 1: name >>> 140: compiler.err.annotation.unrecognized.attribute.name=Annotation @{0} weist das unbekannte Attribut "{1}" auf >> >> Is it intentional to replace single quotes with double quotes? It is done in several locations in this file, but other languages keep using single quotes as in the source English resource file. > > I will consult the language specialist on this. I remember in previous discussions that certain languages would format punctuation differently, but let me ask about this specific instance. The language specialist confirmed the double quotes are locale specific for de. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14430#discussion_r1229881212 From mchung at openjdk.org Wed Jun 14 18:43:57 2023 From: mchung at openjdk.org (Mandy Chung) Date: Wed, 14 Jun 2023 18:43:57 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 04:53:58 GMT, David Holmes wrote: > Updated the version to 22-ea and year to 2024. > > The following unpublished changes will also be included in this update: > - [JDK-8290626](https://bugs.openjdk.org/browse/JDK-8290626): keytool manpage contains a special character > - [JDK-8303928](https://bugs.openjdk.org/browse/JDK-8303928): Update jarsigner man page after [JDK-8303410](https://bugs.openjdk.org/browse/JDK-8303410) > - [JDK-8301207](https://bugs.openjdk.org/browse/JDK-8301207): (jdeps) Deprecate jdeps -profile option > > The following changes, to `javac.1`, were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22: > - [JDK-8296656](https://bugs.openjdk.org/browse/JDK-8296656): java.lang.NoClassDefFoundError exception on running fully legitimate code > - [JDK-8015831](https://bugs.openjdk.org/browse/JDK-8015831): Add lint check for calling overridable methods from a constructor > > Thanks. Marked as reviewed by mchung (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14462#pullrequestreview-1480088199 From lancea at openjdk.org Wed Jun 14 18:51:10 2023 From: lancea at openjdk.org (Lance Andersen) Date: Wed, 14 Jun 2023 18:51:10 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 04:53:58 GMT, David Holmes wrote: > Updated the version to 22-ea and year to 2024. > > The following unpublished changes will also be included in this update: > - [JDK-8290626](https://bugs.openjdk.org/browse/JDK-8290626): keytool manpage contains a special character > - [JDK-8303928](https://bugs.openjdk.org/browse/JDK-8303928): Update jarsigner man page after [JDK-8303410](https://bugs.openjdk.org/browse/JDK-8303410) > - [JDK-8301207](https://bugs.openjdk.org/browse/JDK-8301207): (jdeps) Deprecate jdeps -profile option > > The following changes, to `javac.1`, were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22: > - [JDK-8296656](https://bugs.openjdk.org/browse/JDK-8296656): java.lang.NoClassDefFoundError exception on running fully legitimate code > - [JDK-8015831](https://bugs.openjdk.org/browse/JDK-8015831): Add lint check for calling overridable methods from a constructor > > Thanks. Marked as reviewed by lancea (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14462#pullrequestreview-1480097706 From acobbs at openjdk.org Wed Jun 14 19:23:59 2023 From: acobbs at openjdk.org (Archie Cobbs) Date: Wed, 14 Jun 2023 19:23:59 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 04:53:58 GMT, David Holmes wrote: > The following changes, to javac.1, were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22 Just curious, since you have access to the secret closed sources, can you not backport these changes yourself? Instead of just deleting them and expecting someone else to rescue them from oblivion? To be clear, I'm not blaming you for this situation - the problem is that there exist "closed sources" at all (why??) - but that doesn't seem like a good excuse for deleting work without a clear path to ensuring it is preserved (and the simplest way to do that is to do it yourself). Obviously I'd do it myself if I could. Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14462#issuecomment-1591849483 From amenkov at openjdk.org Wed Jun 14 19:31:04 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 14 Jun 2023 19:31:04 GMT Subject: RFR: 8309663: test fails "assert(check_alignment(result)) failed: address not aligned: 0x00000008baadbabe" In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 09:07:37 GMT, Serguei Spitsyn wrote: > This looks good in general. However, I'm looking at the `stackChunkOopDesc::do_barriers0()` implementation and wonder if a similar trick is needed for interpreter frames (to support `Class Redefinition`): > This code collects references from thread stack, I don't see how class redefinition can affect them ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14460#discussion_r1230074035 From cjplummer at openjdk.org Wed Jun 14 19:35:56 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 14 Jun 2023 19:35:56 GMT Subject: RFR: 8303916: ThreadLists.java inconsistent results In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 09:30:49 GMT, Kevin Walls wrote: > This test fails when additional threads are created, as it compares thread lists from different sources and sees a mismatch. > > It needs to tolerate some threads with certain names appearing, e.g. JVMCI and ForkJoin threads. If there are extra threads in the thread ID list from the ThreadMBean, remove them from the thread count before comparing. > > When it fails, additionally print the ids and names of threads that it got from the MBean. > > I found the test quite confusing, the naming of thread counts, arrays of threads, and arrays of thread stacktraces from different sources, so have renamed some things. Overall looks good, but I'm surprised you only need to filter "ForkJoinPool" and "JVMCI". What about threads like "Common-Cleaner"? https://github.com/openjdk/jdk/blob/57b8251241e2044d5039ce162bf4637a9b2e5466/test/hotspot/jtreg/vmTestbase/nsk/share/jdi/EventFilters.java#L333 ------------- PR Review: https://git.openjdk.org/jdk/pull/14465#pullrequestreview-1480164185 From kevinw at openjdk.org Wed Jun 14 19:46:57 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Wed, 14 Jun 2023 19:46:57 GMT Subject: RFR: 8303916: ThreadLists.java inconsistent results In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 09:30:49 GMT, Kevin Walls wrote: > This test fails when additional threads are created, as it compares thread lists from different sources and sees a mismatch. > > It needs to tolerate some threads with certain names appearing, e.g. JVMCI and ForkJoin threads. If there are extra threads in the thread ID list from the ThreadMBean, remove them from the thread count before comparing. > > When it fails, additionally print the ids and names of threads that it got from the MBean. > > I found the test quite confusing, the naming of thread counts, arrays of threads, and arrays of thread stacktraces from different sources, so have renamed some things. > Thanks - that's a useful reference list. For this test, it's never managed to not have Common-Cleaner in the initial set of threads it observes, I think... But it's very easy to see a couple of ForkJoinPool-x-worker-x threads, and see one more of them by the time it gets threads from the MXBean. That's the way I see it failing for us now. If we do see others appear, I hope having added the extra printing it will be easy to add other names, if they affect this test... ------------- PR Comment: https://git.openjdk.org/jdk/pull/14465#issuecomment-1591874809 From dholmes at openjdk.org Wed Jun 14 21:31:00 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 14 Jun 2023 21:31:00 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 19:21:01 GMT, Archie Cobbs wrote: > Just curious, since you have access to the secret closed sources, can you not backport these changes yourself? Instead of just deleting them and expecting someone else to rescue them from oblivion? @archiecobbs we (Oracle) will take care of restoring this text so please don't be concerned about that. It will just be a temporary glitch. It should have been handled when the original PRs were done. It needs to be handled as a separate issue and PR though - whether that were to happen before or after this PR is somewhat immaterial. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14462#issuecomment-1592014026 From dholmes at openjdk.org Wed Jun 14 21:31:03 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 14 Jun 2023 21:31:03 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 09:25:14 GMT, Serguei Spitsyn wrote: >> Updated the version to 22-ea and year to 2024. >> >> The following unpublished changes will also be included in this update: >> - [JDK-8290626](https://bugs.openjdk.org/browse/JDK-8290626): keytool manpage contains a special character >> - [JDK-8303928](https://bugs.openjdk.org/browse/JDK-8303928): Update jarsigner man page after [JDK-8303410](https://bugs.openjdk.org/browse/JDK-8303410) >> - [JDK-8301207](https://bugs.openjdk.org/browse/JDK-8301207): (jdeps) Deprecate jdeps -profile option >> >> The following changes, to `javac.1`, were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22: >> - [JDK-8296656](https://bugs.openjdk.org/browse/JDK-8296656): java.lang.NoClassDefFoundError exception on running fully legitimate code >> - [JDK-8015831](https://bugs.openjdk.org/browse/JDK-8015831): Add lint check for calling overridable methods from a constructor >> >> Thanks. > > The Serviceability changes look good. > Thanks, > Serguei Thanks for the additional reviews @sspitsyn , @LanceAndersen and @mlchung ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14462#issuecomment-1592014943 From cjplummer at openjdk.org Wed Jun 14 21:32:09 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 14 Jun 2023 21:32:09 GMT Subject: RFR: 8308499: Test vmTestbase/nsk/jdi/MethodExitRequest/addClassExclusionFilter/filter001/TestDescription.java failed: VMDisconnectedException Message-ID: <-0tDx5i1vrT6MQiPY9ZzRMVE6SibI7vqOJMjPuGywqU=.b25f2536-bfbf-4754-a56d-05f5a73fd3ba@github.com> The test gets overloaded with MethodExitEvents, causing them to queue up (in the JDI queue) and continue to be processed by the test after the debuggee exits. This results in a VMDisconnectedException when the test tries to do the following after getting an event: ` String str = ((MethodExitEvent)event).location().declaringType().name(); ` The test is suppose to add filters to execlude events for all non-test classes, but it is only filtering `java.*` and `sun.*`. There are also a very large number of `jdk.*` events coming in, and this is what is causing the event backlog and the processing of events after disconnect. Filtering out `jdk.*` events reduces the total number of events to a few dozen, and the test passes. More details can be found in the CR. Testing in progress: tier5 svc testing, and also running the failing test on all platforms (both debug and product). ------------- Commit messages: - filter jdk.* clases Changes: https://git.openjdk.org/jdk/pull/14480/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14480&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8308499 Stats: 7 lines in 1 file changed: 5 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/14480.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14480/head:pull/14480 PR: https://git.openjdk.org/jdk/pull/14480 From acobbs at openjdk.org Wed Jun 14 21:40:57 2023 From: acobbs at openjdk.org (Archie Cobbs) Date: Wed, 14 Jun 2023 21:40:57 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 21:28:01 GMT, David Holmes wrote: > > Just curious, since you have access to the secret closed sources, can you not backport these changes yourself? Instead of just deleting them and expecting someone else to rescue them from oblivion? > > @archiecobbs we (Oracle) will take care of restoring this text so please don't be concerned about that. It will just be a temporary glitch. OK thanks. Putting my trust in you :) ------------- PR Comment: https://git.openjdk.org/jdk/pull/14462#issuecomment-1592023452 From lmesnik at openjdk.org Wed Jun 14 21:56:56 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Wed, 14 Jun 2023 21:56:56 GMT Subject: RFR: 8308499: Test vmTestbase/nsk/jdi/MethodExitRequest/addClassExclusionFilter/filter001/TestDescription.java failed: VMDisconnectedException In-Reply-To: <-0tDx5i1vrT6MQiPY9ZzRMVE6SibI7vqOJMjPuGywqU=.b25f2536-bfbf-4754-a56d-05f5a73fd3ba@github.com> References: <-0tDx5i1vrT6MQiPY9ZzRMVE6SibI7vqOJMjPuGywqU=.b25f2536-bfbf-4754-a56d-05f5a73fd3ba@github.com> Message-ID: On Wed, 14 Jun 2023 21:24:31 GMT, Chris Plummer wrote: > The test gets overloaded with MethodExitEvents, causing them to queue up (in the JDI queue) and continue to be processed by the test after the debuggee exits. This results in a VMDisconnectedException when the test tries to do the following after getting an event: > > ` String str = ((MethodExitEvent)event).location().declaringType().name(); ` > > The test is suppose to add filters to execlude events for all non-test classes, but it is only filtering `java.*` and `sun.*`. There are also a very large number of `jdk.*` events coming in, and this is what is causing the event backlog and the processing of events after disconnect. Filtering out `jdk.*` events reduces the total number of events to a few dozen, and the test passes. > > More details can be found in the CR. > > Testing in progress: tier5 svc testing, and also running the failing test on all platforms (both debug and product). Changes requested by lmesnik (Reviewer). test/hotspot/jtreg/vmTestbase/nsk/jdi/MethodExitRequest/addClassExclusionFilter/filter001.java line 77: > 75: private String classExclName1 = "java"; > 76: private String classExclName2 = "sun"; > 77: private String classExclName3 = "jdk"; The fix is reasonable. However, I think it would be better to use List with List.of(...) and list.contains() to filter exclusions. ------------- PR Review: https://git.openjdk.org/jdk/pull/14480#pullrequestreview-1480363171 PR Review Comment: https://git.openjdk.org/jdk/pull/14480#discussion_r1230209088 From dholmes at openjdk.org Wed Jun 14 22:11:57 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 14 Jun 2023 22:11:57 GMT Subject: RFR: 8304478: Initial nroff manpage generation for JDK 22 In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 04:53:58 GMT, David Holmes wrote: > Updated the version to 22-ea and year to 2024. > > The following unpublished changes will also be included in this update: > - [JDK-8290626](https://bugs.openjdk.org/browse/JDK-8290626): keytool manpage contains a special character > - [JDK-8303928](https://bugs.openjdk.org/browse/JDK-8303928): Update jarsigner man page after [JDK-8303410](https://bugs.openjdk.org/browse/JDK-8303410) > - [JDK-8301207](https://bugs.openjdk.org/browse/JDK-8301207): (jdeps) Deprecate jdeps -profile option > > The following changes, to `javac.1`, were never applied to the closed sources and are "lost" by this update. These changes will need to be re-applied directly in JDK 21 and JDK 22: > - [JDK-8296656](https://bugs.openjdk.org/browse/JDK-8296656): java.lang.NoClassDefFoundError exception on running fully legitimate code > - [JDK-8015831](https://bugs.openjdk.org/browse/JDK-8015831): Add lint check for calling overridable methods from a constructor > > Thanks. I've filed https://bugs.openjdk.org/browse/JDK-8310067 for the javac manpage fixup. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14462#issuecomment-1592056009 From dholmes at openjdk.org Thu Jun 15 02:17:15 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 15 Jun 2023 02:17:15 GMT Subject: RFR: 8303916: ThreadLists.java inconsistent results In-Reply-To: References: Message-ID: On Wed, 14 Jun 2023 09:30:49 GMT, Kevin Walls wrote: > This test fails when additional threads are created, as it compares thread lists from different sources and sees a mismatch. > > It needs to tolerate some threads with certain names appearing, e.g. JVMCI and ForkJoin threads. If there are extra threads in the thread ID list from the ThreadMBean, remove them from the thread count before comparing. > > When it fails, additionally print the ids and names of threads that it got from the MBean. > > I found the test quite confusing, the naming of thread counts, arrays of threads, and arrays of thread stacktraces from different sources, so have renamed some things. The test assumes that it can find a set of steady-state threads running that does not change, but that is not the case. I don't find this test particularly useful in what it attempts to do. It would make more sense to have a set of test threads that it looks for via each method, and which it can control termination of and thus check that they disappear. Adding an allow-list of dynamically appearing threads seems somewhat fragile, but I guess we've really only seen two problematic scenarios: - Running with virtual threads - Running with JVMCI so the fix "works". ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/14465#pullrequestreview-1480552511 From cjplummer at openjdk.org Thu Jun 15 03:11:58 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 15 Jun 2023 03:11:58 GMT Subject: RFR: 8308499: Test vmTestbase/nsk/jdi/MethodExitRequest/addClassExclusionFilter/filter001/TestDescription.java failed: VMDisconnectedException In-Reply-To: References: <-0tDx5i1vrT6MQiPY9ZzRMVE6SibI7vqOJMjPuGywqU=.b25f2536-bfbf-4754-a56d-05f5a73fd3ba@github.com> Message-ID: On Wed, 14 Jun 2023 21:54:22 GMT, Leonid Mesnik wrote: >> The test gets overloaded with MethodExitEvents, causing them to queue up (in the JDI queue) and continue to be processed by the test after the debuggee exits. This results in a VMDisconnectedException when the test tries to do the following after getting an event: >> >> ` String str = ((MethodExitEvent)event).location().declaringType().name(); ` >> >> The test is suppose to add filters to execlude events for all non-test classes, but it is only filtering `java.*` and `sun.*`. There are also a very large number of `jdk.*` events coming in, and this is what is causing the event backlog and the processing of events after disconnect. Filtering out `jdk.*` events reduces the total number of events to a few dozen, and the test passes. >> >> More details can be found in the CR. >> >> Testing in progress: tier5 svc testing, and also running the failing test on all platforms (bot